Shared String Referencing for Multi-Data-Set Size Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The exponential growth of data storage demands exceeds the capacity of existing storage systems, leading to increased costs and the need to choose between purchasing more storage or reducing data retention periods, as current technologies fail to efficiently compress data sets without significant loss of information.
Innovation Solution
A computing device identifies common strings in multiple data sets, assigns references to these strings, and replaces their occurrences, repeatedly performing this process until a predetermined size reduction threshold is met, thereby compressing data sets and reducing storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If additional storage space is purchased to accommodate increased data amounts, then storage capacity is improved, but storage cost increases
Solution Approach 1:
The patent merges multiple data sets by identifying and replacing common strings with shared references across different data sets. This consolidation reduces the total storage volume required while maintaining data integrity and accessibility, thereby decreasing storage costs without sacrificing capacity.
Solution Approach 2:
Instead of storing duplicate common strings in each data set, the patent creates a single reference copy that is shared across multiple data sets. This eliminates redundant storage while preserving the ability to retrieve and use the original data, effectively reducing storage requirements without losing information.
2Duration of action of stationary object
If data retention period is extended to store data for longer periods, then data availability is improved, but storage cost increases
Solution Approach 1:
By merging common strings across multiple data sets into shared references, the patent reduces the overall storage footprint. This enables organizations to retain data for longer periods (extending the duration) without proportionally increasing storage costs, as the compressed storage format requires less space over time.
3Volume of stationary object
If data compression is applied to reduce storage size, then storage efficiency is improved, but data integrity may be compromised
Solution Approach 1:
The patent uses reference copying rather than lossy compression. Each common string is replaced with a reference that points to the original string stored once. This method reduces data set size while guaranteeing data integrity, as the references can be resolved back to the exact original values without any information loss or corruption.
Data Source
AI summary
A computing device may select a plurality of data sets, determine a set of strings that are included in at least two data sets of the plurality of data sets, and select a particular string of the set of strings. The computing device may replace each occurrence of the particular string in the plurality of data sets to create a modified plurality of data sets such that the modified plurality of data sets is smaller in size than the plurality of data sets. The computing device may assign a reference to the particular string and replace each occurrence of the particular string in the plurality of data sets to create a plurality of modified data sets. The computing device may replace may store the reference and the particular string in a table.