Semantic Redundancy Elimination in Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage and management techniques are inefficient due to the inability to optimally remove data redundancy and compress data structures, leading to high computational and energy costs, as they focus on meta-characteristics and syntactic levels rather than semantic matching.
Innovation Solution
A redundancy elimination system that identifies unique segments within data objects, generates an embedding space based on these segments, estimates semantic proximities, builds a semantic cohesion network, and sorts data objects into semantically cohesive clusters to dynamically eliminate redundant data, thereby reducing energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current data storage techniques focus on meta-characteristics and syntactic levels, then data can be stored in traditional data structures, but redundancy elimination is inefficient and energy costs are high
Solution Approach 1:
The patent segments data objects into constituent elements (smallest meaningful units) and processes them at the element level rather than treating entire data objects as atomic units. This segmentation enables fine-grained redundancy detection where equivalent elements across different data objects can be identified and eliminated, significantly improving redundancy elimination efficiency while reducing energy consumption through more targeted processing.
Solution Approach 2:
The patent transforms data from traditional storage parameters (meta-characteristics like file size, type, date) to semantic parameters (embeddings representing meaning). By changing the parameter space from syntactic to semantic, the system can detect redundancies that are invisible at the syntactic level, achieving better compression ratios and reducing the energy required for meaningful redundancy elimination.
2Measurement precision
If semantic matching is used instead of meta-characteristics, then redundancy elimination accuracy improves, but computational complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-computing embeddings for constituent elements and organizing them in advance. This preprocessing step converts complex semantic matching into simpler vector space operations, reducing the computational complexity during actual redundancy detection while maintaining high accuracy through the use of pre-established semantic representations.
Solution Approach 2:
The patent uses embeddings as simplified copies or representations of the actual data semantics. Instead of performing complex semantic analysis on the original data during redundancy detection, the system works with compact embedding vectors that capture the essential semantic information, significantly reducing computational complexity while preserving detection accuracy.
3Ease of manufacture
If data is stored without semantic compression, then storage operations are simple, but storage space efficiency is low due to redundant data
Solution Approach 1:
The patent segments data into constituent elements and stores only unique elements with references to their occurrences. This segmentation approach maintains operational simplicity by using a straightforward storage model (store unique elements, reference occurrences) while dramatically improving storage space efficiency through the elimination of redundant element copies across multiple data objects.
Data Source
AI summary
A device may identify unique segments within data objects, of an object corpus stored in a data structure, as elements, and may generate an embedding space based on unique elements and mappings of the data objects to embeddings. The device may estimate semantic proximities among the data objects based on the mappings, and may build a semantic cohesion network among the data objects based on the semantic proximities. The device may identify semantically cohesive data clusters in the semantic cohesion network, and may sort the data objects in the semantically cohesive data clusters. The device may determine, from the semantically cohesive and sorted data clusters, a home data cluster for a new data object, and may store bookkeeping details of the new data object in the data structure based on the new data object being semantically similar to the data object in the home data cluster.