Data Storage Redundancy Elimination via Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies due to unwanted redundancy, as they often store multiple copies of identical data fragments, which increases storage requirements and slows down operations, especially in handling large volumes of diverse data types like text documents, images, and multimedia files.
Innovation Solution
The method involves breaking down informational objects into multiple components, analyzing each for uniqueness, and storing only new components while generating relational links for reconstruction, allowing for efficient storage by eliminating redundant data and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple copies of identical data fragments are stored to ensure data availability, then data reliability is improved, but storage space is wasted due to redundancy
Solution Approach 1:
The patent divides informational objects into multiple data objects or chunks, allowing the system to store only unique segments. By segmenting data into identifiable units with unique identifiers, the system can efficiently detect and eliminate redundant copies while maintaining data availability through selective storage of unique segments.
Solution Approach 2:
The patent uses digital signatures or checksums as simplified copies to represent the entire data object. Instead of storing multiple full copies of data, the system stores these verification copies (signatures/checksums) that can identify and reference the original data, thereby reducing storage requirements while ensuring data integrity and availability.
2Reliability
If all data objects are stored in their entirety, then data integrity is maintained, but storage efficiency deteriorates due to redundant storage
Solution Approach 1:
The patent extracts the essential identifying characteristic of each data object (its digital signature or checksum) and stores this extracted feature instead of the entire data object. This extraction process allows the system to verify data integrity through the stored signature while occupying minimal storage space, thereby resolving the contradiction between maintaining integrity and improving storage efficiency.
Solution Approach 2:
The patent transforms the data representation from storing complete data objects to storing derived parameters (digital signatures, checksums, or hash values). This parameter transformation maintains the ability to verify data integrity while dramatically reducing the storage requirements, as these parameter representations are much smaller than the original data objects.
3Adaptability or versatility
If data is divided into multiple parts for storage, then storage flexibility is improved, but system complexity increases due to reconstruction requirements
Solution Approach 1:
The patent performs preliminary organization of divided data parts by storing metadata that describes the structure, relationships, and assembly requirements of data segments before retrieval operations. This preliminary action includes creating and storing information about how divided parts should be reassembled, which simplifies the reconstruction process and reduces system complexity during data retrieval operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for effective data storing allowing reducing storage requirements and speeding up various storage operations by reducing storage of redundant data is disclosed. The method includes receiving new informational objects (such as computer files) for storage and breaking down the new informational objects into multiple data objects. Each data object is analyzed to determine whether an identical data object already exists in a database. If an identical data object already exists in the database, the new data object is not stored. If, on the other hand, an identical data object does not exist in the database already, the new data object is stored. Furthermore, for each informational object to be stored, relational links are generated which associate the data objects and their storage locations with the informational object such that the informational object can be easily recreated when required.