Differential Data Structure Chain for Document Store Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Document databases face inefficiencies in updating data, as they often require duplicating entire documents in memory, leading to prolonged update operations, especially when only a small portion of the data is updated, due to the need for full copies of documents during updates.
Innovation Solution
Implementing a data storage system with chains of linked differential data structures, where each differential data structure contains only the updated members relative to a preceding version, allowing for efficient storage and retrieval by traversing the chain to produce a full updated document version.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire document is duplicated into a new version during update, then the document database maintains data integrity and versioning, but the update operation time increases significantly
Solution Approach 1:
The document update process is segmented into two parts: the base document version and the differential data structure containing only the updated members. Instead of duplicating the entire document, the system divides the storage into a shared base portion and a specific differential portion, reducing the amount of data that needs to be copied and processed during updates.
Solution Approach 2:
The differential data structure extracts and stores only the changed members from the full document. By separating the updated portions from the unchanged base document, the system eliminates the need to process and store redundant data, significantly reducing update operation time while maintaining complete version information.
2Reliability
If the entire document is duplicated into a new version during update, then version history is maintained, but the memory allocation and storage requirements increase
Solution Approach 1:
Storage is segmented into a base document structure and differential data structures. The base document contains the full document data, while subsequent versions store only differential data structures with updated members. This segmentation reduces the total quantity of stored data while preserving complete version history through the chain of differential structures.
Solution Approach 2:
The differential data structures are nested within a chain that references the base document and preceding differential structures. Each differential structure contains a pointer to the previous version, creating a nested hierarchy where the full document can be reconstructed by combining the base document with all differential updates in the chain, reducing overall storage requirements.
3Loss of time
If only updated members are stored in differential data structures, then update operation time decreases, but the data structure complexity increases
Solution Approach 1:
The data structure is segmented into a standardized base document format and standardized differential data structure format. Each differential structure follows a consistent template with identifiers for updated members and pointers to the previous version. This segmentation provides a regular, predictable structure that reduces complexity compared to managing full document copies with varying content.
4Ease of operation
If the entire document is duplicated during update, then retrieval is straightforward, but storage space consumption increases
Solution Approach 1:
Storage space is segmented into a base document containing the full original data and differential data structures containing only updates. This segmentation dramatically reduces the volume of stationary storage objects while retrieval remains straightforward through the systematic chain of differential structures that can be applied sequentially to reconstruct any version.
Data Source
AI summary
A data storage and retrieval system for a computer memory configured according to a document store adapted to contain documents. The document store includes a document data structure containing a header and a document having one or more members. The document store further includes a chain of one or more differential data structures, each having a header and at least one updated member. The updated member has a value which is updated relative to a value in a corresponding member having a same identifier in the document data structure or in a preceding differential data structure in the chain. A header of each of the differential data structures includes a pointer to the document data structure or the preceding differential data structure in the chain, and a flag indicating that each of the differential data structures contains less than all of the members of the document data structure.