Compressed JSON Piecewise Updates Without Full Document Rewrites
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for updating JSON documents in persistent storage often require full document replacement, leading to inefficient performance due to large change logs and increased replication costs, especially when only minor changes are made, as they lack efficient mechanisms for partial updates.
Innovation Solution
Implementing a declarative SQL-based approach for piecewise updates using SQL functions like JSON_MERGEPATCH and JSON_TRANSFORM, and designing a binary format like OSON that supports partial updates by reorganizing itself based on changes, allowing for efficient recording and application of changes without rewriting the entire document.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If full document replacement is used for JSON updates, then update simplicity is maintained, but change log size and replication costs increase significantly
Solution Approach 1:
The patent segments the JSON document into hierarchical paths (e.g., '$.person.name', '$.address.city') and tracks changes at the granular path level rather than treating the entire document as a single unit. This allows the change log to record only the specific paths that were modified, dramatically reducing change log size while maintaining update simplicity through the use of path-based change tracking structures.
2Ease of manufacture
If full document replacement is performed, then update implementation is straightforward, but performance and replication efficiency deteriorate
Solution Approach 1:
The system segments the update operation into path-based modifications rather than full document replacement. By identifying and updating only the specific paths that changed (e.g., updating just '$.person.name' instead of the entire JSON document), the system improves update performance and reduces replication bandwidth requirements while maintaining straightforward implementation through path-based update mechanisms.
Solution Approach 2:
The patent applies partial action by performing only the necessary portion of the update - modifying only the specific paths that changed rather than replacing the entire document. This partial update approach sends smaller change logs for replication and reduces the amount of data that needs to be processed, directly improving update performance and replication efficiency.
3Productivity
If binary formats like BSON are used, then limited partial update capability is provided, but flexibility for general partial updates is restricted
Solution Approach 1:
The patent extends partial update capability by segmenting changes into fine-grained path-level units and providing operators that can compose these segments into complex update patterns. This allows flexible updates such as merging patches, transforming data along paths, and conditional updates based on path matching, going beyond the limited scalar replacement capability of traditional binary formats.
Solution Approach 2:
The system provides a universal path-based update framework that can handle multiple types of updates through a common interface - scalar replacements, array insertions/deletions, object merges, and transformations all operate through the same path-based mechanism. This multi-functional approach provides both the performance of binary formats and the flexibility of high-level update operations.
Data Source
AI summary
Herein are fine grained updates to pieces of JavaScript object notation (JSON) documents by database statements that can update, delete, and insert parts of JSON documents. In an embodiment, a computer receives a request that specifies a modification of a JSON document that is stored in a compressed format in persistent storage. The modification adds additional content to the JSON document, and/or replaces an old value in the JSON document with a new value that is not a same size as the old value. The modification is recorded in a change log. The change log is eventually applied to the compressed format of the JSON document in the persistent storage without entirely rewriting the compressed format of the JSON document in the persistent storage.