Compressed JSON Piecewise Updates With Path-Based Change Logging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing JSON document update methods in persistent storage require full document rewriting, leading to inefficient performance and increased change log size, especially when only minor changes are needed, affecting data manipulation language (DML) and replication performance.
Innovation Solution
Implementing a binary format like OSON that supports piecewise updates by autonomously reorganizing the document based on accumulated changes, using SQL functions for declarative updates and minimizing full document updates, with a change log size proportional to the actual changes made.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If full JSON document replacement is used for updates, then update simplicity is maintained, but update performance deteriorates and change log size increases
Solution Approach 1:
The patent segments the JSON document into hierarchical components (document, object, array, scalar) that can be independently identified and updated. Instead of replacing the entire document, the system locates specific paths (e.g., '$.person.name') and modifies only the necessary segments, thereby improving update performance while maintaining operational simplicity through path-based identification.
Solution Approach 2:
The patent extracts the change log mechanism from the update process, allowing changes to be recorded separately and applied incrementally. This extraction enables the system to track only the modified portions rather than logging entire document replacements, reducing change log size while preserving the simplicity of declarative update statements.
2Ease of manufacture
If full JSON document replacement is used for updates, then update implementation is simplified, but change log size increases
Solution Approach 1:
The patent applies local quality by recording changes at the specific path level rather than globally for the entire document. The change log stores only the modified paths and their new values, creating a sparse representation of changes that is proportional to the actual modification size rather than the full document size, thus reducing change log quantity while maintaining implementation simplicity.
3Productivity
If binary format with partial update support is implemented, then update efficiency improves, but file system complexity increases
Solution Approach 1:
The patent introduces an intermediary layer between the file system and the JSON document that handles path-based navigation and modification. This intermediary translates high-level path operations (e.g., '$.person.name') into low-level binary format modifications, enabling efficient partial updates without requiring the underlying file system to understand or support complex partial update operations, thus maintaining file system simplicity while improving update efficiency.
4Ease of operation
If change log size is proportional to full document size, then transaction logging is simplified, but replication performance deteriorates
Solution Approach 1:
The patent applies partial action by logging only the necessary portion of changes rather than the full document. The change log captures minimal information needed for replication (path and new value) instead of logging entire document replacements, reducing the volume of data to be replicated while maintaining transaction logging simplicity through consistent path-based change representation.
Data Source
AI summary
Herein are fine grained updates to pieces of JavaScript object notation (JSON) documents by database statements that can update, delete, and insert parts of JSON documents. In an embodiment, a computer receives a request that specifies a modification of a JSON document that is stored in a compressed format in persistent storage. The modification adds additional content to the JSON document, and/or replaces an old value in the JSON document with a new value that is not a same size as the old value. The modification is recorded in a change log. The change log is eventually applied to the compressed format of the JSON document in the persistent storage without entirely rewriting the compressed format of the JSON document in the persistent storage.