Graph Storage Engine for Incremental Versioned Data Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph data structures face challenges with time-consuming computations, frequent changes, and the need for tracing previous versions, leading to significant overheads in computation and storage, particularly in large-scale microservices applications.
Innovation Solution
A method and storage engine for graph structure data that supports incremental storage of historical information, allowing efficient management of graph changes by generating version information and mapping nodes/edges with reference to this information, reducing overheads and improving data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complete graph data is stored for version tracking, then version tracing capability is improved, but storage overhead increases
Solution Approach 1:
The patent segments graph data into three distinct components: base data (initial complete graph), incremental data (changes since last version), and delta data (differences between incremental versions). This segmentation allows storing only necessary portions at each level, reducing overall storage overhead while maintaining version tracing capability through selective reconstruction of historical versions.
Solution Approach 2:
The patent extracts and stores only the changing portions of graph data (incremental and delta data) rather than storing complete copies of each version. By taking out only the essential change information and storing it separately from the base data, the system achieves efficient version tracking with minimal storage overhead.
2Reliability
If all graph versions are retained for change analysis, then historical comparison capability is improved, but computation time increases
Solution Approach 1:
The patent segments version history into hierarchical levels (base version, incremental versions, and delta versions). When performing historical comparison, the system only loads and processes the relevant delta data for the specific version range being compared, rather than loading all historical versions. This segmented approach significantly reduces computation time while maintaining full historical comparison capability.
Solution Approach 2:
The patent performs preliminary processing by pre-computing and storing delta data between incremental versions. This preliminary action prepares the data in advance in a compressed differential format, so that when historical comparison is needed, the system can quickly apply these pre-computed deltas to reconstruct historical versions without performing time-consuming computations on the fly.
3Adaptability or versatility
If frequent graph updates are supported, then system adaptability is improved, but storage and computation overhead increases
Solution Approach 1:
The patent implements a dynamic storage strategy where graph data is maintained in different formats depending on the operation type: complete base data for initial storage, incremental data for major version changes, and compact delta data for frequent minor updates. This dynamic approach allows the system to adapt to different update frequencies and types, supporting frequent updates efficiently without incurring proportional storage overhead.
Solution Approach 2:
The patent changes the storage parameter representation by storing graph data in multiple levels of abstraction: full graph structure at the base level, partial update sets at incremental levels, and minimal delta representations at the finest granularity. This parameter transformation allows frequent updates to be represented with progressively smaller data sizes, reducing storage overhead while maintaining full adaptability to handle any type of graph modification.
Data Source
AI summary
Implementations of the present specification disclose a graph structure data processing method and a storage engine and device for graph structure data. The method can be implemented by using the storage engine for graph structure data, and includes: receiving a storage request for first data of a target graph, the storage request including a graph identifier of the target graph and the first data; obtaining historical data of the target graph corresponding to the graph identifier from a storage component based on the graph identifier, determining target data that corresponds to a change of the first data of the target graph relative to the historical data, and generating version information corresponding to the target data, the target data including data of a node in the first data of the target graph and/or data of an edge in the first data of the target graph; and mapping a node and/or an edge associated with the target data in the target graph, and storing the target data in the storage component with reference to the version information.