Knowledge Graph Multi-Version Storage for Historical Data Traceability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current knowledge graphs face challenges in efficiently storing and querying historical data due to data redundancy and expansion issues, making it difficult to trace back to historical data and manage large volumes of daily data effectively.
Innovation Solution
Implementing a multi-version storage method for knowledge graphs based on time granularity, where incremental data is dynamically mapped and stored with reference to a version number, using a protocol like ProtoBuf for efficient storage and querying, and employing dictionary encoding to reduce attribute storage, along with time-to-live and maximum version thresholds for data merging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If full data of knowledge graph are stored at every time point, then historical data can be traced back completely, but storage space is occupied excessively
Solution Approach 1:
The patent extracts only the incremental changes from full data snapshots. Instead of storing complete knowledge graph data at every time point, it stores only the differences (additions, deletions, modifications) between consecutive versions. This extraction principle reduces storage requirements while maintaining the ability to reconstruct historical states by applying increments to base versions.
Solution Approach 2:
The patent implements a nested storage structure where incremental data is organized hierarchically with version numbers and time stamps. Each increment is nested within a version container that references the base data structure. This nesting allows efficient storage organization and retrieval, enabling the system to store large volumes of historical data in a compact, structured format that can be reconstructed on demand.
2Loss of information
If incremental data are stored with detailed attributes, then data completeness is maintained, but data redundancy increases
Solution Approach 1:
The patent applies local quality by storing different levels of detail for different types of data. Common attributes that remain unchanged across versions are stored once in the base data structure, while only the specific attributes that change are stored in incremental data. This selective detail storage reduces redundancy while maintaining completeness of changed information.
Solution Approach 2:
The patent discards redundant attribute information in incremental data by referencing the base data structure for unchanged attributes. When reconstructing historical versions, the system recovers complete data by combining base attributes with incremental changes. This discard and recover approach eliminates redundancy while preserving data completeness through selective storage of only necessary information.
3Reliability
If all versions of incremental data are retained, then version integrity is maintained, but storage efficiency decreases
Solution Approach 1:
The patent implements preliminary action by establishing retention policies and merging mechanisms in advance. Before storage efficiency deteriorates, the system proactively merges redundant incremental data according to predefined time-to-live and maximum version thresholds. This preliminary merging action maintains version integrity for the required retention period while preventing unbounded storage growth, balancing reliability and productivity.
Data Source
AI summary
Constructing knowledge graph data is described. Full data of a knowledge graph at a specified time is obtained. Incremental data of the full data is calculated relative to full data obtained at a previous time granularity. A version number of the incremental data is generated at least partially based on the specified time. By using a protocol and as a dynamic mapping, a node and an edge of the knowledge graph that are associated with the incremental data are dynamically mapped. The incremental data with reference to the version number is stored as stored incremental data.