Graph Key-Value Storage Layout for Adjacent Data Prefetching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing scale and complexity of graph data analysis lead to frequent hard disk reading operations, impacting overall performance due to inefficient data storage and retrieval methods.
Innovation Solution
A key-value pair storage method that organizes graph composition element data based on category identifiers, ensuring same-category data is stored adjacently, and a graph data prefetching method that reads adjacent data during retrieval to improve cache hit rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If graph data is stored using traditional storage methods, then data can be stored, but frequent hard disk reading operations occur leading to poor reading efficiency
Solution Approach 1:
The patent applies preliminary action by pre-reading and caching graph composition element data into memory before it is actually needed. The system determines which data will be needed based on the orderly storage arrangement and prefetches it into memory, so that when the data is required for graph data analysis, it is already available in memory rather than needing to be read from the hard disk at that moment.
Solution Approach 2:
The patent changes the storage organization from a traditional flat or random structure to a multi-dimensional orderly arrangement based on category identifiers. By organizing data with category identifiers as one dimension and arranging same-category data adjacently, the system enables more efficient data location and prefetching strategies, transforming the storage access pattern from random to sequential and predictable.
2Quantity of substance
If graph data scale expands and analysis complexity increases, then more data needs to be processed, but frequent hard disk operations impact overall performance
Solution Approach 1:
The patent applies preliminary action by pre-reading and caching graph composition element data into memory before it is actually needed. The system determines which data will be needed based on the orderly storage arrangement and prefetches it into memory, so that when the data is required for graph data analysis, it is already available in memory rather than needing to be read from the hard disk at that moment.
Solution Approach 2:
The patent ensures continuity of useful action by maintaining data in memory for extended periods through caching. Once data is loaded into memory, it remains available for multiple operations without requiring repeated hard disk reads. The orderly storage arrangement based on category identifiers maximizes cache hit rates, allowing the system to continuously process graph data using in-memory data rather than repeatedly accessing the hard disk.
3Ease of operation
If data is stored without orderly arrangement, then storage is simple, but cache hit rates are low requiring frequent hard disk reads
Solution Approach 1:
The patent applies preliminary action by pre-reading and caching graph composition element data into memory before it is actually needed. The system determines which data will be needed based on the orderly storage arrangement and prefetches it into memory, so that when the data is required for graph data analysis, it is already available in memory rather than needing to be read from the hard disk at that moment.
Solution Approach 2:
The patent changes the storage organization from a traditional flat or random structure to a multi-dimensional orderly arrangement based on category identifiers. By organizing data with category identifiers as one dimension and arranging same-category data adjacently, the system enables more efficient data location and prefetching strategies, transforming the storage access pattern from random to sequential and predictable.
Data Source
Figure 1~2
Figure 3~5
Figure 6~7
AI summary
One or more embodiments of this specification provides key-value pair storage methods for graph data and graph data prefetching methods. The key-value pair storage method for graph data includes: obtaining a plurality of graph composition element key-value pairs, where in any graph composition element key-value pair, a value corresponds to graph composition element data of a corresponding graph composition element in graph data, a key includes a plurality of identifier fields, and one of the identifier fields is a category identifier of the corresponding graph composition element, where graph composition elements corresponding to the plurality of graph composition element key-value pairs include nodes and/or edges; and storing graph composition element key-value pairs in the plurality of graph composition element key-value pairs in an orderly way based on an arrangement sequence of category identifiers included in corresponding keys.