Index Segmentation for Knowledge Map Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage and query systems in knowledge maps face performance degradation due to frequent updates, as conventional solutions either impact write performance with multiple indexes or incur high update overhead when entity data is frequently modified.
Innovation Solution
The proposed solution involves generating a primary index for data and a secondary index based on query rules, storing them in a data table, and delaying the removal of old index rows until actual data queries are executed, ensuring efficient storage and querying of entity data without degrading performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If multiple indexes are created to improve query performance, then query speed is improved, but write performance deteriorates
Solution Approach 1:
The patent divides indexes into two types: primary indexes (one-to-one correspondence with data rows) and secondary indexes (many-to-one correspondence with data rows). This segmentation allows the system to maintain one primary index for fast query while avoiding the overhead of multiple traditional indexes during write operations, thus resolving the contradiction between query speed and write performance.
Solution Approach 2:
The patent extracts the index update logic from the data write process. When data is written or updated, only the primary index is updated immediately, while secondary index updates are deferred and batched. This extraction separates the urgent write operation from the less urgent index maintenance, improving write performance while still maintaining query capability.
2Reliability
If entity data is frequently updated to maintain data freshness, then data accuracy is improved, but system performance deteriorates
Solution Approach 1:
The patent performs preliminary actions by pre-calculating and storing primary indexes during data writes. This allows the system to quickly determine whether updates are necessary before performing full data updates, reducing unnecessary write operations and improving overall system performance while maintaining data accuracy.
Solution Approach 2:
The patent implements a mechanism where old secondary index rows are not immediately removed but are kept until they are no longer needed for queries. This delayed removal reduces the overhead of frequent index maintenance during updates, allowing the system to handle frequent data updates more efficiently while maintaining data accuracy.
3Speed
If traditional indexing methods are used to enable fast querying, then query efficiency is improved, but update overhead increases
Solution Approach 1:
The patent introduces dynamic index management where the system adaptively handles index updates based on operation types. For query operations, both primary and secondary indexes are utilized for fast retrieval. For update operations, only primary indexes are updated immediately while secondary index updates are batched and processed later, dynamically adjusting the indexing strategy to minimize update overhead while maintaining query efficiency.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, apparatus, device, and medium for storing and querying data. The method for storing data includes: generating, for to-be-stored data, a primary index for indexing the data. The method further includes: writing the primary index arid the data into a data row in a data table. The method further includes: generating a secondary index for indexing the primary index based on a query rule of the data table and the data. In addition, the method further includes: writing the secondary index and the primary index into an index row in the data table.