K2 Tree Bitmap Storage for Sparse Graph Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional content recommendation systems face inefficiencies in storage space utilization due to the large size of sparse recommendation matrices, which are further exacerbated by the need to track changes in user interactions, leading to significant storage space requirements and inefficient data management.
Innovation Solution
The implementation of a K2 tree data structure to compactly represent sparse graph data, utilizing bitmaps to efficiently store and update matrix entries, reducing storage needs by at least 50% and enabling efficient traversal of historical data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional matrix storage is used to store recommendation data, then complete data representation is achieved, but storage space requirements become excessively large
Solution Approach 1:
The patent extracts only the non-zero (known) values from the sparse recommendation matrix and stores them in a compressed format using K2-tree data structure. Instead of storing the entire matrix including all zero (unknown) values, the system extracts and stores only the meaningful data points, achieving significant space reduction while preserving all useful information.
Solution Approach 2:
The patent changes the storage parameter from storing full matrix values to storing compressed representations using bitmaps and hierarchical tree structures. By transforming the data representation format and using variable-length encoding based on data sparsity, the system reduces storage requirements while maintaining data integrity.
2Loss of information
If version control is implemented to track changes in user interactions, then historical data tracking capability is improved, but storage space requirements double
Solution Approach 1:
The patent merges the version control functionality with the compressed storage structure by using the same K2-tree and bitmap infrastructure to represent both the current state and historical versions of the recommendation matrix. Changes are tracked by modifying the existing compressed structure rather than creating separate full copies, allowing version history to be maintained with minimal additional space.
Solution Approach 2:
The patent implements a nested structure where historical versions are stored within the same hierarchical K2-tree framework. Each version is nested within the tree structure, allowing efficient storage of multiple temporal states without requiring proportional increases in storage space. The nested structure enables compact representation of version history by sharing common substructures across versions.
Data Source
AI summary
Aspects of the present disclosure include a system comprising a machine-readable storage medium storing at least one program and computer-implemented methods for efficiently storing graph data. Consistent with some embodiments, the method includes using the graph data to construct a tree topology bitmap that represents a topology of a portion of the nodes in a tree data structure representing the graph data. The method further includes creating an archive bit array for the tree data structure that includes elements populated with a binary indicator of whether a corresponding leaf node includes a known entry. The method further includes creating a leaf values array comprising a known value for each leaf node in the tree data structure with a known entry. The method further includes storing, in a memory storage device, the tree topology, the archive bit array, and the leaf values array in association with an initial timestamp.