Directed Link Graph Compression for Large Impression Records
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for storing and accessing large amounts of impression data are inefficient, leading to storage challenges and slow data access due to the size of the data records.
Innovation Solution
The implementation of webgraph-based techniques for compressing data records, which generates a compressed list of impression data records by creating a directed link graph and using properties like locality and similarity to encode duplicate values, allowing for more efficient storage and faster data access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current methods for storing and accessing large amounts of impression data are used, then data can be stored, but storage efficiency is poor and data access is slow due to the size of data records
Solution Approach 1:
The patent segments the data record into multiple components (e.g., user information, impression information, context information) that can be independently compressed and stored. This segmentation allows for targeted compression techniques to be applied to different parts of the record, reducing overall size while maintaining accessibility of individual components.
Solution Approach 2:
The patent merges multiple data records that share common components into a single compressed representation. By identifying and merging duplicate or redundant information across records (such as repeated user profiles or context data), the system reduces total storage requirements while enabling efficient retrieval through the graph structure.
2Quantity of substance
If current methods for storing large amounts of impression data are used, then data can be stored, but storage efficiency is poor leading to storage challenges
Solution Approach 1:
The patent creates a graph-based copy of the data where nodes represent data components and edges represent relationships. This graph structure serves as a compressed representation that can be stored more efficiently than the original data records, reducing storage requirements while preserving the ability to reconstruct and access the original information when needed.
3Productivity
If webgraph-based compression techniques are implemented, then storage efficiency is improved and data access is faster, but system complexity increases
Solution Approach 1:
The patent performs preliminary compression of data records into graph structures during data ingestion or batch processing, before the data needs to be accessed for analysis. This preliminary action transforms the data into a more efficient format in advance, so that subsequent queries and operations can proceed faster without incurring the computational overhead of compression at query time.
Data Source
AI summary
In general, embodiments of the present invention provide systems, methods and computer readable media for data record compression using graph-based techniques.