Webgraph Encoding of Impression Records for In-Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for storing and accessing large amounts of impression data, such as those collected by promotion and marketing services, are inefficient and ineffective, leading to challenges in storage capacity and data access speed.
Innovation Solution
The implementation of webgraph-based techniques for compressing data records, which generates a compressed list of impression data records by creating a directed link graph and using properties like locality and similarity to encode data, allowing for efficient storage and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If current methods are used for storing and accessing large amounts of impression data, then data can be stored, but storage efficiency is poor and memory requirements are high
Solution Approach 1:
The patent segments the impression data into multiple components (e.g., user identifier, advertiser identifier, impression data, timestamp) and processes each component separately through graph-based compression. This segmentation allows for targeted compression of repetitive elements like user and advertiser identifiers while maintaining access efficiency.
Solution Approach 2:
The patent creates compressed representations (copies) of the original data records through graph-based encoding. Instead of storing full duplicate records, it generates compact graph structures that represent the same information more efficiently, reducing memory requirements while preserving data integrity.
2Quantity of substance
If data records are compressed using webgraph-based techniques, then storage efficiency improves and memory requirements reduce, but data access speed may be affected
Solution Approach 1:
The patent performs preliminary compression of the data into graph-based structures before actual data access occurs. This pre-processing step organizes the data in a way that enables efficient querying and retrieval operations, so that when data is accessed, the compressed structure already supports fast operations.
Solution Approach 2:
The patent changes the representation parameters of the data from traditional row-based storage to graph-based encoding. This parameter transformation allows the same data to be stored more compactly while maintaining query capabilities through the graph structure's inherent organization.
3Productivity
If large datasets are processed without compression, then data access is straightforward, but multiple I/O operations are required and processing efficiency decreases
Solution Approach 1:
The patent merges multiple data components and their relationships into a unified graph-based structure. By combining related data elements and their associations into a single compressed representation, the system reduces the number of separate I/O operations needed to retrieve and process related information.
Solution Approach 2:
The patent transitions from traditional one-dimensional row-based data storage to a multi-dimensional graph structure. This dimensional change allows the system to capture and exploit relationships between data elements, enabling more efficient processing by traversing the graph structure rather than performing multiple sequential I/O operations.
Data Source
AI summary
In general, examples of the present disclosure provide systems, methods and computer readable media for data record compression using graph-based techniques. For example, an example method includes retrieving a compound data record list that includes compound data records, identifying a unique second component value set that includes unique second component values, determining a data record position list based at least in part on at least one data record position within the compound data record list, and generating a data record encoding of the compound data records.