Graph-Based Data Record Compression for Faster Impression Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for storing and accessing large amounts of impression data, such as those collected by promotion and marketing services, are inefficient and ineffective, leading to challenges in storage capacity and data access speed.
Innovation Solution
The implementation of webgraph-based techniques for compressing data records, which generates a compressed list of impression data records by creating a directed link graph and using properties like locality and similarity to encode data, allowing for more efficient storage and faster data access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current methods are used for storing and accessing large amounts of impression data, then storage capacity is sufficient, but storage efficiency is low and data access time is long
Solution Approach 1:
The patent segments the large impression data into multiple components (e.g., user identifiers, device identifiers, location data, timestamp) and organizes them into a graph structure where nodes represent data entities and edges represent relationships. This segmentation enables efficient indexing and retrieval of specific data elements without scanning the entire dataset, thereby improving data access speed and reducing access time.
Solution Approach 2:
The patent transforms the traditional flat or hierarchical data storage structure into a multi-dimensional graph structure. By representing data entities as nodes and relationships as edges in a graph, the system adds dimensional relationships (e.g., user-device, user-location, device-time) that enable faster querying and retrieval through graph traversal algorithms, significantly improving data access efficiency.
2Productivity
If current storage methods are used, then storage capacity is available, but storage efficiency is low
Solution Approach 1:
The patent merges related data components into a unified graph structure where connected nodes represent related data entities. By combining multiple data elements (user profiles, device information, location data, interaction records) into an integrated graph model with shared nodes and edges, the system eliminates redundancy and improves storage efficiency while maximizing storage capacity utilization.
Solution Approach 2:
The patent changes the structural parameters of data storage from traditional formats (flat files, relational tables) to a graph-based parameterization where data is represented by nodes, edges, and their attributes. This parameter change enables more efficient storage by allowing the system to store only the necessary relationship information and to compress the representation of repeated data patterns, thereby improving storage efficiency.
Data Source
AI summary
In general, embodiments of the present invention provide systems, methods and computer readable media for data record compression using graph-based techniques. An example method includes determining a plurality of index components; generating a sorted data record list of a plurality of compound data records; generating an ordered unique index component value list associated with a plurality of unique index component values; assigning a plurality of encodings to the plurality of unique index component values; and generating the compressed data record list based on the ordered unique index component value list and the plurality of encodings.