Graph-Based Data Record Compression for Faster Impression Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for storing and accessing large amounts of impression data, such as those collected by promotion and marketing services, are inefficient and ineffective, leading to challenges in storage capacity and data access speed.
Innovation Solution
The implementation of webgraph-based techniques for compressing data records, which generates a compressed list of impression data records by creating a directed link graph and using properties like locality and similarity to encode duplicate values, allowing for multi-dimensional compression and efficient storage and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current storage methods are used for large amounts of impression data, then storage capacity is sufficient, but storage efficiency and data access speed are poor
Solution Approach 1:
The patent segments the storage system into multiple dimensions (e.g., temporal, spatial, categorical dimensions) and applies graph-based compression techniques to each dimension independently. This segmentation allows the system to handle large volumes of impression data efficiently by breaking down the complex storage problem into manageable dimensional components, thereby improving data access speed without overwhelming system complexity.
Solution Approach 2:
The patent introduces multi-dimensional graph structures to represent impression data, transforming the traditional flat storage model into a multi-dimensional space. By organizing data along multiple dimensions (such as time, device, user, and impression characteristics), the system enables faster query processing and data retrieval through dimensional filtering and graph traversal, significantly improving productivity while maintaining manageable complexity through structured organization.
2Quantity of substance
If data is stored in uncompressed format, then data access is simple, but storage space is excessive and retrieval efficiency is low
Solution Approach 1:
The patent applies graph-based compression techniques during the data ingestion phase, performing preliminary compression and organization of impression data before storage. By pre-processing the data to identify patterns, duplicates, and relationships across multiple dimensions, the system reduces storage space requirements upfront and optimizes the data structure for faster retrieval, thereby reducing both storage space consumption and future retrieval time.
Solution Approach 2:
The patent transforms the storage parameters by converting raw impression data into graph-based representations with compressed encodings. This parameter change involves encoding repeated patterns and relationships in the data using graph structures, which significantly reduces the space required to store the same information while enabling efficient querying through graph traversal operations, thus improving both storage space utilization and retrieval speed.
3Productivity
If multiple I/O operations are performed for data retrieval, then complete data can be accessed, but access speed decreases and processing efficiency is reduced
Solution Approach 1:
The patent merges multiple I/O operations into a single efficient retrieval operation by organizing data in a graph-based structure that supports multi-dimensional queries. Instead of performing separate I/O operations for different data dimensions, the system combines the retrieval of related data across multiple dimensions into a unified graph traversal operation, thereby improving processing efficiency while managing I/O complexity through structured data organization.
Solution Approach 2:
The patent introduces a graph-based data structure as an intermediary layer between the storage system and the query processing system. This intermediary structure pre-organizes impression data into graph relationships, allowing the system to retrieve related data across multiple dimensions through a single efficient operation rather than multiple separate I/O operations, thus improving productivity while the graph structure manages the complexity of data relationships.
Data Source
AI summary
In general, embodiments of the present invention provide systems, methods and computer readable media for data record compression using graph-based techniques.