Network Transaction Data Interception and Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face high storage and processing demands when collecting and analyzing transaction data from devices over a network, as they require large amounts of data to be stored and processed, which can be inefficient and limit the usefulness of the data due to the volume of information needed.
Innovation Solution
A system that intercepts communications over a network and determines the type of content using file extensions in URIs, storing more detailed information for page impressions and incrementing counts for less useful content types, thereby reducing storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all transaction data is collected and stored, then complete analysis capability is achieved, but storage and processing requirements become prohibitively large
Solution Approach 1:
The patent segments transaction data into two categories: page impression data (stored completely) and non-page impression data (stored as counts only). This segmentation allows the system to retain complete information for analytically valuable transactions while using compact representations for less valuable transactions, resolving the contradiction between data completeness and storage requirements
Solution Approach 2:
The patent applies different storage qualities to different types of transaction data based on their analytical value. Page impressions receive high-quality storage (complete data), while other transactions receive low-quality storage (counts only). This local differentiation of data quality allows the system to optimize storage usage while preserving analytical capability where it matters most
2Quantity of substance
If sampling is used to reduce data volume, then storage and processing requirements are reduced, but the completeness and usefulness of the data is limited
Solution Approach 1:
The patent applies partial action by storing complete data only for page impressions (the most analytically valuable transactions) while using summary counts for other transactions. This selective approach to data storage reduces overall data volume while maintaining complete information for the transactions that matter most for analysis
Solution Approach 2:
The patent changes the storage parameter for different transaction types: full transaction records for page impressions versus aggregated counts for non-page impressions. This parameter differentiation allows the system to achieve both reduced storage requirements and maintained analytical completeness for critical data
3Measurement precision
If detailed information is stored for all transactions, then analysis precision is improved, but processing power requirements increase
Solution Approach 1:
The patent segments transactions based on their analytical value, storing detailed information only for page impressions where precision is most important, while using aggregated counts for other transactions. This segmentation reduces the overall processing burden while maintaining high analysis precision for the most valuable data
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method for intercepting and storing information relating to communications with at least one device over a network are described. The system comprises: an interceptor configured to intercept communications with one device over a network; and a processing system. The processing system processes each intercepted communication to determine the type of content which is referenced by the intercepted communication. A storage action of one type may be performed if the content satisfies a criterion or a storage action of a second type may be performed if the determined type of content does not satisfy said criterion. More information to be stored about content types most useful for later analysis is stored.