Subscriber Data Deduplication via Frequency Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing volume of reporting data in data networks due to growing network devices and subscriber demand leads to inefficiencies, including redundant data processing and security risks from sensitive information exposure, while scaling resources is costly and time-consuming.
Innovation Solution
A system and method that classify subscriber data into low frequency change and high frequency change data, using data index pointers to reduce data transfer volume and anonymize sensitive information, thereby minimizing redundant data transmission and enhancing security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the volume of reporting data is increased to meet subscriber demand for high granularity data, then the measurement precision and data accuracy are improved, but the data transfer volume and processing burden increase excessively
Solution Approach 1:
The patent extracts only the essential information from the full reporting data by identifying and transmitting only high-frequency change data and data index pointers, while excluding redundant low-frequency change data. This extraction approach maintains measurement precision for critical parameters while significantly reducing data transfer volume.
Solution Approach 2:
The patent segments the reporting data into two categories: high-frequency change data that must be transmitted in detail, and low-frequency change data that can be referenced indirectly through pointers. This segmentation allows the system to transmit only the necessary detailed information while reducing overall data volume.
2Productivity
If multiple network devices collect and transmit reporting data at high speed, then the productivity and data collection capability are improved, but the device complexity and processing requirements increase
Solution Approach 1:
The network devices perform self-optimization by automatically classifying their own reporting data into high-frequency and low-frequency change data, and by generating data index pointers to reduce their own processing burden. This self-service approach maintains high productivity while reducing individual device complexity.
Solution Approach 2:
The system performs preliminary classification and processing of data at the network device level before transmission, pre-sorting data into high-frequency and low-frequency categories and creating pointers in advance. This preliminary action reduces the processing burden during subsequent data transmission and analysis phases.
3Measurement precision
If subscriber identity and location information are included in reporting data for accurate tracking, then the measurement precision is improved, but the security risks and harmful factors increase
Solution Approach 1:
The patent introduces data index pointers as intermediary elements that reference subscriber information without directly transmitting the sensitive subscriber identity and location data. These pointers act as mediators that maintain tracking accuracy while preventing direct exposure of sensitive information to malicious agents.
Solution Approach 2:
The patent extracts and transmits only the essential tracking information through data index pointers, while leaving the actual sensitive subscriber identity and location data in the data storage system. This extraction approach maintains measurement precision for subscriber tracking while removing sensitive information from the transmission path.
4Productivity
If centralized analytics systems scale out to handle more reporting data, then the processing capability and productivity are improved, but the loss of time and cost increase
Solution Approach 1:
The network devices perform preliminary data classification and pointer generation locally before data transmission, pre-processing the data to reduce its volume and complexity. This preliminary action reduces the scaling burden on centralized analytics systems, allowing them to handle more data without requiring proportional increases in processing resources and time.
Data Source
AI summary
Decreasing data transfer over a network may commence with collecting subscriber data. The method may continue with classifying the subscriber data into low frequency change data and high frequency change data based on predetermined criteria. The method may include storing the low frequency change data to a data storage. The method may continue with generating reporting data. The reporting data may include the high frequency change data and at least one data index pointer to the low frequency change data in the data storage. The method may further include providing the reporting data to a data processing node. The low frequency change data may include subscriber identifying data. The data reporting node may be further configured to obfuscate the subscriber identifying data. The at least one data index pointer may include a secure data identifier associated with the obfuscated subscriber identifying data.