Edge Preprocessing for Workload Distribution in Data Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face limitations in managing workload distribution for pre-processing data, leading to inefficient use of computing resources due to the aggregation of redundant and hidden information, which hinders the performance of computer-implemented services.
Innovation Solution
A system and method for pre-processing data by filtering out redundant information and extracting hidden information using edge systems, distributing the workload across a distributed system to enhance data streams with metadata, thereby reducing the computational burden on data users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is aggregated and pre-processed at the edge, then the workload for data users is reduced and service throughput is improved, but redundant and hidden information increases the complexity of data management
Solution Approach 1:
The patent segments the data processing function by introducing edge systems that perform pre-processing locally before data reaches the core system. This divides the workload into edge-level filtering and core-level processing, reducing the complexity of data management at the core while maintaining high throughput.
Solution Approach 2:
The patent extracts redundant and hidden information through filtering operations performed by edge systems. By taking out unnecessary data elements before transmission to the core system, the patent reduces data management complexity while preserving useful information for service processing.
2Loss of information
If more data is transmitted to ensure complete information, then data completeness is improved, but computing resources are consumed more efficiently
Solution Approach 1:
The patent applies preliminary action by having edge systems perform filtering and extraction operations before data transmission to the core system. This pre-processing ensures that only necessary information is transmitted, maintaining data completeness for essential services while improving computing resource efficiency by avoiding unnecessary processing of redundant data.
Data Source
AI summary
Methods and systems for providing computer implemented services are disclosed. To provide the services, information from a variety of systems may be aggregated. Prior to aggregation, the sources of the information may perform processes for enhancing the provided information. The processes may include removing information that is unlikely to benefit the computer implemented services, and identifying hidden information that is not explicitly noted but that is likely to benefit the computer implemented services. The processes may distribute the workload for preprocessing the aggregated information to reduce the likelihood of bottlenecks or other limits on throughput rate of the computer implemented services from occurring.