Predictive Network Data Processing Model Selection Under Throughput Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in meeting throughput requirements when internal resources are insufficient or no borrowed resources are available for processing network data.
Innovation Solution
A data processing apparatus predicts data amounts and resource needs in future time periods, selecting optimal data processing models based on accuracy and throughput trade-offs to improve performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a data processing model with high accuracy is selected, then processing accuracy is improved, but throughput decreases
Solution Approach 1:
The patent applies dynamics by making the data processing model selection adjustable and adaptive rather than fixed. The system dynamically switches between different data processing models (first model with high accuracy/low throughput, second model with low accuracy/high throughput) based on real-time network conditions, traffic patterns, and resource availability. This allows the system to optimize the balance between accuracy and throughput according to varying operational requirements.
Solution Approach 2:
The patent changes the parameters of the data processing system by maintaining multiple data processing models with different accuracy-throughput characteristics. Instead of using a single fixed model, the system varies the model parameters (accuracy level, processing speed) based on predicted data amounts and current resource conditions, enabling flexible adaptation to different operational scenarios.
2Productivity
If more borrowed resources are allocated to meet throughput requirements, then productivity is improved, but resource cost increases
Solution Approach 1:
The patent applies preliminary action by predicting the data amount to be processed in advance using historical data and prediction models. Based on these predictions, the system proactively selects appropriate data processing models and allocates resources before the actual data processing occurs. This allows the system to optimize resource usage by matching resource allocation to predicted workload, avoiding both over-provisioning and under-provisioning.
Solution Approach 2:
The system performs self-service by using its own historical processing data and prediction capabilities to determine future resource needs and model selections. The data processing apparatus autonomously predicts its own workload requirements and selects appropriate processing models without requiring external intervention or over-allocation of resources, thereby optimizing resource efficiency.
3Adaptability or versatility
If multiple data processing models are maintained with different accuracy levels, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the data processing functionality into multiple distinct data processing models, each optimized for specific scenarios (high accuracy vs. high throughput). Instead of creating one complex universal model, the system segments the processing capabilities into specialized models and uses a selection mechanism to choose the appropriate segment based on current conditions, thereby managing complexity through modular design.
Data Source
AI summary
Embodiments of this application disclose a data processing method and apparatus, to improve network data processing performance. The data processing apparatus in the embodiments of this application is configured to process collected network data in a preset network environment. In the embodiments of this application, a method performed by the data processing apparatus includes: The data processing apparatus is configured to obtain first prediction information, and process the collected network data in the preset network environment, where the first prediction information includes a predicted value of a data amount to be processed in a first time period, and the first time period starts from a first moment; select a first data processing model from a data processing model set based on the first prediction information, where the data processing model set includes a second data processing model and the first data processing model; receive collected first network data when the first moment arrives; and process the first network data in the first time period by using the first data processing model.