Data Pipe Traffic Control for Stable Processing Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems face performance degradation due to data congestion caused by fluctuating transmission speeds, leading to reduced processing speed and instability, as existing traffic control methods affect all data partitions equally.
Innovation Solution
Implementing a data processing method that monitors and controls data transmission pipes at a finer granularity level, allowing for independent traffic control on underperforming pipes without impacting other partitions, thereby reducing congestion and stabilizing system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data traffic control operation is performed on data of the entire system, then data congestion is reduced, but processing performance of the data processing system fluctuates
Solution Approach 1:
The patent segments the data processing system into multiple independent data transmission pipes, each monitored and controlled separately. The monitor device tracks processing performance for each individual pipe and triggers traffic control only on pipes experiencing congestion, rather than applying system-wide control. This segmentation allows localized remediation that preserves overall system productivity while ensuring reliability.
Solution Approach 2:
The patent implements local quality control by applying traffic control measures selectively to specific data transmission pipes based on their individual performance metrics. Each pipe receives customized control actions (such as adjusting transmission rates or routing alternatives) tailored to its specific congestion conditions, rather than uniform system-wide control. This localized approach minimizes impact on healthy pipes and maintains overall processing speed.
2Productivity
If data transmission speed increases sharply, then data processing capacity is improved, but data congestion is caused in the data processor
Solution Approach 1:
The patent implements preliminary action through continuous monitoring of processing performance in each data transmission pipe before congestion occurs. The monitor device detects early signs of overload (such as increasing processing time or queue depth) and triggers preventive traffic control measures on affected pipes before actual congestion develops. This proactive approach allows the system to maintain high processing capacity while preventing instability.
Solution Approach 2:
The patent establishes a feedback loop where the monitor device continuously measures processing performance metrics (such as processing speed, throughput, or latency) for each data transmission pipe and uses this information to dynamically adjust traffic control actions. When performance degradation is detected, the system automatically applies control measures on the affected pipe and continues monitoring to ensure the problem is resolved, enabling the system to safely operate at high capacity while maintaining stability.
Data Source
AI summary
A data processing method is provided, including: sending, by an operation node, an alarm signal to a control node when a monitor detects that processing performance of data from a first data transmission pipe is lower than a first threshold (S102), where the alarm signal is used to indicate that the processing performance of the data from the first data transmission pipe is low; and when receiving the alarm signal, reducing, by the control node, a data amount that is from the first data transmission pipe and that is allocated to the operation node (S103). The solution can reduce fluctuation in processing performance of the data processing system during traffic control.