Partitioned Data Traffic Control for Stable Processing Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems face performance degradation due to data congestion caused by fluctuating data transmission speeds, leading to reduced processing speed and efficiency, as existing traffic control methods affect all partitions equally, resulting in unstable performance.
Innovation Solution
Implementing a data processing system with a control node that monitors performance across multiple data transmission pipes, allowing for independent traffic control on each pipe to adjust data allocation based on performance thresholds, thereby isolating traffic control to specific pipes and preventing cross-partition interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data traffic control operation is performed on data of the entire system, then data congestion is reduced, but processing performance of the data processing system fluctuates due to affecting data from different partitions
Solution Approach 1:
The patent divides the data processing system into multiple independent partitions, each with its own traffic control mechanism. The control node separately manages data traffic for each partition based on individual performance conditions, rather than applying uniform control across the entire system. This segmentation allows targeted traffic control that prevents congestion in specific partitions without adversely affecting other partitions, thereby maintaining overall processing performance stability.
2Productivity
If data transmission speed increases sharply, then data processing capacity is improved, but data congestion is caused in the data processor
Solution Approach 1:
The patent implements a feedback mechanism where the control node continuously monitors processing performance conditions of each partition and dynamically adjusts data transmission speed accordingly. When congestion is detected in a specific partition, the control node reduces transmission speed for that partition while maintaining normal speed for other partitions. This feedback-based dynamic control prevents congestion while preserving overall data processing capacity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A data processing method and system are disclosed. The method is applied to a data processing system. The method includes: sending, by an operation node, an alarm signal to a control node when a monitor detects that processing performance of data from a first data transmission pipe is lower than a first threshold (SI02), where the alarm signal is used to indicate that the processing performance of the data from the first data transmission pipe is excessively low, and the first data transmission pipe is any data transmission pipe in a plurality of data transmission pipes; and when receiving the alarm signal, reducing, by the control node, a data amount that is from the first data transmission pipe and that is allocated to the operation node (S103). The solution can reduce fluctuation in processing performance of the data processing system during traffic control.