Data traffic scheduling method and communication system based on traffic characteristics and path selection

Abstract

The application discloses a data flow scheduling method and communication system based on flow characteristics and path selection, which comprises the following steps: presetting a large flow path and a small flow path according to historical state information of each path, wherein path congestion risk and path stability of the large flow path meet preset first requirements, path congestion risk and time delay of the small flow path meet preset second requirements, and the path congestion risk, the path stability and the time delay are determined according to the historical state information; predicting whether the data flow sent by a sending end is a large flow or a small flow according to flow characteristics of historical data flow sent by the sending end, determining that the sending end is a large flow sending end and distributing the data flow to the large flow path if the data flow is a large flow; and determining that the sending end is a small flow sending end and distributing the data flow sent by the small flow sending end to the small flow path if the data flow is predicted to be a small flow. The application can effectively improve bandwidth utilization, response speed of the communication system and reduce congestion and delay.

Description

Technical Field

[0001] This invention relates to the fields of computer and communication technology, and in particular to a data traffic scheduling method and communication system based on traffic characteristics and path selection. Background Technology

[0002] With the rapid development of technologies such as cloud computing, artificial intelligence, and big data, modern data centers are placing increasingly higher demands on network performance. Traffic transmitted in data exchange networks is typically categorized into two types: Elephant Flow and Mice Flow. Elephant Flow refers to large data transmission streams that occupy bandwidth for extended periods, while Mice Flow refers to short data transmission streams that consume minimal resources for short durations. These two types of traffic differ significantly in bandwidth usage, duration, and sensitivity, but they often coexist within the same network infrastructure. Improper scheduling strategies can easily lead to low system resource utilization, congestion, increased latency, and even degraded service quality.

[0003] To mitigate communication delays and congestion caused by improper scheduling of large and small data streams, a widely used scheduling scheme is Random Packet Spray (RPS). The core idea of ​​RPS is to randomly distribute packets from each data stream across multiple available paths to achieve load balancing and maximize the utilization of link resources. This method is simple to implement and easy to deploy, and therefore has been adopted in many data centers. RPS does not distinguish between the type and size of data streams, relying entirely on packet-level random scheduling, which theoretically can avoid single-path congestion problems.

[0004] While RPS can achieve load balancing to some extent, it also has the following problems, especially when facing mixed traffic (elephant flow and mouse flow) scenarios, where it has the following obvious shortcomings:

[0005] (1) Packet out-of-order problem is serious: RPS will cause different data packets of the same data stream to take different paths, especially in elephant streams, which can easily cause packet out-of-order, thereby increasing the reordering overhead of the receiver and even causing TCP performance to decline.

[0006] (2) Lack of traffic awareness mechanism: RPS does not distinguish the size and characteristics of data streams, ignores the high sensitivity of large streams to bandwidth resources and the high sensitivity of small streams to latency, which can easily lead to unreasonable resource allocation;

[0007] (3) Lack of intelligence in path selection: RPS path selection is completely random, without considering the current load or historical congestion of the link, and is prone to dispatching traffic to congested paths, reducing the overall network efficiency.

[0008] The above background information is provided only to assist in understanding the inventive concept and technical solution of this invention. It does not necessarily belong to the prior art of this application, nor does it necessarily provide technical teaching. In the absence of clear evidence that the above information was disclosed before the filing date of this application, the above background information should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0009] The purpose of this invention is to provide a data traffic scheduling method and communication system based on traffic characteristics and path selection, which can effectively improve the bandwidth utilization and response speed of data exchange networks and reduce congestion and latency.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A data traffic scheduling method based on traffic characteristics and path selection, applicable to a data exchange network comprising multiple senders and multiple receivers, wherein the data exchange network includes multiple paths for data transmission from the senders to the receivers, comprising the following steps:

[0012] Based on the historical status information of each path, some paths among multiple paths are preset as high-flow paths, and some paths are preset as low-flow paths. The path congestion risk and path stability of the high-flow paths meet the preset first requirement, and the path congestion risk and latency of the low-flow paths meet the preset second requirement. The historical status information includes multiple of the following: average packet loss rate, data packet queue length, and latency. The path congestion risk, path stability, and latency are determined based on the historical status information.

[0013] Based on the traffic characteristics of the historical data streams sent by each sender, it is predicted whether the data stream sent by each sender is a large stream or a small stream. If the data stream is predicted to be a large stream, the sender is determined to be a large stream sender and the data stream sent by the large stream sender is assigned to the large stream path; if the data stream is predicted to be a small stream, the sender is determined to be a small stream sender and the data stream sent by the small stream sender is assigned to the small stream path.

[0014] Furthermore, based on any one or a combination of the aforementioned technical solutions, for the small stream sending end, the number of small stream paths from the small stream sending end to the receiving end is determined;

[0015] If there are multiple small flow paths, then a dynamic path score is determined for each of the multiple small flow paths. The dynamic path score is determined based on multiple factors, including the instantaneous packet loss rate, the egress port queue length, and the latency of the small flow path.

[0016] The small flow path whose dynamic path score is higher than a preset dynamic path score threshold is identified as the target small flow path, and the data stream sent by the small flow sender is assigned to the target small flow path for transmission.

[0017] Furthermore, following any one or a combination of the aforementioned technical solutions, if there are multiple target small flow paths, then a corresponding traffic allocation weight value is set for each target small flow path based on the dynamic path score of the multiple target small flow paths. The higher the dynamic path score, the higher the traffic allocation weight value.

[0018] The data stream sent by the small stream sending end is allocated to each target small stream path according to the traffic allocation weight value and then sent.

[0019] Furthermore, following any one or a combination of the aforementioned technical solutions, if there are multiple target small flow paths, a random data packet spraying mechanism is used to distribute the data stream sent by the small flow sending end to multiple target small flow paths for transmission.

[0020] Furthermore, based on any one or a combination of the aforementioned technical solutions, for the large-flow sending end, one of the large-flow paths is determined as the first large-flow path, and the large-flow is sent using the first large-flow path;

[0021] Alternatively, for a large stream sender, one of the large stream paths is determined as a first large stream path, the large stream is sent using the first large stream path, and the first large stream path is not configured to transmit data sent by other senders.

[0022] Furthermore, following any one or a combination of the aforementioned technical solutions, the data stream sent by the large-stream sending end is transmitted using the first large-stream path, and the current data transmission quality of the first large-stream path is monitored;

[0023] If the data stream transmission quality meets the preset current data transmission quality requirements, then the first large stream path will continue to be used to send the large stream;

[0024] If the data stream transmission quality does not meet the preset current data transmission quality requirements, then another high-volume path is used to send the high-volume stream.

[0025] Furthermore, following any one or a combination of the aforementioned technical solutions, for the large flow sending end, it is determined whether the number of its corresponding large flow paths is one or more. If there are multiple paths, the large flow path without an occupied port is determined as the target large flow path.

[0026] The target high-flow path with the best historical data transmission quality is determined as the first high-flow path.

[0027] Furthermore, following any of the aforementioned technical solutions or combinations thereof, if each of the major flow paths has occupied ports, it is determined whether there exists a major flow path that satisfies the following preset condition: the sending end corresponding to the data stream forwarded by its occupied port is the same as the sending end of the major flow.

[0028] If it exists, then the major flow path that meets the preset conditions is determined as the first major flow path.

[0029] Furthermore, following any of the aforementioned technical solutions or combinations thereof, if there is no high-current path that meets the preset conditions, then the high-current path with the best historical data transmission quality is determined as the first high-current path.

[0030] or,

[0031] If no high-flow path that meets the preset conditions exists, then the high-flow path with the shortest current queuing time is determined as the first high-flow path.

[0032] or,

[0033] If no high-flow path meets the preset conditions, a comprehensive score for each high-flow path is determined based on the current queuing time and historical data transmission quality, and the high-flow path with the highest comprehensive score is determined as the first high-flow path.

[0034] Furthermore, following any one or a combination of the aforementioned technical solutions, a five-tuple information is configured for the data stream, wherein the five-tuple information includes the source IP address, destination IP address, sender number, receiver number, and transport layer protocol number.

[0035] A historical flow feature library is pre-built, which includes five-tuple information and flow features corresponding to the data streams sent by each sending end. The flow features include flow size and flow frequency.

[0036] For a data stream to be sent from a sending end, extract the five-tuple information of the data stream to be sent as the target five-tuple information, and determine the traffic feature corresponding to the five-tuple information in the historical flow feature library that is the same as the target five-tuple information as the target traffic feature.

[0037] Predict whether the data stream to be sent is a large stream or a small stream based on the target traffic characteristics.

[0038] Furthermore, following any of the aforementioned technical solutions or combinations thereof, if the historical flow feature library does not contain any quintuple information identical to the target quintuple information, then a preset traffic classification model is used to predict whether the data stream to be sent is a large flow or a small flow.

[0039] Furthermore, following any one or a combination of the aforementioned technical solutions, the first requirement includes a first score for the historical data transmission quality of the path not being lower than a first threshold and / or the bandwidth of the path not being lower than a preset bandwidth threshold.

[0040] Furthermore, following any one or a combination of the aforementioned technical solutions, the first score for the historical data transmission quality of each of the paths is calculated using the following formula: S1 = α1 × C risk +β1×H risk , where C risk H represents the probability of congestion risk. risk The path health index is represented by α1, which represents the first congestion risk weight, and β1, which represents the first health weight. α1 > β1 > 0. The path health index is determined based on the packet loss rate and / or the data packet queue length. The congestion risk probability is determined based on the data packet queue length and latency.

[0041] Furthermore, following any one or a combination of the aforementioned technical solutions, the second requirement includes a second score for the historical data transmission quality of the path not being lower than a second threshold and / or the path's latency meeting a preset latency requirement.

[0042] Furthermore, following any one or a combination of the aforementioned technical solutions, the second score for the historical data transmission quality of each of the paths is calculated using the following formula: S2 = α2 × C risk +β2×FCT trend , where C risk FCT represents the probability of congestion risk. trend The latency index is represented by α2, the second congestion risk weight is represented by β2, and the second health index is represented by β2, where 0 < α2 < β2. The path health index is determined based on the packet loss rate and / or the data packet queue length, and the latency index is determined based on the data packet queue length and latency.

[0043] According to another aspect of the present invention, a communication system based on traffic characteristics and path selection is provided, wherein the communication system based on traffic characteristics and path selection performs communication based on the data traffic scheduling method based on traffic characteristics and path selection described in any one or a combination of the above technical solutions.

[0044] The beneficial effects of the technical solution provided by this invention are as follows:

[0045] a. This invention predicts whether each path is more suitable for a large or small flow, and predicts whether the data flow to be sent by the sender is a large or small flow, and makes appropriate path allocation for large and small flows. Through traffic distribution based on characteristics and forward-looking path selection, it avoids blind scheduling and unreasonable resource allocation, making the load distribution more balanced. It can effectively improve the bandwidth utilization and response speed of the data exchange network, reduce congestion and latency, and improve the long-term stability and fairness of the network system and data exchange system.

[0046] b. The predictive large and small flow path allocation proposed in this invention combines the fixed path of the large flow and the real-time scheduling path of the small flow during data flow transmission. It also performs a differentiated path dynamic scheduling method based on the traffic characteristics of the large and small flows during data transmission, which ensures the orderliness of the large flow transmission, reduces the reordering overhead at the receiving end, and improves TCP throughput performance.

[0047] c. This invention prioritizes selecting paths with low congestion risk and stable health as high-flow paths, and sets the first congestion risk weight higher than the first health weight in the high-flow path selection. Through this design principle of prevention over remedy, it can prioritize avoiding high-risk paths to ensure the stability of high-flow transmission, and then use health to ensure the basic data transmission quality, thereby achieving the best balance between high throughput and low interruption probability of large-flow traffic.

[0048] d. This invention, by setting a second congestion risk weight lower than the second health weight in the small flow path selection, ensures that small flows prioritize the path with the fastest historical response, directly optimizing data flow completion time, while retaining an appropriate congestion risk weight to avoid paths that have clearly deteriorated, thus maintaining basic reliability while ensuring low latency. This weight allocation accurately captures the essential needs of small flows for speed priority and strong fault tolerance. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart of a data traffic scheduling method based on traffic characteristics and path selection, provided as an exemplary embodiment of the present invention;

[0051] Figure 2 A flowchart illustrating a method for pre-defined large flow paths and small flow paths provided as an exemplary embodiment of the present invention;

[0052] Figure 3 A flowchart illustrating a method for dynamically adjusting a small flow, provided as an exemplary embodiment of the present invention;

[0053] Figure 4 A block diagram of a communication system based on traffic characteristics and path selection is provided as an exemplary embodiment of the present invention. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0055] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0056] To address the aforementioned problems in existing technologies, this invention proposes a data traffic scheduling method and communication system based on traffic characteristics and path selection. By pre-determining whether each path is more suitable for large or small flows based on the historical data transmission status of the paths, and proactively predicting whether the data to be sent by each sender will be a large or small flow, this method allocates appropriate paths to the data flows to be sent in advance based on path prediction and data flow prediction. This not only enables more rational allocation of data flow transmission paths but also proactively avoids risks, fundamentally alleviating the out-of-order problem and reordering overhead of large flows, improving the low-latency transmission capability of small flows, and ultimately enhancing the stability and efficiency of the entire data exchange network in mixed traffic scenarios.

[0057] In one embodiment of the present invention, a data traffic scheduling method based on traffic characteristics and path selection is provided. This method is applicable to a data exchange network comprising multiple sending ends and multiple receiving ends. Both the sending end (also called the source end or source port) and the receiving end (also called the target end or target port) are electrically connected to terminal devices to be communicated in the data exchange network. The data exchange network includes multiple paths from the sending end to the receiving end, i.e., data transmission paths. The sending end is electrically connected to the source terminal device, and the receiving end is electrically connected to the target terminal device. The source terminal device and the target terminal device are two different terminal devices. (See also...) Figure 1 and Figure 2 The data traffic scheduling method includes the following steps:

[0058] Based on the historical status information of each path, some paths among multiple paths are preset as high-flow paths, and some paths are preset as low-flow paths. The path congestion risk and path stability of the high-flow paths meet the preset first requirement, and the path congestion risk and latency of the low-flow paths meet the preset second requirement. The historical status information includes multiple of the following: average packet loss rate, data packet queue length, and latency. The path congestion risk, path stability, and latency are determined based on the historical status information.

[0059] Based on the traffic characteristics of the historical data streams sent by each sender, it is predicted whether the data stream sent by each sender is a large stream or a small stream. If the data stream is predicted to be a large stream, the sender is determined to be a large stream sender and the data stream sent by the large stream sender is assigned to the large stream path; if the data stream is predicted to be a small stream, the sender is determined to be a small stream sender and the data stream sent by the small stream sender is assigned to the small stream path.

[0060] Compared to existing technologies that rely on real-time traffic detection and path allocation, this application proposes a method for pre-allocating large and small flow paths based on "historical prediction" and predicting large and small flows at ports. When the endpoint of the current flow, i.e., the sender, is found to have frequently generated large flows in the past, the system can predict it as a high-probability large flow at the beginning of the flow, or even when the first data packet arrives. This prediction based on historical behavior patterns provides a more valuable lead time for decision-making compared to previous methods that only analyzed the characteristics of the first few data packets after the flow started or relied on real-time detection and analysis. This lays a solid foundation for early path reservation for large flows and avoiding out-of-order and reordering overhead. It should be noted that both the sender and receiver are ports electrically connected to terminal devices in a data exchange network. When a port is configured to send data, it is a sender; when a port is configured to receive data, it is a receiver.

[0061] In one embodiment of the present invention, see Figure 2Each path is preset as a high-flow path or a low-flow path in the following manner. The first requirement includes that the path's historical data transmission quality score is not lower than a first threshold and / or the path's bandwidth is not less than a preset bandwidth threshold. The second requirement includes that the path's historical data transmission quality score is not lower than a second threshold and / or the path's latency meets a preset latency requirement.

[0062] In this application, the path selection for large flows (high-bandwidth, long-duration data flows) focuses on choosing paths with low congestion risk and stable health to ensure long-term, stable, high-throughput transmission performance. Therefore, in a specific embodiment of the present invention, the first score of the historical data transmission quality of each path is calculated by the following formula: S1=-(α1×C risk +β1×H risk ), where C risk H represents the probability of congestion risk. risk The path health index is represented by α1, which represents the first congestion risk weight, and β1, which represents the first health weight, where α1 > β1 > 0.

[0063] Preferably, the values ​​of α1 and β1 range from (0, 1), more preferably, α1 > β1, and even more preferably, α1 = 0.6 and β1 = 0.4. In this application, the first score is used to filter out paths suitable for large-scale transmission. Given that congestion risk directly triggers the window backoff of the TCP (Transmission Control Protocol) congestion control mechanism, resulting in an exponential decrease in throughput, this is disastrous for long-cycle transmission of large-scale data streams; while health mainly affects the linear improvement of transmission efficiency. Therefore, the advantage of setting the congestion risk weight to 0.6 and the health weight to 0.4 in this application is that this weight allocation reflects the design principle of "prevention is better than cure," ensuring transmission stability by prioritizing the avoidance of high-risk paths, and then supplementing it with health to guarantee the quality of basic services, thereby achieving the best balance between high throughput and low interruption probability for large-scale data streams.

[0064] In this application, path selection for small flows (short lifecycle, small data volume flow) focuses more on end-to-end latency; therefore, path scoring primarily measures latency trends. In a specific embodiment of the invention, the second score for the historical data transmission quality of each path is calculated using the following formula: S2 = -(α2 × C risk +β2×FCT trend ), where C risk FCT represents the probability of congestion risk. trend α represents the latency index, α2 represents the second congestion risk weight, β2 represents the second health weight, and 0 < α2 < β2.

[0065] Preferably, the values ​​of α2 and β2 range from (0,1); more preferably, α2 < β2; even more preferably, α2 = 0.3 and β2 = 0.7. The weight allocation of α2 = 0.3 and β2 = 0.7 for small flow paths is based on their unique transmission characteristics: small flows have short lifecycles (typically completed within 1-3 RTTs), their completion time is extremely sensitive to queuing delays, and the data volume of a single small flow is small, so even if congestion occurs, the cost of retransmission is relatively limited. Therefore, setting the completion time trend as the primary consideration (β2 = 0.7) ensures that small flows prioritize the path with the fastest historical response, directly optimizing their core performance indicator—flow completion time; at the same time, retaining an appropriate congestion risk weight (α2 = 0.3) to avoid paths that have clearly deteriorated, maintaining basic reliability while ensuring low latency. This weight allocation accurately captures the essential requirements of small flows: "speed priority and strong fault tolerance."

[0066] In practical applications, the path health index, congestion risk probability, and latency index can be calculated in various ways. For example, the path health index can be determined based on packet loss rate and data packet queue length, the congestion risk probability can be determined based on data packet queue length and latency, and the latency index can be determined based on data packet queue length and latency. In other specific application examples, the path health index can also be determined by referencing the jitter amplitude of the transmitted data along the path, while the data transmission rate of the path can be used to determine the congestion risk probability and latency index.

[0067] See Figure 2 Iterate through all paths and calculate the first and second scores of historical data transmission quality for each path. For paths with a first score threshold greater than the first threshold and a second score threshold not greater than the second threshold, they are identified as high-flow paths. These paths have good health and stability under low congestion risk, and excellent long-term, stable high-throughput transmission performance, but their end-to-end latency is not good enough. They are suitable as high-flow paths but not as low-flow paths.

[0068] For paths with a first scoring threshold not greater than the first threshold and a second scoring threshold greater than the second threshold, they are identified as small flow paths. These paths have low latency but poor stability, so they are suitable as small flow paths but not as large flow paths.

[0069] For paths where both the first scoring threshold and the second scoring threshold are greater than the first threshold, these paths are suitable for both high-flow and low-flow paths. Preferably, the path is re-evaluated based on its bandwidth. If the path bandwidth is greater than a preset bandwidth threshold, it is determined to be a high-flow path; otherwise, it is determined to be a low-flow path.

[0070] For paths where the first scoring threshold is no greater than the first threshold and the second scoring threshold is no greater than the second threshold, these paths are unsuitable for both high-volume and low-volume paths based on historical data transmission quality scores. For these paths with poor historical data transmission quality, their suitability for high-volume or low-volume paths can be determined based on path bandwidth or latency. For example, if the path bandwidth is greater than a preset bandwidth threshold, it is identified as a high-volume path; otherwise, it is identified as a low-volume path. Alternatively, it can be determined whether the path latency meets the latency requirements for low-volume data transmission. If it does, it is identified as a low-volume path; otherwise, it is identified as a high-volume path.

[0071] For any given sender, predict whether the data stream it sends is a large or small stream. In one embodiment of the invention, the prediction of whether the data stream sent by each sender is a large or small stream is achieved as follows: Configure five-tuple information for the data stream, the five-tuple information including source IP address, destination IP address, sender number, receiver number, and transport layer protocol number; pre-construct a historical stream feature library, the historical stream feature library including five-tuple information and traffic features corresponding to the data streams sent by each sender, the traffic features including traffic size and traffic frequency; for a data stream to be sent by a sender, extract the five-tuple information of the data stream to be sent as target five-tuple information, determine the traffic features corresponding to the five-tuple information in the historical stream feature library that is the same as the target five-tuple information as target traffic features; predict whether the data stream to be sent is a large or small stream based on the target traffic features, specifically, if the historical data stream corresponding to the target traffic features is a large stream, then the data stream to be sent is a large stream; if the historical data stream corresponding to the target traffic features is a small stream, then the data stream to be sent is a small stream. If the historical flow feature library does not contain a quintuple that is identical to the target quintuple information, then a preset traffic classification model is used to predict whether the data stream to be sent is a large flow or a small flow.

[0072] In one specific embodiment, the process of classifying and predicting the large and small streams of data to be sent by each sending end is as follows: An intelligent stream classification and labeling module is pre-built. This module is a predictive stream classification module based on historical behavioral profiles. The core of this module lies in predicting the type of the current stream through historical data. Its technical implementation can be divided into three steps: data structure definition, prediction logic execution, and result labeling and learning.

[0073] First, construct a lightweight historical stream feature library data structure. This feature library is implemented in key-value pair form at the sending end (such as a smart network card or vSwitch). It is preferable to use an efficient in-memory data structure such as a hash table for storage to ensure query speed.

[0074] Key: A fixed-bitwidth hash value calculated from the five-tuple of the data stream. The five-tuple includes: source IP address, destination IP address, sender number, receiver number, and transport layer protocol number. Using the hash value as the key not only ensures uniqueness but also compresses the storage space.

[0075] Value: It is a structure that contains the following historical behavior feature fields: uint64_t historical_avg_size; / / The average size (in bytes) of the historical data stream corresponding to this five-tuple information; uint32_t last_seen_timestamp; / / The timestamp when this data stream was last observed; uint16_t flow_count; / / The cumulative number of historical data streams corresponding to this five-tuple information that have been observed; uint8_t last_flow_type; / / The type of the historical data stream observed last time. The types of data streams include large flows and small flows. For example: 0 represents a small flow, and 1 represents a large flow).

[0076] The working process and specific logic of the predictive classification by the intelligent flow classification and marking module are as follows.

[0077] Step 1: Query. When a new data stream arrives, extract its five-tuple information, generate a hash key using the same hash function, and query it in the historical flow feature library.

[0078] Step 2: Prediction decision. If the query hits, then execute the following judgment logic: IF (historicalavgsize > SIZETHRESHOLD) AND (current time - lastseentimestamp < AGETHRESHOLD) THEN, immediately mark this flow as the corresponding "predicted large flow" or "predicted small flow" based on the prediction result. Here, SIZE_THRESHOLD is the preset size threshold for large flows (such as 1MB), and AGE_THRESHOLD is the time threshold for record validity (such as 30 minutes), which is used to eliminate outdated historical records.

[0079] Miss scenario: If the query misses, then start the real-time backup classifier. This classifier extracts the features of the first N (such as N = 5) data packets of the current flow: the total number of bytes of the first N packets. The variance of the average packet interval time of the first N packets. Input these features into a pre-trained lightweight machine learning model (such as a decision tree or a small-scale neural network), and the model outputs a probability value that this flow is an elephant flow. If the probability value exceeds the preset threshold (such as 0.7), then classify it as a "real-time detection large flow", otherwise, classify it as a "real-time detection small flow".

[0080] Step 3: Identification and learning

[0081] Identification: For data streams classified as high-volume streams, explicit identification is achieved within the network by rewriting the reserved Type / Length fields in the MAC header of the data frame into a conventional field, without modifying the IP or TCP headers, thus ensuring protocol compatibility.

[0082] Learning: Regardless of how the data stream is ultimately classified, the system updates the historical stream feature library when the data stream transmission ends. It updates `historical_avg_size` using an exponentially weighted moving average algorithm and refreshes `last_seen_timestamp` and `flow_count`. This allows the system to adaptively learn changes in application traffic patterns.

[0083] Based on the path prediction results and the size and flow classification prediction results of the data streams to be sent at the sending end, the initial path allocation is performed on the data streams of each sending end.

[0084] For each large stream sender, one of its corresponding large stream paths is designated as the first large stream path, and the large stream is transmitted using the first large stream path. After being assigned to the corresponding large stream, the first large stream path is no longer used to transmit data streams sent by other senders. Preferably, the first large stream path is the optimal large stream path among all the large stream paths from the sender to the receiver.

[0085] See Figure 1 For the large-flow sender, determine whether there is one or more large-flow paths. If there are multiple, determine the large-flow path with no occupied port as the target large-flow path; determine the target large-flow path with the best historical data transmission quality as the first large-flow path.

[0086] If all the major flow paths have occupied ports, it is determined whether there is a major flow path that meets the following preset condition: the sending end corresponding to the data stream forwarded by its occupied port is the same as the sending end of the major flow. If such a major flow path exists, it is determined as the first major flow path. If no major flow path meets the preset condition, it is determined as the major flow path with the best historical data transmission quality. Alternatively, if no major flow path meets the preset condition, it is determined as the major flow path with the shortest current queuing time. Alternatively, if no major flow path meets the preset condition, it is determined as the first major flow path based on the current queuing time and historical data transmission quality. The major flow path with the highest comprehensive score is then determined as the first major flow path. After determining the first major flow path and allocating it to the major flow sending ends for data streams, the port occupancy status on the first major flow path is modified to occupancy, and the major flow is transmitted using the first major flow path until the entire major flow is transmitted, unless the data transmission quality of the first major flow path fails to meet the requirements.

[0087] The data stream sent by the large-stream sender is transmitted using a first large-stream path, and the current data transmission quality of the first large-stream path is monitored. If the data stream transmission quality meets a preset current data transmission quality requirement, the large-stream continues to be transmitted using the first large-stream path; if the data stream transmission quality does not meet the preset current data transmission quality requirement, another large-stream path is used to transmit the large-stream. It should be noted that, after the sender and receiver are determined, the large-stream path refers to all equivalent large-stream paths from the sender to the receiver.

[0088] The evaluation method for the current data transmission quality can be similar to the first-score evaluation method for the historical data transmission quality described above. Specifically, the third score for the current data transmission quality is calculated using the following formula: S1 = -(α1 × C risk_current +β1×H risk ), where C risk_current H represents the current probability of congestion risk. risk The current path health index is represented by α1, where α1 represents the first congestion risk weight and β1 represents the first health weight, and α1 > β1 > 0. If the third score is higher than a preset third threshold, the first high-flow path will continue to be used to send the high flow; otherwise, the first high-flow path will be replaced with another high-flow path.

[0089] Alternatively, the current data transmission quality requirements can be set to meet one or more of the following: (1) the queue length of the exit port on the first major flow path is not higher than the preset queue length threshold; (2) the packet loss rate is not higher than the preset packet loss rate threshold; (3) the round-trip delay is not higher than the preset delay threshold; (4) the data transmission rate is not lower than the preset rate threshold.

[0090] See Figure 3 For the small stream sender, the number of small stream paths from the sender to the receiver is determined. If there are multiple target small stream paths, a random packet spraying mechanism is used to distribute the data stream sent by the small stream sender to multiple target small stream paths for transmission. The advantage of this method is that it requires less computation and consumes fewer system resources, but it lacks rationality in allocation, and its utilization rate of system resources and data transmission efficiency are also relatively low.

[0091] More preferably, if there are multiple small flow paths, then dynamic path scores are determined for each of the multiple small flow paths. These dynamic path scores are determined based on multiple factors, including the instantaneous packet loss rate, the egress port queue length, and the latency of the small flow path. Small flow paths with dynamic path scores higher than a preset dynamic path score threshold are designated as target small flow paths, and the data streams sent by the small flow sender are allocated to these target small flow paths for transmission.

[0092] More preferably, a corresponding traffic allocation weight value is set for the target small flow path based on the dynamic path score of multiple target small flow paths. The higher the dynamic path score, the higher the traffic allocation weight value. The data stream sent by the small flow sending end is allocated to each target small flow path according to the traffic allocation weight value for transmission.

[0093] The following provides a dynamic path scoring and traffic scheduling method based on multi-indicator trend prediction, configuring a dynamic path scoring module and a path dynamic adjustment module. The core of the dynamic path scoring module is to calculate a comprehensive and dynamic evaluation result for each path, integrating the current state and future trends. The path dynamic adjustment module is configured to perform differentiated data traffic scheduling based on the dynamic scoring results of each path output by the dynamic path scoring module.

[0094] The dynamic path scoring module's operation includes path status monitoring and trend calculation.

[0095] The dynamic path scoring module can collect the current status data of each path in real time, or poll each path at a fixed sampling period (e.g., T=100ms). The current status data includes three key physical indicators: instantaneous packet loss rate, egress port queue length, and average round-trip time.

[0096] The dynamic path scoring module performs path trend quantification calculations based on the current status data of the collected paths, including maintaining a fixed-length sliding sampling window for each indicator (instantaneous packet loss rate, egress port queue length, average round-trip delay), such as saving the sampling values ​​of the most recent several periods.

[0097] For each index sequence within the window—namely, instantaneous packet loss rate, egress port queue length, and average round-trip time—a linear regression method is used to fit a straight line to its sampled values. Let the time series be (x1, y1), (x2, y2), ..., (x... k , y k ), where x is the time point, y is the index value (for the packet loss rate sequence, it is the instantaneous packet loss rate corresponding to each sampling time; for the egress port queue length sequence, it is the egress port queue length corresponding to each sampling time; for the average round-trip time sequence, it is the average round-trip time corresponding to each sampling time), and k is the number of sampling points. The slope γ of the fitted execution is calculated using the least squares method: γ=[Σ(x i -x̄)(y i -ȳ)] / Σ(x i -x̄) 2 The slope γ represents the short-term trend of this indicator. γ > 0 indicates that the indicator is deteriorating (e.g., increasing latency), while γ < 0 indicates that it is improving. Specifically, the slope obtained by fitting a straight line to the sampled values ​​of instantaneous packet loss rate is γ1, the slope obtained by fitting a straight line to the sampled values ​​of egress port queue length is γ2, and the slope obtained by fitting a straight line to the sampled values ​​of average round-trip delay is γ3.

[0098] The slope γ is normalized to a trend weight factor (τ) using a sigmoid function, constraining its range to [0.5, 1.5]. For example, the trend weight factor is expressed by the following formula: τ = δ1 + δ2sigmoid(γ × S scale ), where γ takes values ​​γ1, γ2, and γ3, representing the slopes of the fitted lines for instantaneous packet loss rate, exit port queue length, and average round-trip delay, respectively. Correspondingly, τ takes values ​​τ1, τ2, and τ3, where δ1 represents the first trend weight coefficient, δ2 represents the second trend weight coefficient, and the sigmoid function, also called the Logisτic function, has a range of (0, 1) and is a type of S-shaped function. scale The adjustment coefficient is a variable parameter used to adjust the range of values ​​for the trend weight factor. Preferably, δ1=1 and δ2=0.5. This ensures that a deteriorating path has a trend weight factor greater than 1, which penalizes its score; while an improving path has a trend weight factor less than 1, which improves its score.

[0099] Based on the formula τ=δ1+δ2sigmoid(γ×S) scale The instantaneous packet loss rate trend weight factor τ1, the exit port queue length trend weight factor τ2, and the average round-trip delay trend weight factor τ3 of the path are calculated.

[0100] Furthermore, the three instantaneous index values ​​of the path are normalized to the [0,1] interval, that is, the instantaneous packet loss rate, the egress port queue length and the average round-trip time are normalized, and the dynamic score of the path is calculated based on the normalized values.

[0101] Taking instantaneous packet loss rate as an example, its normalized value is: ;in, p Take an integer from 1 to k, where k is the number of sampling points within the sliding window. Indicates the first p The minimum instantaneous packet loss rate for each path at each sampling point. Indicates the first p The maximum instantaneous packet loss rate for each path at each sampling point. Indicates the first p The sampling point i The instantaneous packet loss rate of the path.

[0102] Taking the length of the egress port queue as an example, its normalized value is: ;in, p Take integers from 1 to k. Indicates the first p The minimum exit port queue length for each path at each sampling point. Indicates the first p The minimum exit port queue length for each path at each sampling point. Indicates the first p The sampling point i The length of the exit port queue for each path.

[0103] Taking average round-trip time as an example, its normalized value is: ;in, Indicates the first p The minimum average round-trip time for each path at each sampling point Indicates the first p The maximum average round-trip time for each path at each sampling point Indicates the first p The sampling point i The average round-trip time of the path.

[0104] This application uses a weighted summation model and introduces a trend weight factor to calculate the "bad score" for each path. The dynamic score calculation formula for each path is: S Score_i=ω1×τ1×N(L i )+ω2×τ2×N(Q i )+ ω3×τ3×N(D i ), where ω1, ω2, and ω3 are the first adjustment coefficient, the second adjustment coefficient, and the third adjustment coefficient, respectively. Preferably, ω1 + ω2 + ω3 = 1. The values ​​of ω1, ω2, and ω3 can be adjusted according to the network strategy. For example, in latency-sensitive scenarios, ω3 can be given a higher weight value.

[0105] Based on the dynamic scoring of each path, for small flow paths, the allocation probability value of the data flow assigned to the small flow path is further calculated as follows: Where N represents the total number of the minor flow paths and their equivalent minor flow paths. Based on the allocation probability value To the i Small flows are assigned to small flow paths. Based on this assignment method, small flows can be automatically guided to small flow paths with the best current quality and positive future trends.

[0106] For large flow paths, a large flow path mapping table is constructed based on the aforementioned dynamic scoring. This mapping table includes dynamic scores that correspond one-to-one with each large flow path. Once a large flow is identified, a first large flow path can be determined using the aforementioned initial allocation method and used for data transmission. Alternatively, the large flow path with the highest dynamic score among the current large flow paths can be determined as the first large flow path based on the large flow path mapping table. After determining the first large flow path, all subsequent data packets in the large flow occurring at the sending end will strictly pass through this path until the large flow ends, thereby completely avoiding packet out-of-order delivery.

[0107] After determining the first major flow path, there is another scheduling method. When the first major flow path is used to transmit the major flow, if the data transmission quality does not meet the preset data transmission quality requirements, the major flow path with the highest dynamic score in the current major flow path is determined as the new first major flow path based on the major flow path mapping table.

[0108] Compared to existing static path evaluation methods, this application proposes a dynamic path scoring and traffic scheduling method based on multi-indicator trend prediction, achieving dynamic trend prediction of paths. It not only monitors packet loss, latency, and queue length in real time, but also quantifies the short-term trends of these indicators through sliding windows and linear regression. The calculated "trend factor" is combined with real-time indicators to form a dynamic and forward-looking path evaluation method. This dynamic scoring is directly applied to scheduling decisions: for small flows, the system prioritizes allocating them to paths with high scores (i.e., good quality and positive trends), thereby proactively avoiding links that are about to become congested and significantly reducing transmission latency; for large flows, the most stable path is selected from the best-scoring paths for fixed transmission, effectively avoiding packet out-of-order delivery. Even when a fixed transmission path experiences a serious failure, adjustments can be made accordingly, and a new large flow path can be updated for data transmission.

[0109] In summary, this application utilizes two core technologies—historical prediction of flow type and trend-based prediction of path quality—to form an intelligent closed-loop control system. This system not only identifies traffic more accurately but also proactively mitigates risks, fundamentally alleviating the out-of-order problem of large flows, improving the low-latency transmission capability of smaller flows, and ultimately enhancing the stability and efficiency of the entire data center network in mixed traffic scenarios.

[0110] In one embodiment of the present invention, a communication system based on traffic characteristics and path selection is provided, wherein the communication system performs communication based on the data traffic scheduling method based on traffic characteristics and path selection as described in any of the above embodiments.

[0111] In one embodiment of the present invention, a communication system based on traffic characteristics and path selection is provided, see [link to relevant documentation]. Figure 4 The communication system includes an intelligent flow classification and labeling module, a path performance evaluation and prediction module, an intelligent path selection module, and a dynamic path scoring module.

[0112] The intelligent flow classification and labeling module is configured to predict major and minor flow paths among multiple paths based on the historical state information of each path. For details, please refer to the method for presetting major and minor flow paths described in the above embodiments, which will not be repeated here.

[0113] The path performance evaluation and prediction module is configured to predict whether the data stream sent by each sender is a large flow or a small flow based on the traffic characteristics of the historical data streams sent by each sender. For details, please refer to the prediction methods for large and small flows in the above embodiments, which will not be repeated here.

[0114] The intelligent path selection module includes a path initial allocation module and a path dynamic adjustment module. The path initial allocation module is configured to allocate a corresponding path to the sending end for data stream transmission based on the prediction results of the intelligent flow classification and labeling module and the path performance evaluation and prediction module, as described in the above embodiments, and will not be repeated here.

[0115] The dynamic path scoring module is configured to determine the dynamic score of each path and transmit it to the path dynamic adjustment module. The specific calculation method of the dynamic score of each path is as described in the above embodiment and will not be repeated here.

[0116] The path dynamic adjustment is configured to allocate small flows based on the allocation probability value of the small flow path determined by the dynamic path scoring module, and to schedule large flows based on the dynamic scoring and the current data transmission quality of the large flow path.

[0117] This invention predicts whether each path is more suitable for a large or small flow, and whether the data stream to be sent by the sender is a large or small flow. It then marks and assigns paths accordingly to small flows. During data transmission, it uses a differentiated dynamic path scheduling method based on the traffic characteristics of large and small flows. This effectively improves the bandwidth utilization and response speed of data exchange networks, reducing congestion and latency. Its application prospects are broad, particularly suitable for business scenarios with high real-time and throughput requirements, such as cloud computing, big data analytics, video streaming, and online games. Furthermore, this technology does not rely on dedicated hardware, can be flexibly deployed in existing standard network architectures, and has strong scalability and adaptability. It helps improve the operational efficiency and service quality of data centers, promoting the widespread application of intelligent network scheduling.

[0118] It should be noted that the communication system embodiment based on traffic characteristics and path selection provided by the present invention has the same inventive concept as the above-mentioned data traffic scheduling method embodiment based on traffic characteristics and path selection and the dynamic path scoring and traffic scheduling method embodiment based on multi-index trend prediction. The entire contents of the data traffic scheduling method embodiment based on traffic characteristics and path selection and the dynamic path scoring and traffic scheduling method embodiment based on multi-index trend prediction are incorporated into the communication system embodiment based on traffic characteristics and path selection by means of introduction.

[0119] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0120] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data traffic scheduling method based on traffic characteristics and path selection, characterized in that, Applicable to a data exchange network comprising multiple transmitters and multiple receivers, the data exchange network including multiple paths for data transmission from the transmitters to the receivers, comprising the following steps: Based on the historical status information of each path, some paths are preset as high-flow paths and some paths are preset as low-flow paths. The path congestion risk and path stability of the high-flow paths meet a preset first requirement, which includes a first score for the historical data transmission quality of the path not lower than a first threshold and a bandwidth of the path not less than a preset bandwidth threshold. The path congestion risk and latency of the low-flow paths meet a preset second requirement, which includes a second score for the historical data transmission quality of the path not lower than a second threshold and a latency of the path meeting a preset latency requirement. The historical status information includes multiple parameters such as average packet loss rate, packet queue length, and latency. The path congestion risk, path stability, and latency are determined based on the historical status information. Based on the traffic characteristics of the historical data streams sent by each sender, it is predicted whether the data stream sent by each sender is a large stream or a small stream. If the data stream is predicted to be a large stream, the sender is determined to be a large stream sender, and the data stream sent by the large stream sender is allocated to a large stream path, including: determining one of the large stream paths as a first large stream path, using the first large stream path to send the large stream, and the first large stream path is not configured to transmit data sent by other senders. If the data stream is predicted to be a small stream, then the sending end is determined to be a small stream sending end, and the data stream sent by the small stream sending end is allocated to the small stream path. If there are multiple small stream paths, then the dynamic path score corresponding to the multiple small stream paths is determined. The dynamic path score is determined based on multiple factors of the small stream path, including instantaneous packet loss rate, egress port queue length, and latency. The small stream path with a dynamic path score higher than a preset dynamic path score threshold is determined as the target small stream path, and the data stream sent by the small stream sending end is allocated to the target small stream path for transmission.

2. The data traffic scheduling method based on traffic characteristics and path selection according to claim 1, characterized in that, If there are multiple target small flow paths, then a corresponding traffic allocation weight value is set for each target small flow path based on the dynamic path score of the multiple target small flow paths. The higher the dynamic path score, the higher the traffic allocation weight value. The data stream sent by the small stream sending end is allocated to each target small stream path according to the traffic allocation weight value and then sent.

3. The data traffic scheduling method based on traffic characteristics and path selection according to claim 1, characterized in that, If there are multiple target small flow paths, a random data packet spraying mechanism is used to distribute the data stream sent by the small flow sender to multiple target small flow paths for transmission.

4. The data traffic scheduling method based on traffic characteristics and path selection according to claim 1, characterized in that, The data stream sent by the large stream sender is sent using the first large stream path, and the current data transmission quality of the first large stream path is monitored. If the data stream transmission quality meets the preset current data transmission quality requirements, then the first large stream path will continue to be used to send the large stream; If the data stream transmission quality does not meet the preset current data transmission quality requirements, then another high-volume path is used to send the high-volume stream.

5. The data traffic scheduling method based on traffic characteristics and path selection according to claim 1, characterized in that, For the large flow sending end, determine whether the number of its corresponding large flow paths is one or more. If it is multiple, then determine the large flow path that does not have an occupied port as the target large flow path. The target high-flow path with the best historical data transmission quality is determined as the first high-flow path.

6. The data traffic scheduling method based on traffic characteristics and path selection according to claim 5, characterized in that, If each of the major flow paths has occupied ports, then determine whether there is a major flow path that meets the following preset condition: the sending end of the data stream forwarded by its occupied port is the same as the sending end of the major flow. If it exists, then the major flow path that meets the preset conditions is determined as the first major flow path.

7. The data traffic scheduling method based on traffic characteristics and path selection according to claim 6, characterized in that, If no high-flow path that meets the preset conditions exists, then the high-flow path with the best historical data transmission quality is determined as the first high-flow path. or, If no high-flow path that meets the preset conditions exists, then the high-flow path with the shortest current queuing time is determined as the first high-flow path. or, If no high-flow path meets the preset conditions, a comprehensive score for each high-flow path is determined based on the current queuing time and historical data transmission quality, and the high-flow path with the highest comprehensive score is determined as the first high-flow path.

8. The data traffic scheduling method based on traffic characteristics and path selection according to claim 1, characterized in that, Configure five-tuple information for the data stream, the five-tuple information including source IP address, destination IP address, sender number, receiver number, and transport layer protocol number; A historical flow feature library is pre-built, which includes five-tuple information and flow features corresponding to the data streams sent by each sending end. The flow features include flow size and flow frequency. For a data stream to be sent from a sending end, extract the five-tuple information of the data stream to be sent as the target five-tuple information, and determine the traffic feature corresponding to the five-tuple information in the historical flow feature library that is the same as the target five-tuple information as the target traffic feature. Predict whether the data stream to be sent is a large stream or a small stream based on the target traffic characteristics.

9. The data traffic scheduling method based on traffic characteristics and path selection according to claim 8, characterized in that, If the historical flow feature library does not contain a quintuple that is identical to the target quintuple information, then a preset traffic classification model is used to predict whether the data stream to be sent is a large flow or a small flow.

10. The data traffic scheduling method based on traffic characteristics and path selection according to claim 1, characterized in that, The first score for the historical data transmission quality of each path is calculated using the following formula: S1 = α1 × C risk +β1×H risk , where C risk H represents the probability of congestion risk. risk The path health index is represented by α1, which represents the first congestion risk weight, and β1, which represents the first health weight. α1 > β1 > 0. The path health index is determined based on the packet loss rate and / or the data packet queue length. The congestion risk probability is determined based on the data packet queue length and latency.

11. The data traffic scheduling method based on traffic characteristics and path selection according to claim 1, characterized in that, The second score for the historical data transmission quality of each path is calculated using the following formula: S2 = α2 × C risk +β2×FCT trend , where C risk FCT represents the probability of congestion risk. trend The latency index is represented by α2, the second congestion risk weight is represented by β2, and the second health index is represented by β2, where 0 < α2 < β2. The path health index is determined based on the packet loss rate and / or the data packet queue length, and the latency index is determined based on the data packet queue length and latency.

12. A communication system based on traffic characteristics and path selection, characterized in that, The communication system based on traffic characteristics and path selection communicates based on the data traffic scheduling method based on traffic characteristics and path selection as described in claim 1.

Images