Traffic scheduling method and electronic device

Abstract

The application discloses a flow scheduling method and electronic equipment, and relates to the technical field of flow scheduling. The method comprises the following steps: in response to the fact that a to-be-sent data stream meets a transmission layer protocol and the flow size of the to-be-sent data stream is greater than a preset threshold, a plurality of candidate port values for identifying the to-be-sent data stream at a data sending end are generated; a plurality of probe data packets are created, and the source port fields of the probe data packets are set to different candidate port values; the probe data packets are sent to a data receiving end, and the performance indexes of candidate network paths corresponding to the plurality of probe data packets are obtained based on the response of the data receiving end; a target source port value is determined from the plurality of candidate port values based on the performance indexes; the source port field of a data packet corresponding to the to-be-sent data stream is modified to the target source port value, and the modified data packet corresponding to the to-be-sent data stream is sent to the data receiving end, thereby solving the technical problem of unbalanced multi-path network load in the prior art and improving the reliability of flow scheduling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of traffic scheduling, and in particular to a traffic scheduling method and an electronic device. BACKGROUND

[0002] Network devices rely on hash algorithms to distribute traffic over multiple paths, while hash collisions are inevitable, leading to uneven distribution of network load over multiple paths. This imbalance can cause problems such as congestion on some links, decreased bandwidth utilization, increased network transmission latency, and packet loss, which seriously affect the overall forwarding efficiency and performance of the network.

[0003] To address the problem of uneven load distribution in multi-path networks, related solutions mostly focus on optimizing network core devices (such as switches and routers), such as improving their built-in hash algorithms or implementing weighted cost multi-path routing (WCMP). However, such solutions have inherent limitations, making it difficult to effectively solve the above-mentioned load imbalance problem fundamentally: first, they are not flexible and cost high, requiring upgrading or reconfiguring existing network infrastructure; second, they have poor compatibility and are difficult to deploy uniformly in heterogeneous network environments; most importantly, these solutions are still essentially static or semi-static optimization, which is difficult to perceive the end-to-end path quality in real time, and cannot adapt to dynamic changes in network traffic patterns, thus cannot achieve true dynamic load balancing. SUMMARY

[0004] The present application provides a traffic scheduling method and an electronic device to at least solve the technical problem of uneven load distribution in multi-path networks in related technologies.

[0005] The present application provides a traffic scheduling method applied to a data sending end, the traffic scheduling method comprising:

[0006] In response to the to-be-sent data stream conforming to a transport layer protocol and the traffic size of the to-be-sent data stream being greater than a preset threshold, generating a plurality of candidate port values for identifying the to-be-sent data stream at the data sending end, each candidate port value corresponding to a candidate network path; creating a plurality of probe data packets, wherein the source port field of each probe data packet is set to a different candidate port value; sending the probe data packets to a data receiving end, and obtaining performance indicators of the candidate network paths corresponding to the plurality of probe data packets based on the response of the data receiving end; evaluating the plurality of candidate port values based on the performance indicators, and determining a target source port value from the plurality of candidate port values according to the evaluation result; modifying the source port field of the data packets corresponding to the to-be-sent data stream to the target source port value, and sending the modified data packets corresponding to the to-be-sent data stream to the data receiving end.

[0007] The application further provides an electronic device, comprising a memory for storing a computer program, and a processor for executing the computer program to implement the steps of the traffic scheduling method in the embodiments.

[0008] In response to the to-be-sent data stream conforming to the transport layer protocol and the traffic size of the to-be-sent data stream being greater than a preset threshold, a plurality of candidate port values for identifying the to-be-sent data stream at a data sending end are generated, each candidate port value corresponding to a candidate network path; a plurality of probe data packets are created, wherein the source port fields of the probe data packets are set to different candidate port values; the probe data packets are sent to a data receiving end, and the performance indicators of the candidate network paths corresponding to the plurality of probe data packets are obtained based on the response of the data receiving end; the plurality of candidate port values are evaluated based on the performance indicators, and a target source port value is determined from the plurality of candidate port values according to the evaluation result; the source port field of the data packet corresponding to the to-be-sent data stream is modified to the target source port value, and the modified data packet corresponding to the to-be-sent data stream is sent to the data receiving end.

[0009] The traffic scheduling method provided by the application generates a plurality of candidate port values for identifying the to-be-sent data stream at a data sending end in response to the to-be-sent data stream conforming to the transport layer protocol and the traffic size of the to-be-sent data stream being greater than a preset threshold, each candidate port value corresponding to a candidate network path; a plurality of probe data packets are created, wherein the source port fields of the probe data packets are set to different candidate port values; the probe data packets are sent to a data receiving end, and the performance indicators of the candidate network paths corresponding to the plurality of probe data packets are obtained based on the response of the data receiving end; the plurality of candidate port values are evaluated based on the performance indicators, and a target source port value is determined from the plurality of candidate port values according to the evaluation result; the source port field of the data packet corresponding to the to-be-sent data stream is modified to the target source port value, and the modified data packet corresponding to the to-be-sent data stream is sent to the data receiving end.

[0010] Thus, by setting the data stream to be sent to meet both the transport layer protocol and traffic size criteria, this ensures that data streams with large data volumes that support source port field modification are probed, preventing waste of system resources. Compared to the traditional method of passively accepting default hash results, this application actively creates probe packets, sets the source port field of each probe packet to different candidate port values, simulates data transmission for each candidate port value, and logically establishes a mapping relationship between source port values ​​and network paths. By measuring the performance indicators of simulated transmission, the system can accurately perceive the health status and congestion level of each candidate network path, actively avoid congested paths with high latency and high packet loss, and select the path with the best current quality. This fundamentally avoids the negative impact of hash collisions, achieves dynamic and adaptive load balancing, and achieves traffic scheduling goals by only modifying the source port field. At the same time, it ensures full compatibility with relevant network devices, protocols, and applications without requiring any changes to network infrastructure or applications, resulting in low deployment costs. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a traffic scheduling method provided in an embodiment of this application;

[0013] Figure 2 This is a schematic diagram of the structure of a traffic scheduling system provided in an embodiment of this application;

[0014] Figure 3 This is an internal structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0016] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or network device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or network device. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0017] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] In modern data center, cloud computing, and high-speed enterprise network environments, multi-path routing technologies, such as Equal Cost Multipath (ECMP) or Link Aggregation Groups (LAG), are commonly used to fully utilize the switching capabilities of network devices. In these technologies, network switches typically distribute data flows across multiple physical links by performing hash operations on specific fields in the packet header (such as source IP address, destination IP address, protocol type, source port, and destination port) to achieve load balancing and improve overall bandwidth.

[0019] However, this hash-based load balancing mechanism has an inherent flaw: hash collisions. That is, different data flows may be calculated with the same hash value, thus being mapped to the same physical link. In actual network traffic, due to traffic asymmetry (such as the existence of "elephant flows") or uneven distribution of hash fields (e.g., a large amount of traffic may point to a few destination ports), hash collisions can lead to severe load imbalances on multiple logical links. Some links may become congested, resulting in queuing delays or even packet drop, while other links are lightly loaded or idle. This undoubtedly wastes network bandwidth resources and significantly reduces the overall forwarding efficiency and performance of the network. Currently, industry solutions to this problem mainly focus on the network equipment side. For example, optimizing the hash algorithm of switches (such as using algorithms with more even distribution, such as CRC32), using Weighted Cost Multipath Routing (WCMP), or dynamically adjusting path weights through a centralized controller. However, these solutions all require support from core network equipment, have poor deployment flexibility, and cannot adapt to all dynamically changing traffic patterns, making it difficult to fundamentally solve the load imbalance problem caused by hash collisions.

[0020] In response to the above technical problems, such as Figure 1 As shown, an embodiment of this application provides a traffic scheduling method, which specifically includes the following steps:

[0021] Step 101: In response to the data stream to be sent conforming to the transport layer protocol and the traffic size of the data stream to be sent being greater than a preset threshold, generate multiple candidate port values ​​for identifying the data stream to be sent at the data sending end, with each candidate port value corresponding to a candidate network path.

[0022] In actual business traffic, the source IP (source network address) and destination IP (destination network address) are determined by the two communicating parties and are fixed. The protocol type is determined by the application and is also fixed. The destination port is determined by the listening port of the server application (such as Web-80 / 443, SSH-22 port) and is fixed. Therefore, the only remaining variable is the source port.

[0023] The traffic scheduling method provided in this application is applied at the data sending end. Its specific implementation carrier can be a network card driver running in the operating system kernel, or a smart network card or data processing unit, thereby realizing hardware offloading with zero host resource occupation.

[0024] By shifting the execution of traffic scheduling methods from traditional network core devices (such as switches) to the data sender, this fundamental shift in perspective eliminates the reliance on complex modifications to network infrastructure. Through preprocessing data packets before they enter the network, a distributed and easily deployable optimization mechanism is achieved.

[0025] A data stream to be sent refers to a collection of data packets that have the same five-tuple (source IP, destination IP, protocol type, source port, destination port) and need to be sent over the network.

[0026] Candidate port values ​​are a set of unique source port values ​​generated by the system within the temporary port range of the operating system kernel. Each candidate port value, after being hashed by the network device, may potentially correspond to a candidate network path.

[0027] Specifically, it determines whether the data packet corresponding to the data stream to be sent includes a modifiable source port field, and whether the traffic size of the data stream to be sent exceeds a preset threshold. In response to the data packet corresponding to the data stream to be sent including a modifiable source port field and the traffic size of the data stream to be sent exceeding the preset threshold, multiple non-repeating port values ​​are generated within the temporary port range of the operating system kernel. The port values ​​are bound to key information of the data stream to be sent to generate multiple candidate port values ​​for identifying the data stream to be sent at the data sending end. The key information of the data stream to be sent includes the source network address, destination network address, source port, destination port, and transport layer protocol number.

[0028] By determining whether the data stream to be sent includes a modifiable source port field, whether the data stream conforms to the transport layer protocol (TCP or UDP), and estimating the total number of bytes in the data stream, the system compares this estimate with a preset threshold. When the total number of bytes exceeds the threshold, the data stream is considered to have a larger volume than the preset threshold, i.e., it is identified as an "elephant stream." Subsequent traffic scheduling methods are only executed when both conditions are met. For regular data streams that do not meet the conditions (such as non-TCP / UDP ICMP messages or short streams), the system will maintain its original source port and send it according to default rules, thus ensuring the system's efficiency and economy.

[0029] The system generates multiple unique candidate source port values ​​within the range of temporary ports preset by the operating system kernel. To ensure these candidate port values ​​are correctly applied to the target data stream and tracked, the system binds each candidate port value to a five-tuple of key information in the data stream to be sent. This five-tuple includes the source IP address, destination IP address, source port, destination port, and transport layer protocol. This binding operation generates a set of multiple candidate port values ​​that uniquely identify the data stream to be sent at the data sender. Each candidate port value, after entering the network, is logically mapped to a potential candidate network path through hash calculation by the switch, laying a solid foundation for subsequent path probing and optimization. Thus, by abandoning the strategy of passively accepting default hash results and instead actively generating multiple candidate source port values, the system can proactively detect network hash results by probing the corresponding path status. This mechanism transforms blind sending into a proactive approach, avoiding significant performance losses caused by hash collisions with minimal probing overhead, demonstrating intelligent predictive capabilities.

[0030] In one implementation, multiple candidate source ports can also be generated by a candidate port generator. The candidate port generator can be a functional module located at the data sending end, responsible for executing the task of generating a set of candidate source ports. Its generation strategy aims to ensure the diversity and dispersion of port values, for example, by using random or pseudo-random algorithms to ensure that these ports can be mapped to as many different physical paths as possible after being hashed by network devices.

[0031] Step 102: Create multiple probe packets, where the source port field of each probe packet is set to a different candidate port value.

[0032] Step 103: Send probe packets to the data receiver and obtain the performance metrics of candidate network paths corresponding to multiple probe packets based on the response from the data receiver.

[0033] Probe packets are packets specifically designed for path performance probing. Their destination IP, destination port, protocol type, and other fields are consistent with the real data stream to be sent, but their source port field is set to a candidate port value to simulate the network path if the data stream used this port.

[0034] The source port field is a specific field in the protocol header of a network data packet (such as a TCP or UDP packet) used to identify the port of the sending application. This application uses this field to actively modify a control variable that controls the selection of different network paths for data flow.

[0035] The data receiver is the peer host that communicates with the data sender. It is responsible for receiving probe data packets and returning responses, thus forming a complete end-to-end communication to facilitate the measurement of path performance.

[0036] The core characteristic of probe packets lies in their simulated 5-tuple: except for the source port field, all other fields (including destination IP address, destination port, and protocol type) are completely identical to the real data stream to be scheduled, ensuring the consistency between the probe path and the real data transmission path. Each candidate port value is sequentially filled into the source port field of the probe packet. Thus, each probe packet carrying a different candidate source port becomes a probe targeting a specific potential path. This is the key physical operation for realizing the port-to-path mapping relationship.

[0037] The probe packets created above are sent sequentially or in parallel to the data receiver. When the data receiver receives these probe packets, it will return a response packet according to standard network protocols (such as TCP's ACK mechanism or UDP's custom response). At the data sender, the system uses a high-precision mechanism to capture the latency of this interaction.

[0038] In embodiments executed by a smart network interface card or data processing unit, this mechanism is a hardware timestamp that provides nanosecond-level accuracy, thereby measuring round-trip time (RTT) with extreme precision.

[0039] In a software implementation driven by the operating system, calculations are performed using feedback from the kernel protocol stack and system timestamps. By recording the time from sending each probe packet to receiving a response, the system calculates one or more performance metrics for each candidate source port value (i.e., each candidate network path), the most crucial of which is RTT (Round-Trip Time). Furthermore, the system can also calculate RTT jitter through multiple probes to assess path stability.

[0040] Step 104: Evaluate multiple candidate port values ​​based on performance metrics, and determine the target source port value from the multiple candidate port values ​​based on the evaluation results.

[0041] Specifically, based on the performance metrics of candidate network paths, a state vector representing each candidate network path is constructed; the state vector is used as the input to the reinforcement learning model, and the target source port value is output through the reinforcement learning model.

[0042] The state vector of each candidate network path is represented as follows:

[0043] ;

[0044] Among them, S t (i) Represents the state vector of network path i at time t, RTT t (i) Jitter represents the smoothed round-trip time of network path i. t (i) LossRate represents the latency jitter of network path i. t (i) Q represents the packet loss rate of network path i. t (i) B represents the normalized queue depth corresponding to network path i. t (i) T represents the estimated available bandwidth of the normalized network path i. t This represents normalized time.

[0045] In one implementation, the target source port value is determined as the output of the reinforcement learning model based on the reward function of the model. The reward function is expressed as:

[0046] ;

[0047] Where, r t RTT represents the reward value obtained by the data sender after selecting candidate port value A at time t. t (i) The smoothed round-trip time of network path i is represented by β, which is the latency sensitivity parameter, and LossRate is the loss rate. t (i) Let λ represent the packet loss rate of network path i, λl represent the reliability sensitivity parameter, and B represent the packet loss rate of network path i. t (i) B represents the estimated available bandwidth of the normalized network path i. max λ represents the physical bandwidth limit of the network link, λt represents the throughput sensitivity index, and λs represents the handover penalty coefficient, used to balance the cost of path handover. t A represents the candidate port value selected at time t. t-1 This represents the candidate port value selected at time t-1, 1 {At≠At-1} As an indicator function, when A t With A t-1If the values ​​are different, its value is 1; otherwise, it is 0.

[0048] The evaluation mechanism in this application is driven by a reinforcement learning model. Its core lies in a multi-objective reward function, which aims to select a path that simultaneously achieves low latency, high reliability, and high throughput while maintaining a certain level of stability to avoid the overhead of frequent switching. The reward function merges these competing objectives into a single scalar reward value. Among these, the latency utility term... Ensure severe penalties for high-latency paths; reliability items This means that any packet loss will significantly reduce the reward; throughput item Encourage the use of high-bandwidth paths, while imposing stability penalties. A fixed penalty is then applied to the path selection behavior that differs from the previous one to prevent jitter.

[0049] The process of determining the target source port value as the output of the reinforcement learning model based on the reward function of the reinforcement learning model includes: obtaining a Gaussian distribution corresponding to multiple candidate port values; the reward value of the candidate port value follows the Gaussian distribution corresponding to the candidate port value; in response to the need to determine the target source port value, sampling is performed from the Gaussian distribution corresponding to each candidate port value to obtain the sampled value corresponding to each candidate port value; and the candidate port value with the largest sampled value is taken as the target source port value.

[0050] The Gaussian distribution is represented as:

[0051] ;

[0052] Among them, Q t (A) represents the maximum sampled value, N(μ) A (t), σ A 2 (t) represents a Gaussian distribution, μ A (t) represents the average reward at time t, σ A 2 (t) represents the reward variance at time t.

[0053] In specific decision-making, the system employs the Thompson Sampling algorithm, maintaining a Gaussian distribution of rewards. The reward mean represents an estimate of the port's quality based on historical experience, and the reward variance represents the uncertainty of this estimate. When a decision is needed, the system samples a temporary reward value (Q) from the Gaussian distribution corresponding to each candidate port value. t (A) Subsequently, the system selects the candidate port with the largest sampled value as the target source port value for this operation. This post-sampling selection mechanism cleverly balances the selection of the candidate port value with the best current estimate and the selection of candidate port values ​​with high uncertainty, enabling the system to adaptively discover potential better paths.

[0054] This application further includes setting the candidate port value with the largest sample value as the target source port value, and then including obtaining the reward value corresponding to the target source port value after sending the data packet corresponding to the data stream to be sent based on the target source port value; and updating the Gaussian distribution parameter corresponding to the target source port value based on the reward value corresponding to the target source port value.

[0055] Specifically, the Gaussian distribution parameters corresponding to the target source port value are updated based on the Bayesian function, which is expressed as:

[0056] ;

[0057] ;

[0058] ;

[0059] Where, n A (t+1) represents the total number of times the candidate port value was selected at time t+1, where n is the number of times. A (t) represents the total number of times a candidate port value is selected at time t, μ A (t) represents the mean reward at time t, μ A (t+1) represents the average reward at time t+1, σ A 2 (t+1) represents the reward variance at time t+1, σ0 2 Let σ represent the initial prior variance of the Gaussian distribution. r 2 This represents the noise variance of the observation reward, used to measure the unreliability of a single reward observation.

[0060] After sending a real data stream using the target source port, the performance of that path is continuously measured, and the reward function is used again to calculate the actual reward value corresponding to the target source port value for this decision. Subsequently, the system initiates a Bayesian update process, using the actual reward value to update the Gaussian distribution parameters corresponding to the target source port value. As shown in the Bayesian update formula, the reward mean shifts towards the actual reward value of the newly observed target source port value, with the step size decreasing as the number of selections increases; simultaneously, the reward variance decreases, indicating an increase in the confidence level of the estimated reward value for that port. Through this continuous "decision-observation-update" cycle, the reinforcement learning model can track changes in the network state in real time, making its decision-making agent increasingly accurate, ultimately achieving a dynamic optimal balance of the global load.

[0061] Step 105: Modify the source port field of the data packet corresponding to the data stream to be sent to the target source port value, and send the modified data packet corresponding to the data stream to be sent to the data receiving end.

[0062] The target source port value refers to the candidate source port value that is considered optimal in the current network state and is finally determined through the aforementioned intelligent evaluation and decision-making process (such as the output of the reinforcement learning model). It is the final output of the entire optimization process.

[0063] Source port field modification refers to the technical operation at the data sending end, which replaces the source port field value in the protocol header (such as TCP header or UDP header) of the original data packet to be sent with the target source port value. This operation is completed before the data packet enters the network.

[0064] The modified data stream to be sent refers to the original data stream in which the source port field of all data packets has been uniformly modified to the target source port value. This data stream will be continuously sent via the selected optimal path.

[0065] The system intercepts the raw data stream that would otherwise be sent directly and modifies the source port field of all its packets, changing it from the original port used by the application to the optimized target source port value. These packets with the unified source port are then delivered to the network interface and sent to the data receiver. Because network switching devices use the source port as a key input to their hash algorithm when performing multi-path routing, this modified data stream will be stably mapped to the previously detected, optimal-performing physical path, ensuring the entire data stream enjoys a low-latency, high-throughput transmission experience.

[0066] The source port is a perfectly legal and predictably mutable field in the TCP / UDP protocol. Modifying it will not violate any network protocol specifications, nor will it be considered abnormal and discarded by network devices. This application creatively selects the source port, a standard protocol field, as the core for controlling the data flow path, achieving complete transparency to upper-layer applications. Specifically, from traffic identification and intelligent detection to the final port modification, everything is completed in the operating system kernel or smart network interface card. For the application that generates this data flow, it continues to conduct normal network communication on its own socket, completely unaware of the complex path optimization process behind it. No code modification is required for any existing application, greatly reducing the technical threshold and cost of deployment.

[0067] In one embodiment, for scenarios in large-scale data centers or cloud computing environments where massive data senders need to collaborate on traffic optimization, this application sets up a distributed federated learning mechanism, enabling multiple data senders to share learning experiences and achieve swarm intelligence.

[0068] In this application, multiple data senders are configured, and the traffic scheduling method further includes: establishing communication between multiple data senders; selecting any one of the multiple data senders as a collaborative node; the collaborative node obtains the reinforcement learning model parameters of all data senders, performs a weighted average of the reinforcement learning model parameters of all data senders to obtain collaborative model parameters, and distributes the collaborative model parameters to multiple data senders; for any data sender, its corresponding reinforcement learning model is updated based on the collaborative model parameters.

[0069] It should be explained that a collaborative node is a logical central node that is dynamically elected or designated among multiple data senders. Its function is to collect model parameters from each end, execute the federated averaging algorithm, and distribute the processed collaborative model parameters. It is also a data sender itself.

[0070] The reinforcement model parameters here refer to the internal state data of the local reinforcement learning agent (such as the Thompson Sampling model) at the data sending end. Specifically, this may include the mean reward, variance of the reward, and the number of times each candidate port value is selected.

[0071] The collaborative model parameters include the mean reward, the variance of the reward, and the number of times the data was selected. The collaborative model parameters are obtained by weighted averaging of the reinforcement learning model parameters of all data senders. For any data sender, updating its corresponding reinforcement learning model based on the collaborative model parameters includes: calculating the mean reward, the variance of the reward, and the number of times the data was selected for each candidate port value based on the weighted average calculation formula of the reinforcement learning model parameters; and updating the corresponding reinforcement learning model of each data sender based on the mean reward, the variance of the reward, and the number of times the data was selected.

[0072] The weighted average calculation formula for the parameters of the reinforcement learning model is expressed as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] Where K represents the number of data senders participating in the collaboration, k represents the reinforcement learning model parameters of the k-th data sender, and μ A ' represents the mean reward corresponding to candidate port value A, (σ A ²)' represents the reward variance corresponding to candidate port value A, σ0² represents the initial prior variance, and σ r ² represents the noise variance of the observed reward, N A n represents the total number of times candidate port value A was selected. AThis indicates the reset selection count generated based on the total number of selections.

[0077] This application establishes communication links between multiple data senders, forming a collaborative group. Subsequently, a data sender is elected from within the group as a temporary collaborative node using a lightweight distributed negotiation algorithm (such as the bully algorithm) or designated by a central controller.

[0078] When the predetermined collaboration period arrives or specific conditions are triggered, the collaborating node sends a request to all other data senders within the group to obtain their locally maintained reinforcement learning model parameters. Upon receiving all parameters, the collaborating node does not collect any raw traffic data; instead, it performs a weighted average of these model parameters using a federated averaging algorithm. This weighting ensures that more experienced senders (with more attempts) contribute more to the global model. After calculating the collaborative model parameters, the collaborating node distributes them to all participating data senders. Each data sender, upon receiving the collaborative model parameters, does not completely discard its local model but uses them to update or initialize its local reinforcement learning model. For example, the parameters of the local model can be replaced with the collaborative model parameters, or the two can be merged.

[0079] Thus, by sharing reinforcement learning parameters among multiple data senders, newly joined senders can quickly acquire historical experience, avoiding the overhead of exploring from scratch and accelerating the convergence speed of the entire system. Simultaneously, it enables all senders to collaboratively discover and avoid global network congestion points, achieving a leap from local optima to global optima, thereby realizing true dynamic load balancing at the system level. Furthermore, since only model parameters are exchanged, rather than raw data, this mechanism also fully protects the local data privacy of each data sender.

[0080] In one specific implementation, to adapt to scenarios requiring rapid deployment and being cost-sensitive, such as general-purpose servers and virtualized environments, the traffic scheduling method of this application is implemented through a high-level network interface card (NIC) driver in the server operating system kernel. Its core operation flow is as follows:

[0081] The driver first determines whether the data packet to be sent is a TCP or UDP protocol based on the protocol stack information, and further estimates its traffic volume. When a large data volume is identified, an optimization process is triggered (i.e., the process of generating multiple candidate port values ​​to identify the data stream to be sent at the data sender, and subsequent processes). Then, the driver generates a diverse set of candidate source ports in kernel mode and schedules the sending of a series of probe packets. These probe packets differ only in their source ports; the rest of the content is consistent with the actual data. Based on the kernel protocol stack's ACK confirmation mechanism and system timestamps, the driver accurately calculates the round-trip time (RTT) and its jitter for each candidate port's corresponding path. Finally, the driver selects the candidate port with the lowest and most stable average RTT as the final source port, modifies the source port field of the actual data stream accordingly, and then sends it.

[0082] This embodiment is purely software-based, requiring no dedicated hardware support. It can be deployed simply by upgrading the driver, resulting in extremely low deployment costs and broad compatibility. Furthermore, the entire optimization process is completely transparent to upper-layer applications and is fully compatible with existing network devices.

[0083] In another specific implementation, to meet the needs of scenarios such as high-performance computing and financial transactions that are extremely sensitive to latency and host resource consumption, the traffic scheduling method of this application is offloaded by the smart network interface card (NIC) or data processing unit (DPU). Its core operation flow is as follows:

[0084] The host kernel completely offloads the identified high-volume data transmission task and its context information to the smart network interface card (NIC). The programmable engine within the NIC then automatically generates a set of candidate source ports and sends probe packets using hardware timestamps, independently measuring the RTT of each path with nanosecond-level precision. The entire probing and decision-making process is completed on the NIC itself. Ultimately, the NIC hardware directly modifies the source port of the actual data stream to the optimal selection, updates the checksum, and sends the packet, all without any intervention from the host CPU.

[0085] This hardware offloading implementation achieves exceptional performance: First, it achieves zero host CPU usage, completely isolating computational overhead and ensuring the performance of host services. Second, hardware timestamps and line-speed processing deliver ultra-low and stable processing latency, meeting the performance requirements of even the most demanding applications. Furthermore, thanks to the programmability of the smart NIC, the destination port can be further modified or encapsulation headers can be added to change the protocol type, enabling more flexible traffic manipulation capabilities.

[0086] In another specific implementation, targeting environments with extreme network performance requirements such as large cloud data centers and ultra-large-scale clusters, this application achieves global optimization through collaboration between smart NICs supporting in-band network telemetry (INT) and network switches. The core operation process is as follows: The probe packet generated by the sending smart NIC embeds INT metadata instructions. When this probe packet traverses the network, each INT-enabled switch along the path automatically writes its local queue depth, timestamp, congestion flag, ingress / egress port, and other detailed information into the packet. The receiving end encapsulates the INT metadata containing complete path information in a response packet and returns it. After parsing the response, the sending NIC can not only obtain the end-to-end latency but also clearly understand the real-time health status of each hop along the entire path, thus making more refined and forward-looking decisions than those based solely on RTT, such as proactively avoiding a specific switch with an accumulated internal queue.

[0087] This INT telemetry implementation upgrades decision-making from end-to-end latency to path-wide health. This enables the system to perform predictive optimization, proactively avoiding congestion before it actually impacts end-to-end performance, thereby achieving true global load optimization in complex data center network environments and significantly improving the operational efficiency of large-scale data-intensive applications.

[0088] The beneficial effects of this application are mainly reflected in the following aspects:

[0089] 1. Intelligent Triggering and Resource Optimization: By setting an intelligent triggering mechanism that optimizes only elephant streams of transport layer protocols, the benefits of optimization far outweigh the costs, thereby significantly improving system efficiency, optimizing the utilization of computing and network resources, and fundamentally reducing the overall cost of ownership.

[0090] 2. Active Probe and Dynamic Optimization: By actively sending probe packets to measure path performance, the system can perceive changes in network status in real time and accurately predict the forwarding quality corresponding to different source ports. Subsequently, based on the probe results, it dynamically optimizes and selects the current optimal source port, thereby effectively avoiding the collision problem inherent in traditional hash mechanisms, significantly improving network bandwidth utilization, and significantly reducing transmission latency and packet loss rate.

[0091] 3. Batch Application and Seamless Compatibility: After determining the optimal source port, it is applied in batches to all packets in the entire data stream, ensuring the stability and consistency of the stream transmission. Crucially, the entire solution is implemented by modifying only the standard protocol field of the source port, ensuring seamless compatibility with existing network devices, protocols, and upper-layer applications. This achieves zero-modification deployment, significantly reducing deployment costs and implementation risks.

[0092] 4. Global Optimization and Service Assurance: The above key points work together to enable this solution to intelligently and adaptively adjust traffic distribution, achieving global load balancing. This not only reduces the pressure on core network equipment but also effectively prevents high-volume traffic from interfering with latency-sensitive critical services through intelligent scheduling, thus ensuring the overall service quality.

[0093] In one feasible implementation, to enable traffic scheduling to intelligently adapt to the performance requirements of different applications, a service type identifier for acquiring the data stream to be sent can be set. Based on the service type identifier, the strategy parameters used in the evaluation process are dynamically adjusted. The service types include at least latency-sensitive services and throughput-sensitive services. Specifically, the dynamically adjusted strategy parameters include: when the service type is latency-sensitive, increasing the weights of the latency sensitivity parameter β and the reliability sensitivity parameter λl in the reward function; when the service type is throughput-sensitive, increasing the weight of the throughput sensitivity index λt in the reward function.

[0094] Specifically, in reinforcement learning-based decision-making models, the system dynamically configures the sensitivity parameters of the reward function. For example, when a service type is identified as latency-sensitive (such as video conferencing or online games), the system increases the weights of the latency sensitivity parameter β and the reliability sensitivity parameter λl. This makes the reward function more sensitive to path latency increases and packet loss, driving the system to choose the path with the lowest latency and greatest stability at all costs, ensuring a good user experience for interactive services. When a service type is identified as throughput-sensitive (such as big data backup or scientific computing), the system increases the weight of the throughput sensitivity index λt. This guides the system to prioritize the path with the highest available bandwidth, maximizing data transmission efficiency and shortening job completion time. Thus, by customizing optimization objectives for different service types, resource misallocation is effectively avoided. For example, it prevents massive backup traffic from being scheduled to low-latency but limited-bandwidth paths, or real-time voice traffic from being scheduled to high-bandwidth but congested and jittery paths. This achieves optimal overall service performance at the system level and can adapt to complex and heterogeneous network environments, meeting the differentiated needs of different services, from real-time interaction to batch transmission.

[0095] like Figure 2 As shown, embodiments of this application also provide a traffic scheduling system deployed at a data sending end, specifically including a traffic monitor, a candidate port generator, a probe scheduler, a packet modifier, and a core decision maker.

[0096] The traffic monitor, as the system's trigger point, continuously monitors the data stream to be sent at the data sending end, and triggers subsequent optimization processes when it identifies a data stream that conforms to the transport layer protocol and whose traffic size exceeds a preset threshold.

[0097] The candidate port generator, in response to a traffic monitor's trigger, generates multiple candidate source port values ​​within the operating system kernel's temporary port range to identify the data stream.

[0098] The probe scheduler, in response to the generation of a candidate port set, is responsible for creating probe packets and setting their source port field to different candidate port values. It then schedules the transmission of these probe packets and synchronously notifies the core decision-maker to initiate performance monitoring.

[0099] The core decision-maker, acting as the intelligent hub of the system, responds to notifications from the probe scheduler by receiving and processing probe responses from the data receiver. Based on these responses, it calculates the performance metrics of each candidate path and executes intelligent evaluation algorithms (such as reinforcement learning models) to determine the optimal target source port value.

[0100] The message modifier, in response to the target source port value received from the core decision maker, performs two key operations: first, during the probing phase, it is responsible for sending the probe packets generated by the probe scheduler to the network; second, after the decision is made, it is responsible for modifying the source port field of all data packets in the original data stream to be sent to the target source port value and then sending them.

[0101] The traffic scheduling system deployed at the data sending end provided in this application constructs an efficient and automated closed-loop control process, seamlessly connecting intelligent triggering, active detection, dynamic decision-making and lossless execution, and ultimately realizing intelligent and adaptive scheduling of network traffic at the data sending end.

[0102] Embodiments of this application also provide an electronic device, such as... Figure 3 As shown, it includes a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the above-described traffic scheduling method embodiments.

[0103] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described traffic scheduling method embodiments at runtime.

[0104] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), mobile network device, magnetic disk, or optical disk.

[0105] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] The above provides a detailed description of a traffic scheduling method provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A traffic scheduling method, characterized in that, Applied to the data sending end, the traffic scheduling method includes: In response to the data packet corresponding to the data stream to be sent including a modifiable source port field and the traffic size of the data stream to be sent being greater than a preset threshold, multiple non-repeating port values ​​are generated within the temporary port range of the operating system kernel. The port value is bound to the key information of the data stream to be sent to generate multiple candidate port values ​​for identifying the data stream to be sent at the data sending end. The key information of the data stream to be sent includes the source network address, destination network address, source port, destination port and transport layer protocol number. Each candidate port value corresponds to a candidate network path. Create multiple probe packets, where the source port field of each probe packet is set to a different candidate port value; The probe data packets are sent to the data receiving end, and the performance indicators of the candidate network paths corresponding to the multiple probe data packets are obtained based on the response of the data receiving end. The performance metrics are used to evaluate multiple candidate port values, and the target source port value is determined from the multiple candidate port values ​​based on the evaluation results. Modify the source port field of the data packet corresponding to the data stream to be sent to the target source port value, and send the modified data packet corresponding to the data stream to be sent to the data receiving end; The operation of modifying the source port field of the data packet corresponding to the data stream to be sent to the target source port value is executed in the operating system kernel or smart network card of the data sending end, and does not require code modification of the application that generates the data stream to be sent.

2. The traffic scheduling method according to claim 1, characterized in that, The response to the data packet corresponding to the data stream to be sent includes a modifiable source port field, and the traffic size of the data stream to be sent is greater than a preset threshold. Before generating multiple non-repeating port values ​​within the temporary port range of the operating system kernel, the process includes: Determine whether the data packet corresponding to the data stream to be sent includes a modifiable source port field, and determine whether the traffic size of the data stream to be sent exceeds a preset threshold.

3. The traffic scheduling method according to claim 1, characterized in that, The process of evaluating multiple candidate port values ​​based on the performance metrics and determining the target source port value from the multiple candidate port values ​​based on the evaluation results includes: Based on the performance metrics of candidate network paths, a state vector representing each candidate network path is constructed. The state vector is used as the input to the reinforcement learning model, and the target source port value is output through the reinforcement learning model. The state vector is represented as: ； Among them, S t (i) Represents the state vector of network path i at time t, RTT t (i) Jitter represents the smoothed round-trip time of network path i. t (i) LossRate represents the latency jitter of network path i. t (i) Q represents the packet loss rate of network path i. t (i) B represents the normalized queue depth corresponding to network path i. t (i) T represents the estimated available bandwidth of the normalized network path i. t This represents normalized time.

4. The traffic scheduling method according to claim 3, characterized in that, The output of the target source port value through the reinforcement learning model includes: The target source port value determined by the reward function of the reinforcement learning model is used as the output of the reinforcement learning model; The reward function is expressed as follows: ； Where, r t RTT represents the reward value obtained by the data sender after selecting candidate port value A at time t. t (i) The smoothed round-trip time of network path i is represented by β, which represents the latency sensitivity parameter, and LossRate is the loss rate. t (i) Let λ represent the packet loss rate of network path i, λl represent the reliability sensitivity parameter, and B represent the packet loss rate of network path i. t (i) B represents the estimated available bandwidth of the normalized network path i. max λ represents the physical bandwidth limit of the network link, λt represents the throughput sensitivity index, and λs represents the handover penalty coefficient, used to balance the cost of path handover. t A represents the candidate port value selected at time t. t-1 This represents the candidate port value selected at time t-1, 1 {At≠At-1} As an indicator function, when A t With A t-1 If the values ​​are different, its value is 1; otherwise, it is 0.

5. The traffic scheduling method according to claim 4, characterized in that, The target source port value determined by the reward function based on the reinforcement learning model, as the output of the reinforcement learning model, includes: Obtain a Gaussian distribution corresponding to multiple candidate port values, wherein the reward value of each candidate port value follows a Gaussian distribution corresponding to the candidate port value. In response to the need to determine the target source port value, samples are taken from the Gaussian distribution corresponding to each candidate port value to obtain the sampled value corresponding to each candidate port value; The candidate port value with the largest sampled value is used as the target source port value; The Gaussian distribution is represented as: ； Among them, Q t (A) represents the maximum sampled value, N(μ) A (t), σ A 2 (t) represents a Gaussian distribution, μ A (t) represents the average reward at time t, σ A 2 (t) represents the reward variance at time t.

6. The traffic scheduling method according to claim 5, characterized in that, The step of selecting the candidate port value with the largest sampled value as the target source port value includes: After sending the data packet corresponding to the data stream to be sent based on the target source port value, obtain the reward value corresponding to the target source port value; Based on the reward value corresponding to the target source port value, update the Gaussian distribution parameters corresponding to the target source port value.

7. The traffic scheduling method according to claim 6, characterized in that, The Gaussian distribution parameters corresponding to the updated target source port value include: Update the Gaussian distribution parameters corresponding to the target source port value based on the Bayesian function; The Bayesian function is expressed as: ； ； ； Where, n A (t+1) represents the total number of times the candidate port value was selected at time t+1, where n is the number of times. A (t) represents the total number of times a candidate port value is selected at time t, μ A (t) represents the mean reward at time t, μ A (t+1) represents the average reward at time t+1, σ A 2 (t+1) represents the reward variance at time t+1, σ0 2 Let σ represent the initial prior variance of the Gaussian distribution. r 2 This represents the noise variance of the observation reward, used to measure the unreliability of a single reward observation.

8. The traffic scheduling method according to claim 1, characterized in that, The data sending ends are multiple, and the traffic scheduling method further includes: Establish communication between multiple data senders; Select any one of the multiple data senders as the cooperating node; The collaborative node obtains the reinforcement learning model parameters of all data senders, performs a weighted average of the reinforcement learning model parameters of all data senders to obtain the collaborative model parameters, and distributes the collaborative model parameters to multiple data senders. For any data sender, update its corresponding reinforcement learning model based on the collaborative model parameters.

9. The traffic scheduling method according to claim 8, characterized in that, The collaborative model parameters include the mean reward, the variance of the reward, and the number of times the model is selected. The collaborative model parameters are obtained by weighted averaging of the reinforcement learning model parameters of all data senders. For any data sender, updating its corresponding reinforcement learning model based on the collaborative model parameters includes: The weighted average calculation formula based on the parameters of the reinforcement learning model is used to calculate the mean reward, variance of reward, and number of times each candidate port value is selected. Each data sender updates its corresponding reinforcement learning model based on the mean reward, the variance of the reward, and the number of times it is selected; The weighted average calculation formula for the parameters of the reinforcement learning model is expressed as follows: ； ； ； Where K represents the number of data senders participating in the collaboration, k represents the reinforcement learning model parameters of the k-th data sender, and μ A ' represents the mean reward corresponding to candidate port value A, (σ A ²)' represents the reward variance corresponding to candidate port value A, σ0² represents the initial prior variance, and σ r ² represents the noise variance of the observed reward, N A n represents the total number of times candidate port value A was selected. A This indicates the reset selection count generated based on the total number of selections.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the traffic scheduling method as described in any one of claims 1 to 9 when executing the computer program.

Images