Data processing methods, apparatuses, electronic devices, storage media, and computer program products

Abstract

This application provides a data processing method, apparatus, electronic device, storage medium, and computer program product, relating to the technical field of transmission and bearer technology. The method includes: receiving a data transmission request; wherein the data transmission request includes: data to be processed; dividing the data to be processed into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of each data block is inversely proportional to the congestion level of the corresponding transmission link; and transmitting the corresponding data block to a destination via each transmission link. The technical solutions in this application can improve the transmission efficiency of the data to be processed.

Description

Technical Field

[0001] This application relates to the field of transmission and transmission technology, and in particular to a data processing method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] In related technologies, the rapid development of the digital industry has led to an explosive increase in data volume. Uneven distribution of computing resources has resulted in increasing demands for data transmission between production and consumption, such as East-West data processing, nationwide integrated computing networks, data express delivery, and distributed intelligent computing wide-area interconnection. Currently, for long-distance transmission of large files, the standard Remote Direct Memory Access (RDMA) is commonly used. Because the source and destination port numbers of the same data stream connection are the same, a connection can only select one physical network path for transmission. This introduces two problems:

[0003] 1. Reliability issues. Single physical network paths are prone to link failures, and the reliability issues are even more severe in diverse and heterogeneous wide area networks.

[0004] 2. Bandwidth utilization issues. Wide area networks (WANs) have multiple equivalent paths, and load balancing is generally achieved using hash algorithms. However, a single flow can only be hashed to one physical path, failing to fully utilize the available link bandwidth. If multi-stream transmission is used, hash polarization can easily occur due to the limitations of hash algorithms, resulting in uneven distribution of traffic across multiple links. Congestion on some links leads to low data transmission efficiency. Summary of the Invention

[0005] To address the problem of low data transmission efficiency caused by partial link congestion in wide area networks, embodiments of this application provide a data processing method, apparatus, electronic device, storage medium, and computer program product that can improve the transmission efficiency of data streams.

[0006] The technical solution of this application is implemented as follows:

[0007] This application provides a data processing method, including:

[0008] Receive a data transmission request; wherein the data transmission request includes: data to be processed;

[0009] Based on the congestion level of each transmission link, the data to be processed is divided into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link.

[0010] The corresponding data block is sent to the destination through each of the transmission links.

[0011] In the above scheme, dividing the data to be processed into data blocks corresponding to each transmission link based on the congestion level of each transmission link includes:

[0012] The size of the data block corresponding to each transmission link is determined based on the difference between the round-trip delay and the initial round-trip delay for each transmission link; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link; the round-trip delay is determined when each transmission link transmits data for the previous data.

[0013] Based on the size of each data block, the data to be processed is divided to determine the data block corresponding to each transmission link.

[0014] In the above scheme, before determining the size of the data block corresponding to each transmission link based on the difference between the determined round-trip time of each transmission link and the initial round-trip time, the method further includes:

[0015] The round-trip time and the initial round-trip time for each transmission link are obtained from the network control node, or the difference between the round-trip time and the initial round-trip time for each transmission link is obtained from the network control node.

[0016] In the above scheme, before determining the size of the data block corresponding to each transmission link based on the difference between the determined round-trip time of each transmission link and the initial round-trip time, the method further includes:

[0017] Lightly loaded round-trip delay detection is performed during the initialization of each transmission link to determine the initial round-trip delay corresponding to each transmission link.

[0018] In the above scheme, before determining the size of the data block corresponding to each transmission link based on the difference between the round-trip time of each transmission link and the initial round-trip time, the method further includes:

[0019] During the transmission of the previous data based on historical congestion levels on each of the transmission links, the round-trip time corresponding to each of the transmission links is determined.

[0020] In the above scheme, determining the round-trip time corresponding to each transmission link during the transmission of the previous data based on historical congestion levels includes:

[0021] Based on the historical congestion level of each transmission link, the previous data is divided into historical data blocks corresponding to each transmission link; wherein, the size of the historical data block is inversely proportional to the historical congestion level;

[0022] The historical data block is encapsulated in a transmission message and sent to the destination through each of the transmission links;

[0023] Receive the response message from the destination end to the sent message;

[0024] The round-trip delay is determined based on the transmission duration of the sent message and the returned message.

[0025] In the above scheme, encapsulating the historical data block in a transmission message and sending it to the destination through each transmission link includes:

[0026] Based on the maximum transmission unit of each transmission link, each historical data block is divided into multiple historical data sub-blocks;

[0027] Each historical data sub-block is encapsulated in a sub-transmission message according to the transmission order of the multiple historical data sub-blocks and sent to the destination; wherein, the sub-transmission message includes: the first sub-transmission timestamp of the sub-transmission message, the sub-reception duration for the previous sub-return message, the link identifier, the encoding, and the probe flag; the probe flag is used to indicate whether the destination needs to provide feedback on the sub-return message for the sub-transmission message.

[0028] In the above scheme, the encoding in each of the sub-transmission messages in the same transmission link is monotonically increasing, and the encoding in the sub-transmission messages of different transmission links is decoupled from each other.

[0029] In the above scheme, the return message received by the destination terminal in response to the sent message includes:

[0030] The system receives a first sub-transmission message from the destination end, and a first sub-return message is fed back. The first sub-return message includes: a second sub-transmission timestamp, a first sub-transmission duration, and the corresponding encoding in the first sub-transmission message. The first sub-transmission duration is determined based on the first sub-transmission timestamp and the first sub-receive timestamp of the corresponding first sub-transmission message. The first sub-transmission message is a sub-transmission message that the probe flag is used to characterize the need for the destination end to feed back the first sub-return message in response to the first sub-transmission message.

[0031] In the above scheme, determining the round-trip time based on the transmission duration of the sent message and the returned message includes:

[0032] The first sub-receive duration of each first sub-return message is determined based on the second sub-receive timestamp and the second sub-send timestamp of the first sub-return message, and the round-trip delay is determined based on each first sub-receive duration and the corresponding first sub-send duration.

[0033] The method in the above scheme further includes:

[0034] If at least one of the first sub-transmission duration and the first sub-reception duration is abnormal, the next historical data sub-block is encapsulated in a second sub-transmission message and sent to the destination. The second sub-transmission message includes: a third sub-transmission timestamp, the first sub-reception duration, the link identifier, the encoding, and the probe flag. The probe flag in the second sub-transmission message indicates that the destination needs to send a second sub-return message in response to the second sub-transmission message.

[0035] The destination end responds to the second sub-transmission message with the second sub-return message; wherein the second sub-return message includes: a fourth sub-transmission timestamp for the second sub-return message, a second sub-transmission duration, and the encoding; the second sub-transmission duration is determined based on the third sub-transmission timestamp of the sending end and the third sub-receive timestamp for the second sub-transmission message;

[0036] Based on the fourth sub-receive timestamp and the fourth sub-send timestamp of the received second sub-return message, the second sub-receive duration of the second sub-return message is determined, and the initial round-trip time is determined based on the second sub-receive duration and the second sub-send duration.

[0037] In the above scheme, sending the corresponding data block to the destination through each of the transmission links includes:

[0038] Based on the maximum transmission unit of each transmission link, each data block is divided into multiple data sub-blocks;

[0039] Each of the data sub-blocks is encapsulated in the current transmission message and sent to the destination.

[0040] This application also provides a data processing apparatus for use at a sending end, comprising:

[0041] A receiving unit is configured to receive a data transmission request; wherein the data transmission request includes: data to be processed;

[0042] A data partitioning unit is used to divide the data to be processed into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link.

[0043] A sending unit is used to send the corresponding data block to the destination through each of the transmission links.

[0044] This application also provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the steps in the above-described method.

[0045] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the above-described method.

[0046] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method.

[0047] In this embodiment, a data transmission request is received, comprising: data to be processed; the data to be processed is divided into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of each data block is inversely proportional to the congestion level of the corresponding transmission link; and the corresponding data block is transmitted to the destination through each transmission link. Since the load on each transmission link is inversely proportional to the congestion level of that link, the lower the congestion level, the higher the load. This fully utilizes the bandwidth of multiple equal-cost paths in the wide area network and the reliability provided by these paths, reducing the occurrence of transmission congestion on some transmission links and thus improving the transmission efficiency of the data to be processed. Attached Figure Description

[0048] Figure 1 Flowchart of the data processing method provided in the embodiments of this application Figure 1 ;

[0049] Figure 2 This application provides an illustration of the effects of the data processing method provided in the embodiments. Figure 1 ;

[0050] Figure 3 Flowchart of the data processing method provided in the embodiments of this application Figure 2 ;

[0051] Figure 4 This application provides an illustration of the effects of the data processing method provided in the embodiments. Figure 2 ;

[0052] Figure 5 Flowchart of the data processing method provided in the embodiments of this application Figure 3 ;

[0053] Figure 6 Flowchart of the data processing method provided in the embodiments of this application Figure 4 ;

[0054] Figure 7 Flowchart of the data processing method provided in the embodiments of this application Figure 5 ;

[0055] Figure 8 Flowchart of the data processing method provided in the embodiments of this application Figure 6 ;

[0056] Figure 9 Flowchart of the data processing method provided in the embodiments of this application Figure 7 ;

[0057] Figure 10 This application provides an illustration of the effects of the data processing method provided in the embodiments. Figure 3 ;

[0058] Figure 11 This application provides an illustration of the effects of the data processing method provided in the embodiments. Figure 4 ;

[0059] Figure 12 This application provides an illustration of the effects of the data processing method provided in the embodiments. Figure 5 ;

[0060] Figure 13 This application provides an illustration of the effects of the data processing method provided in the embodiments. Figure 6 ;

[0061] Figure 14 This application provides an illustration of the effects of the data processing method provided in the embodiments. Figure 7 ;

[0062] Figure 15 Flowchart of the data processing method provided in the embodiments of this application Figure 8 ;

[0063] Figure 16 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application;

[0064] Figure 17 This is a schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0067] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0069] In related technologies, the rapid development of the digital industry has led to an explosive growth in data volume, and the uneven distribution of computing resources has resulted in an increasing demand for data transmission between production and consumption. For example, services such as East-West data processing, nationwide integrated computing power networks, data express delivery, and distributed intelligent computing wide-area interconnection currently rely heavily on offline manual hard drive delivery solutions for long-distance transmission of large files. These solutions pose risks such as data leakage and loss, and cannot meet market demands in terms of data security and transmission efficiency.

[0070] Various cloud computing vendors have provided solutions for the efficient transmission of large-scale data over wide area networks (WANs) through global data center deployments and optimized transmission protocols. However, for data transmissions in the tens of terabytes range, online transmission solutions cannot meet timeliness requirements, necessitating offline transmission. Further research is needed on WAN transmission technologies for ultra-large-scale data, such as distributed intelligent computing and wide-area interconnection, which involve large spatiotemporal scales of interactive services.

[0071] RDMA utilizes zero-copy memory, kernel bypass, and Central Processing Unit (CPU) offloading technologies to offload the entire Transmission Control Protocol / Internet Protocol (TCP / IP) stack to the network interface card (NIC). This allows user-mode applications to directly read and write to remote host memory, avoiding data copying and context switching, thus achieving high throughput, low latency, and low CPU overhead. RDMA has been widely deployed in data centers to support various low-latency, high-throughput applications, such as key-value stores, distributed transactions, distributed memory, Remote Procedure Calls (RPC), storage systems, graph computing, and machine learning systems. RDMA's kernel bypass and DMA characteristics significantly reduce CPU overhead at both the sending and receiving ends. Leveraging RDMA's "one-sided" operation, massive data transfers can be achieved with near-invisible latency to the receiving CPU.

[0072] However, because standard RDMA uses the same source and destination port numbers for the same User Datagram Protocol (UDP) connection, a connection can only choose one physical network path for transmission. This introduces two problems:

[0073] 1. Reliability issues. Single physical network paths are prone to link failures (random packet loss, congestion packet loss, one-way link, line disconnection, etc. caused by physical failures or configuration errors). Wide area networks are diverse, heterogeneous, and complex, making reliability issues even more serious.

[0074] 2. Bandwidth utilization issues. Wide area networks (WANs) have multiple equivalent paths, and load balancing is generally achieved using hash algorithms. However, a single flow can only be hashed to one physical path, failing to fully utilize the available link bandwidth. If users adopt a multi-stream transmission approach, hash polarization can easily occur due to the limitations of hash algorithms, resulting in uneven distribution of traffic across multiple links.

[0075] To fully utilize the bandwidth and reliability of multiple equivalent paths in a wide area network (WAN) without increasing application complexity, a multipath transmission mechanism has been introduced. However, finding a balance between transmission efficiency, load balancing, and packet loss recovery costs is a problem that multipath RDMA needs to address.

[0076] This application provides a data processing method. Please refer to the embodiments provided. Figure 1 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 1 , will combine Figure 1The steps shown are explained below:

[0077] S101, Receive a data transmission request; wherein, the data transmission request includes: data to be processed.

[0078] In this embodiment of the application, the sending end receives a data transmission request sent by other devices or user terminals. The data transmission request includes data to be processed.

[0079] The sending end can be a network node in the RDMA transmission network that has data transmission and corresponding data processing functions. There are multiple transmission links between the sending end and the destination end.

[0080] S102. Based on the congestion level of each transmission link, the data to be processed is divided into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link.

[0081] In this embodiment, the sending end divides the data to be processed into data blocks corresponding to each transmission link according to the acquired or determined congestion level of each transmission link. The size of each data block is inversely proportional to the corresponding congestion level.

[0082] In this embodiment, the sending end can pre-send messages to the destination through multiple transmission links and then receive return messages from the destination. The congestion level of each transmission link is determined by the transmission time of the sent and returned messages. The congestion level is directly proportional to the transmission time.

[0083] In this embodiment, the sending end can also obtain congestion information for each transmission link from the network control node. The network control node can determine the congestion information in advance by detecting the round-trip transmission time between the sending end and the destination end.

[0084] S103. The corresponding data block is sent to the destination through each of the transmission links.

[0085] In this embodiment of the application, the sending end sends the corresponding data block to the destination end through each transmission link.

[0086] In this embodiment of the application, the sending end divides each data block into multiple data sub-blocks based on the maximum transmission unit of each transmission link; each data sub-block is encapsulated in the current transmission message and sent to the destination end in sequence.

[0087] The Maximum Transmission Unit (MTU) can be the maximum transmission load of a physical node on a transmission link.

[0088] In this embodiment of the invention, the sending end can further encapsulate each data block in a corresponding current transmission message according to the order of multiple data sub-blocks, and send them to the destination end in sequence. Each current transmission message includes: a transmission timestamp of the current transmission message, a reception duration for the previous current return message, a link identifier, encoding, and a probe flag. The probe flag indicates whether the destination end needs to provide a sub-return message in response to the sub-transmission message. The encoding in each current transmission message within the same transmission link monotonically increases, and the encoding in current transmission messages across different transmission links is decoupled from each other.

[0089] The sending end receives a current return message from the destination end in response to the predetermined current transmission message. The current return message includes: the transmission timestamp of the current return message, the transmission duration for the corresponding predetermined current transmission message, and its encoding. The transmission duration for the corresponding predetermined current transmission message is determined by the destination end based on the transmission timestamp of the corresponding predetermined current transmission message and the corresponding reception timestamp at the destination end. This predetermined current transmission message can be any message from multiple current transmission messages. The probe flag in the predetermined current transmission message indicates that the destination end needs to respond with a current return message in response to the predetermined current transmission message.

[0090] For example, in combination Figure 2 The transmission between the sender and the destination includes n sQP (Subqueue Pair) transmission links. After the sender receives the QP (Queue Pair) data stream (data to be processed), it can divide the QP according to the congestion level of the n transmission links, determining the data block corresponding to each transmission link. The sender can also divide the data block of each transmission link into three data sub-blocks based on the maximum transmission unit. The three data sub-blocks in the sQP1 transmission link are encoded as c1PSN1, c1PSN2, and c1PSN3, respectively. It can be seen that the encodings of the three data sub-blocks in the sQP1 transmission link are monotonically increasing. Furthermore, the encodings of the three data sub-blocks in each of the n sQP transmission links are all different, meaning that the encodings of the data sub-blocks in the N sQP transmission links are decoupled from each other.

[0091] In this embodiment, a data transmission request is received, comprising: data to be processed; the data to be processed is divided into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of each data block is inversely proportional to the congestion level of the corresponding transmission link; and the corresponding data block is transmitted to the destination through each transmission link. Since the load on each transmission link is inversely proportional to the congestion level of that link, the lower the congestion level, the higher the load. This fully utilizes the bandwidth of multiple equal-cost paths in the wide area network and the reliability provided by these paths, reducing the occurrence of transmission congestion on some transmission links and thus improving the transmission efficiency of the data to be processed.

[0092] Please see Figure 3 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 2 , Figure 1 The shown S102 can also be implemented through S201 to S202, which will be explained in conjunction with the steps:

[0093] S201. Based on the difference between the round-trip delay and the initial round-trip delay of each transmission link, determine the size of the data block corresponding to each transmission link; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link.

[0094] In this embodiment, the sending end can calculate the difference between the round-trip time (RTT) and the initial RTT using the known RTT and initial RTT of each transmission link. The difference between the RTT and the initial RTT is used to characterize the congestion level of the corresponding transmission link. The size of the data block corresponding to each transmission link is then determined using the difference between the RTT and the initial RTT; wherein the difference between the RTT and the initial RTT is inversely proportional to the size of the data block corresponding to the transmission link. The RTT is determined when transmitting data to the previous transmission link; the initial RTT is determined during the initialization of each transmission link.

[0095] For example, the sender can allocate the chunk size (data block) of the inflight message (data to be processed) inversely proportional to the round-trip time (RTT) of each transmission link. This achieves load balancing across multiple transmission links per chunk, enabling low-overhead transmission under light load and low-recovery-cost transmission under heavy load. The initial value of the chunk size cn is set to the bandwidth-delay product (BDP) of the corresponding sQP transmission link, and subsequently changed according to the RTT of each sQP transmission link. The relationship between the chunk size cn and RTTn in this scheme can be determined by formula (1):

[0096] cn = K / (RTT) n -RTT bn (1)

[0097] Where cn is the size of the data block, k is a conditional parameter, and RTT is... n For round-trip time, RTT bn This is the initial round-trip time delay.

[0098] S202. Based on the size of each data block, the data to be processed is divided to determine the data block corresponding to each transmission link.

[0099] In this embodiment of the application, after determining the size of each data block, the data to be processed can be divided according to the size of each data block to determine the data block corresponding to each transmission link.

[0100] For example, in combination Figure 4 The data to be processed (QP) can be divided into 9 data blocks (c1-c9) corresponding to sQP transmission links. Among them, the RTT corresponding to data block c1 is... d The difference between the round-trip time and the initial round-trip time is 2ms, while the RTT corresponding to data block c2 is... d The time to turn over (RTT) is 3ms, and as can be seen from the graph, the amount of data in data block c1 is significantly greater than that in data block c2, while the RTT corresponding to data block c1 is... d The RTT corresponding to data block c2 is less than d It can be seen that the data size of the data block and the corresponding RTT d The size is inversely proportional to the size.

[0101] In this embodiment, the size of the data block corresponding to each transmission link is determined based on the difference between the round-trip time (RTT) and the initial RTT of each transmission link; wherein, the difference between the RTT and the initial RTT is inversely proportional to the size of the data block corresponding to the transmission link. Based on the size of each data block, the data to be processed is divided to determine the data block corresponding to each transmission link. Since the load on each transmission link is determined inversely proportional to the difference between the RTT and the initial RTT of that transmission link, the difference can reflect the congestion level of the corresponding transmission link. The lower the congestion level, the higher the load on the transmission link. This can effectively and rationally utilize the transmission resources of each transmission link on the wide area network, reduce the occurrence of transmission link congestion, and thus improve the transmission efficiency of the data to be processed.

[0102] Please see Figure 5 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 3 , Figure 3 The step S201 shown may also include S301, which will be explained in conjunction with the steps:

[0103] S301. Obtain the round-trip time and the initial round-trip time corresponding to each of the transmission links from the network control node, or obtain the difference between the round-trip time and the initial round-trip time corresponding to each of the transmission links from the network control node.

[0104] In this embodiment, the network control node can pre-transmit probe messages through each transmission link and determine the round-trip time of the transmission link based on the transmission duration of the probe messages. It can also transmit probe messages through the transmission link during initialization and determine the initial round-trip time of the transmission link based on these probe messages. When the sending end needs to send data based on the congestion level of the transmission link, it can pre-obtain the round-trip time and the initial round-trip time corresponding to each transmission link from the network control node, or obtain the difference between the round-trip time and the initial round-trip time corresponding to each transmission link from the network control node.

[0105] The network control node may include a centralized network controller.

[0106] For example, the RTT metric can also be obtained through end-to-end network collaboration, by obtaining the static RTTbn and dynamic RTTn of a specific link from the centralized network controller, or by directly obtaining the difference RTTd that represents the degree of congestion.

[0107] In this embodiment, the sending end obtains the round-trip time and the initial round-trip time corresponding to the transmission link, or the difference between the two, from the network control node. This eliminates the need for the sending end to detect the round-trip time and initial round-trip time before sending data, reducing the workload of the sending end and improving its transmission efficiency with limited resources.

[0108] Please see Figure 6 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 4 , Figure 3 The step S201 shown may also include S302, which will be explained in conjunction with the steps:

[0109] S302. When each transmission link is initialized, a light-load round-trip delay detection is performed to determine the initial round-trip delay corresponding to each transmission link.

[0110] In this embodiment, the sending end performs a light-load round-trip delay detection on each transmission link during initialization (i.e., when the transmission link is first established) to determine the initial round-trip delay for each transmission link. Specifically, during the initialization of each transmission link, the sending end sends a message to the destination and then receives a return message from the destination. The initial round-trip delay is determined by the transmission duration of the sent and returned messages.

[0111] In this embodiment, the physical link of each sQP transmission link is relatively fixed (failure triggers switchover), but the physical link length of each sQP is not the same, and some even differ significantly. To determine the congestion level of each sQP transmission link, RTTbn detection is performed during initialization under light load or even no load conditions. Then, the congestion level is determined by the difference between RTTn and RTTbn. If a physical link of an sQP experiences a transmission failure and switchover to a new link, an RTTbn detection under light load or even no load conditions needs to be triggered, and then the congestion level is determined by the difference between RTTn and RTTbn.

[0112] In this embodiment, a light-load round-trip time (RTD) probe is performed during the initialization of each transmission link to determine the initial RTD for each link. Since the load on the transmission link is minimal during initialization, the RTD probe under optimal transmission conditions can accurately determine the initial RTD of that link, providing accurate parameters for subsequent congestion level determination and resulting in a more accurate assessment. Data blocks partitioned based on precise congestion levels are also better matched to the load of the corresponding transmission links, maximizing the utilization of each link's transmission resources and improving the transmission efficiency of the data to be processed.

[0113] Please see Figure 7 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 5 , Figure 3 The step S201 shown may also include S401, which will be explained in conjunction with the steps:

[0114] S401. During the transmission of the previous data based on the historical congestion level on each of the transmission links, the round-trip time corresponding to each of the transmission links is determined.

[0115] In this embodiment, the round-trip time when dividing the data to be processed into blocks is determined when transmitting the previous data. The sending end can transmit the previous data based on the historical congestion level of each transmission link, and determine the round-trip time by the duration of sending and returning messages during the transmission process.

[0116] In this embodiment, during the transmission of the previous data based on historical congestion levels on each transmission link, the round-trip time (RTD) for each transmission link is determined. This allows the congestion level of each transmission link to be calculated using a pre-determined RTD when processing data, and then the data to be processed is divided based on the congestion level. Thus, when transmitting each piece of data, it is unnecessary to first measure the RTD, saving time and improving data transmission efficiency.

[0117] Please see Figure 8 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 6 , Figure 7 The shown S401 can also be implemented by S501 to S504, which will be explained in conjunction with the steps:

[0118] S501. Based on the historical congestion level of each transmission link, the previous data is divided into historical data blocks corresponding to each transmission link; wherein the size of the historical data block is inversely proportional to the historical congestion level.

[0119] In this embodiment, the sending end divides the previous data into historical data blocks corresponding to each transmission link based on a pre-determined historical congestion level of each transmission link. The size of the historical data block is inversely proportional to the historical congestion level.

[0120] S502. The historical data block is encapsulated in a transmission message and sent to the destination through each of the transmission links.

[0121] In this embodiment of the application, the sending end can encapsulate historical data blocks in a transmission message and send them to the destination end through the transmission link corresponding to each historical data block.

[0122] The sent message may include the corresponding sending timestamp.

[0123] S503. Receive the response message from the destination end to the sent message.

[0124] In this embodiment, after receiving the transmitted message, the destination end determines the transmission duration of the transmitted message by using the timestamp of the received transmitted message and the timestamp of the transmitted message. The transmission duration and the timestamp of the returned message are encapsulated in a returned message and fed back to the sending end.

[0125] S504. Determine the round-trip delay based on the transmission duration of the sent message and the returned message.

[0126] In this embodiment, the sending end determines the timestamp of the received return message, and then determines the transmission duration of the return message by combining the timestamp of the received return message with the timestamp of the sent return message. The round-trip delay is determined by combining the transmission duration of the return message and the transmission duration of the sent message.

[0127] In this embodiment, based on the historical congestion level of each transmission link, the previous data is divided into historical data blocks corresponding to each transmission link; wherein the size of the historical data block is inversely proportional to the historical congestion level. The historical data blocks are encapsulated in a transmission message and sent to the destination through each transmission link. The destination responds to the transmission message and sends a return message. The round-trip time (RTD) is determined based on the transmission duration of the transmission and return messages. Similarly, when processing data, the congestion level of each transmission link can be calculated using a pre-determined RTD, and the data to be processed is then divided based on the congestion level. In this way, when transmitting each piece of data, it is not necessary to first measure the RTD, saving the RTD detection process, saving time, and thus improving data transmission efficiency.

[0128] Please see Figure 9 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 7 , Figure 8 The steps S502 to S504 shown can also be implemented by S601 to S604, which will be explained in conjunction with the steps:

[0129] S601. Based on the maximum transmission unit of each transmission link, divide each historical data block into multiple historical data sub-blocks.

[0130] In this embodiment of the application, the sending end divides the historical data block into multiple historical data sub-blocks of the size that the maximum transmission unit can transmit, according to the maximum transmission unit of each transmission link.

[0131] S602. Each historical data sub-block is encapsulated in a sub-transmission message according to the transmission order of the multiple historical data sub-blocks and sent to the destination; wherein, the sub-transmission message includes: the first sub-transmission timestamp of the sub-transmission message, the sub-reception duration for the previous sub-return message, the link identifier, the encoding, and the probe flag.

[0132] In this embodiment, the sending end encapsulates each historical data sub-block in a sub-transmission message according to the transmission order of multiple historical data blocks and sends it to the destination end. The sub-transmission message includes: a first sub-transmission timestamp, a sub-reception duration for the previous sub-return message, a link identifier, encoding, and a probe flag. The probe flag indicates whether the destination end needs to provide a sub-return message in response to the sub-transmission message. The encoding in each sub-transmission message within the same transmission link monotonically increases, and the encodings in sub-transmission messages from different transmission links are decoupled from each other.

[0133] In this embodiment, an independent encoding mechanism for chunk namespaces and message sequence numbers is proposed to decouple sequence numbers such as CPSN between SQP transmission links, ensuring that the CPSN in a single SQP transmission link is monotonically increasing. Previous data is split into n chunks (historical data blocks), and scheduling is performed based on the load of the n SQPs. Each SQP chunk, combined with the PMTU of the SQP link, is further divided into several packets (historical data sub-blocks). The CPSN of each packet is independently and monotonically encoded, ensuring that even in the event of packet loss and retransmission, the CPSN remains strictly monotonically increasing, guaranteeing that packet loss will not cause RTT measurement errors. Independent encoding of chunk namespaces and message sequence numbers, along with decoupling of sequence numbers such as cPSN between SQPs, allows for duplicates.

[0134] For example, in combination Figure 10 This is a schematic diagram of the header information for each sub-transmission message. Within the dashed box, T1 represents the first sub-transmission timestamp of the sub-transmission message, Dest sQP represents the link identifier, A represents the probe flag (which can be 0 or 1 in this embodiment), Tr represents the sub-reception duration of the previous sub-return message, and cPSN represents the encoding.

[0135] S603. Receive the first sub-transmission message from the destination end and the first sub-return message; wherein, the first sub-return message includes: the second sub-transmission timestamp of the first sub-return message, the first sub-transmission duration, and the corresponding encoding in the first sub-transmission message.

[0136] In this embodiment, after receiving each sub-transmission message, the destination end detects the probe flag of the sub-transmission message. If the probe flag of the first sub-transmission message indicates that the destination end needs to send back the sub-return message for the sub-transmission message (probe flag is 1), then the destination end responds to the first sub-transmission message by sending back the first sub-return message. The first sub-return message includes: a second sub-transmission timestamp, a first sub-transmission duration, and the corresponding encoding from the first sub-transmission message. The first sub-transmission duration is determined by the destination end based on the first sub-transmission timestamp and the first sub-receive timestamp of the corresponding first sub-transmission message.

[0137] The first sub-transmission message is a sub-transmission message that the probe flag is used to indicate that the destination needs to send back the first sub-return message in response to the first sub-transmission message.

[0138] For example, in combination Figure 11 This is a schematic diagram of the header information for each first sub-return message. Within the dashed box, T3 represents the second sub-sending timestamp of the first sub-return message, Tr represents the duration of the first sub-sending, and cPSN represents the encoding in the corresponding first sub-sending message.

[0139] In this embodiment of the application, the standard Base Transport Header (BTH) of RDMA over Converged Ethernet (RoCEv2) is combined with... Figure 10 ) and ACK Extended Transport Header (AETH header) (combined with Figure 11 It can be extended or updated to transmit sQP level interactive information and dynamically perform accurate link latency measurement with low overhead. After the aforementioned processing, the measured values ​​guide the user-submitted data to be chunked based on load.

[0140] S604. Determine the first sub-receive duration of each first sub-return message based on the second sub-receive timestamp and the second sub-send timestamp of the first sub-return message, and determine the round-trip delay based on each first sub-receive duration and the corresponding first sub-send duration.

[0141] In this embodiment, the first sub-receive duration of each first sub-return message is determined based on the second sub-receive timestamp and the second sub-send timestamp in each first return message. Then, a round-trip duration is determined based on each first sub-receive duration and its corresponding first sub-send duration. The round-trip delay is determined by averaging the round-trip durations.

[0142] The sending end can determine the round-trip delay based on the average of the various round-trip times, or based on the maximum or minimum of the various round-trip times.

[0143] For example, in combination Figure 12 The sending end determines the first sub-reception duration (T) of each first sub-return message based on the second sub-reception timestamp T4 and the second sub-transmission timestamp T3 in each first return message. r =T4–T3). The destination determines the duration of the first sub-transmission (T) based on the first sub-transmission timestamp T1 and the first sub-reception timestamp T2 of the corresponding first sub-transmission message. s =T2–T1). The sending end then bases the first sub-reception duration T on the data. r and the first sub-transmission duration T received s Determine a round-trip time RTT (RTT = T) r +T s ).

[0144] In this embodiment, the sending end determines the round-trip delay based on the first sub-send duration of the first sub-send message and the first sub-receive duration of the corresponding first sub-return message. Since the encoding in each sub-send message within the same transmission link is monotonically increasing, and the encoding in sub-send messages from different transmission links is decoupled, the corresponding encoding will not be retransmitted in the event of a retransmission of a sub-send message. This avoids errors in the return message caused by duplicate encoding in retransmitted sub-send messages, effectively preventing inaccurate round-trip delay measurements caused by packet loss and retransmission in classic TCP protocols.

[0145] Furthermore, since the destination only sends a first sub-return message to the first sub-transmitted message (which has a probe flag indicating that the destination needs to send a first sub-return message in response to the first sub-transmitted message), the resource overhead caused by sending a feedback message for each sub-transmitted message is avoided. This allows for accurate and low-overhead measurement of the latency changes of multiple equivalent transmission links, saving transmission resources.

[0146] Combination Figure 13 In this embodiment, the user terminal sends multiple data items to the sending terminal via a data transmission request. After each data item is sent, the sending terminal sends a request completion message back to the user terminal. During the sending process, the data can be divided into multiple data blocks, and each data block can be further divided into multiple data sub-blocks. Please refer to... Figure 14The sending end can divide a data block into two sub-blocks and encapsulate them in r15 and r16 messages, sending them sequentially to the destination. The destination responds with an r17 message to the r15 message and an r18 message to the r16 message.

[0147] Please see Figure 15 The following is a flowchart illustrating the data processing method provided in the embodiments of this application. Figure 8 The following steps will be explained:

[0148] S701. If at least one of the first sub-transmission duration and the first sub-reception duration is abnormal, the next historical data sub-block is encapsulated in the second sub-transmission message and sent to the destination. The second sub-transmission message includes: the third sub-transmission timestamp of the second sub-transmission message, the first sub-reception duration, the link identifier, the encoding, and the probe flag.

[0149] In this embodiment of the application, if the duration of the first sub-transmission is greater than the first threshold and / or the duration of the first sub-reception is less than the second threshold, then the next historical data sub-block is encapsulated in the second transmission message and sent to the destination.

[0150] The second sub-transmission message includes: a third sub-transmission timestamp, a first sub-reception duration, the link identifier, the encoding, and a probe flag. The probe flag in the second sub-transmission message indicates that the destination needs to send a second sub-return message in response to the second sub-transmission message. For example, the probe flag in the second sub-transmission message can be 1.

[0151] S702. Receive the second sub-transmission message from the destination end, and the second sub-return message fed back; wherein, the second sub-return message includes: a fourth sub-transmission timestamp, a second sub-transmission duration, and the encoding for the second sub-return message.

[0152] In this embodiment of the application, the sending end receives a second sub-transmission message from the destination end, and a second sub-return message is fed back. The second sub-return message includes: a fourth sub-transmission timestamp for the second sub-return message, a second sub-transmission duration, and the encoding; the second sub-transmission duration is determined based on the third sub-transmission timestamp of the sending end and the third sub-receive timestamp for the second sub-transmission message.

[0153] In this embodiment, the destination actively triggers a second sub-transmission message containing an sQP AETH header by judging changes in Ts (first sub-transmission duration) and / or Tr (first sub-reception duration) (Ts increases to exceed a first threshold, Tr is less than a second threshold). Simultaneously, the sender sets the probe flag in the sQP BTH header as needed based on changes in Ts or Tr and the sender's state (e.g., whether to perform basic RTTbn measurement), requesting acknowledgment from the destination. The Dest sQP and CPSN in the sQP BTH and AETH headers are generated and carried based on the aforementioned chunk namespace and message sequence number independent encoding mechanism.

[0154] S703. Based on the fourth sub-receive timestamp and the fourth sub-send timestamp of the second sub-return message, determine the second sub-receive duration of the second sub-return message, and determine the initial round-trip delay based on the second sub-receive duration and the second sub-send duration.

[0155] In this embodiment, the sending end determines the second sub-reception duration of the second sub-return message by subtracting the fourth sub-send timestamp from the fourth sub-reception timestamp of the received second sub-return message. The initial round-trip time is then determined by the sum of the second sub-reception duration and the received second sub-send duration.

[0156] In this embodiment, if at least one of the first sub-transmission duration and the first sub-reception duration is abnormal, the next historical data sub-block is encapsulated in a second sub-transmission message and sent to the destination. The receiving destination responds to the second sub-transmission message with a second sub-return message. Based on the fourth sub-reception timestamp and the fourth sub-transmission timestamp of the received second sub-return message, the second sub-reception duration of the second sub-return message is determined, and the initial round-trip time is determined based on the second sub-reception duration and the second sub-transmission duration. When at least one of the first sub-transmission duration and the first sub-reception duration is abnormal, it indicates that the physical node state of the corresponding transmission link has changed. In this embodiment, after a change in the transmission link, the corresponding initial round-trip time can be quickly determined by sending the next historical data sub-block, providing the sending end with timely and accurate congestion calculation parameters. Furthermore, the sending end can use timely and accurate congestion information to divide and transmit data, thereby achieving full utilization of the bandwidth of multiple equivalent transmission links in the wide area network and the reliability brought by multiple equivalent paths, avoiding congestion caused by excessive load on individual transmission links, and improving the transmission efficiency of subsequent data.

[0157] Please see Figure 16 This is a schematic diagram of the structure of the data processing device provided in the embodiments of this application.

[0158] This application also provides a data processing device 800, applied at a transmitting end, including: a receiving unit 801, a data partitioning unit 802, and a transmitting unit 803.

[0159] The receiving unit 801 is used to receive a data transmission request; wherein the data transmission request includes: data to be processed;

[0160] The data partitioning unit 802 is used to divide the data to be processed into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link.

[0161] The sending unit 803 is used to send the corresponding data block to the destination through each of the transmission links.

[0162] In this embodiment of the application, the data partitioning unit 802 in the data processing device 800 is used to determine the size of the data block corresponding to each transmission link based on the difference between the round-trip delay of each transmission link and the initial round-trip delay; wherein, the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link; the round-trip delay is determined when each transmission link transmits data to the previous data.

[0163] Based on the size of each data block, the data to be processed is divided to determine the data block corresponding to each transmission link.

[0164] In this embodiment of the application, the receiving unit 801 in the data processing device 800 is used to obtain the round-trip time and the initial round-trip time corresponding to each of the transmission links from the network control node, or to obtain the difference between the round-trip time and the initial round-trip time corresponding to each of the transmission links from the network control node.

[0165] In this embodiment of the application, the sending unit 803 in the data processing device 800 is used to perform light-load round-trip delay detection during the initialization of each transmission link to determine the initial round-trip delay corresponding to each transmission link.

[0166] In this embodiment of the application, the sending unit 803 in the data processing device 800 is used to determine the round-trip delay corresponding to each of the transmission links during the transmission of the previous data based on the historical congestion level of each transmission link.

[0167] In this embodiment of the application, the sending unit 803 in the data processing device 800 is used to divide the previous data into historical data blocks corresponding to each transmission link based on the historical congestion level of each transmission link; wherein, the size of the historical data block is inversely proportional to the historical congestion level;

[0168] The historical data block is encapsulated in a transmission message and sent to the destination through each of the transmission links;

[0169] Receive the response message from the destination end to the sent message;

[0170] The round-trip delay is determined based on the transmission duration of the sent message and the returned message.

[0171] In this embodiment of the application, the sending unit 803 in the data processing device 800 is used to divide each historical data block into multiple historical data sub-blocks based on the maximum transmission unit of each transmission link.

[0172] Each historical data sub-block is encapsulated in a sub-transmission message according to the transmission order of the multiple historical data sub-blocks and sent to the destination; wherein, the sub-transmission message includes: the first sub-transmission timestamp of the sub-transmission message, the sub-reception duration for the previous sub-return message, the link identifier, the encoding, and the probe flag; the probe flag is used to indicate whether the destination needs to provide feedback on the sub-return message for the sub-transmission message.

[0173] In this embodiment of the application, the encoding in each of the sub-transmission messages in the same transmission link is monotonically increasing, and the encoding in the sub-transmission messages of different transmission links is decoupled from each other.

[0174] In this embodiment of the application, the receiving unit 801 in the data processing device 800 is used to receive the first sub-transmission message and the first sub-return message fed back by the destination end in response to the first sub-transmission message; wherein, the first sub-return message includes: the second sub-transmission timestamp of the first sub-return message, the first sub-transmission duration, and the encoding in the corresponding first sub-transmission message; the first sub-transmission duration is determined based on the first sub-transmission timestamp and the first sub-receive timestamp of the corresponding first sub-transmission message; the first sub-transmission message is a sub-transmission message used by the probe flag to characterize that the destination end needs to feed back the first sub-return message in response to the first sub-transmission message.

[0175] In this embodiment of the application, the sending unit 803 in the data processing device 800 is used to determine the first sub-receive duration of each first sub-return message based on the second sub-receive timestamp and the second sub-send timestamp of the first sub-return message, and to determine the round-trip delay based on each first sub-receive duration and the corresponding first sub-send duration.

[0176] In this embodiment, the sending unit 803 in the data processing device 800 is configured to encapsulate the next historical data sub-block in a second sub-transmission message and send it to the destination if at least one of the first sub-transmission duration and the first sub-reception duration is abnormal; wherein, the second sub-transmission message includes: a third sub-transmission timestamp of the second sub-transmission message, the first sub-reception duration, the link identifier, the encoding, and the probe flag; the probe flag in the second sub-transmission message is used to indicate that the destination needs to send a second sub-return message in response to the second sub-transmission message; in this embodiment, the receiving unit in the data processing device 800... Yuan 801 is used to receive the second sub-return message from the destination end in response to the second sub-transmission message; wherein, the second sub-return message includes: a fourth sub-transmission timestamp, a second sub-transmission duration, and the encoding for the second sub-return message; the second sub-transmission duration is determined based on the third sub-transmission timestamp of the sending end and the third sub-receive timestamp for the second sub-transmission message; based on the fourth sub-receive timestamp of the received second sub-return message and the fourth sub-transmission timestamp, the second sub-receive duration of the second sub-return message is determined, and the initial round-trip time is determined based on the second sub-receive duration and the second sub-transmission duration.

[0177] In this embodiment of the application, the sending unit 803 in the data processing device 800 is used to divide each data block into multiple data sub-blocks based on the maximum transmission unit of each transmission link; and to encapsulate each data sub-block in the current transmission message and send it to the destination.

[0178] It should be noted that, in the embodiments of this application, if the above-described item information processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an item information processing device (which may be a personal computer, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0179] Correspondingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method on one side of the data processing apparatus 800.

[0180] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0181] It should be noted that, Figure 17 A schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application, such as... Figure 17 As shown, this application embodiment provides an electronic device 900, including a memory 902 and a processor 901. The memory 902 stores a computer program that can run on the processor 901. When the processor 901 executes the program, it implements the steps in the above-described method, wherein;

[0182] Processor 901 typically controls the overall operation of electronic device 900.

[0183] The memory 902 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the processor 901 and various modules in the electronic device 900. It can be implemented by flash memory or random access memory (RAM).

[0184] Correspondingly, this application also provides a computer program product, including a computer program that can be executed by the processor 901 of the electronic device 900 to complete the steps in the method on one side of the data processing device 800.

[0185] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0186] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the apparatus or units can be electrical, mechanical, or other forms.

[0188] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0189] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0190] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0191] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0192] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized by, Applied to a sending end, comprising: receiving a data sending request; wherein the data sending request comprises: to-be-processed data; based on the congestion degree of each transmission link, the to-be-processed data is divided into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion degree of the corresponding transmission link; the sending end is a network node in the RDMA network; through each transmission link, the corresponding data block is encapsulated in the current sending packet and sent to the destination end; wherein the encoding in each current sending packet in the same transmission link monotonically increases, and the encoding in the current sending packet of different transmission links is decoupled.

2. The data processing method according to claim 1, characterized in that, The method further comprises: based on the difference between the determined round-trip delay of each transmission link and the initial round-trip delay, determining the size of the data block corresponding to each transmission link; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link; the round-trip delay is determined when the previous data transmission on each transmission link; based on the size of each data block, the to-be-processed data is divided to determine the data block corresponding to each transmission link.

3. The data processing method according to claim 2, characterized in that, The method further comprises: obtaining the round-trip delay and the initial round-trip delay corresponding to each transmission link from a network control node, or obtaining the difference between the round-trip delay and the initial round-trip delay corresponding to each transmission link from the network control node.

4. The data processing method according to claim 2, characterized in that, The method further comprises: performing a light-load round-trip delay detection when initializing each transmission link to determine the initial round-trip delay corresponding to each transmission link.

5. The data processing method according to claim 2, characterized in that, The method further comprises: based on the historical congestion degree of each transmission link, the previous data is divided into historical data blocks corresponding to each transmission link; wherein the size of the historical data block is inversely proportional to the historical congestion degree; 6. The data processing method according to claim 5, characterized in that, encapsulating the historical data block in a sending packet and sending it to the destination end through each transmission link; receiving a return packet fed back by the destination end in response to the sending packet; ​ ​ determine the round-trip delay based on transmission durations of the sending packet and the return packet.

7. The data processing method according to claim 6, characterized in that, The encapsulating the historical data block in the sending packet and sending to the destination through each transmission link comprises: dividing each historical data block into a plurality of historical data sub-blocks based on a maximum transmission unit of each transmission link; encapsulating each historical data sub-block in a sub-sending packet in a sending order of the plurality of historical data sub-blocks and sending to the destination, wherein the sub-sending packet comprises a first sub-sending timestamp of the sub-sending packet, a sub-receiving duration for a previous sub-return packet, a link identifier, an encoding, and a detection flag; the detection flag is used to indicate whether the destination needs to feed back the sub-return packet for the sub-sending packet.

8. The data processing method according to claim 7, characterized in that, The encoding in each sub-sending packet in the same transmission link monotonically increases, and the encoding in the sub-sending packets of different transmission links is decoupled.

9. The data processing method according to claim 7, characterized in that, The receiving the return packet fed back by the destination in response to the sending packet comprises: receiving a first sub-return packet fed back by the destination in response to a first sub-sending packet, wherein the first sub-return packet comprises a second sub-sending timestamp of the first sub-return packet, a first sub-sending duration, and the encoding in the corresponding first sub-sending packet; the first sub-sending duration is determined based on the first sub-sending timestamp and a first sub-receiving timestamp of the corresponding first sub-sending packet; the first sub-sending packet is a sub-sending packet in which the detection flag is used to indicate that the destination needs to feed back the first sub-return packet for the first sub-sending packet.

10. The data processing method according to claim 9, characterized in that, The determining the round-trip delay based on transmission durations of the sending packet and the return packet comprises: determining a first sub-receiving duration of each first sub-return packet based on a second sub-receiving timestamp of receiving the first sub-return packet and the second sub-sending timestamp, and determining the round-trip delay based on each first sub-receiving duration and the corresponding first sub-sending duration.

11. The data processing method according to claim 9, characterized in that, The method further comprises: if at least one of the first sub-sending duration and the first sub-receiving duration is abnormal, encapsulating a next historical data sub-block in a second sub-sending packet and sending to the destination, wherein the second sub-sending packet comprises a third sub-sending timestamp of the second sub-sending packet, the first sub-receiving duration, the link identifier, the encoding, and the detection flag; the detection flag in the second sub-sending packet is used to indicate that the destination needs to feed back a second sub-return packet for the second sub-sending packet; receiving the second sub-return packet fed back by the destination in response to the second sub-sending packet, wherein the second sub-return packet comprises a fourth sub-sending timestamp for the second sub-return packet, a second sub-sending duration, and the encoding; the second sub-sending duration is determined based on the third sub-sending timestamp of the sending end and a third sub-receiving timestamp for the second sub-sending packet; determine a second sub-receiving duration of the second sub-return packet based on a fourth sub-receiving timestamp and the fourth sub-sending timestamp of receiving the second sub-return packet, and determine the initial round-trip time based on the second sub-receiving duration and the second sub-sending duration.

12. The data processing method according to any one of claims 1 to 11, characterized in that, The sending, to the destination, of the corresponding data block in the current sending packet through each transmission link comprises: dividing each data block into a plurality of data sub-blocks based on a maximum transmission unit of each transmission link; encapsulating each data sub-block in a current sending packet and sending to the destination.

13. A data processing apparatus, characterized by: Applied to a sending end, comprising: a receiving unit configured to receive a data sending request, wherein the data sending request comprises to-be-processed data; a data dividing unit configured to divide the to-be-processed data into data blocks corresponding to each transmission link based on a congestion degree of each transmission link, wherein a size of the data block is inversely proportional to the congestion degree of the corresponding transmission link; and the sending end is a network node in a RDMA network; a sending unit configured to encapsulate the corresponding data block in a current sending packet through each transmission link and send to a destination, wherein an encoding in each current sending packet in a same transmission link monotonically increases, and the encodings in the current sending packets of different transmission links are decoupled from each other.

14. An electronic device, comprising: The computer program is executed by the processor to implement the steps in the method of any one of claims 1 to 12.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps in the method of any one of claims 1 to 12.

16. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps in the method of any one of claims 1 to 12.

Images