Data processing method and device, electronic equipment, storage medium and computer program product
By dividing data blocks according to the congestion degree of transmission links and dynamically adjusting their size, the problem of low data stream transmission efficiency in the wide area network is solved, and more efficient bandwidth utilization and reliability are achieved, avoiding link congestion.
Patent Information
- Application Number
- CN202411045648.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-31
AI Technical Summary
In wide-area networks, the reliability problems of a single physical network path and bandwidth utilization problems lead to low data stream transmission efficiency, especially due to the limitations of the hashing algorithm, the traffic distribution of multiple streams is uneven and some links are blocked. Causes inefficient data transmission.
By dividing data blocks according to the congestion degree of each transmission link, the size of the data block is inversely proportional to the congestion degree, and the corresponding data block is sent to the destination through each transmission link, and the network control nodes are used to obtain the round-trip delay and The initial round-trip delay difference value is used to determine the size of the data block and realize the dynamic segmentation and transmission of the data block.
It improves the transmission efficiency of data streams, makes full use of multiple equivalent path bandwidths of the wide area network, reduces the situation of transmission link blockage, and enhances the reliability of the network.
Smart Images

Figure CN119071246A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of transmission and bearing technology, and in particular to a data processing method, device, electronic device, storage medium and computer program product. Background Art
[0002] In related technologies, the rapid development of the digital industry has led to an explosive increase in the amount of data, and the uneven distribution of computing resources has caused more and more transmission needs between data production and consumption, such as East-West computing, national integrated computing network, data express, distributed intelligent computing wide area interconnection and other services. Currently, standard remote direct memory access (RDMA) is usually used for long-distance transmission of large files. Because the source and destination port numbers of the same data stream connection are the same, a connection can only choose one physical network path for transmission. This will introduce two problems:
[0003] 1. Reliability issues: A single physical network path is prone to link failures, and wide area networks are diverse, heterogeneous, and complex, making reliability issues even more serious.
[0004] 2. Bandwidth utilization problem. There are multiple equal-cost paths in the wide area network. Generally, the hash algorithm is used for load balancing. A single flow can only be hashed to one physical path, and the effective link bandwidth cannot be fully utilized. If multi-flow transmission is adopted, due to the limitations of the hash algorithm, hash polarization is prone to occur, resulting in uneven distribution of traffic of multiple flows among multiple links, and some links are blocked, resulting in low transmission efficiency of data flows. Summary of the invention
[0005] In order to solve the problem of low data flow transmission efficiency caused by the blockage of some links in a wide area network, the embodiments of the present application provide a data processing method, device, electronic device, storage medium and computer program product, which can improve the transmission efficiency of data flow.
[0006] The technical solution of this application is implemented as follows:
[0007] The present application provides a data processing method, including:
[0008] receiving a data transmission request; wherein the data transmission request includes: data to be processed;
[0009] Based on the congestion level of each transmission link, the data to be processed is divided into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link;
[0010] The corresponding data block is sent to the destination end through each transmission link.
[0011] In the above solution, the method of dividing the to-be-processed data into data blocks corresponding to each transmission link based on the congestion level of each transmission link includes:
[0012] Based on the determined difference between the round-trip delay of each transmission link and the initial round-trip delay, determine the size of the data block corresponding to each transmission link; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link; the round-trip delay is determined when each transmission link transmits the previous data;
[0013] The data to be processed is divided based on the size of each of the data blocks, and the data block corresponding to each of the transmission links is determined.
[0014] In the above solution, before determining the size of the data block corresponding to each transmission link based on the difference between the determined round-trip delay of each transmission link and the initial round-trip delay, the method further includes:
[0015] The round trip delay and the initial round trip delay corresponding to each of the transmission links are obtained from a network control node, or a difference between the round trip delay and the initial round trip delay corresponding to each of the transmission links is obtained from the network control node.
[0016] In the above solution, before determining the size of the data block corresponding to each transmission link based on the difference between the determined round-trip delay of each transmission link and the initial round-trip delay, the method further includes:
[0017] When each of the transmission links is initialized, a light-load round-trip delay detection is performed to determine the initial round-trip delay corresponding to each of the transmission links.
[0018] In the above solution, before determining the size of the data block corresponding to each transmission link based on the difference between the determined round-trip delay of each transmission link and the initial round-trip delay, the method further includes:
[0019] In the transmission process of the previous data on each transmission link based on the historical congestion level, the round trip delay corresponding to each transmission link is determined.
[0020] In the above solution, determining the round trip delay corresponding to each transmission link during the transmission process of the previous data based on the historical congestion level of each transmission link includes:
[0021] Based on the historical congestion level of each transmission link, the previous data is divided into historical data blocks corresponding to each transmission link; wherein the size of the historical data block is inversely proportional to the historical congestion level;
[0022] Encapsulating the historical data block in a transmission message, and sending the message to the destination end through each transmission link;
[0023] Receiving a return message fed back by the destination end in response to the sent message;
[0024] The round-trip delay is determined based on the transmission duration of the sent message and the returned message.
[0025] In the above solution, encapsulating the historical data block in a transmission message and sending it to the destination through each transmission link includes:
[0026] Based on the maximum transmission unit of each transmission link, each of the historical data blocks is divided into a plurality of historical data sub-blocks;
[0027] Each historical data sub-block is encapsulated in a sub-sending message according to the sending order of multiple historical data sub-blocks, and sent to the destination end; wherein the sub-sending message includes: the first sub-sending timestamp of the sub-sending message, the sub-receiving duration for the previous sub-return message, the link identifier, the encoding and the detection flag; the detection flag is used to indicate whether the destination end needs to feedback the sub-return message for the sub-sending message.
[0028] In the above scheme, the codes in the sub-messages in the same transmission link are monotonically increased, and the codes in the sub-messages in different transmission links are decoupled from each other.
[0029] In the above scheme, the receiving of the return message fed back by the destination end in response to the sent message includes:
[0030] Receive the first sub-return message fed back by the destination in response to the first sub-sending message; wherein the first sub-return message includes: the second sub-sending timestamp of the first sub-return message, the first sub-sending duration and the corresponding encoding in the first sub-sending message; the first sub-sending duration is determined based on the first sub-sending timestamp and the first sub-receiving timestamp of the corresponding first sub-sending message; the first sub-sending message is the detection flag used to indicate that the destination end needs to feed back the first sub-return message in response to the first sub-sending message.
[0031] In the above solution, determining the round trip delay based on the transmission duration of the sent message and the returned message includes:
[0032] The first sub-receiving duration of each first sub-return message is determined based on the second sub-receiving timestamp and the second sub-sending timestamp of receiving the first sub-return message, and the round-trip delay is determined based on each first sub-receiving duration and the corresponding first sub-sending duration.
[0033] In the above scheme, the method further includes:
[0034] If at least one of the first sub-sending duration and the first sub-receiving duration is abnormal, the next sub-block of the historical data is encapsulated in a second sub-sending message and sent to the destination end; wherein the second sub-sending message includes: a third sub-sending timestamp of the second sub-sending message, the first sub-receiving duration, the link identifier, the encoding and the detection flag; the detection flag in the second sub-sending message is used to indicate that the destination end needs to feedback a second sub-return message for the second sub-sending message;
[0035] Receive the second sub-return message fed back by the destination in response to the second sub-send message; wherein the second sub-return message includes: a fourth sub-send timestamp, a second sub-send duration and the encoding for the second sub-return message; the second sub-send duration is determined based on the third sub-send timestamp of the sender and the third sub-receive timestamp for the second sub-send message;
[0036] Based on the fourth sub-receiving timestamp and the fourth sub-sending timestamp of receiving the second sub-return message, the second sub-receiving duration of the second sub-return message is determined, and the initial round-trip delay is determined based on the second sub-receiving duration and the second sub-sending duration.
[0037] In the above solution, sending the corresponding data block to the destination end through each transmission link includes:
[0038] Based on the maximum transmission unit of each transmission link, each of the data blocks is divided into a plurality of data sub-blocks;
[0039] Each of the data sub-blocks is encapsulated in a current sending message and sent to the destination end.
[0040] The embodiment of the present application further provides a data processing device, applied to a sending end, comprising:
[0041] A receiving unit, configured to receive a data transmission request; wherein the data transmission request includes: data to be processed;
[0042] A data partitioning unit, configured to partition the to-be-processed data into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link;
[0043] A sending unit is used to send the corresponding data block to a destination end through each transmission link.
[0044] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the computer program.
[0045] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are implemented.
[0046] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps in the above method when executed by a processor.
[0047] In the embodiment of the present application, a data transmission request is received; wherein the data transmission request includes: data to be processed; based on the congestion level of each transmission link, the data to be processed is divided into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link; and the corresponding data block is sent to the destination through each transmission link. Since the load on each transmission link is determined in inverse proportion to the congestion level on the transmission link, the load is greater when the congestion level is smaller, and the bandwidth of multiple equivalent paths in the wide area network and the reliability brought by multiple equivalent paths can be fully utilized to reduce the occurrence of transmission congestion in some transmission links, thereby improving the transmission efficiency of the data to be processed. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 1 ;
[0049] Figure 2 The effect of the data processing method provided in the embodiment of the present application is shown in FIG. Figure 1 ;
[0050] Figure 3 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 2 ;
[0051] Figure 4 The effect of the data processing method provided in the embodiment of the present application is shown in FIG. Figure 2 ;
[0052] Figure 5 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 3 ;
[0053] Figure 6 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 4 ;
[0054] Figure 7 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 5 ;
[0055] Figure 8 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 6 ;
[0056] Fig. 9 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 7 ;
[0057] Fig.10 The effect of the data processing method provided in the embodiment of the present application is shown in FIG. Figure 3 ;
[0058] Fig.11 The effect of the data processing method provided in the embodiment of the present application is shown in FIG. Figure 4 ;
[0059] Fig.12 The effect of the data processing method provided in the embodiment of the present application is shown in FIG. Figure 5 ;
[0060] Fig.13 The effect of the data processing method provided in the embodiment of the present application is shown in FIG. Figure 6 ;
[0061] Fig.14 The effect of the data processing method provided in the embodiment of the present application is shown in FIG. Figure 7 ;
[0062] Fig.15 Schematic diagram of the data processing method provided in the embodiment of the present application Figure 8 ;
[0063] Fig.16 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0064] Fig.17 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0065] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0066] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0067] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0069] In related technologies, the rapid development of digital industries has led to an explosive growth in data volume, and the uneven distribution of computing resources has resulted in an increasing demand for data transmission between production and consumption. For example, services such as East-West Computing, the National Integrated Computing Network, data express, and distributed intelligent computing wide area interconnection currently use offline manual hard disk express solutions for large file long-distance transmission, which poses risks such as data leakage and loss, and cannot meet market demand in terms of data security and transmission efficiency.
[0070] Cloud computing vendors have provided solutions for the efficient transmission of large-scale data in wide area networks through global data center layout and optimized transmission protocols. However, for data transmission of tens of TB, online transmission solutions cannot meet the timeliness requirements and offline transmission is still required. For large-scale interactive services such as distributed intelligent computing and wide-area interconnection, it is also necessary to further study the wide-area transmission technology of ultra-large-scale data.
[0071] RDMA uses technologies such as zero memory copy, kernel bypass, and CPU offload to completely offload the TCP / IP protocol stack to the network card, allowing user-mode applications to directly read and write remote host memory, avoiding data copying and context switching, and achieving high throughput, low latency, and low CPU computing power loss. RDMA has been widely deployed in data centers to support a variety of low-latency and high-throughput applications. For example, key-value storage, distributed transactions, distributed memory, remote procedure call (RPC), storage systems, graph computing, and machine learning systems. RDMA's kernel bypass and DMA features can greatly reduce the CPU overhead of the sender and receiver. Using RDMA's "one-sided" operation, massive data transmission can be achieved with almost no perception of the receiving CPU.
[0072] However, since the source and destination port numbers of the User Datagram Protocol (UDP) for the same data stream connection are the same for standard RDMA, a connection can only choose one physical network path for transmission. This will introduce two problems:
[0073] 1. Reliability issues. A single physical network path is prone to link failures (random packet loss, congestion packet loss, single link, line disconnection, etc. caused by physical failure or configuration errors). The reliability problem is even more serious due to the diversity, heterogeneity and complexity of wide area networks.
[0074] 2. Bandwidth utilization problem. There are multiple equal-cost paths in the wide area network. Generally, the hash algorithm is used for load balancing. A single flow can only be hashed to one physical path, which cannot fully utilize the effective link bandwidth. If users use multi-flow transmission, hash polarization is prone to occur due to the limitations of the hash algorithm, resulting in uneven distribution of traffic of multiple flows among multiple links.
[0075] In order to fully utilize the bandwidth of multiple equal-cost paths in the wide area network and the reliability brought by multiple equal-cost paths without increasing the complexity of the application, a multi-path transmission mechanism was introduced. However, how to find a balance between transmission efficiency, load balancing, and packet loss recovery cost is a problem that multi-path RDMA needs to solve.
[0076] This application embodiment provides a data processing method, see Figure 1 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 1 , will combine Figure 1The steps shown are explained:
[0077] S101. Receive a data sending request; wherein the data sending request includes: data to be processed.
[0078] In the embodiment of the present application, the transmitting end receives a data transmission request sent by other devices or a user end, wherein the data transmission request includes: data to be processed.
[0079] The sending end may be a network node in the RDMA transmission network that has data transmission and corresponding data processing functions. There are multiple transmission links between the sending end and the destination end.
[0080] S102: based on the congestion level of each transmission link, divide the data to be processed into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link.
[0081] In the embodiment of the present application, the transmitting end divides the to-be-processed data into data blocks corresponding to each transmission link according to the acquired or determined congestion level of each transmission link, wherein the size of the data block of each transmission link is inversely proportional to the corresponding congestion level.
[0082] In the embodiment of the present application, the sending end may send a message to the destination end through multiple transmission links in advance, and then receive a return message fed back by the destination end, and determine the congestion degree of each transmission link by the transmission time of the sent message and the return message. The congestion degree is proportional to the transmission time.
[0083] In the embodiment of the present application, the transmitting end may also obtain the congestion degree information of each transmission link from the network control node. The network control node may determine the congestion degree information in advance by detecting the round-trip transmission time between the transmitting end and the destination end.
[0084] S103: Send the corresponding data block to the destination end through each transmission link.
[0085] In the embodiment of the present application, the sending end sends the corresponding data block to the destination end through each transmission link.
[0086] In an embodiment of the present application, the transmitting end divides each of the data blocks into a plurality of data sub-blocks based on the maximum transmission unit of each of the transmission links; encapsulates each of the data sub-blocks in a current transmission message, and sends them to the destination end in sequence.
[0087] The maximum transmission unit (MTU) may be the maximum transmission load of a physical node on the transmission link.
[0088] In an embodiment of the present invention, the transmitting end can also encapsulate each data word block in the corresponding current transmission message according to the order of multiple data sub-blocks, and send it to the destination end in order. Each current transmission message includes: the transmission timestamp of the current transmission message, the reception duration for the previous current return message, the link identifier, the coding and the detection flag; the detection flag is used to indicate whether the destination end needs to feedback the sub-return message for the sub-transmission message. The coding in each current transmission message in the same transmission link increases monotonically, and the coding in the current transmission messages of different transmission links are decoupled from each other.
[0089] The sending end receives the current return message fed back by the destination end in response to the scheduled current sending message. The current return message includes: the sending timestamp of the current return message, the sending duration and code for the corresponding scheduled current sending message; the sending duration for the corresponding scheduled current sending message is determined by the destination end based on the sending timestamp of the corresponding scheduled current sending message and the receiving timestamp corresponding to the destination end; the scheduled current sending message can be any message among multiple currently sending messages, and the detection flag in the scheduled current sending message is used to indicate that the destination end needs to feed back the current return message for the scheduled current sending message.
[0090] Exemplary, combined Figure 2 , there are n sQP (Subqueue pair) transmission links between the sender and the destination. After the sender obtains the QP (Queue Pair) data stream (data to be processed), it can divide the QP according to the congestion level of the n transmission links to determine the data block corresponding to each transmission link. The sender can also divide the data block of each transmission link into three data sub-blocks according to the maximum transmission unit. Among them, the codes corresponding to the three data sub-blocks in the sQP1 transmission link are: c1PSN1, c1PSN1 and c1PSN3. It can be seen that the codes corresponding to the three data sub-blocks in the sQP1 transmission link are monotonically increasing. And the codes of the three data sub-blocks corresponding to the n sQP transmission links are different from each other, that is, the codes of the data sub-blocks corresponding to the N sQP transmission links are decoupled from each other.
[0091] In the embodiment of the present application, a data transmission request is received; wherein the data transmission request includes: data to be processed; based on the congestion level of each transmission link, the data to be processed is divided into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link; and the corresponding data block is sent to the destination through each transmission link. Since the load on each transmission link is determined in inverse proportion to the congestion level on the transmission link, the load is greater when the congestion level is smaller, and the bandwidth of multiple equivalent paths in the wide area network and the reliability brought by multiple equivalent paths can be fully utilized to reduce the occurrence of transmission congestion in some transmission links, thereby improving the transmission efficiency of the data to be processed.
[0092] See also Figure 3 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 2 , Figure 1 The illustrated S102 can also be implemented through S201 to S202, which will be described in conjunction with the steps:
[0093] S201. Determine the size of the data block corresponding to each transmission link based on the difference between the determined round-trip delay of each transmission link and the initial round-trip delay; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link.
[0094] In an embodiment of the present application, the transmitting end can calculate the difference between the round-trip delay and the initial round-trip delay by predicting the round-trip delay of each transmission link and the initial round-trip delay. The difference between the round-trip delay and the initial delay is used to characterize the congestion level of the corresponding transmission link. The size of the data block corresponding to each of the transmission links is then determined by the difference between the round-trip delay and the initial delay; the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link. The round-trip delay is determined when each of the transmission links transmits the previous data; the initial round-trip delay is determined when each transmission link is initialized.
[0095] Exemplarily, the sender can allocate the chunk (data block) size of the inflight message (data to be processed) inversely proportional to the round-trip time (RTT) of each transmission link. This achieves load balancing of multiple transmission links by chunk, with low overhead for light load and low recovery cost for heavy load. The initial value cn of the chunk size is set to the bandwidth-delay product (BDP) of the corresponding sQP transmission link, and is subsequently changed according to the change of the RTT of each sQP transmission link. The relationship between the chunk size cn and RTTn of this scheme can be determined by formula (1):
[0096] cn = K / (RTT n -RTT bn ) (1)
[0097] Among them, cn is the size of the data block, k is the conditional parameter, and RTT n is the round-trip time, RTT bn is the initial round trip delay.
[0098] S202: Divide the data to be processed based on the size of each of the data blocks, and determine the data block corresponding to each of the transmission links.
[0099] In the embodiment of the present application, after the size of each data block is determined, the data to be processed may be divided according to the size of each data block to determine the data block corresponding to each transmission link.
[0100] Exemplary, combined Figure 4 , the data to be processed (QP) can be divided into 9 data blocks (c1-c9) corresponding to the sQP transmission links. Among them, the RTT corresponding to the data block c1 is d (The difference between the round-trip delay and the initial round-trip delay) is 2ms, and the RTT corresponding to data block c2 is d It is 3ms, and it can be seen from the figure that the amount of data in data block c1 is significantly larger than that in data block c2, and the RTT corresponding to data block c1 d Smaller than the RTT corresponding to data block c2 d , we can know the data volume of the data block and the corresponding RTT d is inversely proportional to the size.
[0101] In an embodiment of the present application, the size of the data block corresponding to each transmission link is determined based on the difference between the determined round-trip delay of each transmission link and the initial round-trip delay; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link. Based on the size of each data block, the data to be processed is divided to determine the data block corresponding to each transmission link. Since the load on each transmission link is determined in inverse proportion to the difference between the round-trip delay of the transmission link and the initial round-trip delay, the difference between the round-trip delay and the initial round-trip delay can reflect the congestion level of the corresponding transmission link. The lower the congestion level, the greater the load on the transmission link. In this way, the transmission resources of each transmission link on the wide area transmission network can be effectively and reasonably utilized, reducing the occurrence of transmission congestion in some transmission links, thereby improving the transmission efficiency of the data to be processed.
[0102] See also Figure 5 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 3 , Figure 3 The process shown in S201 may further include S301, which will be described in combination with the steps:
[0103] S301. Obtain the round-trip delay and the initial round-trip delay corresponding to each of the transmission links from a network control node, or obtain the difference between the round-trip delay and the initial round-trip delay corresponding to each of the transmission links from the network control node.
[0104] In an embodiment of the present application, the network control node may transmit a detection message through each transmission link in advance, and determine the round-trip delay of the transmission link based on the transmission duration of the detection message. And the detection message may be transmitted through the transmission link when the transmission link is initialized, and the initial round-trip delay of the transmission link may be determined based on the detection message. When the transmitting end needs to send data based on the congestion level of the transmission link, the round-trip delay and the initial round-trip delay corresponding to each of the transmission links may be obtained in advance from the network control node, or the difference between the round-trip delay and the initial round-trip delay corresponding to each of the transmission links may be obtained from the network control node.
[0105] The network control node may include a centralized network controller.
[0106] Exemplarily, the RTT indicator can also be obtained through end-to-network collaboration, and the static RTTbn and dynamic RTTn of a specific link can be obtained from a centralized network controller, or the difference RTTd representing the degree of congestion can be directly obtained.
[0107] In the embodiment of the present application, the transmitting end obtains the round-trip delay and the initial round-trip delay corresponding to the transmission link, or the difference between the two, from the network control node. In this way, the transmitting end saves the step of detecting the round-trip delay and the initial round-trip delay before sending data, reduces the workload of the transmitting end, and can improve the transmission efficiency of the transmitting end under limited resources.
[0108] See also Figure 6 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 4 , Figure 3 The process shown in S201 may further include S302, which will be described in combination with the steps:
[0109] S302: Perform a light-load round-trip delay detection when each transmission link is initialized to determine the initial round-trip delay corresponding to each transmission link.
[0110] In the embodiment of the present application, the transmitting end performs a light-load round-trip delay detection on the transmission link when each transmission link is initialized (that is, when the transmission link is just established), and determines the initial round-trip delay corresponding to each transmission link. That is, when each transmission link is initialized, the transmitting end sends a message to the destination end, and then receives a return message fed back by the destination end, and determines the initial round-trip delay through the transmission time of the sent message and the return message.
[0111] In the embodiment of the present application, the physical link of each sQP transmission link is relatively fixed (failure will trigger switching), but the physical link length of each sQP is different, and some are even very different. In order to judge the congestion level of each sQP transmission link, it is necessary to first detect RTTbn when lightly loaded or even unloaded during initialization, and then judge the congestion level by the difference between RTTn and RTTbn. If the physical link of a sQP fails and switches to a new link, it is necessary to trigger a detection of RTTbn when lightly loaded or even unloaded, and then judge the congestion level by the difference between RTTn and RTTbn.
[0112] In an embodiment of the present application, a light-load round-trip delay detection is performed when each of the transmission links is initialized to determine the initial round-trip delay corresponding to each of the transmission links. Since the load of the transmission link is the least during initialization, the initial round-trip delay of the transmission link under the optimal transmission state can be accurately determined through the light-load round-trip delay detection, thereby providing accurate parameter guarantees for the subsequent determination of the congestion level, and the determined congestion level is also more accurate. The data blocks divided by the precise congestion level are also more matched with the load conditions of the corresponding transmission links, thereby maximizing the use of the transmission resources of each transmission link and improving the transmission efficiency of the data to be processed.
[0113] See also Figure 7 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 5 , Figure 3 The process shown in S201 may further include S401, which will be described in combination with the steps:
[0114] S401. In a transmission process of the previous data on each transmission link based on a historical congestion level, determine the round-trip delay corresponding to each transmission link.
[0115] In the embodiment of the present application, the round-trip delay when the data to be processed is divided into blocks is determined when the previous data is transmitted. The sender can transmit the previous data based on the historical congestion level of each transmission link, and determine the round-trip delay during the transmission process by the length of time for sending and returning the message.
[0116] In the embodiment of the present application, in the transmission process of each transmission link for the previous data based on the historical congestion level, the round-trip delay corresponding to each transmission link is determined. In this way, the congestion level of each transmission link can be calculated based on the predetermined round-trip delay for the data to be processed, and then the data to be processed can be divided based on the congestion level. In this way, when transmitting each data, there is no need to measure the round-trip delay first, which saves the round-trip delay detection process, saves time, and thus improves the data transmission efficiency.
[0117] See also Figure 8 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 6 , Figure 7 The illustrated S401 can also be implemented through S501 to S504, which will be described in conjunction with the steps:
[0118] S501. Based on the historical congestion level of each of the transmission links, divide the previous data into historical data blocks corresponding to each of the transmission links; wherein the size of the historical data block is inversely proportional to the historical congestion level.
[0119] In the embodiment of the present application, the transmitting end divides the previous data into historical data blocks corresponding to each transmission link based on the predetermined historical congestion level of each transmission link, wherein the size of the historical data block is inversely proportional to the historical congestion level.
[0120] S502: Encapsulate the historical data block in a sending message, and send it to the destination end through each of the transmission links.
[0121] In the embodiment of the present application, the sending end may encapsulate the historical data blocks in a sending message, and send the historical data blocks to the destination end through the transmission link corresponding to each historical data block.
[0122] The sent message may include a corresponding sending timestamp.
[0123] S503: Receive a return message fed back by the destination end in response to the sent message.
[0124] In the embodiment of the present application, after receiving the sent message, the destination determines the transmission duration of the sent message by receiving the timestamp of the sent message and the sending timestamp in the sent message, and encapsulates the transmission duration and the sending timestamp of the return message in the return message and feeds it back to the sender.
[0125] S504: Determine the round-trip delay based on the transmission duration of the sent message and the returned message.
[0126] In the embodiment of the present application, the transmitting end determines the timestamp of receiving the return message, and then determines the transmission time of the return message by the timestamp of receiving the return message and the sending timestamp of the return message in the return message. The round trip delay is determined by combining the transmission time of the return message and the transmission time of the sent message.
[0127] In an embodiment of the present application, based on the historical congestion level of each transmission link, the previous data is divided into historical data blocks corresponding to each transmission link; wherein the size of the historical data block is inversely proportional to the historical congestion level. The historical data block is encapsulated in a transmission message and sent to the destination through each transmission link. The destination responds to the transmission message and the feedback return message is received. The round-trip delay is determined based on the transmission time of the transmission message and the return message. Similarly, when the data to be processed is to be processed, the congestion level of each transmission link can be calculated using the predetermined round-trip delay, and then the data to be processed can be divided based on the congestion level. In this way, when transmitting each data, there is no need to measure the round-trip delay first, which saves the round-trip delay detection process, saves time, and thus improves the data transmission efficiency.
[0128] See also Fig. 9 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 7 , Figure 8 S502 to S504 shown can also be implemented by S601 to S604, which will be described in combination with the steps:
[0129] S601. Divide each of the historical data blocks into a plurality of historical data sub-blocks based on a maximum transmission unit of each of the transmission links.
[0130] In the embodiment of the present application, the transmitting end divides the historical data block into multiple historical data sub-blocks of the size of the data amount that can be transmitted by the maximum transmission unit according to the maximum transmission unit of each transmission link.
[0131] S602. Encapsulate each historical data sub-block in a sub-sending message according to the sending order of the multiple historical data sub-blocks, and send it to the destination; wherein the sub-sending message includes: the first sub-sending timestamp of the sub-sending message, the sub-receiving duration for the previous sub-return message, the link identifier, the encoding and the detection flag.
[0132] In an embodiment of the present application, the transmitting end encapsulates each historical data sub-block in a sub-transmission message according to the transmission order of multiple historical data word blocks, and sends it to the destination end. The sub-transmission message includes: the first sub-transmission timestamp of the sub-transmission message, the sub-reception duration for the previous sub-return message, the link identifier, the coding and the detection flag; the detection flag is used to indicate whether the destination end needs to feedback the sub-return message for the sub-transmission message. The coding in each sub-transmission message in the same transmission link is monotonically increasing, and the coding in the sub-transmission messages of different transmission links are decoupled from each other.
[0133] In the embodiment of the present application, a chunk namespace and message sequence number independent encoding mechanism is proposed to achieve decoupling of CPSN and other sequence numbers between sQP transmission links, and the CPSN in a single SQP transmission link is monotonically increasing. The previous data is split into n chunks (historical data blocks), which are scheduled and dispatched in combination with the load conditions of n SQPs. Each chunk of sQP is split into several packets (historical data sub-blocks) in combination with the PMTU of the sQP link. The CPSN of each packet is independently and incrementally encoded. Even if packet loss and retransmission occur, the CPSN is still strictly monotonically increasing, ensuring that packet loss will not cause RTT measurement errors. The chunk namespace and message sequence number are independently encoded, and the CPSN and other sequence numbers between sQPs are decoupled, allowing duplication.
[0134] Exemplary, combined Fig.10 , which is a schematic diagram of the header information of each sub-sent message. Among them, T1 in the dotted box represents the first sub-sent timestamp of the sub-sent message, Dest sQP is used to represent the link identifier, A is used to represent the detection flag (which can be 0 or 1 in this embodiment), Tr is used to represent the sub-receiving time length of the previous sub-return message, and cPSN is used to represent the code.
[0135] S603, receiving the first sub-return message fed back by the destination in response to the first sub-sending message; wherein the first sub-return message includes: the second sub-sending timestamp of the first sub-return message, the first sub-sending duration and the corresponding code in the first sub-sending message.
[0136] In an embodiment of the present application, after receiving each sub-sending message, the destination end detects the detection flag of the sub-sending message. If the detection flag of the first sub-sending message is detected to indicate that the destination end needs to feedback the sub-return message for the sub-sending message (the detection flag is 1), the destination end responds to the first sub-sending message and feeds back the first sub-return message. The first sub-return message includes: the second sub-sending timestamp of the first sub-return message, the first sub-sending duration, and the corresponding encoding in the first sub-sending message. The first sub-sending duration is determined by the destination end based on the first sub-sending timestamp and the first sub-receiving timestamp of the corresponding first sub-sending message.
[0137] The first sub-sending message is a sub-sending message in which the detection flag is used to indicate that the destination end needs to feed back the first sub-returning message in response to the first sub-sending message.
[0138] Exemplary, combined Fig.11 , which is a schematic diagram of the header information of each first sub-return message. Among them, T3 in the dotted box represents the second sub-sending timestamp of the first sub-return message, Tr is used to represent the first sub-sending duration, and cPSN is used to represent the code in the corresponding first sub-sending message.
[0139] In the embodiment of the present application, the base transport header (Base Transport Header, BTH) (combined with the base transport header of the standard based on converged Ethernet (RDMA over Converged Ethernet, RoCEv2) Fig.10 ) and ACK Extended Transport Header (ACKExtended Transport Header, AETH header) (combined Fig.11 ) is expanded or updated to transmit sQP-level interaction information and to dynamically and low-overheadly perform accurate link delay measurements. After the aforementioned processing, the measured values guide the user-submitted data to be chunked based on the load.
[0140] S604: Determine the first sub-receiving duration of each first sub-return message based on the second sub-receiving timestamp and the second sub-sending timestamp of receiving the first sub-return message, and determine the round-trip delay based on each first sub-receiving duration and the corresponding first sub-sending duration.
[0141] In an embodiment of the present application, the first sub-receiving duration of each first sub-return message is determined based on the second sub-receiving timestamp and the second sub-sending timestamp in each first return message. A round trip duration is then determined based on each first sub-receiving duration and the corresponding first sub-sending duration. The round trip delay is determined by the average of each round trip duration.
[0142] The transmitting end may determine the round trip delay based on the average of each round trip time, or determine the round trip delay based on the maximum or minimum value of each round trip time.
[0143] Exemplary, combined Fig.12 The sending end determines the first sub-receiving time length (T r =T4-T3). The destination determines the first sub-transmission duration (T4-T3) based on the first sub-transmission timestamp T1 and the first sub-reception timestamp T2 of the corresponding first sub-transmission message. s =T2-T1). The transmitting end then calculates the first sub-receiving time based on each first sub-receiving time T r and the first sub-transmission duration T received s Determine a round trip time RTT (RTT = T r +T s ).
[0144] In the embodiment of the present application, the transmitting end determines the round-trip delay based on the first sub-sending duration of the first sub-sending message and the first sub-receiving duration of the corresponding first sub-returning message. Since the codes in each sub-sending message in the same transmission link are monotonically increasing, and the codes in the sub-sending messages of different transmission links are decoupled from each other, when a sub-sending message is retransmitted, its corresponding code will not be retransmitted, thereby avoiding the return message error caused by the repeated code in the retransmitted sub-sending message at the destination end, and can effectively avoid the round-trip delay measurement inaccuracy caused by packet loss and retransmission of TCP classic protocols.
[0145] Furthermore, since the destination end will only feed back the first sub-return message corresponding to the first sub-sent message (having a detection flag bit indicating that the destination end needs to feed back the first sub-return message in response to the first sub-sent message), the resource overhead caused by feeding back messages for each sub-sent message is avoided, and the delay changes of multiple equivalent transmission links can be accurately measured with low overhead, saving transmission resources.
[0146] Combination Fig.13 In the embodiment of the present application, the user end sends multiple data to the sending end through a data sending request for transmission operation. After the sending end completes the sending of each data, it feeds back a request completion message to the user end. In the process of sending each data by the sending end, the data can be divided into multiple data blocks, and each data block can be divided into multiple data sub-blocks. Fig.14, the sender can divide a data block into two data sub-blocks, and pack the two data sub-blocks into r15 and r16 messages and send them to the destination in sequence. The destination feeds back r17 message for r15 message, and feeds back r18 message for r16 message.
[0147] See also Fig.15 , which is a flow chart of the data processing method provided in the embodiment of the present application Figure 8 , will be explained in combination with the steps:
[0148] S701. If at least one of the first sub-sending duration and the first sub-receiving duration is abnormal, the next sub-block of the historical data is encapsulated in a second sub-sending message and sent to the destination; wherein the second sub-sending message includes: the third sub-sending timestamp of the second sub-sending message, the first sub-receiving duration, the link identifier, the encoding and the detection flag.
[0149] In the embodiment of the present application, if the first sub-sending duration is greater than the first threshold and / or the first sub-receiving duration is less than the second threshold, the next historical data sub-block is encapsulated in the second sending message and sent to the destination.
[0150] The second sub-sending message includes: the third sub-sending timestamp of the second sub-sending message, the first sub-receiving duration, the link identifier, the encoding and the detection flag. The detection flag in the second sub-sending message is used to indicate that the destination end needs to feed back a second sub-return message for the second sub-sending message. Exemplarily, the detection flag in the second sub-sending message can be 1.
[0151] S702, receiving the second sub-return message fed back by the destination in response to the second sub-send message; wherein the second sub-return message includes: a fourth sub-send timestamp, a second sub-send duration and the code for the second sub-return message.
[0152] In an embodiment of the present application, the transmitting end receives the second sub-return message fed back by the destination end in response to the second sub-sending message. The second sub-return message includes: a fourth sub-sending timestamp, a second sub-sending duration, and the encoding for the second sub-return message; the second sub-sending duration is determined based on the third sub-sending timestamp of the transmitting end and the third sub-receiving timestamp for the second sub-sending message.
[0153] In the embodiment of the present application, the destination terminal actively triggers the second sub-transmission message containing the sQP AETH header by judging the change of Ts (first sub-transmission duration) and / or Tr (first sub-reception duration) (Ts increases to exceed the first threshold, Tr is less than the second threshold). At the same time, the sender sets the detection flag in the sQP BTH header information as needed according to the change of Ts or Tr and the state of the sender (such as whether to measure the basic RTTbn), and requests the destination terminal to confirm. The Dest sQP and CPSN in the sQP BTH and AETH headers are generated and carried based on the aforementioned chunk namespace and message sequence number independent encoding mechanism.
[0154] S703: Determine the second sub-receiving duration of the second sub-return message based on the fourth sub-receiving timestamp and the fourth sub-sending timestamp of receiving the second sub-return message, and determine the initial round-trip delay based on the second sub-receiving duration and the second sub-sending duration.
[0155] In the embodiment of the present application, the transmitting end determines the second sub-receiving duration of the second sub-return message based on the fourth sub-receiving timestamp of the second sub-return message received minus the fourth sub-sending timestamp, and determines the initial round-trip delay by the sum of the second sub-receiving duration and the received second sub-sending duration.
[0156] In the embodiment of the present application, if at least one of the first sub-sending duration and the first sub-receiving duration is abnormal, the next historical data sub-block is encapsulated in the second sub-sending message and sent to the destination. The destination receives the second sub-sending message in response to the second sub-sending message and the second sub-return message fed back. Based on the fourth sub-receiving timestamp and the fourth sub-sending timestamp of the second sub-return message, the second sub-receiving duration of the second sub-return message is determined, and the initial round-trip delay is determined based on the second sub-receiving duration and the second sub-sending duration. When at least one of the first sub-sending duration and the first sub-receiving duration is abnormal, it means that the state of the physical node of the corresponding transmission link has changed. In the embodiment of the present application, after the transmission link changes, the corresponding initial round-trip delay can be quickly determined by sending the next historical data sub-block to provide the sending end with timely and accurate congestion degree calculation parameters. Then the sending end can divide and transmit the data through timely and accurate congestion degree information, thereby realizing the full utilization of the bandwidth of multiple equivalent transmission links in the wide area network and the reliability brought by multiple equivalent paths, avoiding congestion caused by excessive load of individual transmission links, and improving the transmission efficiency of subsequent data.
[0157] See also Fig.16 , which is a structural diagram of a data processing device provided in an embodiment of the present application.
[0158] The embodiment of the present application further provides a data processing device 800 , which is applied to a sending end and includes: a receiving unit 801 , a data dividing unit 802 and a sending unit 803 .
[0159] The receiving unit 801 is used to receive a data transmission request; wherein the data transmission request includes: data to be processed;
[0160] The data partitioning unit 802 is configured to partition the to-be-processed data into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link;
[0161] The sending unit 803 is configured to send the corresponding data block to a destination end through each transmission link.
[0162] In the embodiment of the present application, the data partitioning unit 802 in the data processing device 800 is used to determine the size of the data block corresponding to each transmission link based on the difference between the determined round-trip delay of each transmission link and the initial round-trip delay; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link; the round-trip delay is determined when each transmission link transmits the previous data;
[0163] The data to be processed is divided based on the size of each of the data blocks, and the data block corresponding to each of the transmission links is determined.
[0164] In an embodiment of the present application, the receiving unit 801 in the data processing device 800 is used to obtain the round-trip delay and the initial round-trip delay corresponding to each of the transmission links from the network control node, or to obtain the difference between the round-trip delay and the initial round-trip delay corresponding to each of the transmission links from the network control node.
[0165] In the embodiment of the present application, the sending unit 803 in the data processing device 800 is used to perform light-load round-trip delay detection when each of the transmission links is initialized, and determine the initial round-trip delay corresponding to each of the transmission links.
[0166] In the embodiment of the present application, the sending unit 803 in the data processing device 800 is used to determine the round-trip delay corresponding to each transmission link during the transmission process of the previous data on each transmission link based on the historical congestion level.
[0167] In the embodiment of the present application, the sending unit 803 in the data processing device 800 is used to divide the previous data into historical data blocks corresponding to each of the transmission links based on the historical congestion level of each of the transmission links; wherein the size of the historical data block is inversely proportional to the historical congestion level;
[0168] Encapsulating the historical data block in a transmission message, and sending the message to the destination end through each transmission link;
[0169] Receiving a return message fed back by the destination end in response to the sent message;
[0170] The round-trip delay is determined based on the transmission duration of the sent message and the returned message.
[0171] In the embodiment of the present application, the sending unit 803 in the data processing device 800 is used to divide each of the historical data blocks into a plurality of historical data sub-blocks based on the maximum transmission unit of each of the transmission links;
[0172] Each historical data sub-block is encapsulated in a sub-sending message according to the sending order of multiple historical data sub-blocks, and sent to the destination end; wherein the sub-sending message includes: the first sub-sending timestamp of the sub-sending message, the sub-receiving duration for the previous sub-return message, the link identifier, the encoding and the detection flag; the detection flag is used to indicate whether the destination end needs to feedback the sub-return message for the sub-sending message.
[0173] In the embodiment of the present application, the codes in each of the sub-transmission messages in the same transmission link are monotonically increased, and the codes in the sub-transmission messages of different transmission links are decoupled from each other.
[0174] In an embodiment of the present application, the receiving unit 801 in the data processing device 800 is used to receive the first sub-return message fed back by the destination end in response to the first sub-sending message; wherein the first sub-return message includes: the second sub-sending timestamp, the first sub-sending duration and the corresponding encoding in the first sub-sending message of the first sub-return message; the first sub-sending duration is determined based on the first sub-sending timestamp and the first sub-receiving timestamp of the corresponding first sub-sending message; the first sub-sending message is the detection flag used to indicate that the destination end needs to feed back the first sub-return message in response to the first sub-sending message.
[0175] In an embodiment of the present application, the sending unit 803 in the data processing device 800 is used to determine the first sub-receiving duration of each first sub-return message based on the second sub-receiving timestamp and the second sub-sending timestamp of receiving the first sub-return message, and determine the round-trip delay based on each first sub-receiving duration and the corresponding first sub-sending duration.
[0176] In an embodiment of the present application, the sending unit 803 in the data processing device 800 is used to encapsulate the next sub-block of the historical data in a second sub-sending message and send it to the destination end if at least one of the first sub-sending duration and the first sub-receiving duration is abnormal; wherein the second sub-sending message includes: a third sub-sending timestamp of the second sub-sending message, the first sub-receiving duration, the link identifier, the encoding and the detection flag; the detection flag in the second sub-sending message is used to indicate that the destination end needs to feedback a second sub-return message for the second sub-sending message; in an embodiment of the present application, the receiving unit 803 in the data processing device 800 is used to encapsulate the next sub-block of the historical data in a second sub-sending message and send it to the destination end if at least one of the first sub-sending duration and the first sub-receiving duration is abnormal; wherein the second sub-sending message includes: a third sub-sending timestamp of the second sub-sending message, the first sub-receiving duration, the link identifier, the encoding and the detection flag; the detection flag in the second sub-sending message is used to indicate that the destination end needs to feedback a second sub-return message for the second sub-sending message; Element 801 is used to receive the second sub-return message fed back by the destination in response to the second sub-send message; wherein the second sub-return message includes: a fourth sub-send timestamp, a second sub-send duration and the encoding for the second sub-return message; the second sub-send duration is determined based on the third sub-send timestamp of the sender and the third sub-receive timestamp for the second sub-send message; based on the fourth sub-receive timestamp and the fourth sub-send timestamp of receiving the second sub-return message, the second sub-receive duration of the second sub-return message is determined, and the initial round-trip delay is determined based on the second sub-receive duration and the second sub-send duration.
[0177] In an embodiment of the present application, the sending unit 803 in the data processing device 800 is used to divide each of the data blocks into multiple data sub-blocks based on the maximum transmission unit of each of the transmission links; encapsulate each of the data sub-blocks in the current sending message and send it to the destination.
[0178] It should be noted that in the embodiments of the present application, if the above-mentioned method for processing item information is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium and includes several instructions for enabling an item information processing device (which can be a personal computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.
[0179] Correspondingly, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method on the data processing device 800 side are implemented.
[0180] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.
[0181] It should be noted that Fig.17 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Fig.17 As shown, an embodiment of the present application provides an electronic device 900, including a memory 902 and a processor 901, wherein the memory 902 stores a computer program that can be run on the processor 901, and the processor 901 implements the steps in the above method when executing the program, wherein;
[0182] The processor 901 generally controls the overall operation of the electronic device 900 .
[0183] The memory 902 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or processed by the processor 901 and various modules in the electronic device 900 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).
[0184] Correspondingly, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the processor 901 of the electronic device 900 to complete the steps in the method on the data processing device 800 side.
[0185] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.
[0186] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0187] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0188] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0189] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0190] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a magnetic disk or an optical disk, and other media that can store program codes.
[0191] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a mobile storage device, a ROM, a magnetic disk, or an optical disk.
[0192] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data processing method, characterized in that: Applied to the sending end, including: receiving a data transmission request; wherein the data transmission request includes: data to be processed; Based on the congestion level of each transmission link, the data to be processed is divided into data blocks corresponding to each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link; The corresponding data block is sent to the destination end through each transmission link.
2. The data processing method according to claim 1, characterized in that: The step of dividing the to-be-processed data into data blocks corresponding to each transmission link based on the congestion level of each transmission link includes: Based on the determined difference between the round-trip delay of each transmission link and the initial round-trip delay, determine the size of the data block corresponding to each transmission link; wherein the difference between the round-trip delay and the initial round-trip delay is inversely proportional to the size of the data block corresponding to the transmission link; the round-trip delay is determined when each transmission link transmits the previous data; The data to be processed is divided based on the size of each of the data blocks, and the data block corresponding to each of the transmission links is determined.
3. The data processing method according to claim 2, characterized in that: Before determining the size of the data block corresponding to each of the transmission links based on the difference between the determined round-trip delay of each of the transmission links and the initial round-trip delay, the method further includes: The round trip delay and the initial round trip delay corresponding to each of the transmission links are obtained from a network control node, or a difference between the round trip delay and the initial round trip delay corresponding to each of the transmission links is obtained from the network control node.
4. The data processing method according to claim 2, characterized in that: Before determining the size of the data block corresponding to each of the transmission links based on the difference between the determined round-trip delay of each of the transmission links and the initial round-trip delay, the method further includes: When each of the transmission links is initialized, a light-load round-trip delay detection is performed to determine the initial round-trip delay corresponding to each of the transmission links.
5. The data processing method according to claim 2, characterized in that: Before determining the size of the data block corresponding to each of the transmission links based on the difference between the determined round-trip delay of each of the transmission links and the initial round-trip delay, the method further includes: In the transmission process of the previous data on each transmission link based on the historical congestion level, the round trip delay corresponding to each transmission link is determined.
6. The data processing method according to claim 5, characterized in that: The determining, during the transmission process of the previous data on each transmission link based on the historical congestion degree, of the round trip delay corresponding to each transmission link comprises: Based on the historical congestion level of each transmission link, the previous data is divided into historical data blocks corresponding to each transmission link; wherein the size of the historical data block is inversely proportional to the historical congestion level; Encapsulating the historical data block in a transmission message, and sending the message to the destination end through each transmission link; Receiving a return message fed back by the destination end in response to the sent message; The round-trip delay is determined based on the transmission duration of the sent message and the returned message.
7. The data processing method according to claim 6, characterized in that: The encapsulating the historical data block in a sending message and sending the message to the destination through each transmission link includes: Based on the maximum transmission unit of each transmission link, each of the historical data blocks is divided into a plurality of historical data sub-blocks; Each historical data sub-block is encapsulated in a sub-sending message according to the sending order of multiple historical data sub-blocks, and sent to the destination end; wherein the sub-sending message includes: the first sub-sending timestamp of the sub-sending message, the sub-receiving duration for the previous sub-return message, the link identifier, the encoding and the detection flag; the detection flag is used to indicate whether the destination end needs to feedback the sub-return message for the sub-sending message.
8. The data processing method according to claim 7, characterized in that: The codes in the sub-transmission messages in the same transmission link are monotonically increased, and the codes in the sub-transmission messages of different transmission links are decoupled from each other.
9. The data processing method according to claim 7, characterized in that: The receiving of the return message fed back by the destination end in response to the sent message includes: Receive the first sub-return message fed back by the destination in response to the first sub-sending message; wherein the first sub-return message includes: the second sub-sending timestamp of the first sub-return message, the first sub-sending duration and the corresponding encoding in the first sub-sending message; the first sub-sending duration is determined based on the first sub-sending timestamp and the first sub-receiving timestamp of the corresponding first sub-sending message; the first sub-sending message is the detection flag used to indicate that the destination end needs to feed back the first sub-return message in response to the first sub-sending message.
10. The data processing method according to claim 9, characterized in that: The determining the round trip delay based on the transmission duration of the sent message and the returned message includes: The first sub-receiving duration of each first sub-return message is determined based on the second sub-receiving timestamp and the second sub-sending timestamp of receiving the first sub-return message, and the round-trip delay is determined based on each first sub-receiving duration and the corresponding first sub-sending duration.
11. The data processing method according to claim 9, characterized in that: The method further comprises: If at least one of the first sub-sending duration and the first sub-receiving duration is abnormal, the next sub-block of the historical data is encapsulated in a second sub-sending message and sent to the destination end; wherein the second sub-sending message includes: a third sub-sending timestamp of the second sub-sending message, the first sub-receiving duration, the link identifier, the encoding and the detection flag; the detection flag in the second sub-sending message is used to indicate that the destination end needs to feedback a second sub-return message for the second sub-sending message; Receive the second sub-return message fed back by the destination in response to the second sub-send message; wherein the second sub-return message includes: a fourth sub-send timestamp, a second sub-send duration and the encoding for the second sub-return message; the second sub-send duration is determined based on the third sub-send timestamp of the sender and the third sub-receive timestamp for the second sub-send message; Based on the fourth sub-receiving timestamp and the fourth sub-sending timestamp of receiving the second sub-return message, the second sub-receiving duration of the second sub-return message is determined, and the initial round-trip delay is determined based on the second sub-receiving duration and the second sub-sending duration.
12. The data processing method according to any one of claims 1 to 11, characterized in that: The sending the corresponding data block to the destination end through each transmission link includes: Based on the maximum transmission unit of each transmission link, each of the data blocks is divided into a plurality of data sub-blocks; Each of the data sub-blocks is encapsulated in a current sending message and sent to the destination end.
13. A data processing device, characterized in that: Applied to the sending end, including: A receiving unit, configured to receive a data transmission request; wherein the data transmission request includes: data to be processed; A data partitioning unit, configured to partition the to-be-processed data into data blocks corresponding to each transmission link based on the congestion level of each transmission link; wherein the size of the data block is inversely proportional to the congestion level of the corresponding transmission link; A sending unit is used to send the corresponding data block to a destination end through each transmission link.
14. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 12 when executing the computer program.
15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 12 are implemented.
16. A computer program product comprising a computer program, characterized in that The computer program implements the steps of the method according to any one of claims 1 to 12 when executed by a processor.
Citation Information
Patent Citations
Video processing method and system
CN106792270A
Data packet distribution method and device, mobile terminal and storage medium
CN109803318A
Shunting method for multi-connection communication
CN112187654A
Data message transmission method and device, electronic equipment and computer storage medium
CN113992307A
Data transmission method, device and equipment, transmission network management and control system and terminal
CN114513865A
