Method for controlling re-injection of multipath transmission, electronic device and storage medium

Abstract

The embodiment of the application provides a kind of multi-path transmission re-injection control method, electronic equipment and storage medium, in the re-injection process of multi-path transmission in the embodiment, the expected transmission time corresponding to application request is perceived, and the expected transmission time corresponding to application request is used as the time basis for allowing to execute re-injection operation, so that the data packet on the non-idle path can be retransmitted (i.e. execute re-injection operation) using the idle path in multi-path at more reasonable time. Since re-injection operation is executed in multi-path transmission, the performance of multi-path transmission can be guaranteed, and since the execution of re-injection operation is based on the expected transmission time corresponding to application request, the user experience of the application can be guaranteed, and the data packet executed by re-injection can be reduced to a certain extent, the amount of redundant data in multi-path transmission is reduced, and traffic resources are saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a re-injection control method for multi-path transmission, an electronic device and a storage medium. BACKGROUND

[0002] With the development of wireless technology and mobile terminals, a terminal device can simultaneously use two or more paths (such as Wi-Fi and 4G) for data transmission, which is referred to as multi-path transmission. Using multi-path for data transmission can improve the throughput of data transmission by utilizing the bandwidth of multiple paths, and can also improve the robustness of data transmission, that is, when the performance of one path decreases, data transmission can be completed on other paths.

[0003] The most important performance problem in multi-path transmission is the head-of-line blocking (MP-HoL Blocking) problem of multi-path, that is, when one path encounters blocking and cannot transmit the data packets allocated thereto, the transmission layer cannot deliver data to the application layer due to the absence of these data packets, and at this time, subsequent data packets cannot be sent using other paths, thereby causing the overall transmission performance to decrease.

[0004] Re-injection is widely used to solve the head-of-line blocking problem, and the principle is to retransmit data packets on a non-idle path using an idle path. In the case where the performance of the idle path is better than that of the blocked path, the transmission can be completed in advance, and the transmission performance of the multi-path can be effectively improved. However, re-injection can cause redundancy of transmitted data, resulting in waste of traffic resources. Therefore, how to reasonably perform re-injection to reduce the amount of redundant data transmitted while ensuring the performance of multi-path transmission is a difficult problem faced by multi-path transmission. SUMMARY

[0005] Aspects of the present application provide a re-injection control method for multi-path transmission, an electronic device and a storage medium, to reduce the amount of redundant data transmitted while ensuring the performance of multi-path transmission.

[0006] The present application provides a re-injection control method for multi-path transmission, applied to a first end, and the method comprises the following steps.

[0007] Determine an expected transmission time corresponding to an application request, the application request being used to request to-be-transmitted data from the first end; send data packets in the to-be-transmitted data to a second end through multiple transmission paths between the first end and the second end; and according to the expected transmission time, retransmit data packets on a transmission path in a non-idle state to the second end through a transmission path in an idle state.

[0008] The embodiment of the present application provides a kind of re-injection control device of multipath transmission, it is applied to first end, the device includes: determination module, for determining the expected transmission time corresponding to application request, the application request is used to request to be transmitted data;Multipath transmission module, for sending the data packet in the to-be-transmitted data to the second end by the multiple transmission paths between the first end and the second end;Re-injection control module, for according to the expected transmission time, by the transmission path in idle state to the second end re-sending the data packet on the transmission path in non-idle state.

[0009] The embodiment of the present application also provides an electronic device, comprising: a memory and a processor;The memory is used to store computer programs;The processor is coupled with the memory, and is used to execute the computer programs to execute the steps in the re-injection control method of multipath transmission.

[0010] The embodiment of the present application also provides a computer readable storage medium storing computer programs, when the computer programs are executed by the processor, the processor can realize the steps in the re-injection control method of multipath transmission.

[0011] In the embodiment, in the re-injection process of multipath transmission, the expected transmission time corresponding to application request is perceived, and the expected transmission time corresponding to application request is used as the time basis for allowing the execution of re-injection operation, so that the idle path in the multipath can be used to retransmit (i.e. execute re-injection operation) the data packet on the non-idle path at a more reasonable time. Since the re-injection operation is executed in the multipath transmission, the performance of the multipath transmission can be guaranteed. Since the execution of the re-injection operation is based on the expected transmission time corresponding to the application request, the user experience of the application can be guaranteed, and the data packets executed by the re-injection can be reduced to a certain extent, the amount of redundant data in the multipath transmission can be reduced, and the flow resources can be saved. BRIEF DESCRIPTION OF DRAWINGS

[0012] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0013] Figure 1a A process diagram of exemplary multipath transmission is shown in FIG. 1.

[0014] Figure 1b A schematic diagram of multipath transmission from a client device to a server device is shown in FIG. 2.

[0015] Figure 1c A schematic diagram of multipath transmission from a server device to a client device is shown in FIG. 3.

[0016] Figure 2a A flow chart of a multi-path transmission re-injection control method provided by an embodiment of the present application is shown in FIG. 1.

[0017] Figure 2b A state diagram in which the expected transmission time is earlier than the sending completion time is shown in FIG. 4.

[0018] Figure 2c A state diagram in which the expected transmission time is earlier than the sending completion time is shown in FIG. 4.

[0019] Figure 3 An application scenario is shown in FIG. 5, taking adaptive video streaming as an example.

[0020] Figure 4 A structural diagram of a multi-path transmission re-injection control device provided by an embodiment of the present application is shown in FIG. 6.

[0021] Figure 5 A structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in detail with reference to the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0023] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0024] The following briefly introduces several terms related to the embodiments of the present application:

[0025] MPTCP (Multipath TCP): is a multi-path TCP (Transmission Control Protocol, Transmission Control Protocol), which uses multiple transmission paths to transmit data compared with the traditional single-path TCP, so as to improve the utilization rate of bandwidth resources.

[0026] UDP (User Datagram Protocol): A transport layer protocol that can send encapsulated data packets without establishing a connection.

[0027] QUIC (Quick Udp Internet Connection): A transport layer protocol based on UDP.

[0028] MPQUIC (Mutipath QUIC): Supports simultaneous transmission of data on multiple QUIC-based transmission paths, improving overall transmission efficiency through multi-path transmission.

[0029] QoE (Quality of Experience): Reflects the quality of service provided by devices, networks, systems, or applications to users. In adaptive video streaming, QoE typically includes video quality, stall time, and smoothness of video quality.

[0030] DASH (Dynamic Adaptive Streaming over HTTP): A transmission protocol for adaptive streaming media.

[0031] ABR (Adaptive BitRate) algorithm: An algorithm that dynamically selects the quality level of video blocks based on network environment and playback buffer level.

[0032] Adaptive video streaming: A video stream whose bitrate can be dynamically adjusted based on network environment and playback buffer level. For on-demand scenarios (such as under the DASH standard), the adjustment bitrate granularity is usually at the video block level.

[0033] Reinjection: In a multi-path transmission scenario, retransmit data packets on other transmission paths to optimize multi-path transmission performance. The reinjected data packets can be any unsent or sent but not acknowledged by the receiving end.

[0034] RTT (Round-Trip Time): The time delay experienced from the start of sending data from the sender to receiving an acknowledgement from the receiver.

[0035] ACK (Acknowledgement): In data communication, a transmission control message sent by the receiver to the sender indicating that the received data has been acknowledged.

[0036] Bit rate: also known as bit rate or bit speed, is the amount of data (such as video data) transmitted per unit of time. The unit is bps (bit per second), and kbps (kilobits per second) or Mbps (megabits per second) are commonly used.

[0037] Application: An application that provides application services to users, has user interface functions for network transmission, and can provide communication services between itself and peer applications through the transport layer. The application resides in the application layer as defined by the network protocol.

[0038] Transport layer: This is the transport layer defined in network protocols, primarily responsible for providing communication services between applications at both ends. Since multiple applications may run simultaneously on the same end, the transport layer has multiplexing and demultiplexing functions.

[0039] In practical applications, multipath transmission methods, such as QUIC (MPQUIC), are commonly used in video services or other services involving large-scale data transmission. This leverages the bandwidth of multiple paths to increase data throughput and improves robustness, allowing data transmission to continue even when one path's performance degrades. However, research has shown that multipath head-of-line congestion degrades multipath transmission performance. This is because when head-of-line congestion occurs, data packets must wait on non-idle paths, making multipath transmission performance dependent on the slowest path.

[0040] by Figure 1a For example, the packet scheduler in the server device schedules two paths, sending packets to the client device from two different paths. Some packets, such as... Figure 1a Data packets 2, 3, and 4 are transmitted via a relatively fast path, allowing them to arrive at the client device earlier; some data packets, such as... Figure 1a Data packet 1 in the multipath queue is transmitted via a relatively slow, non-idle path, resulting in it arriving at the client device later. Even though data packet 1 was sent earlier than data packets 2, 3, and 4, due to the non-idle path, data packets 2, 3, and 4, which have already arrived at the client device, must wait for data packet 1 to arrive before they can be submitted to the application on the client device for processing. This is known as multipath head-of-line blocking. Therefore, multipath head-of-line blocking leads to a decrease in the overall transmission performance of multipath systems.

[0041] Reinjection is widely used to solve the head-of-line blocking problem. The principle is to use the idle path to retransmit the data packets on the non-idle path. In the case that the performance of the idle path is better than that of the blocked path, the transmission can be completed in advance, effectively improving the transmission performance of the multi-path. Taking the client device requesting the server device to transmit a video block as an example, the client device requests the server device for a video block, and the server device sends all data packets of the video block to the client device through the multiple transmission paths between the server device and the client device according to the request of the client device. After sending all the data packets, the server device waits for the last 1 RTT (the time for returning ACK to the client device for the last sent data packet) for the return of ACK. At this time, the server device starts to perform the reinjection operation. When performing the reinjection operation, the server device checks which data packets on the non-idle path have not received ACK. If there are data packets on the non-idle path that have not received ACK and the fast path is in an idle state, the data packets that have not received ACK on the non-idle path can be retransmitted through the fast path so as to be quickly transmitted to the client device, thereby improving the overall transmission efficiency.

[0042] In the reinjection process, the reinjected data packets will cause data redundancy, and the transmission of the redundant data will consume additional traffic resources. In addition, the retransmission of the data packets needs to select a transmission path for retransmission. If the transmission performance of the selected transmission path becomes worse later, it may cause the retransmitted data packets to be blocked again, which not only consumes additional traffic, but also wastes bandwidth resources. Furthermore, after sending all the data packets, the reinjection operation is automatically performed, which may perform unnecessary reinjection, which also causes the waste of traffic and bandwidth resources. Therefore, it is necessary to provide a reinjection control method that can reduce the amount of redundant data transmitted while ensuring the performance of multi-path transmission and saving the waste of traffic and bandwidth resources caused by reinjection.

[0043] Based on the above, the embodiments of the present application provide a multi-path transmission reinjection control method, an electronic device and a storage medium. In the embodiments, the expected transmission time corresponding to the application request is introduced in the reinjection process of the multi-path transmission, and the expected transmission time corresponding to the application request is used as the time basis for allowing the reinjection operation to be performed, so that the data packets on the non-idle path can be retransmitted (i.e., the reinjection operation is performed) using the idle path in a more reasonable time. Since the reinjection operation is performed in the multi-path transmission, the performance of the multi-path transmission can be ensured. Since the performance of the reinjection operation is based on the expected transmission time corresponding to the application request, the user experience of the application can be ensured, and the data packets that are executed by the reinjection can be reduced to a certain extent, the amount of redundant data in the multi-path transmission can be reduced, and the traffic and bandwidth resources can be saved.

[0044] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0045] The re-injection control method for multipath transmission provided in this application can be applied to multipath-based data transmission between a first end and a second end. The first end can be understood as the sending end in the data transmission process, and the second end can be understood as the receiving end in the data transmission process. The first end can be, for example, a client device, and correspondingly, the second end can be, for example, a server device. That is, the re-injection control method provided in this application can be used when the client device sends data to the server device through multiple paths; or, the first end can be, for example, a server device, and the second end can be, for example, a client device. That is, the re-injection control method provided in this application can also be used when the server device sends data to the client device through multiple paths, but this is not limited to these methods.

[0046] It should be noted that in this embodiment, the first and second ends run corresponding applications. These applications can perform multipath data transmission through multiple transmission paths between them. During multipath transmission, the re-injection control method provided in this embodiment is used to solve the retransmission problem of data packets on congested paths, thereby improving the performance of multipath transmission. In other words, the multipath transmission and the re-injection control method mentioned in this embodiment are applied between the transmission layers of the first and second ends. The application on the client device can act as an application client, primarily facing the user, supporting interaction with the user, responding to user requests, and requesting relevant services or data from the application on the server device. Correspondingly, the application on the server device can act as an application server, primarily responding to and processing requests from the application client, providing various services such as data storage for the application client. Furthermore, regarding the functions implemented by the application, this embodiment does not limit the implementation form of the application; for example, it can be a video application, an email service application, an online shopping application, a game application, etc.

[0047] The following is combined Figure 1b and Figure 1c The multipath transmission scenario provided in the embodiments of this application will be described.

[0048] like Figure 1b As shown, the first end is the client, the second end is the server, and there are multiple transmission paths between the client and the server. Figure 1bThis diagram illustrates three transmission paths: Wi-Fi (Wireless Fidelity), 5G (5 Generation), and 4G (4 Generation), but it is not limited to these paths. An application on a client device needs to send first data to an application on a server device. This first data may include, for example, a data request to the application on the server device for second data, or application data that needs to be uploaded to the server device. In this case, the client device can respond to the data transmission request of the application on the client device by sending the first data to the server device through multiple transmission paths, a process referred to as multi-path transmission of the first data. During multi-path transmission of the first data, the first data can be encapsulated into at least one data packet, and the data packet within the first data can be transmitted through multiple transmission paths.

[0049] like Figure 1c As shown, the first end is the server device, the second end is the client device, and there are multiple transmission paths between the client device and the server device. Figure 1c The diagram illustrates three transmission paths: Wi-Fi, 5G, and 4G. However, it is not limited to these paths and may also include other transmission paths such as Bluetooth and infrared. An application on the server device sends second data to an application on the client device. This second data may include application data returned to the application on the client device or notification messages sent to the application on the client device. In response to the data transmission request from the application on the server device, the server device can send the second data to the client device through multiple transmission paths, a process referred to as multi-path transmission of the second data. During multi-path transmission of the second data, the second data can be encapsulated into at least one data packet, which is then transmitted through multiple transmission paths.

[0050] In the various embodiments of this application, the method of multipath transmission of the first data or the second data using multiple transmission paths is not limited. For example, the first data or the second data can be transmitted using a fully redundant transmission method or an auxiliary redundant transmission method. The fully redundant transmission method means that for any data packet to be transmitted, when the data packet is sent for the first time, the data packet is transmitted simultaneously on two or more transmission paths. The auxiliary redundant transmission method means that for any data packet to be transmitted, when the data packet is sent for the first time, one transmission path is selected and the data packet is transmitted on the selected transmission path, and data packets for which no acknowledgment information has been received from the other end are retransmitted.

[0051] In the above process of multi-path transmission of the first data or the second data, the re-injection control method provided by the embodiments of the present application can be used to retransmit the data packets on the transmission path in the non-idle state by using the transmission path in the idle state. The following will be described in different cases:

[0052] For the case of multi-path transmission of the first data, as shown in Figure 1b , the client device can determine the expected transmission time of the application to the first data according to the performance requirement of the application, the expected transmission time refers to the time length required by the application on the client device to transmit the first data to the server device; the data packets in the first data are transmitted to the server device through multiple transmission paths between the client device and the server device; the re-injection operation is started based on the expected transmission time, that is, the data packets on the transmission path in the non-idle state are retransmitted to the server device through the transmission path in the idle state.

[0053] For the case of multi-path transmission of the second data, as shown in Figure 1c , the client device determines the expected transmission time of the application to the second data according to the performance requirement of the application, the expected transmission time refers to the time length required by the application on the client device to transmit the second data from the server device to the client device; the expected transmission time of the application to the second data is transmitted to the server device through at least one of the multiple transmission paths between the client device and the server device. For the server device, the expected transmission time of the application to the second data transmitted by the client device can be received through at least one of the multiple transmission paths; the data packets in the second data are transmitted to the client device through the multiple transmission paths between the server device and the client device; the re-injection operation is started based on the expected transmission time, that is, the data packets on the transmission path in the non-idle state are retransmitted to the server device through the transmission path in the idle state.

[0054] In the embodiments of the present application, the re-injection control process for full redundancy transmission is different from the re-injection control process for auxiliary redundancy transmission, which will be briefly described below. For the case of full redundancy transmission, when the data packets on the transmission path in the non-idle state are retransmitted to the opposite end through the transmission path in the idle state, the data packets retransmitted on the transmission path in the idle state include the sent data packets on the transmission path in the non-idle state which have been sent but have not received the confirmation information from the opposite end, and the data packets on the transmission path in the non-idle state which have not been sent. For the case of auxiliary redundancy transmission, when the data packets on the transmission path in the non-idle state are retransmitted to the opposite end through the transmission path in the idle state, the data packets retransmitted on the transmission path in the idle state include the sent data packets on the transmission path in the non-idle state which have been sent but have not received the confirmation information from the opposite end.

[0055] The detailed description of the re-injection control method when the first data or the second data is transmitted in the multi-path manner can be found in the following embodiments, in which the second end requests the data from the first end, and the first end transmits the data to the second end.

[0056] Figure 2a A flowchart of the re-injection control method of the multi-path transmission provided in the embodiments of the present application. The method is applied to the first end, and the method can include the following steps: Figure 2a

[0057] 201. The first end determines an expected transmission time corresponding to an application request, and the application request is used to request the data to be transmitted from the first end to the second end.

[0058] 202. The first end sends the data packets in the data to be transmitted to the second end through the multiple transmission paths between the first end and the second end.

[0059] 203. The first end re-sends the data packets on the transmission path in the non-idle state to the second end through the transmission path in the idle state according to the expected transmission time.

[0060] In the embodiments, the first end needs to send the data to be transmitted to the second end, and the data to be transmitted is different according to the application scenarios of the first end and the second end, for example, but not limited to, audio and video data, text data, image data, application installation package or data compression package.

[0061] In actual application, the data to be transmitted is determined according to the application request sent by the second end and transmitted to the second end. In the case where the first end determines the data to be transmitted according to the application request of the second end, the second end sends the application request to the first end through at least one transmission path between the first end and the second end, the application request includes the identification information of the data to be transmitted, the first end receives the application request sent by the second end through the at least one transmission path, determines the data to be transmitted according to the identification information of the data to be transmitted included in the application request, and then transmits the data to be transmitted to the second end through the multiple transmission paths between the first end and the second end.

[0062] ​Further optionally, in some adaptive data transmission scenarios, such as adaptive video streaming scenarios, the first end can encode the to-be-transmitted data into multiple versions of different code rates, such as 5 Mbps, 8 Mbps, 16 Mbps, and the like. Different transmission code rates correspond to different application layer processing effects. Generally speaking, the higher the code rate, the better the application layer processing effect. In the adaptive video streaming scenario, the application layer processing effect refers to the video quality, resolution, and / or clarity, or can also be represented by the user's QoE. In order to better optimize the data processing effect of the application layer of the second end, the second end can also determine the code rate of the to-be-transmitted data and request the first end to return the to-be-transmitted data of the required code rate. For the convenience of description and differentiation, the code rate version requested by the first end to use is referred to as the target code rate. Based on this, the first end can also receive the target code rate sent by the second end through at least one of the multiple transmission paths between the first end and the second end. In an optional embodiment, the second end can send the target code rate in the application request to the first end, that is, the first end can receive the application request sent by the second end through at least one of the transmission paths, and the application request includes the identification information of the to-be-transmitted data and the target code rate. From the to-be-transmitted data corresponding to multiple code rates, the to-be-transmitted data corresponding to the target code rate is determined, so that the to-be-transmitted data corresponding to the target code rate can be sent to the second end.

[0063] Taking adaptive video streaming as an example, the adaptive video streaming includes multiple video blocks, and each video block can be encoded into multiple versions of different code rates, and different code rates correspond to different clarity and quality in terms of playing effect. Different code rates, such as 5 Mbps, 8 Mbps, and 16 Mbps, correspond to resolutions of 480P, 720P, and 1080P respectively, and the higher the resolution, the higher the picture quality (such as clarity) of video playing. In the adaptive video streaming scenario, the first end is a server device, the second end is a client device, and the to-be-transmitted data is any video block. The client device can run an ABR algorithm to dynamically select the target code rate of the to-be-transmitted video block based on the network throughput of the application (i.e., the player) on the client device. Since the target code rate is adapted to the network throughput of the player, the video playing quality is improved, and the QoE of the player is also improved.

[0064] In the above embodiments, the second end does not limit the at least one transmission path used by the second end to send the application request to the first end. Alternatively, the second end can randomly select one transmission path from a plurality of transmission paths, or can select a transmission path with better quality from the plurality of transmission paths according to path quality parameters of the plurality of transmission paths, or can use a plurality of transmission paths to perform multi-path transmission of the application request, such as a fully redundant multi-path transmission mode or an auxiliary redundant multi-path transmission mode, and perform the re-injection operation on the transmission process of the application request in the multi-path transmission by using the re-injection control method provided in the embodiments of the present application.

[0065] In the present embodiment, the first end needs to determine the expected transmission time corresponding to the application request. In the case where the first end is a server device and the second end is a client device, the expected transmission time corresponding to the application request can be obtained by the client device (i.e., the second end). The expected transmission time refers to the time length that the client device (i.e., the second end) hopes to take for the transmission of the to-be-transmitted data from the server device to the client device, which is counted from the beginning of the transmission of the first data packet of the to-be-transmitted data by the server device.

[0066] In the present embodiment, the first end needs to determine the expected transmission time corresponding to the application request. In the case where the first end is a server device and the second end is a client device, the expected transmission time corresponding to the application request can be obtained by the client device (i.e., the second end). The expected transmission time refers to the time length that the client device (i.e., the second end) hopes to take for the transmission of the to-be-transmitted data from the server device to the client device, which is counted from the beginning of the transmission of the first data packet of the to-be-transmitted data by the server device.

[0067] Further, when generating the expected transmission time, a certain numerical calculation can be performed on the network throughput and the data size of the to-be-transmitted data to obtain the expected transmission time. Alternatively, the ratio of the size of the to-be-transmitted data to the network throughput can be taken as the expected transmission time. Assuming that the expected transmission time is denoted as D, the size of the to-be-transmitted data is denoted as K, and the network throughput of the application is denoted as T, then D = K / T. Alternatively, the size of the to-be-transmitted data can be multiplied by a certain weight to obtain the expected transmission time according to the formula D = w*K / T, where w is the weight. In the case where the to-be-transmitted data is a video block, the size K of the video block can be estimated according to the target code rate S corresponding to the video block, and the size of the video block can be obtained according to the formula K = S*L, where L is the time length of the video block.

[0068] For the case that the first end is a server device and the second end is a client device, the second end needs to send the expected transmission time corresponding to the application request to the first end. Optionally, the second end can send the expected transmission time to the first end through at least one of the plurality of transmission paths, and correspondingly, the first end can receive the expected transmission time sent by the second end through at least one of the plurality of transmission paths. In this embodiment, the at least one transmission path used by the second end to send the expected transmission time to the first end is not limited. Optionally, the second end can randomly select a transmission path from the plurality of transmission paths, or can select a transmission path with better quality from the plurality of transmission paths according to the path quality parameters of the plurality of transmission paths, or can use the plurality of transmission paths to perform multi-path transmission of the expected transmission time, such as a fully redundant multi-path transmission mode or an auxiliary redundant multi-path transmission mode, and perform re-injection operation on the transmission process of the expected transmission time by using the re-injection control method provided in this embodiment.

[0069] Further, in the case that the first end is a server device and the second end is a client device, the application embodiment does not limit the way in which the second end sends the expected transmission time to the first end, nor the order in which the second end sends the expected transmission time to the first end and sends the application request to the first end. For example, the existing QoE frame of the transmission layer protocol can be used to carry the expected transmission time, and the QoE frame is sent to the second end through at least one transmission path. Correspondingly, the second end receives the QoE frame sent by the first end through at least one transmission path, and parses the expected transmission time from the QoE frame. The QoE frame is independent of the application request sent by the second end to the first end, and the application request is an HTTP request, which can be determined according to the transmission layer protocol used. Alternatively, the application request can be expanded to add a new field to carry the expected transmission time, and the first end carries the expected transmission time in the expanded field of the application request and sends it to the second end through at least one transmission path. Correspondingly, the second end receives the application request sent by the first end through at least one transmission path, and obtains the identification information of the to-be-transmitted data, the target code rate and the expected transmission time from the application request.

[0070] In this embodiment, the transmission layer protocol used by the plurality of transmission paths is not limited, for example, but not limited to: QUIC, UDP or TCP, etc. Further optionally, in order to better improve the QoE of the application layer, the transmission layer protocol used by the plurality of transmission paths is QUIC, and the multi-path transmission is MPQUIC.

[0071] In the embodiment, the to-be-transmitted data is encapsulated into at least one data packet, and the first end can send the data packets in the to-be-transmitted data to the second end through a plurality of transmission paths between the first end and the second end. In actual application, when the data packets in the to-be-transmitted data are sent to the second end through the plurality of transmission paths, a full redundancy transmission mode or an auxiliary redundancy transmission mode can be used. That is, for any data packet in the at least one data packet, the first end can send the data packet to the second end through part of the plurality of transmission paths to implement auxiliary redundancy transmission, or send the data packet to the second end through all of the plurality of transmission paths to implement full redundancy transmission.

[0072] In the embodiment, the first end controls the re-injection of the multi-path transmission process of the to-be-transmitted data according to the expected transmission time, that is, the first end re-sends the data packets on the transmission paths in the non-idle state to the second end through the transmission paths in the idle state according to the expected transmission time, so as to improve the multi-path transmission performance through the re-injection operation.

[0073] Further optionally, when the first end re-sends the data packets on the transmission paths in the non-idle state to the second end through the transmission paths in the idle state according to the expected transmission time, the sending completion time of the to-be-transmitted data can be obtained; the re-injection execution time is determined according to the expected transmission time and the sending completion time; and the data packets on the transmission paths in the non-idle state are re-sent to the second end through the transmission paths in the idle state when the re-injection execution time arrives.

[0074] In the embodiment, the sending completion time of the to-be-transmitted data refers to the time when the first end sends the last data packet contained in the to-be-transmitted data. It should be noted that the sending completion time is not equal to the transmission completion time of the at least one data packet. Specifically, after the first end transmits the last data packet in the at least one data packet to the second end, the first end records the sending time of the last data packet, and takes the recorded time as the sending completion time of the to-be-transmitted data, which is commonly referred to as the last RTT time. Of course, the sending completion time of the at least one data packet in the embodiment can also be defined according to other modes, for example, a preset floating time can be added to the sending time of the last data packet as the sending completion time of the to-be-transmitted data, which is not limited.

[0075] In the embodiment, the first end determines the re-injection execution time according to the expected transmission time and the sending completion time of the data to be transmitted, and performs the re-injection operation when the re-injection execution time arrives. In actual application, the re-injection execution time can be the sending completion time or later than the sending completion time. When the re-injection execution time arrives, the first end performs the re-injection operation. Since the expected transmission time is determined according to the performance requirement of the application, the re-injection execution time determined in this way can meet the performance requirement of the application. Optionally, the performance requirement of the application can be represented by some performance parameters that can reflect user experience. In addition, when the re-injection execution time is later than the sending completion time, compared with performing the re-injection operation according to the sending completion time, the time for performing the re-injection operation is delayed, which can leave more transmission time for the data packets on the multiple transmission paths on the basis of meeting the performance requirement of the application, so that more data packets can be transmitted to the second end before the re-injection operation is performed, thereby reducing the number of data packets that need to be retransmitted when the re-injection operation is performed, reducing the amount of redundant data, and thereby saving the flow and bandwidth resources.

[0076] Further optionally, in order to better control the re-injection, when the re-injection execution time is determined according to the expected transmission time and the sending completion time, the re-injection degree adjustment parameter can be used to adjust the re-injection execution time. The re-injection degree adjustment parameter is a preset value or is obtained in real time. When the re-injection execution time is later than the sending completion time, the re-injection execution time is determined as the re-injection execution time. Further optionally, when the re-injection execution time is earlier than the sending completion time, the sending completion time is determined as the re-injection execution time. In this case, the re-injection operation is performed according to the sending completion time, which can give priority to the normal sending process of the data to be transmitted and reduce the impact of the re-injection operation on the normal data transmission process.

[0077] In the embodiment, the re-injection degree adjustment parameter is used to control the aggressiveness of the re-injection. Through the re-injection degree adjustment parameter, the time for allowing the re-injection operation can be controlled to a certain extent. For example, if the re-injection degree adjustment parameter is 1, it means that the re-injection operation is allowed when the expected transmission time arrives. If the re-injection degree adjustment parameter is less than 1, it means that the re-injection operation is allowed before the expected transmission time (which can be understood as allowing the re-injection in advance). If the re-injection degree adjustment parameter is greater than 1, it means that the re-injection operation is allowed after the expected transmission time (which can be understood as allowing the re-injection with a delay). In the embodiment, through the re-injection degree adjustment parameter, the aggressiveness of the re-injection can be flexibly controlled before, after or at the expected transmission time according to the application scenario or the application requirement, which has higher flexibility.

[0078] With respect to the expected transmission time, the final determined re-injection execution time can be earlier than the sending completion time or later than the sending completion time, and the relatively later time is taken as the re-injection execution time, whether in the case of early re-injection permission or in the case of late re-injection permission. Figure 2b In the case of early re-injection permission, the relationship between the re-injection permission time, the expected transmission time, the re-injection execution time and the sending completion time is exemplarily illustrated in Figure 2c Figure 2b As shown in, the expected transmission time is later than the sending completion time, and in the case of early re-injection permission, the re-injection permission time is earlier than the expected transmission time but later than the sending completion time, so the re-injection permission time is taken as the re-injection execution time, which means that the re-injection operation is executed at the re-injection permission time. Figure 2c As shown in , the expected transmission time is earlier than the sending completion time, and in the case of early re-injection permission, the re-injection permission time is earlier than the expected transmission time and earlier than the sending completion time, so the sending completion time is taken as the re-injection execution time, which means that the re-injection operation is executed at the sending completion time.

[0079] Further optionally, in order to better control the re-injection, when the re-injection degree adjustment parameter is acquired in real time, the network state information of the multiple transmission paths and / or the buffer level of the data buffer area used by the application layer can be acquired in real time when the expected transmission time is determined; and the re-injection degree adjustment parameter is determined according to the network state information and / or the buffer level of the data buffer area.

[0080] If the transmission speed of the transmission path is fast based on the network state information, the re-injection degree adjustment parameter can be adjusted in the direction of late re-injection permission, that is, the re-injection degree adjustment parameter is determined to be greater than 1, and the greater the value of the re-injection degree adjustment parameter, the later the re-injection operation is allowed to be executed, as long as the performance requirement of the application layer can be met; if the transmission speed of the transmission path is slow based on the network state information, the re-injection degree adjustment parameter can be adjusted in the direction of early re-injection permission, that is, the re-injection degree adjustment parameter is determined to be less than 1, and the smaller the value of the re-injection degree adjustment parameter, the earlier the re-injection operation is allowed to be executed, mainly in order to meet the performance requirement of the application layer.

[0081] The cache level of the data cache area reflects the amount of data cached by the data cache area. If the amount of cached data is small, in order to allow the application layer to have enough data for processing, the re-injection degree adjustment parameter can be adjusted in the direction of allowing re-injection in advance, that is, the re-injection degree adjustment parameter is determined to be less than 1, and the smaller the value of the re-injection degree adjustment parameter, the earlier the re-injection operation is allowed to be performed, which is beneficial to transmitting data required by the application layer as soon as possible and meeting the performance requirement of the application program. If the amount of cached data is large, the re-injection degree adjustment parameter can be adjusted in the direction of allowing re-injection in the future, that is, the re-injection degree adjustment parameter is determined to be greater than 1, and the greater the value of the re-injection degree adjustment parameter, the later the re-injection operation is allowed to be performed, as long as the performance requirement of the application layer can be met.

[0082] In actual applications, various types of numerical calculations such as multiplication and addition can be performed on the expected transmission time and the re-injection degree adjustment parameter to obtain the allowed re-injection time. Further, in order to better control re-injection, when generating the allowed re-injection time according to the expected transmission time and the re-injection degree adjustment parameter, the expected transmission time and the re-injection degree adjustment parameter can be subjected to numerical calculation to obtain an intermediate state time; the reference delay is determined according to the round-trip delays of the multiple paths; and the difference between the intermediate state time and the reference delay is taken as the allowed re-injection time.

[0083] Suppose that the allowed re-injection time is denoted as E, the re-injection degree adjustment parameter is denoted as β, the expected transmission time is denoted as D, and the reference delay is denoted as R. Wherein, E = β × D - R. When β is greater than 1, for example, 1.2, it means that the re-injection operation is allowed to be performed in the future; when β is less than 1, for example, 0.8, it means that the re-injection operation is allowed to be performed in advance. The reference delay R can be obtained according to the round-trip delays of the multiple transmission paths, for example, the round-trip delays of the multiple transmission paths can be averaged, weighted summed, or the maximum round-trip time or the minimum round-trip delay can be selected as the reference delay R.

[0084] In the embodiments of the present application, the first end performing the re-injection operation means that the first end re-sends the data packet on the transmission path in the non-idle state through the transmission path in the idle state to the second end.

[0085] Specifically, the first end can identify whether the transmission path is in an idle state or in a non-idle state according to the network state information of the plurality of transmission paths. The transmission path in the idle state can be referred to as an idle path, and the transmission path in the non-idle state can be referred to as a non-idle path. The non-idle path refers to a transmission path with poor transmission performance and is relatively busy. The poor transmission performance can be limited by the physical performance of the path itself, or can be caused by congestion. In an optional embodiment, the transmission of the data packet is performed in a transmission window manner for each transmission path, and each transmission path has a certain number of transmission windows. For the transmission path with an idle window, it is an idle path, and for the transmission path without an idle window, it is a non-idle path.

[0086] The embodiments of the present application do not limit the number of data packets transmitted on the idle path and the non-idle path. The idle path can be because the number of data packets to be transmitted is small, or because the path quality is good, so it becomes an idle path. Correspondingly, the non-idle path can be because the number of data packets to be transmitted is large, or because the path quality is poor and is relatively busy, so it is referred to as a non-idle path.

[0087] In the embodiment, the network state information of each transmission path includes, but is not limited to, the number of idle windows. According to whether each transmission path has an idle window and the number of idle windows, the idle path in the idle state in the plurality of transmission paths can be determined. The definition of the idle path is not limited in the embodiments of the present application. For example, the transmission path with an idle window can be referred to as an idle path, or the transmission path with an idle window and the number of idle windows reaching a first number threshold can be referred to as an idle path. Correspondingly, the definition of the non-idle path is also not limited. For example, the transmission path without an idle window can be referred to as a non-idle path, or the transmission path without an idle window and having data packets to be transmitted and the number of data packets to be transmitted being greater than a second number threshold can be referred to as a non-idle path.

[0088] Further, the network state information can also include path quality parameters. The path quality parameters include, but are not limited to, bandwidth, packet loss rate, latency, and jitter. The bandwidth, also referred to as throughput, refers to the amount of data transmitted per unit of time, and the unit is usually bits per second (bps). The bandwidth reflects the transmission capability of the network, and the greater the better. The packet loss rate refers to the ratio of the number of lost data packets to the number of transmitted data packets. The packet loss rate reflects the reliability of the network, and the smaller the better. The latency refers to the time taken from the start of data packet transmission to the reception, and the unit is usually milliseconds (ms). The latency reflects the speed of the network, and the smaller the better. The jitter reflects the stability of the network, and the smaller the better. In this case, the definition of the idle path and the non-idle path can also be combined with the path quality. For example, a transmission path with an idle window and a path quality satisfying a certain quality requirement can be regarded as an idle path. Correspondingly, a transmission path without an idle window and a path quality not satisfying the quality requirement can be regarded as a non-idle path, and so on.

[0089] In this case, according to the network state information of the plurality of transmission paths, the transmission paths in the idle state (i.e., the idle paths) and the transmission paths in the non-idle state (i.e., the non-idle paths) are determined from the plurality of transmission paths. Further, according to the definition of the idle path, one or more idle paths can be identified. If there is only one idle path, the idle path is directly selected as the target transmission path. If there are multiple idle paths, the idle path with the better quality is selected as the target transmission path according to the path quality parameters of the multiple idle paths. Then, the data packets on the non-idle path are retransmitted to the second end through the target transmission path.

[0090] In actual applications, if there are multiple idle paths and one path quality parameter, at least one idle path with a path quality satisfying a corresponding requirement (e.g., greater than a set quality threshold) can be selected as the target transmission path according to the path quality parameter. If there are multiple path quality parameters, the multiple path quality parameters can be weighted and summed for each idle path to obtain the overall path quality of each idle path, and at least one idle path with an overall path quality satisfying a corresponding requirement (e.g., greater than a set quality threshold) can be selected as the target transmission path. In addition to the above method, if there are multiple path quality parameters, the multiple path quality parameters can be prioritized. Then, the target transmission path can be selected from the multiple idle paths according to the priority of the path quality parameters. When the target transmission path is selected from the multiple idle paths according to the priority of the path quality parameters, the priority of the bandwidth is higher than that of the packet loss rate, and the priority of the packet loss rate is higher than that of the latency. Therefore, the higher the bandwidth, the higher the priority of the corresponding idle path. In the case of the same bandwidth, the smaller the packet loss rate, the higher the priority of the idle path. In the case of the same bandwidth and packet loss rate, the smaller the latency, the higher the priority of the idle path.

[0091] In the case of one non-idle path, the data packets on the non-idle path can be retransmitted to the second end through the target transmission path. In the case of multiple non-idle paths, a non-idle path with poor quality can be selected from the multiple non-idle paths according to the path quality parameters of the multiple non-idle paths, and the data packets on the non-idle path with poor quality selected from the multiple non-idle paths can be retransmitted to the second end through the target transmission path.

[0092] In either case, when retransmitting the data packets on the non-idle path to the second end through the target transmission path, if the full redundancy transmission mode is adopted, the transmitted data packets and the non-transmitted data packets on the transmission path in the non-idle state that have not received the confirmation information of the second end can be retransmitted to the second end through the target transmission path; if the auxiliary redundancy transmission mode is adopted, the transmitted data packets on the transmission path in the non-idle state that have not received the confirmation information of the second end can be retransmitted to the second end through the target transmission path.

[0093] The technical scheme provided by the embodiments of the present application introduces the expected transmission time corresponding to the application request in the re-injection process of the multi-path transmission, uses the expected transmission time corresponding to the application request as the time basis for allowing the execution of the re-injection operation, and can retransmit (i.e., perform the re-injection operation) the data packets on the non-idle path using the idle path in the multi-path at a more reasonable time. Since the re-injection operation is performed in the multi-path transmission, the performance of the multi-path transmission can be ensured, and since the execution of the re-injection operation is based on the expected transmission time corresponding to the application request, the user experience of the application can be ensured, and the data packets subjected to the re-injection, the amount of redundant data in the multi-path transmission, and the flow resources can be reduced to a certain extent.

[0094] It is worth noting that the case where the first end is a server device and the second end is a client device is different from the case where the first end is a client device and the second end is a server device. In the case where the first end is a client device and the second end is a server device, the first end can actively send the to-be-transmitted data to the second end, at this time, the first end determines the expected transmission time corresponding to the to-be-transmitted data instead of the expected transmission time corresponding to the application request, the to-be-transmitted data refers to the data that the first end needs to transmit to the second end, and the expected transmission time corresponding to the to-be-transmitted data refers to the time required for the application on the first end to transmit the to-be-transmitted data to the second end, and other operations are the same as or similar to the foregoing embodiments and will not be described one by one.

[0095] In order to better understand the technical scheme provided by the embodiments of the present application, the adaptive video streaming scenario is taken as an example, and the Figure 3 The re-injection process of the adaptive video streaming scenario based on multi-path transmission is described. Referring to Figure 3, according to the layered architecture of TCP / IP protocol, the client device includes an application layer and a transport layer from top to bottom, and the server device includes an application layer and a transport layer from top to bottom. The application program of the client device includes a player, the application layer of the server device includes a web server, the transport layer of the client device includes a transport client, and the transport layer of the server device includes a transport server; the HTTP protocol is used between the player and the web server, and multiple transmission paths exist between the transport client and the transport server, so that the client device and the server device can transmit based on multiple paths, for example, transmission path 1 is a Wi-Fi link, transmission path 2 is a Cellular link, and the transmission layer protocol is illustrated by using QUIC as an example.

[0096] The player in the client device plays the adaptive video stream, and the played adaptive video stream is provided by the web server in the server device, that is, the player requests the web server to send the adaptive video stream. The web server locally caches the adaptive video stream, the adaptive video stream is divided into multiple video blocks, and the web server caches multiple code rate versions of the same video block, for example, simultaneously caches the video blocks with code rates of 5Mbps, 8Mbps and 16Mbps for the same video block.

[0097] The process that the player requests the video block from the web server and transmits the video block based on the multiple paths and performs the re-injection operation in the multiple path transmission process is as follows:

[0098] S1, the player in the client device runs the ABR algorithm to determine the target code rate from multiple code rates based on the network throughput of the player, and generates the size of the video block according to the target code rate, and generates the expected transmission time corresponding to the requested video block according to the network throughput of the player and the size of the video block.

[0099] S2, the player in the client device provides the expected transmission time to the transport client in the client device, and the transport client in the client device sends a QoE frame including the expected transmission time to the transport server of the server device through the transmission path 1 and / or the transmission path 2. In Figure 3 , transmission of the expected transmission time through the transmission path 1 is taken as an example for illustration. Of course, the HTTP request for requesting to obtain the video block can also be expanded, and the expected transmission time can be transmitted by using the expansion field in the HTTP request.

[0100] S3, the transport server of the server device parses the expected transmission time from the QoE frame, and transmits the expected transmission time to the web server of the server device.

[0101] S4, the player in the client device sends a HTTP request for requesting the video block to the transmission server of the server device through the transmission path 1 and / or the transmission path 2, the HTTP request including the video block identifier and the target code rate. Figure 3 In the embodiment, the transmission of the HTTP request through the transmission path 2 is taken as an example for illustration.

[0102] S5, the transmission server of the server device forwards the HTTP request for requesting the video block to the web server of the server device.

[0103] S6, the web server of the server device responds to the HTTP request, acquires the video block corresponding to the video block identifier at the target code rate, splits the video block into a plurality of data packets, and transmits the data packets of the video block to the transmission server of the server device, which transmits the plurality of data packets to the player of the client device through the transmission path 1 and the transmission path 2.

[0104] S7, the transmission server of the server device starts to transmit the data packets in the video block, takes the current time as the start transmission time of the video block, and takes the transmission time of the last data packet as the transmission completion time of the video block after the last data packet is transmitted from the server device.

[0105] S8, during the transmission of the plurality of data packets, the transmission server of the server device adjusts the allowed re-injection time according to the expected transmission time and the re-injection degree, judges whether the allowed re-injection time arrives before the transmission completion time, takes the transmission completion time as the re-injection execution time if yes, and starts to perform the re-injection operation after the last data packet is transmitted; or takes the allowed re-injection time as the re-injection execution time if no, and starts to perform the re-injection operation when the allowed re-injection time arrives.

[0106] For the re-injection control of the adaptive video streaming scene based on the multi-path transmission, the re-injection behavior in the data transmission process is controlled by sensing the performance requirements such as the network throughput of the player, so that the data transmission process can be optimized together with the player to optimize the video watching experience (for example, QoE) of the user, reduce redundant data, improve the quality of the video and reduce the lag, thereby helping to improve the user watching rate and the user retention rate.

[0107] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps 201 to 203 can be device A; for another example, the execution subject of steps 201 and 202 can be device A, and the execution subject of step 203 can be device B; and the like.

[0108] In addition, in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations are included in a specific order, but it should be clear that the operations can be executed in the order in which they appear in this document or in parallel, and the serial numbers of the operations such as 201, 202, etc. are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. described in this document are used to distinguish different messages, devices, modules, etc. and do not represent the order of precedence. Also, "first" and "second" are not of different types.

[0109] Figure 4 A structure diagram of a multi-path transmission re-injection control device provided by an embodiment of the present application. The device can be configured at a first end, and the first end and a second end perform data transmission through multiple paths. Wherein, the first end is a client device, and the second end is a server device, or the first end is a server device, and the second end is a client device. As shown in the figure, the device includes: Figure 4

[0110] The determining module 41 is configured to determine an expected transmission time corresponding to an application request, the application request being used to request the first end for to-be-transmitted data;

[0111] The multi-path transmission module 42 is configured to send data packets in the to-be-transmitted data to the second end through multiple transmission paths between the first end and the second end;

[0112] The re-injection control module 43 is configured to resend the data packets on the transmission path in the non-idle state to the second end through the transmission path in the idle state according to the expected transmission time.

[0113] Further optionally, the re-injection control module 43 is specifically configured to: obtain a sending completion time of the to-be-transmitted data; determine a re-injection execution time according to the expected transmission time and the sending completion time; and resend the data packets on the transmission path in the non-idle state to the second end through the transmission path in the idle state at the re-injection execution time.

[0114] Further optionally, when the re-injection control module 43 determines the re-injection execution time according to the expected transmission time and the sending completion time, it is specifically configured to: generate an allowed re-injection time according to the expected transmission time and a re-injection degree adjustment parameter, the re-injection degree adjustment parameter being a preset value or being obtained in real time; determine the allowed re-injection time as the re-injection execution time in the case where the allowed re-injection time is later than the sending completion time; and determine the sending completion time as the re-injection execution time in the case where the allowed re-injection time is earlier than the sending completion time. ​

[0115] Further, when the re-injection control module 43 acquires the re-injection degree adjustment parameter in real time, the specific operation is as follows: acquiring network state information of the multiple transmission paths and / or a cache level of a data cache area used by the application layer in real time under the condition of determining the expected transmission time; and determining the re-injection degree adjustment parameter according to the network state information and / or the cache level of the data cache area.

[0116] Further, when the re-injection control module 43 generates the allowed re-injection time according to the expected transmission time and the re-injection degree adjustment parameter, the specific operation is as follows: performing numerical calculation on the expected transmission time and the re-injection degree adjustment parameter to obtain an intermediate state time; determining a reference delay according to round-trip delays of the multiple paths; and taking a difference between the intermediate state time and the reference delay as the allowed re-injection time.

[0117] Further, the determining module 41 is specifically configured to receive the expected transmission time sent by the second end through at least one of the multiple transmission paths between the first end and the second end. The second end acquires the expected transmission time in the following manner: determining a target code rate corresponding to the to-be-transmitted data according to a network throughput of an application corresponding to the application request, determining a data size of the to-be-transmitted data according to the target code rate; and generating the expected transmission time corresponding to the application request according to the network throughput and the data size of the to-be-transmitted data.

[0118] Further, when the multi-path transmission module 42 receives the expected transmission time sent by the second end through at least one of the multiple transmission paths between the first end and the second end, the specific operation is as follows: receiving a QoE frame sent by the second end through the at least one transmission path, the QoE frame including the expected transmission time; or acquiring the expected transmission time from an extension field of the application request.

[0119] In an optional embodiment, the transmission layer protocol used by the multiple transmission paths is not limited to QUIC, UDP or TCP.

[0120] Further, before the multi-path transmission module 42 transmits a data packet in the to-be-transmitted data to the second end through the multiple transmission paths between the first end and the second end, the specific operation is as follows: receiving an application request sent by the second end through at least one of the multiple transmission paths between the first end and the second end, the application request including identification information and a target code rate of the to-be-transmitted data; and determining the to-be-transmitted data corresponding to the target code rate from multiple to-be-transmitted data corresponding to multiple code rates.

[0121] Further, when the re-injection control module 43 re-sends the data packet on the transmission path in the non-idle state to the second end through the transmission path in the idle state, the re-injection control module 43 is specifically configured to: determine the transmission path in the idle state and the transmission path in the non-idle state from the plurality of transmission paths according to network state information of the plurality of transmission paths, the network state information including the number of idle windows and path quality parameters; when the transmission path in the idle state is multiple, select a target transmission path from the plurality of transmission paths in the idle state according to the path quality parameters of the plurality of transmission paths in the idle state, and re-send the data packet on the transmission path in the non-idle state to the second end through the target transmission path.

[0122] Further, when the re-injection control module 43 re-sends the data packet on the transmission path in the non-idle state to the second end through the target transmission path, the re-injection control module 43 is specifically configured to: in the full redundancy transmission mode, re-send the sent data packet and the unsent data packet on the transmission path in the non-idle state to the second end through the target transmission path, the sent data packet not receiving the acknowledgement information from the second end; or in the auxiliary redundancy transmission mode, re-send the sent data packet on the transmission path in the non-idle state to the second end through the target transmission path, the sent data packet not receiving the acknowledgement information from the second end.

[0123] Figure 4 The apparatus shown can perform the method shown Figure 2a The method shown, the implementation principle and technical effect will not be described again. For the Figure 4 The specific manner in which each module performs operations shown has been described in detail in the foregoing embodiments, and will not be described in detail here.

[0124] Figure 5 A structural schematic diagram of an electronic device is provided for the embodiments of the present application. As shown in Figure 5 The electronic device includes a memory 51 and a processor 52.

[0125] The memory 51 is configured to store computer programs and can be configured to store other various data to support operations on the electronic device. Examples of these data include instructions of any application program or method for operating on the electronic device, messages, pictures, videos, etc.

[0126] The processor 52 is coupled to the memory 51 and is configured to execute the computer programs in the memory 51, so as to: determine an expected transmission time corresponding to an application request, the application request being used to request to-be-transmitted data from a first end; send a data packet in the to-be-transmitted data to a second end through a plurality of transmission paths between the second end; and re-send the data packet on the transmission path in the non-idle state to the second end through the transmission path in the idle state according to the expected transmission time.

[0127] In an optional embodiment, the processor 52 retransmits the data packet on the transmission path in the non-idle state to the second end through the transmission path in the idle state according to the expected transmission time, comprising: obtaining a sending completion time of the data to be transmitted; determining a re-injection execution time according to the expected transmission time and the sending completion time; when the re-injection execution time arrives, retransmitting the data packet on the transmission path in the non-idle state to the second end through the transmission path in the idle state.

[0128] In an optional embodiment, the processor 52 determines the re-injection execution time according to the expected transmission time and the sending completion time, comprising: generating an allowed re-injection time according to the expected transmission time and a re-injection degree adjustment parameter, the re-injection degree adjustment parameter being a preset value or being obtained in real time; in the case that the allowed re-injection time is later than the sending completion time, determining that the allowed re-injection time is the re-injection execution time; in the case that the allowed re-injection time is earlier than the sending completion time, determining that the sending completion time is the re-injection execution time.

[0129] In an optional embodiment, the processor 52 obtains the re-injection degree adjustment parameter in real time, comprising: obtaining network state information of the plurality of transmission paths and / or a cache level of a data cache area used by the application layer in real time when the expected transmission time is determined; determining the re-injection degree adjustment parameter according to the network state information and / or the cache level of the data cache area.

[0130] In an optional embodiment, the processor 52 generates the allowed re-injection time according to the expected transmission time and the re-injection degree adjustment parameter, comprising: performing numerical calculation on the expected transmission time and the re-injection degree adjustment parameter to obtain an intermediate state time; determining a reference delay according to a round-trip delay of the plurality of paths; taking a difference between the intermediate state time and the reference delay as the allowed re-injection time.

[0131] In an optional embodiment, the processor 52 determines the expected transmission time corresponding to the application request, comprising: receiving the expected transmission time sent by the second end through at least one of the plurality of transmission paths between the first end and the second end. Wherein, the second end obtains the expected transmission time of the data to be transmitted by the application layer in the following manner: determining a target code rate corresponding to the data to be transmitted according to a network throughput of an application corresponding to the application request, and determining a data size of the data to be transmitted according to the target code rate; generating the expected transmission time corresponding to the application request according to the network throughput and the data size of the data to be transmitted.

[0132] In an optional embodiment, the processor 52 is further configured to, before sending at least one data packet in the to-be-transmitted data to the second end through at least one of the plurality of transmission paths between the second end, receive an application request sent by the second end through at least one of the plurality of transmission paths between the second end, the application request comprising identification information and a target code rate of the to-be-transmitted data; and determine the to-be-transmitted data corresponding to the target code rate from the to-be-transmitted data corresponding to a plurality of code rates.

[0133] In an optional embodiment, the processor 52 receives the expected transmission time sent by the second end through at least one of the plurality of transmission paths between the second end, comprising: receiving a quality of experience (QoE) frame sent by the second end through at least one of the transmission paths, the QoE frame comprising the expected transmission time; or obtaining the expected transmission time from an extension field of the application request.

[0134] In an optional embodiment, the plurality of transmission paths can use a QUIC protocol, a UDP protocol or a TCP protocol, without limitation.

[0135] In an optional embodiment, the processor 52 re-sends the data packet on the transmission path in the non-idle state to the second end through the transmission path in the idle state, comprising: determining the transmission path in the idle state and the transmission path in the non-idle state from the plurality of transmission paths according to network state information of the plurality of transmission paths, the network state information comprising a number of idle windows and a path quality parameter; in the case of multiple transmission paths in the idle state, selecting a target transmission path in the idle state from the plurality of transmission paths in the idle state according to the path quality parameter of the plurality of transmission paths in the idle state; and re-sending the data packet on the transmission path in the non-idle state to the second end through the target transmission path.

[0136] Further optionally, when the re-injection control module 43 re-sends the data packet on the transmission path in the non-idle state to the second end through the target transmission path, it is specifically configured to: in the full redundancy transmission mode, re-sending the sent data packet and the unsent data packet on the transmission path in the non-idle state to the second end through the target transmission path, which have not received the acknowledgement information from the second end; or in the auxiliary redundancy transmission mode, re-sending the sent data packet on the transmission path in the non-idle state to the second end through the target transmission path, which have not received the acknowledgement information from the second end.

[0137] Further, as shown in Figure 5 the electronic device further comprises a communication component 53, a display 54, a power supply component 55, an audio component 56 and other components. Figure 5 Some components are only schematically shown in the figure, and it does not mean that the electronic device only comprises Figure 5 the components shown in the figure. In addition, Figure 5The components in the dashed box are optional components, not mandatory components, and can be determined according to the product form of the electronic device. The electronic device in the embodiment can be implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone or an IOT (Internet of things) device, or a server device such as a general server, a cloud server or a server array. If the electronic device in the embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone or an IOT (Internet of things) device, the electronic device can include Figure 5 components in the dashed box; if the electronic device in the embodiment is implemented as a server device such as a general server, a cloud server or a server array, the electronic device can not include Figure 5 components in the dashed box.

[0138] The detailed implementation process of the processor performing each action can refer to the related description in the foregoing method embodiment or device embodiment, and will not be described here.

[0139] Correspondingly, the embodiment of the present application further provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor can implement each step in the foregoing method embodiment.

[0140] The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0141] The communication component is configured to facilitate wired or wireless communication between the device on which the communication component is installed and other devices. The device on which the communication component is installed can access a wireless network based on a communication standard or standards, such as Wi-Fi, 2G (2 Generation), 3G (3 Generation), 4G (4 Generation) / LTE (Long Term Evolution), 5G (5 Generation), or the like, or a combination thereof. In an example embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0142] The display includes a screen, which can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touch or a slide action, but also detect a duration and a pressure associated with the touch or slide operation.

[0143] The power supply component provides power to various components of the device on which the power supply component is installed. The power supply component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device on which the power supply component is installed.

[0144] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when the device on which the audio component is installed is in a particular mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in memory or transmitted via the communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0145] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a system, or a computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable code.

[0146] The application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0147] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0149] In one typical configuration, the computing device includes one or more processors (Central Processing Units, CPUs), input / output interfaces, network interfaces, and memory.

[0150] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), or electrically erasable programmable read only memory (EEPROM), for the storage of software that is read during runtime. The memory is an example of computer readable media.

[0151] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0152] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements does not only include those elements, but also includes other elements not explicitly listed, or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0153] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the scope of claims of the present application.

Claims

1. A method of controlling re-injection of a multi-path transmission, characterized by, The method applied to the first end comprises: According to the network throughput of the application corresponding to the application request, the target code rate corresponding to the to-be-transmitted data is determined; the data size of the to-be-transmitted data is generated according to the target code rate; the expected transmission time is generated according to the data size of the to-be-transmitted data and the network throughput; the application request is used to request the to-be-transmitted data from the first end; Through a plurality of transmission paths between the first end and the second end, the data packets in the to-be-transmitted data are sent to the second end; The sending completion time of the to-be-transmitted data is obtained; the re-injection execution time is determined according to the expected transmission time and the sending completion time; when the re-injection execution time arrives, the data packets on the transmission path in the non-idle state are re-sent to the second end through the transmission path in the idle state; Wherein, according to the expected transmission time and the sending completion time, the re-injection execution time is determined, comprising: In the case of determining the expected transmission time, the network state information of the plurality of transmission paths and / or the cache level of the data cache area used by the application layer are obtained in real time; the re-injection degree adjustment parameter is determined according to the network state information and / or the cache level of the data cache area; if the transmission speed of the transmission path is fast based on the network state information, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in lag; if the transmission speed of the transmission path is slow based on the network state information, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in advance; the cache level of the data cache area reflects the amount of data cached in the data cache area, if the amount of data cached is less, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in advance; if the amount of data cached is more, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in lag; the expected transmission time and the re-injection degree adjustment parameter are numerically calculated to obtain an intermediate state time; the reference delay is determined according to the round-trip delay of the plurality of paths; the difference between the intermediate state time and the reference delay is taken as the allowed re-injection time; in the case that the allowed re-injection time is later than the sending completion time, the allowed re-injection time is determined as the re-injection execution time; in the case that the allowed re-injection time is earlier than the sending completion time, the sending completion time is determined as the re-injection execution time.

2. The method of claim 1, wherein, Before sending at least one data packet in the to-be-transmitted data to the second end through a plurality of transmission paths between the first end and the second end, further comprising: Through at least one transmission path in the plurality of transmission paths between the first end and the second end, an application request sent by the second end is received, and the application request comprises identification information and a target code rate of the to-be-transmitted data; From a plurality of code rates corresponding to the to-be-transmitted data, the to-be-transmitted data corresponding to the target code rate is determined.

3. The method of claim 2, wherein, Determining the expected transmission time corresponding to the application request comprises: Receiving a quality of experience (QoE) frame sent by the second end through the at least one transmission path, wherein the QoE frame comprises the expected transmission time; Or, obtaining the expected transmission time from an extended field of the application request.

4. The method according to any one of claims 1 to 3, characterized in that, retransmitting the data packet on the non-idle transmission path to the second end through the transmission path in the idle state, comprising: determining the transmission path in the idle state and the transmission path in the non-idle state from the plurality of transmission paths according to network state information of the plurality of transmission paths, wherein the network state information comprises a number of idle windows and a path quality parameter; in the case that the transmission path in the idle state is multiple, selecting a target transmission path from the plurality of transmission paths in the idle state according to the path quality parameter of the plurality of transmission paths in the idle state; retransmitting the data packet on the non-idle transmission path to the second end through the target transmission path.

5. The method of claim 4, wherein, retransmitting the data packet on the non-idle transmission path to the second end through the target transmission path, comprising: retransmitting the transmitted data packet and the non-transmitted data packet on the non-idle transmission path to the second end through the target transmission path, wherein the transmitted data packet and the non-transmitted data packet do not receive the acknowledgement information from the second end; or retransmitting the transmitted data packet on the non-idle transmission path to the second end through the target transmission path, wherein the transmitted data packet does not receive the acknowledgement information from the second end.

6. A multi-path transmission re-injection control apparatus characterized by comprising: applied to a first end, the apparatus comprises: a determination module configured to determine a target code rate corresponding to to-be-transmitted data according to network throughput corresponding to an application request of an application; generate a data size of the to-be-transmitted data according to the target code rate; and generate an expected transmission time according to the data size of the to-be-transmitted data and the network throughput; the application request is used to request the to-be-transmitted data; a multi-path transmission module configured to transmit a data packet contained in the to-be-transmitted data to a second end through a plurality of transmission paths between the first end and the second end; a re-injection control module configured to obtain a transmission completion time of the to-be-transmitted data; determine a re-injection execution time according to the expected transmission time and the transmission completion time; and retransmit the data packet on a non-idle transmission path to the second end through a transmission path in an idle state when the re-injection execution time arrives; wherein the determination of the re-injection execution time according to the expected transmission time and the transmission completion time comprises: In the case of determining the expected transmission time, the network state information of the plurality of transmission paths and / or the cache level of the data cache area used by the application layer is acquired in real time; the re-injection degree adjustment parameter is determined according to the network state information and / or the cache level of the data cache area; wherein, if the transmission speed of the transmission path is fast based on the network state information, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in lag; if the transmission speed of the transmission path is slow based on the network state information, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in advance; the cache level of the data cache area reflects the amount of data cached by the data cache area, if the amount of cached data is less, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in advance; if the amount of cached data is more, the re-injection degree adjustment parameter is adjusted towards the direction of allowing re-injection in lag; The expected transmission time and the re-injection degree adjustment parameter are numerically calculated to obtain an intermediate state time; a reference delay is determined according to the round-trip delay of the plurality of paths; the difference between the intermediate state time and the reference delay is taken as the re-injection allowed time; In the case of the re-injection allowed time being later than the sending completion time, the re-injection allowed time is determined as the re-injection execution time; in the case of the re-injection allowed time being earlier than the sending completion time, the sending completion time is determined as the re-injection execution time.

7. An electronic device, comprising: Comprise: Memory and processor; The memory is used to store computer programs; the processor is coupled with the memory and is used to execute the computer programs to execute the steps in the method of any one of claims 1-5.

8. A computer readable storage medium storing a computer program, characterized in that, When the computer programs are executed by the processor, the processor can realize the steps in the method of any one of claims 1-5.

Images