Multi-channel-based Data Packet Transmission Method, Device, Equipment and Medium

Abstract

The present application discloses a multi-channel-based data packet transmission method, apparatus, device, medium, and program product, belonging to the field of network transmission services. The multi-channel-based data packet transmission method includes: determining at least two data packet sets based on the data to be transmitted, each data packet set including at least one data packet; allocating the at least two data packet sets to at least two transmission channels for transmission according to the set granularity, and the data packets in the same data packet set are transmitted via the same transmission channel; in the case where the transmission of the first data packet set fails on the first transmission channel among the at least two transmission channels, reallocating the first data packet set to the second transmission channel for transmission. The above solution improves the service quality by reallocating the transmission channels, selecting the transmission channels according to the actual situation for content distribution, and timely sending the data packets. The embodiments of the present application can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, and assisted driving.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of network transmission services, and particularly to a multi-channel based data packet transmission method, apparatus, device, medium, and program product. Background Art

[0002] Video transmission refers to the transmission of video data from a data source to a data terminal through a connection according to certain regulations.

[0003] In related technologies, most general video servers use a single wireless local area network connection or a mobile network connection to send video streams, that is, single connection transmission. Another multi-connection transmission method, such as the Multi-Path Transmission Control Protocol (MPTCP), uses multiple connections to send video streams. MPTCP uses the same Transmission Control Protocol (TCP) packet format and the same transmission algorithm.

[0004] However, even in the multi-connection transmission method, the transmission rate is still limited by the connection with the lowest transmission rate. When this connection has a problem, it is necessary to wait for it to be repaired before the entire multi-connection transmission can continue to transmit data. Summary of the Invention

[0005] The present application provides a multi-channel based data packet transmission method, apparatus, device, medium, and program product, which can flexibly select a transmission channel according to the actual situation, send data packets in a timely manner, and improve service quality. The technical solutions are as follows:

[0006] According to one aspect of the embodiments of the present application, a multi-channel based data packet transmission method is provided. This method is executed by a sending device, and the method includes:

[0007] Determine at least two data packet sets based on the data to be transmitted, where each data packet set includes at least one data packet;

[0008] Allocate at least two data packet sets to at least two transmission channels for transmission according to the set granularity, and the data packets in the same data packet set are transmitted through the same transmission channel;

[0009] In the case where the transmission of the first data packet set fails on the first transmission channel among at least two transmission channels, re-allocate the first data packet set to the second transmission channel for transmission.

[0010] According to another aspect of the embodiments of the present application, a multi-channel based data packet transmission method is provided. This method includes:

[0011] Allocate at least two data packet sets to at least two transmission channels for transmission according to the network speed ratio of the at least two data packet sets at the set granularity;

[0012] Among them, the ratio of the number of data packets allocated to each transmission channel matches the network speed ratio. The types of the at least two transmission channels include at least one of a wireless local area network transmission channel, a mobile network transmission channel, a wired network transmission channel, and an optical fiber transmission channel.

[0013] According to another aspect of the embodiments of the present application, a multi-channel-based data packet transmission method is provided. The method includes:

[0014] Allocate the data packet set related to the key data in the at least two data packet sets to the transmission channel with the lowest packet loss rate among the at least two transmission channels for transmission;

[0015] Or, allocate the data packet set related to the key data in the at least two data packet sets to the transmission channel with the fastest network speed among the at least two transmission channels for transmission;

[0016] Among them, the key data includes at least one of key frame data in a cloud video scenario, operation data in a cloud game scenario, and voice data in a video call scenario. The key data has the highest transmission priority.

[0017] According to another aspect of the embodiments of the present application, a multi-channel-based data packet transmission method is provided. The method is executed by a receiving end device. The method includes:

[0018] Receive at least two data packet sets through at least two transmission channels. Each data packet set includes at least one data packet, and the data packets in the same data packet set are transmitted via the same transmission channel;

[0019] In the case where the reception of the first data packet set fails on the first transmission channel among the at least two transmission channels, receive the first data packet set again through the second transmission channel.

[0020] According to another aspect of the embodiments of the present application, a multi-channel-based data packet transmission method for selecting a second transmission channel is provided. The method includes:

[0021] The second transmission channel is the transmission channel with the fastest network speed among the transmission channels other than the first transmission channel;

[0022] Or, the second transmission channel is the transmission channel with the lowest load among the transmission channels other than the first transmission channel;

[0023] Or, the second transmission channel is the transmission channel with the lowest packet loss rate among the transmission channels other than the first transmission channel as the second transmission channel.

[0024] According to another aspect of the embodiments of the present application, a multi-channel based data packet transmission method is provided. The method determines that the data packets in the first data packet set fail to be transmitted, including:

[0025] The number of retransmissions of the data packets in the first data packet set in the first transmission channel reaches a first threshold;

[0026] Or, the transmission duration after the data packets in the first data packet set start to be transmitted reaches a second threshold and the acknowledgment feedback of the first data packet set has not been sent;

[0027] Or, the allocated duration of the first data packet set allocated to the first transmission channel reaches a third threshold and the acknowledgment feedback of the first data packet set has not been sent;

[0028] Or, a retransmission request or a negative acknowledgment feedback of the first data packet set is sent.

[0029] According to another aspect of the embodiments of the present application, a multi-channel based data packet transmission method is provided. The method includes:

[0030] The retransmission priority of the data packets in the first data packet set is higher than the priority of the initially transmitted data packets in the second transmission channel.

[0031] The transmitted data includes video stream data;

[0032] At least two data packet sets are obtained by dividing based on video frame granularity;

[0033] Or, at least two data packet sets are obtained by dividing based on the data block granularity in the video frame.

[0034] According to another aspect of the embodiments of the present application, a multi-channel based data packet transmission method is provided. The method includes:

[0035] When receiving the second data packet set transmitted by the second transmission channel, an acknowledgment feedback of the second data packet set is sent.

[0036] According to another aspect of the embodiments of the present application, a multi-channel based data packet transmission method is provided. The method includes:

[0037] The receiving end reorders the data packets received by at least two transmission channels.

[0038] According to another aspect of the embodiments of the present application, a multi-channel based data packet transmission device is provided. The device includes:

[0039] A determination module, configured to determine at least two data packet sets based on the data to be transmitted, and each data packet set includes at least one data packet;

[0040] A transmission module, configured to allocate at least two sets of data packets to at least two transmission channels for transmission according to the set granularity, and the data packets in the same set of data packets are transmitted via the same transmission channel;

[0041] A retransmission module, configured to re-allocate the first set of data packets to the second transmission channel for transmission when the transmission of the first set of data packets fails on the first transmission channel among at least two transmission channels.

[0042] According to another aspect of the embodiments of the present application, there is provided a multi-channel-based data packet transmission device, and the device includes:

[0043] A receiving module, configured to receive at least two sets of data packets through at least two transmission channels, each set of data packets includes at least one data packet, and the data packets in the same set of data packets are transmitted via the same transmission channel;

[0044] A re-receiving module, configured to re-receive the first set of data packets through the second transmission channel when the reception of the first set of data packets fails on the first transmission channel among at least two transmission channels.

[0045] According to another aspect of the embodiments of the present application, there is provided a computer device, and the computer device includes: a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the multi-channel-based data packet transmission method in the above-mentioned various aspects.

[0046] According to another aspect of the embodiments of the present application, there is provided a computer-readable storage medium, and at least one instruction, at least one segment of program, code set or instruction set is stored in the computer-readable storage medium, and at least one instruction, at least one segment of program, code set or instruction set is loaded and executed by the processor to implement the multi-channel-based data packet transmission method in the above-mentioned various aspects.

[0047] According to another aspect of the embodiments of the present application, there is provided a computer program product (or a computer program), and the computer program product (or the computer program) includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the multi-channel-based data packet transmission method in the above-mentioned various aspects.

[0048] The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

[0049] By designing to reallocate the transmission channel in the case of the failure of a certain data packet set transmission, the transmission channel is flexibly selected according to the actual situation for content distribution. For example, select the transmission channel that is faster and has better transmission quality, so as to send the data packet in time, ensure the reliability and orderliness of the data packet transmission, and improve the service quality. Brief Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0051] Figure 1 Shows a schematic diagram of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application;

[0052] Figure 2 Shows a schematic diagram of the architecture of a multi-channel-based data packet transmission system provided by an exemplary embodiment of the present application;

[0053] Figure 3 Shows a flowchart of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application;

[0054] Figure 4 Shows a flowchart of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application;

[0055] Figure 5 Shows a schematic diagram of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application;

[0056] Figure 6 Shows a schematic diagram of the architecture of a multi-channel-based data packet transmission system provided by an exemplary embodiment of the present application;

[0057] Figure 7 Shows a flowchart of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application;

[0058] Figure 8 Shows a schematic diagram of the architecture of a multi-channel-based data packet transmission system provided by an exemplary embodiment of the present application;

[0059] Figure 9 Shows a schematic diagram of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application;

[0060] Figure 10The figure shows a schematic diagram of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application;

[0061] Figure 11 The figure shows a block diagram of a multi-channel-based data packet transmission device provided by an exemplary embodiment of the present application;

[0062] Figure 12 The figure shows a block diagram of a multi-channel-based data packet transmission device provided by an exemplary embodiment of the present application;

[0063] Figure 13 The figure shows a schematic structural diagram of a computer device provided by an exemplary embodiment of the present application. Detailed implementation manners

[0064] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0065] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0066] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. The singular forms "a", "the", and "that" used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the input images involved in the present application are obtained under the authorization of the user or with the full authorization of all parties.

[0068] It should be understood that although the terms first, second, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, the first parameter may also be referred to as the second parameter, and similarly, the second parameter may also be referred to as the first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0069] Embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, assisted driving, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The user terminal includes but is not limited to mobile phones, computers, intelligent voice interaction devices, intelligent home appliances, vehicle-mounted terminals, aircraft, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here.

[0070] Cloud computing refers to the delivery and usage model of Internet technology infrastructure, which means obtaining the required resources in a on-demand and easily scalable manner through the network; in a broad sense, cloud computing refers to the delivery and usage model of services, which means obtaining the required services in a on-demand and easily scalable manner through the network. Such services can be software, Internet-related, or other services. Cloud computing is the product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage, virtualization, and load balancing.

[0071] With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the promotion of demands such as search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Different from the previous parallel distributed computing, the emergence of cloud computing will, in concept, drive a revolutionary change in the entire Internet model and enterprise management model.

[0072] A brief introduction to some key terms in this application is as follows:

[0073] Cloud video: A video stream service with content integrated in the cloud, such as on-demand, live broadcast, video call, cloud game, etc.

[0074] Transmission channel: A channel established by end-to-end confirmation at both ends at the transport layer. In this application, the transmission channel has the same meaning as the transmission connection.

[0075] Session: Represents a management unit at the client level. One client corresponds to one session, and one session can have multiple transmission channels.

[0076] Transmission protocol: A system standard that allows information to propagate between two or more terminals in a transmission system in any physical medium, and also refers to the common language of computer communication or network devices.

[0077] Cloud video server: A server architected in the cloud for transmitting cloud videos.

[0078] Multi-channel transmission: Refers to simultaneously using multiple transmission channels such as WIFI + Long Term Evolution (LTE), WIFI + WIFI, WIFI + LTE + 5G, etc. to jointly transmit data.

[0079] Transmission channel switching: Refers to when a transmission problem is encountered on one transmission channel, enabling the data to be promptly switched to another transmission channel for transmission.

[0080] Packet loss rate: The proportion of packets lost within a certain period on the network.

[0081] Transmission blocking: The phenomenon where transmitted data accumulates and cannot obtain an acknowledgment from the peer end.

[0082] Sub-flow: A concept introduced by MPTCP, which is a single-channel data stream running on one channel.

[0083] Figure 1 The figure shows a schematic diagram of a multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application. The data transmission system in the embodiment of the present application includes: a sending end 100 and a receiving end 200. Among them, the sending end 100 includes a sending end session layer 120, and the receiving end 200 includes a receiving end session layer 180. The sending end 100 can send data packets to the receiving end 200 through multiple channels. In addition to implementing a transmission guarantee mechanism at the packet level in the channel layer, the present application also adds session layers at both ends to implement a transmission guarantee mechanism at the data block (packet set) level through the session layer.

[0084] In this embodiment, the multiple channels include at least two channels. This embodiment takes the multiple channels including a first channel and a second channel as an example for illustration. Taking the sending end 100 sending data packets to the receiving end 200 as an example, the exemplary process includes:

[0085] The sending - end session layer 120 splits the data into data block 1, data block 2, data block 3, and data block 4. Data block 1 includes data packets ①②, data block 2 includes data packets ③④, data block 3 includes data packets ⑤⑥, and data block 4 includes data packets ⑦⑧. Based on the data blocks as the scheduling unit, the sending - end session layer 120 allocates multiple data packets in each data block to the same channel for transmission.

[0086] Exemplarily, the sending - end session layer 120 allocates data block 1 (including data packets ①②) to be transmitted from the first channel to the receiving - end session layer 180, and allocates data block 2 (including data packets ③④) to be transmitted from the second channel to the receiving - end session layer 180. Data packets ①②③ are successfully transmitted to the receiving - end session layer 180, but data packet ④ fails to be transmitted. That is, data block 1 is successfully transmitted, and data packet ④ in data block 2 fails to be successfully transmitted. After data block 1 is completely transmitted to the receiving - end session layer 180, the sending - end session layer 120 destroys data block 1. Since data block 2 is not completely transmitted to the receiving - end session layer 180, the sending - end session layer 120 does not destroy data block 2. At this time, the receiving - end session layer 180 has data packets ①②③, and the sending - end session layer 120 has data packets ③④⑤⑥⑦⑧.

[0087] The sending - end session layer 120 continues to allocate data block 3 (including data packets ⑤⑥) to be transmitted from the first channel to the receiving - end session layer 180. The second channel still attempts to transmit data packet ④ to the receiving - end session layer 180. Data packet ⑤ is successfully transmitted to the receiving - end session layer 180. Assuming that data packet ④ still fails to be transmitted. At this time, the receiving - end session layer 180 has data packets ①②③⑤, and the sending - end session layer 120 has data packets ③④⑤⑥⑦⑧.

[0088] The sending - end session layer 120 re - allocates data block 2 (including data packets ③④) to be transmitted from the first channel to the receiving - end session layer 180. At the same time, data packet ④ continues to attempt to be transmitted from the second channel to the receiving - end session layer 180. Since data packet ④ fails to be transmitted from the second channel multiple times, the sending - end session layer 120 allocates data block 2 to the first channel for transmission. And because the sending - end session layer 120 transmits data packets in order, data packets ③④ have a higher transmission priority than data packet ⑥. After data block 2 is completely transmitted to the receiving - end session layer 180, the sending - end session layer 120 destroys data block 2. At this time, the receiving - end session layer 180 has data packets ①②③④⑤, and the sending - end session layer 120 has data packets ⑤⑥⑦⑧.

[0089] The sending - end session layer 120 sequentially sends data packets ⑥ and ⑦⑧ to the receiving - end session layer 180 through the first channel, and data packet ④ is transmitted to the receiving - end session layer 180 through the second channel during this period. After data blocks 3 and 4 are completely transmitted to the receiving - end session layer 180, the sending - end session layer 120 destroys data blocks 3 and 4. At this time, the receiving - end session layer 180 has data packets ①②③④⑤⑥⑦⑧, and the sending - end session layer 120 has no data packets. After data packet ④ is transmitted to the receiving - end session layer 180 through the second channel, since the receiving - end session layer 180 has previously received data packet 4 sent from the first channel, it performs merging and deduplication processing on data packet 4 sent through the second channel, only retaining data packet 4 sent from the first channel. After that, the sending - end session layer 120 can send data packets through the second channel again.

[0090] In summary, the method provided in this embodiment flexibly selects and re - transmits data packets by re - allocating transmission channels and choosing transmission channels with faster transmission speed and better transmission quality, ensuring the reliable transmission of data packets and improving the service quality.

[0091] Figure 2 The schematic structural diagram of a multi - channel - based data - packet transmission system provided by an exemplary embodiment of the present application is shown. The multi - channel - based data - packet transmission system includes a sending - end device 100 and a receiving - end device 200. The sending - end device can be simply referred to as the sending end, and the receiving - end device can be simply referred to as the receiving end. The sending - end device 100 includes a traditional physical server, a cloud server, etc., and the receiving - end device 200 includes a mobile phone, a computer, etc.

[0092] Data is transmitted between the sending - end device 100 and the receiving - end device 200 through network transmission. The sending - end device 100 determines the transmission data as at least two data - packet sets, each data - packet set includes at least one data packet, and then distributes the at least two data - packet sets to at least two transmission channels for transmission according to the set granularity. The types of the at least two transmission channels include at least one of a wireless local - area network transmission channel, a mobile - network transmission channel, a wired - network transmission channel, and an optical - fiber transmission channel. The data - packet sets are transmitted to the receiving - end device 200 through the transmission channels, and the receiving - end device 200 sorts the data - packet sets to obtain the correct data.

[0093] Figure 3 The flowchart of a multi - channel - based data - packet transmission method provided by an exemplary embodiment of the present application is shown. This method is executed by the sending - end device 100 and the receiving - end device 200, and the method includes:

[0094] Step 310: The sending - end device 100 determines at least two data - packet sets based on the data to be transmitted, and each data - packet set includes at least one data packet.

[0095] The data to be transmitted includes video stream data, audio stream data, etc. Among them, taking the video stream data as an example to illustrate the situation of the data packet set, the video stream data is composed of multiple video frame data. The data packet set can be divided according to the video frame granularity or the data block granularity in the video frame. The data packets included in different data packet sets can be one or more. For example, data packet set 1 includes data packet 1, and data packet set 2 includes data packets 2 to 5. The data packets in different data packet sets can also be repeated. For example, data packet set 1 includes data packets 1 to 10, and data packet set 1 includes data packets 5 to 15.

[0096] Step 320: The sending device 100 distributes at least two data packet sets to at least two transmission channels for transmission according to the set granularity, and the data packets in the same data packet set are transmitted via the same transmission channel.

[0097] Taking the data packet set divided according to the data block granularity in the video frame as an example, data block 1 includes data packet ① and data packet ②, and data block 2 includes data packet ③ and data packet ④. The sending device 100 distributes data block 1 to the first transmission channel and distributes data block 2 to the second transmission channel. Data packet ① and data packet ② are transmitted via the first transmission channel, and data packet ③ and data packet ④ are transmitted via the second transmission channel.

[0098] Step 330: The receiving device 200 receives at least two data packet sets through at least two transmission channels. Each data packet set includes at least one data packet, and the data packets in the same data packet set are transmitted via the same transmission channel.

[0099] In this embodiment, after the receiving device 200 receives the data packets in at least two data packet sets, it is necessary to reorder the data packets. Taking the data packets in at least two data packet sets as data packets ①②③④ as an example, when the receiving device 200 receives these data packets, it is not necessarily received in the correct order. It may be in the order of ①③②④, ①③④②, etc. The receiving device 200 needs to rearrange these data packets in the order of ①②③④.

[0100] Step 340: When the transmission of the first data packet set fails on the first transmission channel among at least two transmission channels, the sending device 100 re - distributes the first data packet set to the second transmission channel for transmission.

[0101] The sending device 100 needs to determine whether the first data packet set fails to be transmitted. In the case of determining the transmission failure, the sending device 100 re - distributes the data packets in the first data packet set, such as data packets ①②, to the second transmission channel for re - transmission.

[0102] The retransmission priority of the data packets in the first data packet set is higher than that of the initial transmission data packets in the second transmission channel. Taking the data packets ① and ② in the first data packet set and the initial transmission data packets ⑤ and ⑥ in the second transmission channel as an example, since the sending device 100 transmits the data packets in order, the data packets ① and ② in the first data packet set should be transmitted before the initial transmission data packets ⑤ and ⑥ in the second transmission channel according to the order. Therefore, the data packets ① and ② have a higher priority during retransmission.

[0103] Step 350: When the receiving device 200 fails to receive the first data packet set on the first transmission channel among at least two transmission channels, the receiving device 200 re-receives the first data packet set through the second transmission channel.

[0104] When the receiving device 200 receives a data packet set, it needs to send an acknowledgment feedback of the data packet set to the sending device 100. In addition to sending the acknowledgment feedback of the second data packet set, after the first data packet set is re-allocated by the sending device 100 to the second transmission channel for transmission, when the receiving device 200 receives the first data packet set transmitted through the second transmission channel, it needs to send the acknowledgment feedback of the first data packet set.

[0105] In summary, the method provided in this embodiment distributes content by reallocating the transmission channel, selects the transmission channel according to the actual situation, sends the data packets in a timely manner, ensures the reliability and orderliness of the data packets, and improves the service quality.

[0106] Figure 4 The flowchart of the multi-channel-based data packet transmission method provided by an exemplary embodiment of the present application is shown. This method is executed by the sending device 100 and the receiving device 200. The method includes:

[0107] Step 310: The sending device 100 determines at least two data packet sets based on the data to be transmitted, and each data packet set includes at least one data packet.

[0108] The data to be transmitted includes video stream data, audio stream data, etc. Among them, taking the video stream data as an example to illustrate the situation of the data packet set, the video stream data is composed of multiple video frame data.

[0109] In some embodiments, if the number of data packets of the video stream data is small, then based on the video frame granularity, the video stream data is divided into at least two data packet sets.

[0110] In some embodiments, if the number of data packets of the video stream data is large, then based on the data block granularity in the video frame, the video stream data is divided into at least two data packet sets.

[0111] Step 320: The sending device 100 distributes at least two data packet sets to at least two transmission channels according to the set granularity for transmission, and the data packets in the same data packet set are transmitted via the same transmission channel.

[0112] In this embodiment, the types of transmission channels include at least one of a wireless local area network transmission channel, a mobile network transmission channel, a wired network transmission channel, and an optical fiber transmission channel. Among them, the wireless local area network is divided into 2.5G, 5G, etc., the mobile network is divided into LTE, New Radio (NR), etc., the wired network includes a standard 8-bit modular interface, etc., and the optical fiber transmission is the data and signal transmission using an optical fiber as the medium.

[0113] Step 330: The receiving device 200 receives at least two data packet sets through at least two transmission channels. Each data packet set includes at least one data packet, and the data packets in the same data packet set are transmitted via the same transmission channel.

[0114] In this embodiment, after the receiving device 200 receives the data packets in at least two data packet sets, it is necessary to reorder the data packets. Taking the data packets in at least two data packet sets as data packets ①②③④ as an example, when the receiving device 200 receives these data packets, it is not necessarily in the correct order. It may be in the order of data packets ①③②④, data packets ①③④②, etc. The receiving device 200 needs to rearrange these data packets in the order of data packets ①②③④.

[0115] Step 341: The sending device 100 determines that the first data packet set fails to be transmitted.

[0116] The sending device 100 needs to determine that the first data packet set fails to be transmitted. Exemplarily, the failure of the first data packet set to be transmitted includes at least one of the following situations:

[0117] In some embodiments, when the number of retransmissions of the data packets in the first data packet set in the first transmission channel reaches the first threshold, it is determined that the first data packet set fails to be transmitted, that is, the data packets in the first data packet set still cannot be transmitted after multiple retransmissions. At this time, it is determined that the first data packet set fails to be transmitted.

[0118] In some embodiments, when the transmission duration after the data packets in the first data packet set start to be transmitted reaches the second threshold and no acknowledgment feedback of the first data packet set is received, it is determined that the first data packet set fails to be transmitted, that is, the transmission duration is too long and no acknowledgment feedback sent by the receiving end session layer is received. Only a long transmission time may be due to problems such as slow transmission speed. Therefore, it is also necessary to meet the condition of not receiving the acknowledgment feedback sent by the receiving end session layer to determine that the first data packet set fails to be transmitted.

[0119] In some embodiments, when the allocated duration for the first data packet set to be assigned to the first transmission channel reaches the third threshold and no acknowledgement feedback for the first data packet set is received, it is determined that the transmission of the first data packet set has failed, that is, the elapsed time after allocation is too long and no acknowledgement feedback sent by the receiving end session layer is received. Merely the elapsed time after allocation being too long may be due to problems such as slow transmission speed. Therefore, it is also necessary to satisfy the condition of not receiving the acknowledgement feedback sent by the receiving end session layer to determine that the transmission of the first data packet set has failed.

[0120] In some embodiments, when a retransmission request or negative acknowledgement feedback for the first data packet set is received, it is determined that the transmission of the first data packet set has failed, that is, when the receiving end sends a retransmission request or negative acknowledgement feedback to the sending end, it clearly indicates that the first data packet set has not been received. At this time, it is determined that the transmission of the first data packet set has failed.

[0121] Step 342: The sending end device 100 re - allocates the first data packet set to the second transmission channel for transmission.

[0122] Based on historical transmission information, calculate the network speed of the transmission channel, that is, the number of bits of binary information transmitted per second, with the unit of bits per second, denoted as bps. The calculation formula: S=(1 / T)*log2 N , where T is the width or repetition period of a digital pulse signal, with the unit of seconds; N is the number of discrete values taken by a symbol.

[0123] Based on historical transmission information, calculate the load of the transmission channel: the number of data packets not sent on the transmission channel.

[0124] Based on historical transmission information, calculate the packet loss rate of the transmission channel: [(number of input data packets - number of output data packets) / number of input data packets]*100%, where the input data packets refer to the data packets allocated by the session layer, and the output data packets refer to the data packets successfully sent.

[0125] The selection of the second transmission channel includes at least one of the following cases:

[0126] Select the transmission channel with the fastest network speed among the transmission channels other than the first transmission channel as the second transmission channel. For example, if the network speed of transmission channel A is 5 times that of the first transmission channel and the network speed of transmission channel B is 2 times that of the first transmission channel, then select transmission channel A as the second transmission channel, so as to ensure timely data transmission;

[0127] Alternatively, select the transmission channel with the smallest load among the transmission channels other than the first transmission channel as the second transmission channel. For example, if there are 5 transmission data packets remaining in transmission channel A and 10 transmission data packets remaining in transmission channel B, then select transmission channel A as the second transmission channel. This can minimize the pressure on the transmission channel and minimize the overall time to complete the task.

[0128] Alternatively, select the transmission channel with the lowest packet loss rate among the transmission channels other than the first transmission channel as the second transmission channel. For example, if the packet loss rate of transmission channel A is 1% and the packet loss rate of transmission channel B is 3%, then select transmission channel A as the second transmission channel. This can ensure the reliability of data transmission.

[0129] Step 350: The receiving device 200 re-receives the first data packet set through the second transmission channel.

[0130] After the sending device 100 re-transmits the data packets in the first data packet set, the receiving device 200 re-receives the data packets in the first data packet set through the corresponding transmission channel.

[0131] Step 360: The receiving device 200 sends an acknowledgement feedback for the second data packet set.

[0132] When the receiving device 200 receives the second data packet set, it needs to send an acknowledgement feedback for the second data packet set to the sending device 100 to ensure that all the data packets in the second data packet set arrive reliably.

[0133] Step 362: The receiving device 200 sends an acknowledgement feedback for the first data packet set.

[0134] After the first data packet set is re-allocated by the sending device 100 to the second transmission channel for transmission, when the receiving device 200 receives the first data packet set transmitted through the second transmission channel, it needs to send an acknowledgement feedback for the first data packet set.

[0135] Step 370: When the sending device 100 receives the acknowledgement feedback for the second data packet set transmitted through the second transmission channel, the second transmission channel allocates the third data packet set for transmission and destroys the data packets in the cached second data packet set.

[0136] When the sending device 100 receives the acknowledgement feedback for the data packet set transmitted through the transmission channel, it determines that the transmission channel is working properly, continues to allocate other data packet sets for transmission to the transmission channel, and at the same time destroys the data packets in the cached transmitted data packet set, so as to ensure that the data packets can be delivered to the receiving device 200 in a timely and reliable manner.

[0137] Step 380: The receiving device 200 sends an acknowledgment feedback for the third packet set.

[0138] When the receiving device 200 receives the third packet set, it needs to send an acknowledgment feedback for the third packet set to the sending device 100 to ensure that all the packets in the third packet set arrive reliably.

[0139] In summary, the method provided in this embodiment flexibly selects the retransmission channel: when selecting the transmission channel with the fastest network speed as the retransmission channel, it ensures timely data transmission; when selecting the transmission channel with the least load as the retransmission channel, it minimizes the pressure on the transmission channel and the overall time to complete the task is the shortest; when selecting the transmission channel with the lowest packet loss rate as the retransmission channel, it ensures the reliability of data transmission; and by using a data processing mechanism that destroys only after receiving an acknowledgment from the receiving end, it can timely sense whether the packet is successfully received, which is more in line with the actual situation.

[0140] Based on Figure 4 the optional embodiments, there are also other possible designs:

[0141] In one possible design, step 320 can allocate at least two packet sets to at least two transmission channels for transmission according to the network speed ratio of the at least two packet sets. Among them, the ratio of the number of packets allocated to each transmission channel matches the network speed ratio. For example, if the network speed of transmission channel A is 10 Mbps and the network speed of transmission channel B is 5 Mbps, then the network speed of transmission channel A is twice that of transmission channel B, and the number of packets allocated to transmission channel A is twice that of transmission channel B;

[0142] In one possible design, step 320 can also allocate the packet set related to the key data in the at least two packet sets to the transmission channel with the lowest packet loss rate among at least two transmission channels for transmission. For example, if the packet loss rate of transmission channel A is 1% and the packet loss rate of transmission channel B is 3%, then select transmission channel A as the transmission channel, so as to ensure the reliable transmission of key data;

[0143] In one possible design, step 320 can also allocate the packet set related to the key data in the at least two packet sets to the transmission channel with the fastest network speed among at least two transmission channels for transmission. For example, if the network speed of transmission channel A is twice that of transmission channel B, then select transmission channel A as the transmission channel, so as to ensure that the key data is transmitted to the receiving end earliest;

[0144] In a possible design, step 320 may also allocate the set of data packets related to the critical data in at least two sets of data packets to two or more transmission channels for transmission. For example, the set of data packets related to the critical data is allocated to transmission channel A and transmission channel B for joint transmission. In this way, when one transmission channel has a problem, the other transmission channel can transmit the critical data to the receiving end, ensuring the reliable transmission of the critical data;

[0145] Among them, the critical data includes at least one of the following: key frame data in the cloud video scenario, operation data in the cloud game scenario, and voice data in the video call scenario. The critical data has the highest transmission priority.

[0146] Figure 5 The figure shows a schematic diagram of a multi-channel based data packet transmission method provided by an exemplary embodiment of the present application.

[0147] The present application can be applied to the cloud video scenario. Cloud video refers to a video stream service with content integrated in the cloud, such as video on demand, live broadcast, video call, cloud game, etc. The cloud video server stores the video stream data. When the client sends a request for the video stream data, the cloud video server sends the video stream data to the client. At this time, the cloud video server is the sending end device, and the client is the receiving end device. In this embodiment, the transmission of video stream data is used as an example for explanation.

[0148] In the related art, when a single channel is used to transmit video stream data, if network jitter, packet loss, etc. occur, one can only wait for the retransmission of the data packets on the current network channel and the slow recovery of the network, resulting in a poor user experience. Although multi-channel transmission such as MPTCP uses multiple channels to transmit multiple sub-streams, taking the transmission of data block 1 in video frame 1 by MPTCP as an example, assuming that data block 1 has 20 data packets, MPTCP pre-allocates 10 data packets to be transmitted from the first channel and 10 data packets to be transmitted from the second channel. If 2 data packets are lost in the first channel, then the 10 data packets that were originally supposed to be transmitted on the first channel will never be transmitted, and thus the entire video frame 1 will never be transmitted. Because each data packet has been allocated to a specific channel, the transmission time is reduced compared to single-channel transmission, but there is still the head-of-line blocking problem, that is, the first data packet in a column is blocked, resulting in the blocking of the entire column of data packets. MPTCP will keep waiting for the first channel to be repaired before transmitting the data packets.

[0149] The multi-channel based data packet transmission system provided by this embodiment includes a cloud video server 510 and a client 530. Among them, the proxy layer of the cloud video server 510 provides a proxy service for the session layer 520 to work. The session layer 520 includes a first channel module 610 and a second channel module 620. The session layer 520 transmits video stream data by allocating data packet transmission tasks to the first channel module 610 and the second channel module 620.

[0150] In the multi-channel based data packet transmission method provided by this embodiment, through the service architecture of multi-channel transmission, multiple channels are established. These channels can be wireless local area network transmission channels and mobile network transmission channels, or TCP channels and User Datagram Protocol (UDP) channels. In this embodiment, TCP channels and UDP channels are taken as examples for illustration. The cloud video server 510 transmits video stream data to the session layer 520 through TCP channels and UDP channels. The session layer 520 splits the video stream data into several data packets and then allocates them to the first channel module 610 and the second channel module 620 for transmission. The session layer 520 can establish a peer connection with the client 530, that is, the client 530 establishes a one-to-one channel with the cloud video server 510, and the session layers at both ends correspond to the channel modules one by one. The cloud video server 510 sends the data packets to the client 530 through the TCP channel, and finally sends the complete video stream data. In this embodiment, the cloud video server 510 transmits one video stream data through multiple channels. When data packet transmission fails in some channels, the channels can be replaced in time to retransmit the data packets, effectively improving the fluency and real-time performance of the video.

[0151] In summary, the method provided by this embodiment dynamically selects a better channel to transmit data packets by pre-establishing two channels between the client and the cloud video server when needed, so as to improve the service quality of cloud video products without disturbing the user.

[0152] Figure 6 The schematic diagram of the architecture of the multi-channel based data packet transmission system provided by an exemplary embodiment of the present application is shown.

[0153] In the related art, multi-channel transmission taking MPTCP as an example works at the transport layer and does not work at the application layer. Therefore, it cannot be closely attached to the application service and cannot meet the actual needs. And for the data at the application layer, once it is sent to the kernel, it will be immediately destroyed, and the application layer does not know whether the data is delivered to the client in a timely and reliable manner.

[0154] The multi-channel-based data packet transmission system provided in this embodiment can work on both the client side and the server side. The multi-channel-based data packet transmission system includes: an application layer 600, a transport layer, a network layer, a data link layer, and a physical layer. Among them, the application layer 600 includes: a session layer 520, a first channel module 610, a second channel module 620, a first socket 612, a second socket 622, and a third socket 632. The session layer 520, the first channel module 610, and the second channel module 620 all send data through sockets.

[0155] In this embodiment, the service architectures for multi-channel-based data packet transmission all work in the application layer 600, which can be closely integrated with the application service to meet actual needs. After the channels are established, the service architecture aggregates multiple channel modules into the session layer 520 to transmit the same video stream. In this embodiment, the first channel module 610 and the second channel module 620 are arbitrary channel modules. All channel modules send data packets through sockets and have the same working process. Here, the first channel module 610 and the second channel module 620 are taken as examples for illustration, and the remaining channel modules will not be elaborated.

[0156] When a video stream of data is transmitted, the session layer 520 divides it into video frame form, and then divides the video frames into data block form. Each data block contains the same or different numbers of data packets. The number of data blocks and the number of data packets contained in each data block are determined by the session layer 520. The session layer 520 distributes the data blocks to different channel modules through the third socket 632. For example, after the first channel module 610 receives the allocated data block 1, it sends all the data packets contained in the data block 1 to the channel module of the client through the first socket 612. The channel module of the client then sends all the data packets to the session layer of the client. After the session layer of the client receives all the data packets, it returns the confirmation of data block 1, so as to ensure that data block 1 reaches the client reliably. After that, the first channel module 610 sends the information that data block 1 has reached the client reliably to the session layer 520. If a certain channel module fails to send all the data packets contained in a data block to the client and the session layer 520 does not receive the information that the data block has reached the client reliably, for example, the packet 4 in the data block 2 sent by the second channel module 620 fails to be sent, the session layer 520 will allocate the data block 2 responsible for the second channel module 620 to the first channel module 610 for sending, so as to ensure that the final complete video stream data reaches the client reliably.

[0157] In summary, the system provided in this embodiment builds the service architecture in the application layer, introduces a new reliable transmission mechanism, and the data block will be destroyed only after receiving the confirmation from the client, which ensures that the data block can be reliably delivered to the client and improves the reliability of the video stream data.

[0158] Figure 7 The flowchart of a multi-channel based data packet transmission method provided by an exemplary embodiment of the present application is shown. Taking the transmission of video frame data as an example, the method is executed by the sending end, and the transmission process includes:

[0159] Step 710: Establish a first channel.

[0160] The sending end establishes a first channel in advance for transmitting data packets. The first channel is any one of the channels in the sending end, and its working process is the same as that of all channels of the same type.

[0161] Step 712: Establish a second channel.

[0162] The sending end establishes a second channel in advance for transmitting data packets. The second channel is any one of the channels in the sending end, and its working process is the same as that of all channels of the same type.

[0163] Step 720: The sending end sends video frame data.

[0164] The video stream data is composed of video frame data. The sending end sequentially sends the video frame data for session layer processing, and finally sends the entire video stream data to the receiving end.

[0165] Step 730: The session layer splits the video frame into data blocks.

[0166] The session layer splits a video frame data into multiple data blocks. Each data block contains the same or different numbers of data packets. The number of data blocks and the number of data packets contained in the data blocks are determined by the session layer.

[0167] Step 740: Find the latest data block to be sent or retransmitted.

[0168] The session layer distributes the data packets in the data block to be sent to different channels for transmission. If a data block fails to be sent, then the session layer needs to retransmit the data block.

[0169] Step 750: Whether to use the current channel.

[0170] The session layer judges whether to use a certain channel. For example, it judges according to whether it can transmit data packets, selects a transmission channel that can transmit data packets to ensure the stable transmission of data packets; or judges according to the network speed, selects the transmission channel with the fastest network speed to ensure the timely arrival of data packets; or judges according to the packet loss rate, selects the transmission channel with the smallest packet loss rate to ensure the reliable transmission of data packets.

[0171] If the first channel does not have a situation where data packets cannot be transmitted, then it is judged to use the first channel and step 760 is executed; if the second channel has a situation where data packets cannot be transmitted, then it is judged not to use the second channel and step 752 is executed.

[0172] Step 752: Select a new channel.

[0173] Taking the case where the session layer determines that the second channel cannot transmit data packets as an example, the session layer does not use the second channel and re - allocates the data block 2 that was originally transmitted through the second channel to the first channel for transmission.

[0174] Step 760: Transmit data packets through the channel.

[0175] When both the first channel and the second channel are working properly, the first channel transmits data packets 1 and 2 in data block 1, and the second channel transmits data packets 3 and 4 in data block 2; when the second channel has a situation where it cannot transmit data packets, the first channel sequentially transmits the data packets in data block 1 and data block 2. If the first channel has already started transmitting data packets in data block 3 at this time, after the first channel receives the data block 2 re - allocated by the session layer, it preferentially transmits data packets 3 and 4 in data block 2.

[0176] Step 770: Whether the data packets are acknowledged.

[0177] When all data packets are acknowledged, execute Step 780; when a certain data packet, for example, data packet 4 in data block 2 transmitted by the second channel, is not acknowledged, execute Step 752.

[0178] Step 780: Whether the data blocks are acknowledged.

[0179] When all data blocks are acknowledged, the video frame data reaches the receiving end reliably, and the above - mentioned process is repeated to finally transmit the complete video stream data; when a certain data block, for example, data block 2 transmitted by the second channel, is not acknowledged, execute Step 782.

[0180] Step 782: Feedback to the session layer.

[0181] When a certain data block is not acknowledged, for example, data block 2 transmitted by the second channel is not acknowledged, the second - channel module needs to feedback this situation to the session layer, and the session layer executes Step 740.

[0182] In summary, the method provided in this embodiment transmits video stream data through multiple transmission channels, flexibly selects the transmission channel, switches to other channels for transmission when a problem occurs in a certain channel, ensures the user's video viewing experience, and improves the service quality.

[0183] Figure 8 Shows the schematic architecture diagram of a multi - channel - based data packet transmission system provided by an exemplary embodiment of the present application.

[0184] The data transmission system according to the embodiment of the present application includes: a sending end 100 and a receiving end 200. Among them, the sending end 100 includes a sending end session layer 120, and the sending end session layer 120 includes a sending end first channel module 130 and a sending end second channel module 140. The sending end session layer 120 is a part of the application layer of the sending end 100; the receiving end 200 includes a receiving end session layer 180, and the receiving end session layer 180 includes a receiving end first channel module 160 and a receiving end second channel module 170. The receiving end session layer 180 is a part of the application layer of the receiving end 200. The sending end first channel module 130 corresponds to the receiving end first channel module 160, and the sending end second channel module 140 corresponds to the receiving end second channel module 170. These four channel modules are any corresponding channel modules. In this embodiment, these four channel modules are taken as examples for illustration, and the remaining channel modules will not be elaborated. All channel modules are managed by the session layer. The sending end 100 includes a traditional physical server, a cloud server, etc., and the receiving end 200 includes a mobile phone, a computer, etc.

[0185] Figure 9 and Figure 10 FIG. shows a schematic diagram of a multi-channel based data packet transmission method provided by an exemplary embodiment of the present application.

[0186] In this embodiment, taking the sending end 100 sending video stream data to the receiving end 200 as an example, the processing flow includes:

[0187] Step 1.1: The sending end session layer 120 splits the video frame 1 into data blocks 1, 2, 3, 4. The data block 1 includes data packets 1, 2, the data block 2 includes data packets 3, 4, the data block 3 includes data packets 5, 6, and the data block 4 includes data packets 7, 8. The video stream data is composed of video frame data. The sending end session layer 120 sends the video frame data in sequence and finally sends out the entire video stream data. The sending end session layer 120 splits a video frame data into multiple data blocks, and each data block contains the same or different numbers of data packets. The number of data blocks and the number of data packets contained in the data block are determined by the sending end session layer 120;

[0188] Step 2.1: The sending end session layer 120 allocates the data block 1 to the sending end first channel module 130. After the sending end session layer 120 splits the video frame data into data blocks, it sends the data packets in the data blocks to different channel modules in sequence. The working processes of different channel modules are the same. In this embodiment, the sending end session layer 120 allocating the data block to the sending end first channel module 130 and the sending end second channel module 140 are used for illustration. The sending end session layer 120 allocates the data block 1 including data packets 1, 2 to the sending end first channel module 130;

[0189] Step 2.2: The sender session layer 120 allocates data block 2 to the sender second channel module 140. The sender session layer 120 allocates data block 2, which includes data packets 3 and 4, to the sender second channel module 140;

[0190] Step 3.1: The sender first channel module 130 sends data packet 1 to the receiver first channel module 160, and the receiver first channel module 160 sends data packet 1 to the receiver session layer 180. The sender first channel module 130 sends data packet 1 and data packet 2 to the receiver first channel module 160 in sequence, and the receiver first channel module 160 sends data packet 1 and data packet 2 to the receiver session layer 180 in sequence;

[0191] Step 3.2: The sender second channel module 140 sends data packet 3 to the receiver second channel module 170, and the receiver second channel module 170 sends data packet 3 to the receiver session layer 180. The sender second channel module 140 sends data packet 3 and data packet 4 to the receiver second channel module 170 in sequence, and the receiver second channel module 170 sends data packet 3 and data packet 4 to the receiver session layer 180 in sequence;

[0192] Step 4.1: The receiver first channel module 160 sends an acknowledgement 1 (ACK1) of data packet 1 to the sender first channel module 130. After receiving data packet 1, the receiver first channel module 160 needs to return an acknowledgement 1 to the sender first channel module 130 to ensure the reliable arrival of data packet 1;

[0193] Step 4.2: The receiver second channel module 170 sends an acknowledgement 3 of data packet 3 to the sender second channel module 140. After receiving data packet 3, the receiver second channel module 170 needs to return an acknowledgement 3 to the sender second channel module 140 to ensure the reliable arrival of data packet 3;

[0194] Step 5.1: The sender first channel module 130 sends data packet 2 to the receiver first channel module 160, and the receiver first channel module 160 sends data packet 2 to the receiver session layer 180. The sender first channel module 130 sends data packet 1 and data packet 2 to the receiver first channel module 160 in sequence, and the receiver first channel module 160 sends data packet 1 and data packet 2 to the receiver session layer 180 in sequence;

[0195] Step 5.2: The sender second channel module 140 sends data packet 4 to the receiver second channel module 170. The sender second channel module 140 sends data packet 3 and data packet 4 to the receiver second channel module 170 in sequence;

[0196] Step 6.1: The receiving - end first - channel module 160 sends the acknowledgment 2 of data packet 2 to the sending - end first - channel module 130. After receiving data packet 2, the receiving - end first - channel module 160 needs to return the acknowledgment 2 to the sending - end first - channel module 130 to ensure the reliable arrival of data packet 2;

[0197] Step 6.2: The receiving - end second - channel module 170 sends the non - acknowledgment 4 (NACK4) of data packet 4 to the sending - end second - channel module 140. Due to packet loss, the receiving - end second - channel module 170 does not receive data packet 4. Therefore, the receiving - end second - channel module 170 sends the non - acknowledgment 4 of data packet 4;

[0198] Step 6.3: The receiving - end session layer 180 sorts the data packets and sends the acknowledgment 10 of data block 1 to the receiving - end first - channel module 160. The receiving - end session layer 180 arranges the data packets in the order of data packet 1, data packet 2, and data packet 3. And because all the data packets included in data block 1 are received, it sends the acknowledgment 10 of data block 1 to the receiving - end first - channel module 160 to ensure the reliable arrival of data block 1. And the receiving - end session layer 180 can, according to the network transmission situation, select other receiving - end channel modules with good transmission quality to send the acknowledgment;

[0199] Step 6.4: The receiving - end first - channel module 160 sends the acknowledgment 10 of data block 1 to the sending - end session layer 120. After receiving the acknowledgment 10, the receiving - end first - channel module 160 sends the acknowledgment 10 to the sending - end first - channel module 130. After receiving the acknowledgment 10, the sending - end first - channel module 130 sends the acknowledgment 10 to the sending - end session layer 120, and the sending - end session layer 120 destroys data block 1. This can ensure that the session layer can timely sense whether data block 1 has been acknowledged, so as to decide whether to re - transmit data block 1;

[0200] Step 7.1: The sending - end session layer 120 allocates data block 3 to the sending - end first - channel module 130. The sending - end session layer 120 allocates data block 3, which includes data packets 5 and 6, to the sending - end first - channel module 130;

[0201] Step 8.1: The sending - end first - channel module 130 sends data packet 5 to the receiving - end first - channel module 160, and the receiving - end first - channel module 160 sends data packet 5 to the receiving - end session layer 180. The sending - end first - channel module 130 sends data packet 5 and data packet 6 to the receiving - end first - channel module 160 in order, and the receiving - end first - channel module 160 sends data packet 5 and data packet 6 to the receiving - end session layer 180 in order;

[0202] Step 8.2: The second channel module 140 at the sending end re - sends packet 4 to the second channel module 170 at the receiving end. The second channel module 140 at the sending end sends packet 4 to the second channel module 170 at the receiving end again;

[0203] Step 9.1: The first channel module 160 at the receiving end sends an acknowledgment 5 of packet 5 to the first channel module 130 at the sending end. After receiving packet 5, the first channel module 160 at the receiving end needs to return an acknowledgment 5 to the first channel module 130 at the sending end to ensure the reliable arrival of packet 5;

[0204] Step 9.2: The second channel module 170 at the receiving end sends a non - acknowledgment 4 of packet 4 to the second channel module 140 at the sending end. Due to packet loss, the second channel module 170 at the receiving end did not receive packet 4, so the second channel module 170 at the receiving end sends a non - acknowledgment 4 of packet 4;

[0205] Step 10.1: The session layer 120 at the sending end re - allocates data block 2 to the first channel module 130 at the sending end. Since it always receives a non - acknowledgment 4 of packet 4, the session layer 120 at the sending end allocates data block 2 to the first channel module 130 at the sending end for transmission. At the same time, the second channel module 140 at the sending end continuously sends packet 4 to the second channel module 170 at the receiving end;

[0206] Step 11.1: The first channel module 130 at the sending end preferentially sends packet 3 to the first channel module 160 at the receiving end, and the first channel module 160 at the receiving end sends packet 3 to the session layer 180 at the receiving end. The first channel module 130 at the sending end sends packet 3 and packet 4 to the first channel module 160 at the receiving end in order, and the first channel module 160 at the receiving end sends packet 3 and packet 4 to the session layer 180 at the receiving end in order. Since the session layer sends packets in order, the re - transmitted packets should be sent before the packets being sent by the first channel module 130 at the sending end in order, that is, packet 3 and packet 4 have a higher priority than packet 6;

[0207] Step 12.1: The first channel module 160 at the receiving end sends an acknowledgment 3 of packet 3 to the first channel module 130 at the sending end. After receiving packet 3, the first channel module 160 at the receiving end needs to return an acknowledgment 3 to the first channel module 130 at the sending end to ensure the reliable arrival of packet 3;

[0208] Step 13.1: The sending - end first - channel module 130 preferentially sends data packet 4 to the receiving - end first - channel module 160, and the receiving - end first - channel module 160 sends data packet 4 to the receiving - end session layer 180. The sending - end first - channel module 130 sends data packets 3 and 4 to the receiving - end first - channel module 160 in sequence, and the receiving - end first - channel module 160 sends data packets 3 and 4 to the receiving - end session layer 180 in sequence;

[0209] Step 14.1: The receiving - end first - channel module 160 sends an acknowledgment 4 of data packet 4 to the sending - end first - channel module 130. After receiving data packet 4, the receiving - end first - channel module 160 needs to return an acknowledgment 4 to the sending - end first - channel module 130 to ensure the reliable arrival of data packet 4;

[0210] Step 15.1: The receiving - end session layer 180 sorts the data packets and sends an acknowledgment 20 of data block 2 to the receiving - end first - channel module 160. The receiving - end session layer 180 arranges the data packets in the order of data packets 1, 2, 3, 4, 5, and because all the data packets included in data block 2 are received, it sends an acknowledgment 20 of data block 2 to the receiving - end first - channel module 160 to ensure the reliable arrival of data block 2;

[0211] Step 15.2: The receiving - end first - channel module 160 sends an acknowledgment 20 of data block 2 to the sending - end session layer 120. After receiving the acknowledgment 20, the receiving - end first - channel module 160 sends the acknowledgment 20 to the sending - end first - channel module 130. After receiving the acknowledgment 20, the sending - end first - channel module 130 sends the acknowledgment 20 to the sending - end session layer 120, and the sending - end session layer 120 destroys data block 2;

[0212] Step 16.1: The sending - end first - channel module 130 sends data packet 6 to the receiving - end first - channel module 160, and the receiving - end first - channel module 160 sends data packet 6 to the receiving - end session layer 180. The sending - end first - channel module 130 sends data packets 5 and 6 to the receiving - end first - channel module 160 in sequence, and the receiving - end first - channel module 160 sends data packets 5 and 6 to the receiving - end session layer 180 in sequence;

[0213] Step 17.1: The receiving - end first - channel module 160 sends an acknowledgment 6 of data packet 6 to the sending - end first - channel module 130. After receiving data packet 6, the receiving - end first - channel module 160 needs to return an acknowledgment 6 to the sending - end first - channel module 130 to ensure the reliable arrival of data packet 6;

[0214] Step 18.1: The receiving - end session layer 180 sorts the data packets and sends the acknowledgment 30 of data block 3 to the receiving - end first channel module 160. The receiving - end session layer 180 arranges the data packets in the order of data packets 1, 2, 3, 4, 5, 6. And because all the data packets included in data block 3 are received, it sends the acknowledgment 30 of data block 3 to the receiving - end first channel module 160 to ensure the reliable arrival of data block 3;

[0215] Step 18.2: The receiving - end first channel module 160 sends the acknowledgment 30 of data block 3 to the sending - end session layer 120. After receiving the acknowledgment 30, the receiving - end first channel module 160 sends the acknowledgment 30 to the sending - end first channel module 130. After receiving the acknowledgment 30, the sending - end first channel module 130 sends the acknowledgment 30 to the sending - end session layer 120, and the sending - end session layer 120 destroys data block 3;

[0216] Step 19.1: The sending - end session layer 120 allocates data block 4 to the sending - end first channel module 130. Since the sending - end session layer 120 determines that there is a problem with the sending - end second channel module 140, it allocates data block 4 to the sending - end first channel module 130 for sending;

[0217] Step 20.1: The sending - end first channel module 130 sends data packet 7 to the receiving - end first channel module 160, and the receiving - end first channel module 160 sends data packet 7 to the receiving - end session layer 180. The sending - end first channel module 130 sends data packets 7 and 8 to the receiving - end first channel module 160 in order, and the receiving - end first channel module 160 sends data packets 7 and 8 to the receiving - end session layer 180 in order;

[0218] Step 21.1: The receiving - end first channel module 160 sends the acknowledgment 7 of data packet 7 to the sending - end first channel module 130. After receiving data packet 7, the receiving - end first channel module 160 needs to return the acknowledgment 7 to the sending - end first channel module 130 to ensure the reliable arrival of data packet 7;

[0219] Step 22.1: The sending - end first channel module 130 sends data packet 8 to the receiving - end first channel module 160, and the receiving - end first channel module 160 sends data packet 8 to the receiving - end session layer 180. The sending - end first channel module 130 sends data packets 7 and 8 to the receiving - end first channel module 160 in order, and the receiving - end first channel module 160 sends data packets 7 and 8 to the receiving - end session layer 180 in order;

[0220] Step 23.1: The receiving end's first channel module 160 sends the acknowledgment 8 of data packet 8 to the sending end's first channel module 130. After receiving data packet 8, the receiving end's first channel module 160 needs to return the acknowledgment 8 to the sending end's first channel module 130 to ensure the reliable arrival of data packet 8;

[0221] Step 24.1: The receiving end's session layer 180 sorts the data packets and sends the acknowledgment 40 of data block 4 to the receiving end's first channel module 160. The receiving end's session layer 180 arranges the data packets in the order of data packets 1, 2, 3, 4, 5, 6, 7, 8, and because all the data packets included in data block 4 have been received, it sends the acknowledgment 40 of data block 4 to the receiving end's first channel module 160 to ensure the reliable arrival of data block 4;

[0222] Step 24.2: The receiving end's first channel module 160 sends the acknowledgment 40 of data block 4 to the sending end's session layer 120. After receiving the acknowledgment 40, the receiving end's first channel module 160 sends the acknowledgment 40 to the sending end's first channel module 130. After receiving the acknowledgment 40, the sending end's first channel module 130 sends the acknowledgment 40 to the sending end's session layer 120, and the sending end's session layer 120 destroys data block 4;

[0223] Step 25.1: The sending end's second channel module 140 sends data packet 4 to the receiving end's second channel module 170, and the receiving end's second channel module 170 sends data packet 4 to the receiving end's session layer 180. During the operation of other channel modules, the sending end's second channel module 140 continuously sends data packet 4 to the receiving end's second channel module 170 until the sending is successful. After the successful sending, the sending end's session layer 120 can allocate another data block to the sending end's second channel module 140;

[0224] Step 26.1: The receiving end's second channel module 170 sends the acknowledgment 4 of data packet 4 to the sending end's second channel module 140. After receiving data packet 4, the receiving end's second channel module 170 needs to return the acknowledgment 4 to the sending end's second channel module 140 to ensure the reliable arrival of data packet 4;

[0225] Step 27.1: The receiving end's session layer 180 merges and deduplicates data packet 4 and sends the acknowledgment 20 of data block 2. After the receiving end's second channel module 170 sends data packet 4 to the receiving end's session layer 180, since the receiving end's session layer 180 has already received data packet 4 sent by the receiving end's first channel module 160 before, it merges and deduplicates the data packet 4 sent by the receiving end's second channel module 170, only keeping the data packet 4 sent by the receiving end's first channel module 160, and then sends the acknowledgment 20 of data block 2 to the receiving end's second channel module 170;

[0226] Step 27.2: The receiving - end second - channel module 170 sends the acknowledgment 20 of data block 2 to the sending - end session layer 120. After the receiving - end second - channel module 170 receives the acknowledgment 20, it sends the acknowledgment 20 to the sending - end second - channel module 140. After the sending - end second - channel module 140 receives the acknowledgment 20, it sends the acknowledgment 20 to the sending - end session layer 120, informing the sending - end session layer 120 that it can allocate data blocks to the sending - end second - channel module 140 again.

[0227] In summary, the method provided in this embodiment performs data transmission through multiple transmission channels, flexibly selects the transmission channels, and adds a reliable transmission mechanism at the session layer to ensure the reliability of video frames. At the same time, by using the re - ordering and merging and duplicate - checking mechanisms at the receiving - end session layer, the integrity and correctness of data packets are ensured, effectively improving the smoothness and real - time performance of the video.

[0228] Figure 11 The block diagram of a data - packet transmission device based on multiple channels provided by an exemplary embodiment of the present application is shown. The device includes:

[0229] A determination module 1110, configured to determine at least two data - packet sets based on the data to be transmitted, and each data - packet set includes at least one data packet;

[0230] A transmission module 1120, configured to allocate at least two data - packet sets to at least two transmission channels for transmission according to the set granularity, and the data packets in the same data - packet set are transmitted via the same transmission channel;

[0231] A re - transmission module 1130, configured to re - allocate the first data - packet set to the second transmission channel for transmission when the transmission of the first data - packet set fails in the first transmission channel among at least two transmission channels.

[0232] In a possible design of this embodiment, the types of at least two transmission channels are at least one of a wireless local - area network transmission channel, a mobile network transmission channel, a wired - network transmission channel, and an optical - fiber transmission channel.

[0233] In a possible design of this embodiment, the re - transmission module 1130 is configured to select the transmission channel with the fastest network speed among the transmission channels other than the first transmission channel based on historical transmission information as the second transmission channel;

[0234] Or, select the transmission channel with the smallest load among the transmission channels other than the first transmission channel as the second transmission channel;

[0235] Or, select the transmission channel with the smallest packet - loss rate among the transmission channels other than the first transmission channel as the second transmission channel.

[0236] In a possible design of this embodiment, the determining module 1110 is configured to determine that the transmission of the first data packet set fails when the number of retransmissions of the data packets in the first data packet set in the first transmission channel reaches a first threshold;

[0237] Or, determine that the transmission of the first data packet set fails when the transmission duration after the data packets in the first data packet set start to be transmitted reaches a second threshold and no acknowledgment feedback for the first data packet set is received;

[0238] Or, determine that the transmission of the first data packet set fails when the allocated duration for the first data packet set to be allocated to the first transmission channel reaches a third threshold and no acknowledgment feedback for the first data packet set is received;

[0239] Or, determine that the transmission of the first data packet set fails when a retransmission request or a negative acknowledgment feedback for the first data packet set is received.

[0240] In a possible design of this embodiment, the retransmission priority of the data packets in the first data packet set is higher than the priority of the initial transmission data packets in the second transmission channel.

[0241] In a possible design of this embodiment, the transmitted data includes video stream data, and the transmission module 1120 is configured to divide the video stream data into at least two data packet sets based on the video frame granularity;

[0242] Or, divide the video stream data into at least two data packet sets based on the data block granularity in the video frame.

[0243] In a possible design of this embodiment, the transmission module 1120 is configured to allocate at least two data packet sets to at least two transmission channels for transmission according to the network speed ratio of the at least two data packet sets;

[0244] Wherein, the ratio of the number of data packets allocated to each transmission channel matches the network speed ratio.

[0245] In a possible design of this embodiment, the transmission module 1120 is configured to allocate the data packet set related to the key data in the at least two data packet sets to the transmission channel with the lowest packet loss rate among at least two transmission channels for transmission;

[0246] Or, allocate the data packet set related to the key data in the at least two data packet sets to the transmission channel with the fastest network speed among at least two transmission channels for transmission;

[0247] Wherein, the key data includes at least one of key frame data in the cloud video scenario, operation data in the cloud game scenario, and voice data in the video call scenario. The key data has the highest transmission priority.

[0248] A destruction module 1140, configured to, when receiving an acknowledgement feedback for a second data packet set transmitted through a second transmission channel, allocate a third data packet set to the second transmission channel for transmission, and destroy the data packets in the cached second data packet set.

[0249] Figure 12 The block diagram of a multi-channel-based data packet transmission device provided by an exemplary embodiment of the present application is shown. The device includes:

[0250] A receiving module 1210, configured to receive at least two data packet sets through at least two transmission channels, each data packet set including at least one data packet, and the data packets in the same data packet set being transmitted through the same transmission channel;

[0251] A re-receiving module 1220, configured to re-receive a first data packet set through a second transmission channel when the first transmission channel among at least two transmission channels fails to receive the first data packet set.

[0252] In a possible design of this embodiment, the types of at least two transmission channels include at least one of a wireless local area network transmission channel, a mobile network transmission channel, a wired network transmission channel, and an optical fiber transmission channel.

[0253] In a possible design of this embodiment, the data to be transmitted includes video stream data, and at least two data packet sets are obtained by partitioning based on video frame granularity;

[0254] Or, at least two data packet sets are obtained by partitioning based on the data block granularity in a video frame.

[0255] In a possible design of this embodiment, the re-receiving module 1220 is configured to select the transmission channel with the fastest network speed among the transmission channels other than the first transmission channel as the second transmission channel;

[0256] Or, select the transmission channel with the least load among the transmission channels other than the first transmission channel as the second transmission channel;

[0257] Or, select the transmission channel with the lowest packet loss rate among the transmission channels other than the first transmission channel as the second transmission channel.

[0258] In a possible design of this embodiment, the retransmission priority of the data packets in the first data packet set is higher than the priority of the initially transmitted data packets in the second transmission channel.

[0259] The sending module 1230 is configured to send an acknowledgment feedback for the second data packet set when receiving the second data packet set transmitted through the second transmission channel; and send an acknowledgment feedback for the first data packet set when receiving the first data packet set transmitted through the second transmission channel.

[0260] The reordering module 1240 is configured to reorder data packets received through at least two transmission channels.

[0261] Figure 13 FIG. shows a schematic structural diagram of a computer device provided by an exemplary embodiment of the present application. The structures of the sending device and the receiving device are the same as that of the computer device. Generally, the computer device 1300 includes a processor 1301 and a memory 1302.

[0262] The processor 1301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1301 may be implemented in at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the Central Processing Unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1301 may be integrated with a Graphics Processing Unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1301 may further include an Artificial Intelligence (AI) processor for processing computational operations related to machine learning.

[0263] The memory 1302 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 1302 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1302 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1301 to implement the method and / or usage method of the image classification model provided in the method embodiments of the present application.

[0264] In some embodiments, the server 1300 may further optionally include: an input interface 1303 and an output interface 1304. The processor 1301, the memory 1302, the input interface 1303, and the output interface 1304 may be connected through a bus or signal lines. Each peripheral device may be connected to the input interface 1303 and the output interface 1304 through a bus, signal lines, or a circuit board. The input interface 1303 and the output interface 1304 may be used to connect at least one peripheral device related to input / output (I / O) to the processor 1301 and the memory 1302. In some embodiments, the processor 1301, the memory 1302, the input interface 1303, and the output interface 1304 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1301, the memory 1302, the input interface 1303, and the output interface 1304 may be implemented on a separate chip or circuit board, and the embodiments of the present application do not limit this.

[0265] Those skilled in the art can understand that the structure shown above does not constitute a limitation on the server 1300. The server 1300 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component layout.

[0266] In an exemplary embodiment, there is also provided a computer program product, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor reads and executes the computer instructions to implement the training method and / or the usage method of the image classification model provided by the above-mentioned method embodiments.

[0267] In an exemplary embodiment, there is also provided a computer-readable storage medium, in which a computer program is stored, and the computer program is loaded and executed by a processor to implement the training method and / or the usage method of the image classification model provided by the above-mentioned method embodiments.

[0268] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a disk, or an optical disc, etc.

[0269] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. The computer-readable medium includes computer storage media and communication media, where the communication media includes any medium that facilitates the transfer of a computer program from one place to another. The storage media can be any available medium accessible by a general-purpose or special-purpose computer.

[0270] The above are only alternative embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A multi-channel based data packet transmission method, characterized in that, The method is executed by a sending device, and at least two transmission channels are established between the sending device and the receiving device. The method includes: Determining at least two data packet sets based on the data to be transmitted, where each data packet set includes at least one data packet; the number of the at least two data packet sets and the number of data packets included in each data packet set are determined by the session layer of the sending device; Allocating the at least two data packet sets to the at least two transmission channels for transmission according to the set granularity by the session layer of the sending device, and the data packets in the same data packet set are transmitted via the same transmission channel; In the case where the transmission of the first data packet set fails on the first transmission channel among the at least two transmission channels, reallocating the first data packet set to the second transmission channel for transmission, and continuously attempting to transmit the data packets that have failed to be transmitted in the first data packet set through the first transmission channel; if the transmission order of the first data packet set is before the initially transmitted data packets in the second transmission channel, the retransmission priority of the data packets in the first data packet set is higher than the priority of the initially transmitted data packets; If a first acknowledgment feedback of the first data packet set transmitted by the second transmission channel is received, allocating other data packet sets to the second transmission channel for transmission, and at the same time destroying the data packets in the cached first data packet set; the first acknowledgment feedback is sent by the receiving device when it determines that all the data packets in the first data packet set have been received after reordering the received data packets; If a second acknowledgment feedback of the first data packet set transmitted by the first transmission channel is received, allocating other data packet sets to the first transmission channel for transmission; the second acknowledgment feedback is sent by the receiving device after receiving all the data packets in the first data packet set transmitted by the first transmission channel and merging and deduplicating the data packets in the first data packet set; The sending device, the receiving device, and at least two transmission channels operate at the application layer.

2. The method according to claim 1, characterized in that, The step of reallocating the first data packet set to the second transmission channel for transmission in the case where the transmission of the first data packet set fails on the first transmission channel among the at least two transmission channels includes: In the case where the transmission of the first data packet set fails on the first transmission channel among the at least two transmission channels, selecting the second transmission channel from the transmission channels other than the first transmission channel based on historical transmission information; Reallocating the first data packet set to the second transmission channel for transmission.

3. The method according to claim 2, characterized in that The step of selecting the second transmission channel from the transmission channels other than the first transmission channel based on historical transmission information includes: Selecting the transmission channel with the fastest network speed from the transmission channels other than the first transmission channel as the second transmission channel; or, Selecting the transmission channel with the lowest load from the transmission channels other than the first transmission channel as the second transmission channel; or, Selecting the transmission channel with the lowest packet loss rate from the transmission channels other than the first transmission channel as the second transmission channel.

4. The method according to any one of claims 1 to 3, characterized in that The method further includes: In the case that the number of retransmissions of the data packets in the first data packet set in the first transmission channel reaches a first threshold, it is determined that the transmission of the first data packet set fails; or, In the case that the transmission duration after the start of transmission of the data packets in the first data packet set reaches a second threshold and no acknowledgement feedback of the first data packet set is received, it is determined that the transmission of the first data packet set fails; or, In the case that the allocated duration for the first data packet set allocated to the first transmission channel reaches a third threshold and no acknowledgement feedback of the first data packet set is received, it is determined that the transmission of the first data packet set fails; or, In the case that a retransmission request or a negative acknowledgement feedback of the first data packet set is received, it is determined that the transmission of the first data packet set fails.

5. The method according to any one of claims 1 to 3, characterized in that, The data to be transmitted includes video stream data; determining at least two data packet sets based on the data to be transmitted includes: Dividing the video stream data into at least two data packet sets based on video frame granularity; Or, Dividing the video stream data into at least two data packet sets based on data block granularity in a video frame.

6. The method according to any one of claims 1 to 3, characterized in that The method further includes: In the case that an acknowledgement feedback of a second data packet set transmitted by the second transmission channel is received, allocating a third data packet set to the second transmission channel for transmission.

7. The method according to any one of claims 1 to 3, characterized in that The method further includes: In the case that an acknowledgement feedback of a second data packet set transmitted by the second transmission channel is received, destroying the data packets in the cached second data packet set.

8. A multi-channel based data packet transmission method, characterized in that, The method is executed by a receiving end device, and at least two transmission channels are established between the receiving end device and a sending end device. The method includes: Receiving at least two data packet sets through the at least two transmission channels, each data packet set includes at least one data packet, and the data packets in the same data packet set are transmitted through the same transmission channel; the number of the at least two data packet sets and the number of data packets included in each data packet set are determined by the session layer of the sending end device; the allocation of the at least two data packet sets to the at least two transmission channels for transmission is executed by the session layer of the sending end device; In the case that the first transmission channel in the at least two transmission channels fails to receive a first data packet set, receiving the first data packet set again through a second transmission channel, and continuously attempting to receive the data packets that fail to be transmitted in the first data packet set through the first transmission channel; if the transmission order of the first data packet set is before the initially transmitted data packets in the second transmission channel, the retransmission priority of the data packets in the first data packet set is higher than the priority of the initially transmitted data packets; When it is determined that all the data packets in the first data packet set are received through the second transmission channel after reordering the received data packets, transmitting a first acknowledgement feedback of the first data packet set to the sending end device through the second transmission channel, so that the sending end device allocates other data packet sets to the second transmission channel for transmission, and at the same time destroying the data packets in the cached first data packet set; After determining that all data packets in the first data packet set are received through the first transmission channel and the data packets in the first data packet set are merged and de-duplicated, transmit a second acknowledgment feedback of the first data packet set to the sending device through the first transmission channel, so that the sending device allocates other data packet sets to the first transmission channel for transmission; The sending device, the receiving device, and at least two transmission channels operate at the application layer.

9. A multi-channel-based data packet transmission device, characterized in that, The apparatus includes: A determination module, configured to determine at least two data packet sets based on data to be transmitted, each data packet set including at least one data packet; the number of the at least two data packet sets and the number of data packets included in each data packet set are determined by the session layer of the sending device; A transmission module, configured to allocate the at least two data packet sets to at least two transmission channels established between the receiving device and the sending device according to the set granularity by the session layer of the sending device for transmission, and data packets in the same data packet set are transmitted through the same transmission channel; A retransmission module, configured to, when the transmission of a first data packet set fails on a first transmission channel among the at least two transmission channels, re-allocate the first data packet set to a second transmission channel for transmission; if the transmission order of the first data packet set is before the initially transmitted data packets in the second transmission channel, the retransmission priority of the data packets in the first data packet set is higher than the priority of the initially transmitted data packets; A module for performing the following steps: when the first data packet set is re-allocated to the second transmission channel for transmission, continuously attempt to transmit the data packets that failed to be transmitted in the first data packet set through the first transmission channel; A module for performing the following steps: if a first acknowledgment feedback of the first data packet set transmitted through the second transmission channel is received, allocate other data packet sets to the second transmission channel for transmission, and simultaneously destroy the data packets in the cached first data packet set; the first acknowledgment feedback is sent when the receiving device determines that all data packets in the first data packet set have been received after re-ordering the received data packets; A module for performing the following steps: if a second acknowledgment feedback of the first data packet set transmitted through the first transmission channel is received, allocate other data packet sets to the first transmission channel for transmission; the second acknowledgment feedback is sent after the receiving device receives all data packets in the first data packet set transmitted through the first transmission channel and merges and de-duplicates the data packets in the first data packet set; The sending device, the receiving device, and at least two transmission channels operate at the application layer.

10. The device according to claim 9, wherein The retransmission module is configured to: When the transmission of a first data packet set fails on a first transmission channel among the at least two transmission channels, select the second transmission channel from the transmission channels other than the first transmission channel based on historical transmission information; Re-allocate the first data packet set to the second transmission channel for transmission.

11. The device according to claim 10, wherein, The retransmission module is configured to: Select the transmission channel with the fastest network speed among the transmission channels other than the first transmission channel as the second transmission channel; or, Select the transmission channel with the lowest load among the transmission channels other than the first transmission channel as the second transmission channel; Or, Select the transmission channel with the lowest packet loss rate among the transmission channels other than the first transmission channel as the second transmission channel.

12. The device according to any one of claims 9 to 11, characterized in that, The determining module is further configured to: Determine that the first data packet set transmission fails when the number of retransmissions of the data packets in the first data packet set in the first transmission channel reaches a first threshold; or, Determine that the first data packet set transmission fails when the transmission duration after the start of transmission of the data packets in the first data packet set reaches a second threshold and no acknowledgment feedback of the first data packet set is received; Or, Determine that the first data packet set transmission fails when the allocated duration of the first data packet set allocated to the first transmission channel reaches a third threshold and no acknowledgment feedback of the first data packet set is received; Or, Determine that the first data packet set transmission fails when a retransmission request or negative acknowledgment feedback of the first data packet set is received.

13. The device according to any one of claims 9 to 11, characterized in that The data to be transmitted includes video stream data; the determining module is configured to: Divide the video stream data into at least two data packet sets based on video frame granularity; Or, Divide the video stream data into at least two data packet sets based on the data block granularity in the video frame.

14. The device according to any one of claims 9 to 11, characterized in that, The device further includes a module for performing the following steps: Allocate a third data packet set to the second transmission channel for transmission when an acknowledgment feedback of the second data packet set transmitted by the second transmission channel is received.

15. The device according to any one of claims 9 to 11, characterized in that The device further includes a destruction module for: Destroy the data packets in the cached second data packet set when an acknowledgment feedback of the second data packet set transmitted by the second transmission channel is received.

16. A multi-channel-based data packet transmission device, characterized in that, The device includes: A receiving module, configured to receive at least two data packet sets through at least two transmission channels established between a receiving end device and a sending end device. Each data packet set includes at least one data packet, and the data packets in the same data packet set are transmitted via the same transmission channel; the number of the at least two data packet sets and the number of data packets included in each data packet set are determined by the session layer of the sending end device; the allocation of the at least two data packet sets to the at least two transmission channels for transmission is performed by the session layer of the sending end device; A re-receiving module, configured to re-receive the first data packet set through a second transmission channel when the first transmission channel in the at least two transmission channels fails to receive the first data packet set; if the transmission order of the first data packet set is before the initially transmitted data packets in the second transmission channel, the retransmission priority of the data packets in the first data packet set is higher than the priority of the initially transmitted data packets; A module for performing the following steps: when the first set of data packets is received again through the second transmission channel, continuously attempt to receive the data packets that failed to be transmitted in the first set of data packets through the first transmission channel at the same time; A module for performing the following steps: after reordering the received data packets, when it is determined that all the data packets in the first set of data packets are received through the second transmission channel, transmit a first acknowledgment feedback of the first set of data packets to the sending device through the second transmission channel, so that the sending device allocates other sets of data packets to the second transmission channel for transmission, and at the same time destroy the data packets in the cached first set of data packets; A module for performing the following steps: when it is determined that all the data packets in the first set of data packets are received through the first transmission channel and the data packets in the first set of data packets are merged and deduplicated, transmit a second acknowledgment feedback of the first set of data packets to the sending device through the first transmission channel, so that the sending device allocates other sets of data packets to the first transmission channel for transmission; The sending device, the receiving device, and at least two transmission channels operate at the application layer.

17. A transmitting device, characterized in that, The sending device includes: a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the data packet transmission method according to any one of claims 1 to 7.

18. A receiving-end device, characterized in that, The receiving device includes: a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the data packet transmission method according to claim 8.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the data packet transmission method according to any one of claims 1 to 8.

20. A computer program product, characterized in that, The computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium, and the processor obtains the computer instructions from the computer-readable storage medium, so that the processor loads and executes to implement the data packet transmission method according to any one of claims 1 to 8.

Images