Video data transmission method, apparatus, device, and storage medium

Abstract

The application discloses a video data transmission method and device, equipment and a storage medium, and relates to the technical field of data transmission. A plurality of parameters, such as actual round-trip delay and packet loss detection duration, can directly reflect real-time network status. The compensation duration required for retransmitting a target video data packet if the target video data packet is lost under the current network status is accurately determined. Then, by comparing the size between the local cache duration fed back by a video receiving end and the packet loss compensation duration, it is determined whether the transmission strategy of retransmitting the target data packet for packet loss compensation will affect the video playing effect of the video receiving end. Then, a suitable target transmission strategy is selected from a transmission strategy set, the network service quality of a video transmission service is improved, and the video experience of a user is ensured.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, specifically to a video data transmission method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of network technology, the demand for real-time video transmission services such as video-on-demand, live streaming, and multi-person real-time communication is increasing. The Quality of Service (QoS) of real-time video transmission directly affects the user experience, so the quality requirements for low latency and smoothness of real-time video transmission services are also getting higher and higher.

[0003] However, real-time video transmission services rely on the network, and network conditions have a significant impact on these services. When network conditions fluctuate, it can also affect real-time video transmission, leading to issues such as data packet loss and low transmission efficiency. Ultimately, this can result in intermittent audio and stuttering video, severely impacting the user experience.

[0004] Therefore, improving the network service quality of video transmission services and ensuring user experience are urgent issues that need to be addressed. Summary of the Invention

[0005] This application provides a video data transmission method, apparatus, device, and storage medium to improve video data transmission efficiency and enhance the network service quality of video transmission services.

[0006] On the one hand, a video data transmission method is provided, applied at a video sending end, the method comprising:

[0007] Determine whether the target video data packet to be transmitted belongs to the first video frame in the video stream;

[0008] If the target video data packet does not belong to the first video frame, then the packet loss compensation duration of the target video data packet is determined based on the actual round-trip time of each video data packet transmitted in the video stream and the packet loss detection duration fed back by the video receiver; wherein, the packet loss detection duration represents the reference duration required for historical packet loss feedback of the video stream, and the packet loss compensation duration represents the compensation duration required to retransmit the target video data packet if it is lost;

[0009] Based on the relationship between the local buffer duration sent by the video receiver and the packet loss compensation duration, a corresponding target transmission strategy is determined from the transmission strategy set, wherein the local buffer duration represents the number of video data packets buffered by the video receiver.

[0010] The target video data packet is transmitted using the target transmission strategy.

[0011] On the one hand, a video data transmission device is provided, comprising:

[0012] The determining unit is used to determine whether the target video data packet to be transmitted belongs to the first video frame in the video stream.

[0013] The calculation unit is configured to determine the packet loss compensation duration of the target video data packet if the target video data packet does not belong to the first video frame, based on the actual round-trip time of each video data packet already transmitted in the video stream and the packet loss detection duration fed back by the video receiver; wherein, the packet loss detection duration represents the reference duration required for historical packet loss feedback of the video stream, and the packet loss compensation duration represents the compensation duration required for retransmitting the target video data packet if it is lost;

[0014] The decision unit is used to determine the corresponding target transmission strategy from the transmission strategy set based on the relationship between the local buffer duration sent by the video receiver and the packet loss compensation duration, wherein the local buffer duration represents the number of video data packets buffered by the video receiver.

[0015] A transmission unit is used to transmit the target video data packet using the target transmission strategy.

[0016] Optionally, the computing unit is specifically used for:

[0017] For each of the transmitted video data packets, perform the following operations respectively:

[0018] For a given video data packet, the actual round-trip time of the video data packet is determined based on the sending and receiving times of the video data packet.

[0019] Wherein, the sending time is the time when the video sending end sends the data packet, and the receiving time is the time when the video sending end receives the confirmation information of the video data packet.

[0020] Optionally, the computing unit is specifically used for:

[0021] The reference round-trip time of the video stream is determined based on the actual round-trip time of the video data packet and the actual round-trip time of the video data packet sent before the video data packet.

[0022] The packet loss compensation duration is determined based on the reference round-trip time and the packet loss detection duration.

[0023] Optionally, the decision-making unit is specifically used for:

[0024] When the local buffer duration is greater than or equal to the packet loss compensation duration, the first transmission strategy is determined as the target transmission strategy;

[0025] When the local buffer duration is less than the packet loss compensation duration, the second transmission strategy is determined as the target transmission strategy.

[0026] Optionally, the transmission unit is specifically used for:

[0027] Send the target video data packet to the video receiving end;

[0028] If a packet loss indication is received from the video receiver regarding the target video data packet, the target video data packet will be retransmitted to the video receiver.

[0029] Optionally, the transmission unit is specifically used for:

[0030] Based on the video frames to which the target video data packet belongs, a redundant data packet is generated.

[0031] Send the target video data packet to the video receiving end; and

[0032] The redundant data packets are sent to the video receiver so that the video receiver can recover the target video data packet based on the redundant data packets when the target video data packet is lost.

[0033] Optionally, the transmission unit is specifically used for:

[0034] If the target video data packet belongs to the first video frame, then the second transmission strategy is determined as the target transmission strategy;

[0035] The target video data packet is transmitted using the target transmission strategy.

[0036] On one hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0037] On the one hand, a computer storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of any of the above methods.

[0038] On one hand, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the methods described above.

[0039] The beneficial effects of the embodiments of this application are as follows:

[0040] In this embodiment, the method determines whether the target video data packet to be transmitted belongs to the first video frame in the video stream. If it does not, the packet loss compensation duration for the target video data packet is determined based on the actual round-trip time of each video data packet already transmitted in the video stream and the packet loss detection duration reported by the video receiver. According to the relationship between the local buffer duration sent by the video receiver and the packet loss compensation duration, a corresponding target transmission strategy is determined from the transmission strategy set, and the target video data packet is transmitted using this target transmission strategy. This method accurately determines the compensation duration required to retransmit the target video data packet if it is lost under the current network conditions by using multiple parameters that directly reflect the real-time network status, such as the actual round-trip time and packet loss detection duration. Furthermore, by comparing the local buffer duration reported by the video receiver with the packet loss compensation duration, it determines whether the transmission strategy of retransmitting the target data packet for packet loss compensation will affect the video playback effect at the video receiver. Finally, a suitable target transmission strategy is selected from the transmission strategy set to improve the network service quality of video transmission services and ensure the user's video experience.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 This is a schematic diagram illustrating an application scenario provided in the embodiments of this application;

[0044] Figure 2 A flowchart illustrating a video data transmission method provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the structure of a video data packet provided in an embodiment of this application;

[0046] Figure 4 A schematic diagram illustrating a packet loss compensation process provided in an embodiment of this application;

[0047] Figure 5 A schematic diagram of a local buffer for a video receiving device provided in an embodiment of this application;

[0048] Figure 6 A flowchart illustrating another video data transmission method provided in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram of the structure of a video data transmission device provided in an embodiment of this application;

[0050] Figure 8 This is a schematic diagram of the composition structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0052] To facilitate understanding of the technical solutions provided in the embodiments of this application, some key terms used in the embodiments of this application will be explained below:

[0053] Automatic Repeat Request (ARQ): also known as packet loss retransmission, is a channel protection algorithm that corrects errors by retransmitting critical data packets.

[0054] Forward error correction (FEC): A channel coding algorithm that recovers lost data packets by adding redundant data.

[0055] The design concept of the embodiments of this application is briefly introduced below:

[0056] With the rapid development of network technology, the demand for real-time video transmission services such as video-on-demand, live streaming, and multi-person real-time communication is increasing. The QoS (Quality of Service) of real-time video transmission directly impacts the user experience, thus placing increasingly higher demands on the low latency and smoothness of real-time video transmission. However, real-time video transmission relies on the network, and network conditions have a significant impact on it. Fluctuations in network conditions can affect real-time video transmission, leading to issues such as data packet loss and low transmission efficiency. Ultimately, this results in intermittent audio and stuttering video, severely impacting the user experience.

[0057] To address the issue of data packet loss during data transmission, related technologies often employ either the Forward Error Correction (FEC) strategy or the Advanced Data Query (ARQ) strategy to ensure normal data transmission even in the event of packet loss. The FEC strategy involves inserting redundant data packets into the source data packet to be transmitted and sending both the source and redundant packets to the receiving end. This allows the receiving end to repair the lost source data packets using the redundant packets, provided the number of lost packets does not exceed the forward error correction capability limit. However, this method does not consider rapidly changing network conditions. When no data packets are lost, the redundant packets are not utilized and instead consume network bandwidth, resulting in a waste of network resources.

[0058] The ARQ strategy involves the receiver reporting packet loss to the sender upon detecting it, and the sender then retransmitting the packet to resolve the issue. However, this method consumes time in both the receiver's detection and reporting of packet loss, and the sender's receipt of the feedback and retransmission. This increases data transmission time and reduces efficiency. In video transmission scenarios, if the video data buffered at the receiver is played out during packet retransmission, it can cause intermittent audio and stuttering in real-time video, severely impacting user experience.

[0059] In view of the above problems, this application provides a video data transmission method. It determines whether the target video data packet to be transmitted belongs to the first video frame in the video stream. If not, it determines the packet loss compensation duration of the target video data packet based on the actual round-trip time of each transmitted video data packet in the video stream and the packet loss detection duration reported by the video receiver. Based on the relationship between the local buffer duration sent by the video receiver and the packet loss compensation duration, a corresponding target transmission strategy is determined from a set of transmission strategies, and the target video data packet is transmitted using this target transmission strategy. This method accurately determines the compensation duration required to retransmit the target video data packet if it is lost under the current network conditions by using multiple parameters that directly reflect the real-time network status, such as the actual round-trip time and packet loss detection duration. Furthermore, by comparing the local buffer duration reported by the video receiver with the packet loss compensation duration, it determines whether the transmission strategy of retransmitting the target data packet for packet loss compensation will affect the video playback effect at the video receiver. Finally, it selects a suitable target transmission strategy from the set of transmission strategies, improving the network service quality of video transmission services and ensuring the user's video experience.

[0060] Furthermore, in order to accurately select a suitable transmission strategy for the target data packet and improve the efficiency of video data transmission, this embodiment of the application determines the actual round-trip time of each video data packet by using the sending and receiving times of each transmitted video data packet. Based on the actual round-trip time of a video data packet and the actual round-trip time of video data packets sent before it, the reference round-trip time of the video stream is accurately determined. By using the reference round-trip time, which can reflect the current network status in real time, and the packet loss detection duration, the packet loss compensation duration of the target data packet is finally determined, so as to select a suitable transmission strategy for the target data packet and improve the efficiency of video data transmission.

[0061] To further improve video data transmission efficiency, this application embodiment forces the video data packet of the first video frame of the video stream to be transmitted using a second transmission strategy. By sending redundant data packets and video data packets together to the video receiver, the video receiver can recover the video data packets through the redundant data packets when the video data packets are lost, thus solving the problem of data packet loss during data transmission and ensuring that the transmission time of the first video frame is short, achieving the effect of low-latency transmission of the first frame. This avoids the first transmission strategy based on retransmission mechanism, which would increase the data transmission time of the first frame of the video stream, causing increased latency in the first frame and thus affecting the user's video experience.

[0062] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0063] The solution provided in this application can be applied to any video data transmission scenario, such as video surveillance, live streaming, video conferencing, video-on-demand, and multi-person communication, etc., and will not be listed one by one here. Figure 1 The diagram shown is an application scenario provided by an embodiment of this application. In this scenario, a video sending device 101 and a video receiving device 102 may be included.

[0064] The video transmitting device 101 can be a computer device with certain processing capabilities, such as a mobile phone, personal computer (PC), camera, surveillance camera, or server, etc., which can be configured to execute any of the devices provided in the embodiments of this application. For ease of description, the following describes the implementation of the method using a server capable of executing the method as the executing subject. It should be understood that using a server as the executing subject is merely an illustrative example and should not be construed as a limitation of the method. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, but is not limited to these. Furthermore, the video transmitting device 101 can also be configured with a database, which can be used to store data such as video data packets, actual round-trip time, packet loss detection duration, packet loss compensation duration, local buffer duration, and transmission strategy set involved in the scheme provided in the embodiments of this application.

[0065] The video receiving device 102 can be any computer device with certain processing capabilities, such as a mobile phone, personal computer (PC), tablet computer (PAD), laptop computer, desktop computer, mobile internet device (MID), smart TV, smart in-vehicle device, and smart wearable device, etc., that can be connected to the video sending device 101 to provide local video services to users. This embodiment does not make any specific limitations.

[0066] During the process of transmitting a video stream to a video receiving device 102, the video transmitting device 101 determines whether the target video data packet to be transmitted belongs to the first video frame in the video stream using the video data transmission method provided in this application embodiment. If it is determined that it does not belong to the first video frame, the video transmitting device 101 calculates the packet loss compensation duration of the target video data packet, obtains the local buffer duration at the current moment from the video receiving device 102, and determines the corresponding target transmission strategy from the transmission strategy set based on the relationship between the local buffer duration and the packet loss compensation duration, and transmits the target video data packet to the video receiving device 102.

[0067] In this embodiment, the video transmitting device 101 and the video receiving device 102 can be directly or indirectly connected via one or more networks 103. The network 103 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network, or any other possible network. This embodiment of the invention does not limit the types of networks used.

[0068] It should be noted that, Figure 1 The examples shown are merely illustrative. In reality, the number of video transmitting devices 101 and video receiving devices 102 is not limited and is not specifically limited in this embodiment.

[0069] The video data transmission method provided by the exemplary embodiments of this application will be described below with reference to the accompanying drawings and the application scenarios described above. It should be noted that the application scenarios described above are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect.

[0070] See Figure 2 The diagram shown is a flowchart illustrating the video data transmission method provided in this application embodiment. The example uses a video transmitting device as the execution subject. The specific implementation flow of this method is as follows:

[0071] Step 201: Determine whether the target video data packet to be transmitted belongs to the first video frame in the video stream. If yes, proceed to step 205; otherwise, proceed to step 202.

[0072] In this embodiment of the application, during the video data transmission process of the video stream to the video receiving device, the video sending device determines the subsequent processing flow corresponding to the video data packet by judging whether the target video data packet to be transmitted belongs to the first video frame (i.e. the first video frame) of the video stream.

[0073] In one possible implementation, since each video frame in the video stream has a large amount of data, the video transmitting device divides each video frame of the video stream to be transmitted into a specific number of video data packets for transmission during the video data transmission process, in order to reduce network bandwidth pressure and improve the transmission efficiency of video data.

[0074] Specifically, such as Figure 3 The video transmitting device shown can divide each video frame into N video data packets according to the size of the Maximum Transmission Unit (MTU), where N is the number of video data packets into which each video frame is divided, as shown below:

[0075]

[0076] Step 202: If the target video data packet does not belong to the first video frame, then determine the packet loss compensation duration of the target video data packet based on the actual round-trip time of each video data packet already transmitted in the video stream and the packet loss detection duration reported by the video receiver.

[0077] In this embodiment of the application, after the video transmitting device determines that the target video data to be transmitted does not belong to the first video frame, it will calculate the compensation time required to retransmit the target video data packet when the target video data packet is lost by using the actual round-trip time of each transmitted video data packet and the packet loss detection time that represents the historical packet loss feedback of the video stream.

[0078] In one possible implementation, Round-Trip Time (RTT) is an important performance indicator in network transmission. It represents the total delay from the time the sender begins sending data until the sender receives an acknowledgment from the receiver (the receiver sends an acknowledgment immediately after receiving the data). RTT varies with network congestion and, to some extent, reflects the real-time status of the network transmission link. A video transmitting device can determine the actual RTT for each video data packet by using its respective send and receive times. The send time is the moment the video sender transmits the data packet, and the receive time is the moment the video sender receives the acknowledgment information for the video data packet.

[0079] Specifically, during video data transmission, when the video receiving device receives the video data packet transmitted by the video sending device, it will send back a corresponding response data packet to the video sending device. When the video sending device receives the response data packet, it records the time point of the response data packet (i.e., the receiving time t1). Combined with the previously recorded sending time t2 of the video data packet, the RTT value of each video data packet can be calculated as t1-t2.

[0080] In one possible implementation, due to the variability of network conditions during actual data transmission, the Round-Trip Time (RTT) of data packets transmitted between the receiving and sending ends changes with variations in network conditions. The difference between the maximum and minimum RTT is defined as network jitter, used to characterize network stability. Specifically, if the maximum RTT is 20 milliseconds and the minimum RTT is 5 milliseconds, then network jitter is 15 milliseconds; a higher jitter value indicates a more unstable network. Therefore, to reduce the impact of network jitter and avoid fluctuations caused by drastic changes in RTT values, after each video data packet is sent and its actual RTT is determined, the video transmitting device can determine the latest reference RTT of the video stream based on the actual RTT of the video data packet and the actual RTT of video data packets sent before it. This allows the reference RTT value to more accurately reflect the real-time status of the network transmission link.

[0081] In one possible implementation, the video transmitting device calculates the reference round-trip time of the video stream based on the actual round-trip time of each video data packet that has been transmitted, and iteratively updates the reference round-trip time of the video stream using the actual round-trip time of the video data packet after each new video data packet is transmitted, so as to obtain the latest reference round-trip time of the video stream.

[0082] Specifically, a preset smoothing factor can be used to statistically smooth the actual round-trip time of the target video data packet at the current moment, as well as the reference round-trip time of the video stream before the target video data packet was sent, to obtain the reference round-trip time of the video stream characterizing the network transmission link status at the current moment, as shown below:

[0083] RTT′(t)=α×RTT(t)+(1-α)×RTT′(t-1)

[0084] Where t represents the target video data packet sent at time t, RTT(t) represents the actual round-trip time of the target video data packet sent at time t, and RTT′(t) is the reference round-trip time of the video stream at time t. t-1 represents the video data packets sent before the target video data packet is sent, and RTT′(t-1) is the reference round-trip time of the video stream at the time preceding time t. α is a pre-set smoothing factor, whose value can be 0.8-0.9, etc., without much limitation. Using the smoothing factor α as the weight in the weighted average calculation, RTT(t) and RTT′(t-1) are weighted and averaged according to the above formula to obtain the reference round-trip time RTT′(t) of the video stream at the current time.

[0085] In one possible implementation, the video transmitting device determines the reference round-trip time of the video stream, and can determine the packet loss compensation time for the target data packet by using the reference round-trip time and the packet loss detection duration.

[0086] Specifically, such as Figure 4 As shown, when a target video data packet is lost, the packet loss compensation process for retransmitting the target video data packet is as follows: The video receiving device determines that the target video data packet is lost and immediately sends a packet loss indication to the video sending device. Upon receiving the packet loss indication, the video sending device immediately retransmits the target video data packet to the video receiving device. Therefore, the packet loss compensation duration for the target data packet is the total value of the time required from the time the video sending device sends the target video data packet to the video receiving device until the video receiving device determines that the data packet is lost (i.e., the packet loss detection duration t3), the time required from the time the video receiving device determines that the data packet is lost until the video sending device receives its packet loss indication t4, and the time required from the time the video sending device receives the packet loss indication to the time required to retransmit the target video data packet to the video receiving device t5. The sum of t4 and t5 is equal to the round-trip time (RTT) of the current network transmission link. Therefore, the sum of the reference RTT′ and the packet loss detection duration t3 can be used as the packet loss compensation duration for the target data packet. For example, the video receiving device determines that its packet loss detection time is 40ms and feeds it back to the video sending device. The video sending device determines the reference round-trip time (RTT) of the current network transmission link as 100ms by sending data packets and receiving response data packets, and thus calculates the packet loss compensation time for retransmitting the target data packet as 140ms.

[0087] In one possible implementation, at the end of each video frame transmission in the video stream, the video receiving device determines the latest packet loss detection duration at the current moment and feeds it back to the video sending device. The packet loss detection duration can be determined by the video receiving device based on the time of the most recent detected packet loss, or it can be the average duration determined based on the detection time of each packet loss in history.

[0088] Step 203: Based on the relationship between the local buffer duration and the packet loss compensation duration sent by the video receiver, determine the corresponding target transmission strategy from the transmission strategy set.

[0089] In this embodiment of the application, after determining the packet loss compensation duration of the target data packet, the video transmitting device compares the local buffer duration of the current moment fed back by the video receiving device with the packet loss compensation duration, and determines the corresponding target transmission strategy from the transmission strategy set according to the preset mapping relationship based on the comparison result.

[0090] In one possible implementation, such as Figure 5 As shown, the local buffering time of a video receiving device represents the number of video data packets buffered at the video receiving end. The video player in the video receiving device usually buffers a certain number of video data packets before starting video playback, and adaptively adjusts the local buffering time according to the real-time network conditions. For example, when the network conditions are good, the local buffering time will be reduced to achieve low-latency video playback and improve the user experience, while the local buffering time will be increased when the network conditions are poor to resist network jitter.

[0091] Specifically, the video receiving device can determine the number of buffered video frames by counting the number of video data packets in the buffer at the current moment, and calculate the duration of each video frame according to the preset frame rate, thereby determining the local buffer duration at the current moment. For example, if the frame rate is 25 frames per second, one video frame represents 40ms.

[0092] In one possible implementation, the transmission strategy set may include a first transmission strategy based on a retransmission mechanism and a second transmission strategy based on redundant data transmission. When the local buffer duration is greater than or equal to the packet loss compensation duration, the first transmission strategy is determined as the target transmission strategy. When the local buffer duration is less than the packet loss compensation duration, the second transmission strategy is determined as the target transmission strategy.

[0093] In one possible implementation, the first transmission strategy based on the retransmission mechanism can be, but is not limited to, an ARQ transmission strategy. After the video transmitting device sends the target video data packet to the video receiving device, if it receives a packet loss indication from the video receiving device, it retransmits the target video data packet to the video receiving device to resolve the data packet loss issue. Since retransmitting the target video data packet requires a certain packet loss compensation period, if all the video data packets already buffered locally by the video receiving device are played during this period, and packet loss compensation is not completed, playback delays and video playback stuttering will occur until packet loss compensation is complete, affecting the user experience. Therefore, when the local buffering time is determined to be greater than or equal to the packet loss compensation time, the first transmission strategy based on the retransmission mechanism can be used for data transmission. Even if a certain packet loss compensation time is consumed, the video data already buffered locally by the video receiving device will not be played out completely, preventing video playback delays.

[0094] Specifically, when the video receiving device determines that its packet loss detection duration is 40ms and its local buffer duration is 500ms, it feeds this information back to the video sending device. The video sending device determines that the reference round-trip time (RTT) of the current network transmission link is 100ms, and thus calculates that the packet loss compensation duration for retransmitting the target data packet is 140ms. Since the local buffer duration is 500ms, even after playing 140ms of video data packets, there are still 360ms of video data packets available for playback. Therefore, adopting the first transmission strategy based on retransmission will not cause video playback delay. However, if the second transmission strategy is used for transmission at this time, it will introduce redundant data packets that occupy bandwidth, resulting in a waste of bandwidth resources.

[0095] In one possible implementation, the second transmission strategy based on redundant data transmission can be, but is not limited to, the FEC transmission strategy. The video transmitting device generates redundant data packets from the various video data packets included in the video frame to which the target video data packet belongs, sends the target video data packet to the video receiving end, and also sends the redundant data packets to the video receiving end. This allows the video receiving end to recover the target video data packet based on the redundant data packets when packet loss occurs. Therefore, when the video transmitting device determines that the local buffer duration is less than the packet loss compensation duration, it can use the second transmission strategy based on redundant data transmission for data transmission, thus avoiding retransmission of the target data packet and preventing unplayable buffered data packets during retransmission, which could lead to video playback delay.

[0096] In one possible implementation, when the video receiving device determines that its packet loss detection duration is 40ms and its local buffering duration is 100ms and feeds this information back to the video sending device, the video sending device determines that the reference round-trip time (RTT′) of the current network transmission link is 100ms, and thus calculates that the packet loss compensation duration for retransmitting the target data packet is 140ms. During the 140ms retransmission of the target data packet, the video receiving device has already played all the buffered data packets that were already in the buffer at 100ms, resulting in a 40ms video stutter. When the target data packet is retransmitted after 140ms, the video receiving device will also experience an additional 40ms delay when resuming playback. Therefore, the video sending device decides to adopt a second transmission strategy based on redundant data transmission to avoid using the first transmission strategy, which would increase the video data transmission time and lead to increased video latency.

[0097] Step 204: Transmit the target video data packet using the target transmission strategy.

[0098] In this embodiment of the application, after the video sending device determines the target transmission strategy from the transmission strategy set, it uses the target transmission strategy to send the target video data packet to the video receiving device, so as to solve the data packet loss problem and avoid increasing video latency and affecting user experience.

[0099] In one possible implementation, if the video transmitting device adopts a first transmission strategy for data transmission, the video transmitting device can assign a sequence identifier to each video data packet when transmitting each video data packet of the video frame. The sequence identifier indicates the order in which the video data packet is transmitted in the video frame. The video receiving device detects whether there is a data packet loss by monitoring the sequence identifier of each video data packet. Therefore, when the video receiving device does not receive or receives an incorrect data packet, it can send a retransmission request to the video transmitting device through the signaling channel. When the video transmitting device receives the retransmission request, it will retransmit the video data packet.

[0100] In one possible implementation, if the video transmitting device uses the second transmission strategy for data transmission, it can generate nk redundant data packets by performing FEC encoding on k source video data packets included in the video frame to which the target video data packet belongs. The k source video data packets and nk redundant data packets are then merged into an FEC data block, where the ratio of the number of source video data packets to redundant data packets is fixed. The video transmitting device sends these n FEC data blocks together to the video receiving device, so that after receiving the FEC data blocks, the video receiving device can recover the lost or erroneous source video data packets using the FEC decoding algorithm and the redundant data packets. The FEC encoding and decoding algorithm can be, but is not limited to, the Reeds-Solomon algorithm, the Raptor algorithm, and the Tornado algorithm.

[0101] Step 205: Determine the second transmission strategy as the target transmission strategy, and use the second transmission strategy to transmit the target video data packets.

[0102] In this embodiment of the application, if the video transmitting device determines that the target video data packet belongs to the first video frame data packet of the video stream to be transmitted, then the second transmission strategy is determined as the target transmission strategy and the target video data packet is transmitted.

[0103] Specifically, since the video receiving device does not have any buffered video data packets available for playback when the video sending device starts transmitting the first video frame of the video stream to the video receiving device, the second transmission strategy is used to transmit all video data packets for the first video frame. The lost video data packets are recovered through redundant data packets, without the need to retransmit data packets and cause video playback delay. This allows the first frame to be transmitted in the shortest possible time, achieving a low-latency video playback effect for the first frame.

[0104] In one possible implementation, such as Figure 6 The image shows another video data transmission method provided in an embodiment of this application. The specific implementation process of this method is as follows:

[0105] Step 601: The video transmitting device determines whether the target video data packet to be transmitted belongs to the first video frame of the video stream. If so, proceed to step 602; otherwise, proceed to step 603.

[0106] Step 602: Transmit the target video data packet to the video receiving device using the FEC transmission strategy.

[0107] Step 603: Obtain and update the local buffer duration t6 and packet loss detection duration t3 at the current moment from the video receiving device.

[0108] Step 604: Determine the reference round-trip time (RTT′) of the video stream at the current moment.

[0109] Step 605: Determine whether t6 is not less than the sum of t3 and RTT′. If so, proceed to step 606; otherwise, proceed to step 602.

[0110] Step 606: Transmit the target video data packet to the video receiving device using the ARQ transmission strategy.

[0111] In summary, the embodiments of this application enforce the FEC strategy during the transmission of the first frame, and dynamically implement the FEC or ARQ transmission strategy based on information such as the client's playback buffer duration, packet loss detection duration, and network round-trip latency during the transmission of subsequent video frames. This ensures that no additional latency is added during the video frame transmission process, and avoids transmitting redundant packets unnecessarily, saving valuable bandwidth resources and achieving the goal of low-latency transmission and playback without introducing excessive redundant bandwidth.

[0112] Please see Figure 7 Based on the same inventive concept, embodiments of this application also provide a video data transmission device 70, which includes:

[0113] The determining unit 701 is used to determine whether the target video data packet to be transmitted belongs to the first video frame in the video stream;

[0114] The calculation unit 702 is used to determine the packet loss compensation duration of the target video data packet if the target video data packet does not belong to the first video frame, based on the actual round-trip time of each video data packet that has been transmitted in the video stream and the packet loss detection duration fed back by the video receiver; wherein, the packet loss detection duration represents the reference duration required for historical packet loss feedback of the video stream, and the packet loss compensation duration represents the compensation duration required for retransmitting the target video data packet if it is lost.

[0115] Decision unit 703 is used to determine the corresponding target transmission strategy from the transmission strategy set based on the relationship between the local buffer duration and the packet loss compensation duration sent by the video receiver. The local buffer duration represents the number of video data packets buffered by the video receiver.

[0116] The transmission unit 704 is used to transmit target video data packets using a target transmission strategy.

[0117] Optionally, the computing unit 702 is specifically used for:

[0118] For each transmitted video data packet, perform the following operations respectively:

[0119] For a given video data packet, determine the actual round-trip time based on the sending and receiving times of the video data packet;

[0120] The sending time is the moment when the video sending end sends the data packet, and the receiving time is the moment when the video sending end receives the confirmation information of the video data packet.

[0121] Optionally, the computing unit 702 is specifically used for:

[0122] The reference round-trip time of the video stream is determined based on the actual round-trip time of the video data packets and the actual round-trip time of the video data packets sent before the video data packets.

[0123] The packet loss compensation duration is determined based on the reference round-trip time and packet loss detection duration.

[0124] Optional, decision unit 703, specifically used for:

[0125] When the local buffer duration is greater than or equal to the packet loss compensation duration, the first transmission strategy is determined as the target transmission strategy;

[0126] When the local buffer duration is less than the packet loss compensation duration, the second transmission strategy will be determined as the target transmission strategy.

[0127] Optionally, the transmission unit 704 is specifically used for:

[0128] Send the target video data packet to the video receiving end;

[0129] If a packet loss indication is received from the video receiver regarding the target video data packet, the target video data packet will be retransmitted to the video receiver.

[0130] Optionally, the transmission unit 704 is specifically used for:

[0131] Based on the video frames to which the target video data packet belongs, generate redundant data packets.

[0132] Send the target video data packet to the video receiving end; and,

[0133] Redundant data packets are sent to the video receiver so that the video receiver can recover the target video data packet based on the redundant data packets when the target video data packet is lost.

[0134] Optionally, the transmission unit 704 is specifically used for:

[0135] If the target video data packet belongs to the first video frame, then the second transmission strategy is determined as the target transmission strategy;

[0136] The target video data packets are transmitted using a target transmission strategy.

[0137] Using the aforementioned device, multiple parameters that directly reflect the real-time network status, such as actual round-trip time and packet loss detection duration, can accurately determine the compensation time required to retransmit the target video data packet if it is lost under the current network conditions. Then, by comparing the local buffer duration reported by the video receiver with the packet loss compensation time, it can be determined whether the transmission strategy of retransmitting the target data packet for packet loss compensation will affect the video playback effect at the video receiver. Finally, a suitable target transmission strategy can be selected from the set of transmission strategies to improve the network service quality of video transmission services and ensure the user's video experience.

[0138] For ease of description, the above sections are divided into functional units (or modules) and described separately. Of course, in implementing this application, the functions of each unit (or module) can be implemented in one or more software or hardware components. This device can be used to execute the methods shown in the embodiments of this application; therefore, the functions that each functional module of the device can achieve can be referred to the descriptions of the foregoing embodiments, and will not be repeated here.

[0139] Please see Figure 8 Based on the same technical concept, embodiments of this application also provide a computer device. In one embodiment, as shown in the figure, the computer device may include a memory 801, a communication module 803, and one or more processors 802.

[0140] The memory 801 is used to store computer programs executed by the processor 802. The memory 801 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, and the data storage area can store various operation instruction sets, etc.

[0141] Memory 801 may be volatile memory, such as random-access memory (RAM); memory 801 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 801 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 801 may be a combination of the above-described memories.

[0142] The processor 802 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 802 is used to implement the aforementioned video data transmission method when it calls the computer program stored in the memory 801.

[0143] The communication module 803 is used to communicate with video receiving devices or other network devices.

[0144] This application embodiment does not limit the specific connection medium between the memory 801, communication module 803, and processor 802 described above. This application embodiment... Figure 8 The memory 801 and the processor 802 are connected via a bus 804, and the bus 804 is in Figure 8The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 804 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 8 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0145] The memory 801 stores a computer storage medium, which stores computer-executable instructions for implementing the video data transmission method of the embodiments of this application. The processor 802 is used to execute the video data transmission methods of the above embodiments.

[0146] Based on the same inventive concept, embodiments of this application also provide a storage medium storing a computer program, which, when executed on a computer, causes a computer processor to perform the steps in the video data transmission method according to various embodiments of this application described above.

[0147] In some possible implementations, various aspects of the video data transmission method provided in this application can also be implemented as a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the video data transmission method according to the various exemplary embodiments of this application described above. For example, the computer device can perform the steps of the various embodiments.

[0148] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0149] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a computing device. However, the program product of this application is not limited thereto. In this application, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0150] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0151] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0152] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0153] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0154] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0155] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0157] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A video data transmission method, characterized in that, Applied to a video sending end, the method includes: Determine whether the target video data packet to be transmitted belongs to the first video frame in the video stream; If the target video data packet does not belong to the first video frame, then the packet loss compensation duration of the target video data packet is determined based on the actual round-trip time of each video data packet transmitted in the video stream and the packet loss detection duration fed back by the video receiver; wherein, the packet loss detection duration represents the reference duration required for historical packet loss feedback of the video stream, and the packet loss compensation duration represents the compensation duration required to retransmit the target video data packet if it is lost; Based on the relationship between the local buffer duration sent by the video receiver and the packet loss compensation duration, a corresponding target transmission strategy is determined from the transmission strategy set, wherein the local buffer duration represents the number of video data packets buffered by the video receiver. The target video data packet is transmitted using the target transmission strategy.

2. The method as described in claim 1, characterized in that, Before determining the packet loss compensation duration for the target video data packet based on the actual round-trip time of each video data packet transmitted in the video stream and the packet loss detection duration reported by the video receiver, the method further includes: For each of the transmitted video data packets, perform the following operations respectively: For a given video data packet, the actual round-trip time of the video data packet is determined based on the sending and receiving times of the video data packet. Wherein, the sending time is the time when the video sending end sends the data packet, and the receiving time is the time when the video sending end receives the confirmation information of the video data packet.

3. The method as described in claim 2, characterized in that, After determining the actual round-trip time of the video data packet based on the transmission and reception times, the method further includes: The reference round-trip time of the video stream is determined based on the actual round-trip time of the video data packet and the actual round-trip time of the video data packet sent before the video data packet. Based on the actual round-trip time of each video data packet transmitted in the video stream and the packet loss detection duration reported by the video receiver, the packet loss compensation duration for the target video data packet is determined, including: The packet loss compensation duration is determined based on the reference round-trip time and the packet loss detection duration.

4. The method according to any one of claims 1 to 3, characterized in that, The transmission strategy set includes a first transmission strategy based on a retransmission mechanism and a second transmission strategy based on redundant data transmission. Then, based on the relationship between the local buffer duration sent by the video receiver and the packet loss compensation duration, a corresponding target transmission strategy is determined from the transmission strategy set, including: When the local buffer duration is greater than or equal to the packet loss compensation duration, the first transmission strategy is determined as the target transmission strategy; When the local buffer duration is less than the packet loss compensation duration, the second transmission strategy is determined as the target transmission strategy.

5. The method as described in claim 4, characterized in that, When the target transmission strategy is the first transmission strategy, then transmitting the target video data packet using the target transmission strategy includes: Send the target video data packet to the video receiving end; If a packet loss indication is received from the video receiver regarding the target video data packet, the target video data packet will be retransmitted to the video receiver.

6. The method as described in claim 4, characterized in that, When the target transmission strategy is the second transmission strategy, then transmitting the target video data packet using the target transmission strategy includes: Based on the video frames to which the target video data packet belongs, a redundant data packet is generated. Send the target video data packet to the video receiving end; and The redundant data packets are sent to the video receiver so that the video receiver can recover the target video data packet based on the redundant data packets when the target video data packet is lost.

7. The method as described in claim 4, characterized in that, After determining whether the target video data packet to be transmitted belongs to the first video frame in the video stream, the method further includes: If the target video data packet belongs to the first video frame, then the second transmission strategy is determined as the target transmission strategy; The target video data packet is transmitted using the target transmission strategy.

8. A video data transmission device, characterized in that, The device includes: The determining unit is used to determine whether the target video data packet to be transmitted belongs to the first video frame in the video stream. The calculation unit is configured to determine the packet loss compensation duration of the target video data packet if the target video data packet does not belong to the first video frame, based on the actual round-trip time of each video data packet already transmitted in the video stream and the packet loss detection duration fed back by the video receiver; wherein, the packet loss detection duration represents the reference duration required for historical packet loss feedback of the video stream, and the packet loss compensation duration represents the compensation duration required for retransmitting the target video data packet if it is lost; The decision unit is used to determine the corresponding target transmission strategy from the transmission strategy set based on the relationship between the local buffer duration sent by the video receiver and the packet loss compensation duration, wherein the local buffer duration represents the number of video data packets buffered by the video receiver. A transmission unit is used to transmit the target video data packet using the target transmission strategy.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer storage medium storing computer program instructions thereon, characterized in that, When executed by a processor, the computer program instructions implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product comprising computer program instructions, characterized in that, When executed by a processor, the computer program instructions implement the steps of the method according to any one of claims 1 to 7.

Images