Data transmission method and device, medium, electronic equipment and program product

Abstract

The invention provides a data transmission method and device, a medium, electronic equipment and a program product, and relates to the technical field of data transmission, the data transmission method comprises the following steps: determining an original transmission rate according to feedback information sent by a receiving end; the feedback information is feedback information sent by the receiving end based on a historical video frame received before a target video frame, and the target video frame is a video frame to be transmitted currently; determining the frame type of the target video frame, wherein the frame type represents the size of the target video frame; determining a target transmission rate according to the original transmission rate and the frame type of the target video frame; and transmitting the target video frame to the receiving end based on the target transmission rate. By determining the frame type of the current to-be-transmitted target video frame, the more accurate target transmission rate is obtained, so that the target transmission rate can be more adaptive to the dynamic change of the traffic, the problem of data transmission congestion caused by traffic suddenness and intermittency is avoided, and the quality of a live video is improved.

Description

Technical Field

[0001] This disclosure relates to the field of data transmission technology, specifically to a data transmission method, apparatus, medium, electronic device, and program product. Background Technology

[0002] With the increasing number of live streaming users and their share of internet traffic, data transmission in dynamic network environments faces increasingly severe challenges. This trend requires us to achieve lower transmission latency, higher transmission bitrate, and lower playback stuttering rates to ensure a smooth and high-quality live streaming experience. However, due to its bursty and intermittent traffic patterns, live video streams are prone to congestion during data transmission, leading to latency and stuttering. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, this disclosure provides a data transmission method, including: The original transmission rate is determined based on the feedback information sent by the receiving end; the feedback information is the feedback information sent by the receiving end based on historical video frames received before the target video frame, and the target video frame is the video frame to be transmitted at present. Determine the frame type of the target video frame, wherein the frame type represents the size of the target video frame; The target transmission rate is determined based on the original transmission rate and the frame type of the target video frame; Based on the target transmission rate, the target video frame is transmitted to the receiving end.

[0005] Secondly, this disclosure provides a data transmission apparatus, comprising: The first determining module is configured to determine the original transmission rate based on feedback information sent by the receiving end; the feedback information is feedback information sent by the receiving end based on historical video frames received before the target video frame, and the target video frame is the video frame to be transmitted. The second determining module is configured to determine the frame type of the target video frame, wherein the frame type represents the size of the target video frame; The third determining module is configured to determine the target transmission rate based on the original transmission rate and the frame type of the target video frame; The transmission module is configured to transmit the target video frame to the receiving end based on the target transmission rate.

[0006] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0007] Fourthly, this disclosure provides an electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] Based on the above technical solution, a basic original transmission rate can be determined in advance based on the feedback information sent by the receiving end, and the frame type of the target video frame to be transmitted, i.e. the size of the target video frame, can be determined in real time to determine the current data transmission pressure. Then, the original transmission rate can be corrected according to the frame type of the target video frame to obtain a more accurate target transmission rate. This allows the target transmission rate to better adapt to the dynamic changes in traffic volume, avoid data transmission congestion caused by the suddenness and intermittency of traffic, and improve the quality of live video.

[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart illustrating a data transmission method according to some embodiments.

[0012] Figure 2 This is a data processing schematic diagram illustrating a data transmission method according to some embodiments.

[0013] Figure 3 This is a schematic diagram of the structure of a data transmission device according to some embodiments.

[0014] Figure 4 This is a schematic diagram of the structure of an electronic device according to some embodiments. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0017] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0018] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0019] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0020] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0021] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0022] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0023] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0024] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0025] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0026] Figure 1 This is a flowchart illustrating a data transmission method according to some embodiments. Figure 2 This is a data processing diagram illustrating a data transmission method according to some embodiments, such as... Figure 1 and Figure 2 As shown, this disclosure provides a data transmission method, which can be executed by a data transmission device, which can be implemented in software and / or hardware. Figure 1 As shown, the method may include the following steps.

[0027] In step S110, the original transmission rate is determined based on the feedback information sent by the receiving end; the feedback information is the feedback information sent by the receiving end based on the historical video frames received before the target video frame, and the target video frame is the video frame to be transmitted.

[0028] In this embodiment, the data transmission method can be applied to the sending end, which sends data to the receiving end so that the receiving end can load and display the data upon receipt. The data to be transmitted can be video data, including multiple video frames, such as video data in a live broadcast scenario. The original transmission rate for transmitting the target video frame can be determined based on feedback information sent by the receiving end.

[0029] The historical video frame can be at least one video frame received by the receiving end before the target video frame; for example, the historical video frame can be a single video frame. During data transmission, each video frame can be split into multiple data packets for transmission. After the receiving end receives a complete video frame each time, it can send feedback information to the sending end based on that video frame. This feedback information may include the receiving rate of the received video frame, the queuing delay of the multiple transmitted video frames during transmission, and the size of the playback buffer in the receiving end.

[0030] While providing feedback in data packets allows for more timely responses to network conditions, it also incurs bandwidth overhead. In video transmission, video is decoded and played frame by frame. The receiving end's buffer collects a complete frame before delivering it to the decoder, focusing more on the transmission time of each frame. Using frame-level feedback, sending a feedback message to the sending end as soon as the receiving end has collected a complete video frame, better meets the needs of video transmission.

[0031] In step S120, the frame type of the target video frame is determined, and the frame type represents the size of the target video frame.

[0032] In this embodiment, the frame type can be distinguished by the size of the video frame. For example, the frame type may include large frames or small frames, and the frame type of the target video frame can be determined based on the size of the target video frame.

[0033] Optionally, the frame type of the target video frame can be determined directly based on the frame size of the target video frame, or it can be determined based on the number of data packets of the target video frame and the receiving rate of the video frame received by the receiving end.

[0034] It should be noted that when the frame type of the target video frame is determined directly based on the frame size of the target video frame, the execution order of steps S110 and S120 is not limited. Step S110 can be executed first and then step S120, or step S120 can be executed first and then step S110, or steps S110 and S120 can be executed simultaneously.

[0035] When determining the frame type of a target video frame based on the number of data packets in the target video frame and the receiving rate of the video frame received by the receiving end, step S110 must be executed first to obtain the receiving rate in the feedback information, and then the frame type of the target video frame is determined based on the receiving rate.

[0036] In step S130, the target transmission rate is determined based on the original transmission rate and the frame type of the target video frame.

[0037] In this embodiment, the frame type of the target video frame can characterize the current transmission traffic volume and can be used to determine the current data transmission pressure. Different frame types can correspond to different rate correction parameters. Based on the frame type of the target video frame, the corresponding rate correction parameter can be determined to further correct the original transmission rate, resulting in a more accurate target transmission rate. This allows the target transmission rate to better adapt to dynamic changes in traffic volume, avoiding data transmission congestion caused by sudden and intermittent traffic, and improving the quality of live video.

[0038] In step S140, the target video frame is transmitted to the receiving end based on the target transmission rate.

[0039] In this embodiment, after determining the target transmission rate corresponding to the target video frame, the target video frame can be transmitted to the receiving end based on the target transmission rate, so that the transmission rate can better meet the current video transmission requirements, avoid congestion caused by excessive transmission rate during data transmission, and avoid video playback stuttering caused by insufficient transmission rate.

[0040] In this embodiment, a basic initial transmission rate can be determined in advance based on feedback information sent by the receiving end, and the frame type of the target video frame to be transmitted, i.e., the size of the target video frame, can be determined in real time to assess the current data transmission pressure. The initial transmission rate is then adjusted according to the frame type of the target video frame to obtain a more accurate target transmission rate. This allows the target transmission rate to better adapt to dynamic changes in traffic volume, avoiding data transmission congestion caused by sudden and intermittent traffic spikes, and improving the quality of live video.

[0041] In some possible implementations, the feedback information may include the reception rate of the video frames received by the receiver.

[0042] Determine the frame type of the target video frame, including: Based on the receiving rate, determine the bottleneck link bandwidth; based on the bottleneck link bandwidth, the maximum data packet size in the historical transmission process, and the target transmission time, obtain the classification threshold; based on the number of data packets in the target video frame and the classification threshold, obtain the frame type of the target video frame.

[0043] In this embodiment, the frame type of the target video frame can be determined based on the number of data packets in the target video frame and the receiving rate of the video frame received by the receiving end. The receiving rate of the video frame received by the receiving end can include the receiving rates of multiple video frames received by the receiving end within a preset time period before the target video frame; for example, the preset time period can be 2 seconds. For any video frame received by the receiving end, the receiving rate of that video frame can be determined based on the time from the start of reception to the completion of reception. Since there are frame intervals between video frames, calculating the receiving rate on a data packet basis would include the time when no data is transmitted; calculating the receiving rate on a frame basis is more accurate.

[0044] After obtaining the reception rate, the bottleneck link bandwidth can be determined based on the reception rates of multiple video frames. Optionally, the bottleneck link bandwidth can be determined based on the maximum value among the reception rates of multiple video frames.

[0045] Furthermore, a classification threshold can be obtained based on the bottleneck link bandwidth, combined with the maximum data packet size during historical transmissions and the target transmission time. The target transmission time can be the maximum allowed transmission time for a small frame, or half the frame interval. For example, at 30fps (Frames Per Second), with a current frame interval of 33ms, the target transmission time could be half the frame interval, i.e., 16ms.

[0046] The formula for calculating the classification threshold is as follows:

[0047] in, The classification threshold, For the target transmission time, For bottleneck link bandwidth, This represents the maximum data packet size during the historical transmission process.

[0048] The classification threshold is a threshold representing the number of data packets within a video frame. Based on the number of data packets in the target video frame and the classification threshold, the frame type of the target video frame can be determined. The frame type can include a first type and a second type; the first type can be a large frame, and the second type can be a small frame. If the number of data packets in the target video frame is greater than or equal to the classification threshold, the frame type of the target video frame can be determined as the first type; if the number of data packets in the target video frame is less than the classification threshold, the frame type of the target video frame can be determined as the second type.

[0049] Since the link bandwidth is variable, the above method determines the classification threshold based on the real-time bottleneck link bandwidth, thereby enabling the classification result of the target video frame to change with the link bandwidth, making the classification result of the target video frame more accurate, so as to obtain more accurate rate correction parameters and thus more accurate target transmission rate.

[0050] In some possible implementations, the frame type of the target video frame can be determined directly based on its frame size. For example, a frame size threshold can be set. If the frame size of the target video frame is greater than or equal to the threshold, the frame type can be determined to be a first type; if the frame size is less than the threshold, the frame type can be determined to be a second type. For example, the frame size threshold could be 10KB.

[0051] In some possible implementations, the target transmission rate is determined based on the original transmission rate and the frame type of the target video frame, including: Determine the rate correction parameters based on the frame type of the target video frame; determine the target transmission rate based on the original transmission rate and the rate correction parameters.

[0052] In this embodiment, the rate correction parameter is used to correct the original transmission rate in order to obtain a more accurate target transmission rate. The rate correction parameter can be determined based on the frame type of the target video frame, and can be directly determined based on the correspondence between the rate correction parameter and the frame type.

[0053] In some possible implementations, the rate correction parameters are determined based on the frame type of the target video frame, including: If the target video frame is of type 1, the rate correction parameter is determined to be the first correction parameter, where type 1 indicates that the number of data packets in the target video frame is greater than or equal to the classification threshold; or, if the target video frame is of type 2, the rate correction parameter is determined to be the second correction parameter, where type 2 indicates that the number of data packets in the target video frame is less than the classification threshold, and the first correction parameter is less than the second correction parameter.

[0054] In this embodiment, when the frame size of the target video frame is greater than or equal to a frame size threshold, the frame type of the target video frame can be determined to be a first type; when the frame size of the target video frame is less than the frame size threshold, the frame type of the target video frame can be determined to be a second type. The first type corresponds to a first correction parameter, and the second type corresponds to a second correction parameter. The first correction parameter is less than the second correction parameter, wherein the first correction parameter can be greater than or equal to 1. For example, the first correction parameter is 1, and the second correction parameter is 1.5. That is, when the target video frame is a large frame, it can be transmitted directly at the original transmission rate; when the target video frame is a small frame, it is transmitted at 1.5 times the original transmission rate. After a small frame is transmitted, there is more remaining time in the frame interval than with a large frame, so the small frame can be sent at a faster rate.

[0055] In some possible implementations, the feedback information includes at least one of the queuing delay of historical video frames during transmission and the size of the playback buffer at the receiving end.

[0056] Optionally, the original transmission rate can be determined based on the queuing delay of historical video frames during transmission.

[0057] Alternatively, the original transmission rate can be determined based on the size of the playback buffer in the receiving end.

[0058] Optionally, the original transmission rate can be determined based on the queuing delay of historical video frames during transmission and the size of the playback buffer at the receiving end.

[0059] In some possible implementations, the feedback information includes the queuing delay of multiple transmitted video frames during transmission and the size of the playback buffer at the receiving end.

[0060] Based on the feedback information sent by the receiving end, determine the original transmission rate, including: The first transmission rate is determined based on the queuing delay; the second transmission rate is determined based on the playback buffer size; and the original transmission rate is determined based on the first and second transmission rates.

[0061] In this embodiment, a first transmission rate and a second transmission rate can be determined based on queuing delay and playback buffer size, respectively, and then a final target transmission rate can be determined based on the first transmission rate and the second transmission rate. Optionally, the larger of the first transmission rate and the second transmission rate can be determined as the final target transmission rate.

[0062] In some possible implementations, determining the first transmission rate based on queuing delay includes: Based on the queuing delay, determine the queuing delay difference of multiple adjacent video frames that have been transmitted; based on the queuing delay difference of multiple adjacent video frames and the transmission rate of historical video frames, determine the first transmission rate.

[0063] In this embodiment, the queuing delays of multiple video frames transmitted before the target video frame can be obtained, thereby determining the queuing delay difference between any two adjacent video frames, resulting in at least one queuing delay difference. Then, based on the queuing delay difference and historical rate change rule factors, such as the rate change rule factor of the previous transmission cycle, the current rate change rule factor is determined. Finally, based on the current rate change rule factor and combined with the transmission rates of historical video frames, such as the transmission rate of video frames in the previous transmission cycle, a first transmission rate is determined.

[0064] Congestion control signals can be extracted from the QoE (Quality of Experience) metric: In video transmission, the frame interval between two adjacent frames reflects the smoothness of the video and directly affects the user's QoE. A shorter frame interval indicates an increase in QoE, but in network transmission, a shorter frame interval does not necessarily mean better network conditions.

[0065] The frame interval formula is shown below: IFD = rt2 - rt1 = st2 - st1 + d2 - d1 = d2-d1+IFDexp=q2+mrtt2-q1-mrtt1+IFDexp=q2-q1+mrtt2-mrtt1+IFDexp The frame interval (IFD) between two adjacent frames at the receiving end consists of the transmission interval of the first data packet of the two frames at the sending end and the one-way delay difference between the two frames. Here, rt1 and rt2 are the times when the receiving end receives the first data packet of frame 1 and frame 2, st1 and st2 are the times when the sending end transmits the first data packet of frame 1 and frame 2, d1 and d2 are the delays experienced by the data packets in the network, q1 and q2 are the queuing delays experienced by the data packets in the network, rtt is the round-trip time, mrtt is the minimum path rtt, mrtt1 is the minimum path rtt of frame 1, mrtt2 is the minimum path rtt of frame 2, and IFDexp is the ideal transmission frame interval under ideal conditions.

[0066] The transmission interval between two frames at the sending end is determined by the sending end's rate and cannot directly reflect the network condition. Ideally, the transmission interval between two frames at the sending end is equal to IFDexp. Even when the minimum path time (RTT) changes, the one-way delay difference cannot accurately reflect network congestion. Removing these two terms leaves the queuing delay difference between adjacent frames, which is proportional to IFD. In network transmission, a larger queuing delay indicates more severe congestion, while a smaller queuing delay indicates that congestion has eased. Therefore, the queuing delay difference between adjacent frames can be used as a congestion control signal to calculate the first transmission rate, and it also reflects changes in QoE to some extent.

[0067] The formula for calculating the rate change rule factor is as follows:

[0068] in, The rate change rule factor is the rate at time t, where T is the duration of one period. and These are the weighting coefficients. Let t be the queuing delay. This is the queuing delay difference between the two most recent video frames.

[0069] The formula for calculating the first transmission rate is as follows:

[0070] in, For the first transmission rate, The transmission rate of historical video frames. The adjustment coefficient has a value range of 0-1. Change the rule factor for historical rates.

[0071] Alternatively, the first transmission rate can be directly determined as the original transmission rate.

[0072] In some possible implementations, the second transmission rate is determined based on the playback buffer size, including: The second transmission rate is determined based on the playback buffer size, the maximum round-trip time of historical video frames, and the frame size of the target video frame.

[0073] In this embodiment, taking live video streaming as an example, user QoE is crucial, especially regarding buffering during viewing. Related technologies reduce the bitrate when buffering occurs, but ignore the optimization that the transmission protocol can perform when the bitrate remains constant. This embodiment proposes a QoE-based rate guarantee mechanism to transmit data packets with minimal congestion without changing the encoding rate, while ensuring that QoE remains unaffected.

[0074] At 30fps, user QoE decreases when the frame interval exceeds 33ms. When network congestion occurs, if the user's (i.e., the receiver's) playback buffer has sufficient data, the sender can transmit data at a rate lower than the frame rate without causing stuttering, thus alleviating network congestion. Therefore, a frame interval threshold can be calculated based on the amount of data in the receiver's playback buffer; transmitting at a rate greater than this frame interval will not cause stuttering.

[0075] For each frame received, the receiver's playback buffer increases by 33ms, regardless of how long it takes for the frame to arrive. The receiver takes 1 / 2 RTT (Round-Trip Time) to send a frame back to the sender, and the sender takes another 1 / 2 RTT to send the data packet after receiving the feedback. Assuming the worst-case scenario where the receiver doesn't receive any data within this one RTT, after one RTT, the receiver's playback buffer will have a remaining playback duration equal to the receiver's playback buffer size minus the RTT. Considering network congestion, the maximum RTT is used, and the QoE frame interval threshold is the playback buffer size minus the maximum RTT.

[0076] When the playback buffer size minus the maximum RTT is less than or equal to the buffer threshold, the QoE impact phase begins. At this point, the amount of data in the client's playback buffer is insufficient, which can easily cause playback stuttering. Therefore, a minimum guaranteed frame interval that does not affect the user's QoE is used to calculate the QoE rate. The formula for calculating the second transmission rate is as follows: Second transmission rate = Frame size / Minimum guaranteed frame interval At 30fps, the buffer threshold is 66ms, and the minimum frame interval is 33ms. The reason for setting the threshold to 66ms is to tell the sender to send frames at least 33ms apart when there are still 2 frames remaining in the playback buffer, thus providing a warning of stuttering.

[0077] When the playback buffer size minus the maximum RTT exceeds the buffer threshold, the network impact phase begins. At this point, the client's playback buffer contains a large amount of data. When network congestion occurs, the transmission rate can be reduced to alleviate the congestion without causing stuttering. In this situation, the QoE guaranteed rate at the sending end, i.e., the second transmission rate, can be calculated using the following formula: Second transmission rate = frame size / (playback buffer size - maximum RTT) Alternatively, the first transmission rate can be directly determined as the original transmission rate.

[0078] Figure 3 This is a schematic diagram of the structure of a data transmission device according to some embodiments. For example... Figure 3 As shown, this embodiment of the disclosure provides a data transmission device 300, which includes: The first determining module 301 is configured to determine the original transmission rate based on feedback information sent by the receiving end; the feedback information is feedback information sent by the receiving end based on historical video frames received before the target video frame, and the target video frame is the video frame to be transmitted. The second determining module 302 is configured to determine the frame type of the target video frame, wherein the frame type represents the size of the target video frame; The third determining module 303 is configured to determine the target transmission rate based on the original transmission rate and the frame type of the target video frame; The transmission module 304 is configured to transmit the target video frame to the receiving end based on the target transmission rate.

[0079] Optionally, the feedback information includes the reception rate of the video frames received by the receiving end; The second determining module 302 is configured as follows: Determine the bottleneck link bandwidth based on the received rate; The classification threshold is obtained based on the bottleneck link bandwidth, the maximum data packet size in the historical transmission process, and the target transmission time; The frame type of the target video frame is obtained based on the number of data packets in the target video frame and the classification threshold.

[0080] Optionally, the third determining module 303 is configured to: Determine the rate correction parameters based on the frame type of the target video frame; The target transmission rate is determined based on the original transmission rate and the rate correction parameter.

[0081] Optionally, the third determining module 303 is configured as follows: If the frame type of the target video frame is a first type, the rate correction parameter is determined to be a first correction parameter, where the first type indicates that the number of data packets in the target video frame is greater than or equal to a classification threshold; or, If the target video frame is of type two, the rate correction parameter is determined to be a second correction parameter. The second type indicates that the number of data packets in the target video frame is less than a classification threshold, and the first correction parameter is less than the second correction parameter.

[0082] Optionally, the feedback information includes at least one of the queuing delay of the historical video frames during transmission and the playback buffer size in the receiving end.

[0083] Optionally, the feedback information includes the queuing delay of multiple transmitted video frames during transmission and the size of the playback buffer in the receiving end; The first determining module 301 is configured as follows: The first transmission rate is determined based on the queuing delay; The second transmission rate is determined based on the playback buffer size; The original transmission rate is determined based on the first transmission rate and the second transmission rate.

[0084] Optionally, the first determining module 301 is configured to: Based on the queuing delay, determine the queuing delay difference of multiple adjacent video frames that have been transmitted; The first transmission rate is determined based on the queuing delay difference of the plurality of adjacent video frames and the transmission rate of the historical video frames.

[0085] Optionally, the first determining module 301 is configured to: Determining the second transmission rate based on the playback buffer size includes: The second transmission rate is determined based on the playback buffer size, the maximum round-trip time of the historical video frames, and the frame size of the target video frame.

[0086] Optionally, the first determining module 301 is configured to: Determining the original transmission rate based on the first transmission rate and the second transmission rate includes: The larger of the first transmission rate and the second transmission rate is determined as the original transmission rate.

[0087] The functional logic executed by each functional module in the aforementioned data transmission device 300 has been described in detail in the section on methods, and will not be repeated here.

[0088] The following is for reference. Figure 4 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 400 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0089] like Figure 4As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. Processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0090] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0091] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0092] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0093] In some implementations, quality assurance systems and business systems can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communications of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0094] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0095] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine an original transmission rate based on feedback information sent by a receiving end; wherein the feedback information is feedback information sent by the receiving end based on historical video frames previously received before the target video frame, and the target video frame is the currently to be transmitted video frame; determine the frame type of the target video frame, wherein the frame type characterizes the size of the target video frame; determine a target transmission rate based on the original transmission rate and the frame type of the target video frame; and transmit the target video frame to the receiving end based on the target transmission rate.

[0096] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0098] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the module itself.

[0099] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0102] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0103] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A data transmission method, characterized in that, include: Determine the original transmission rate based on the feedback information sent by the receiving end; The feedback information is feedback information sent by the receiving end based on historical video frames received before the target video frame, where the target video frame is the video frame to be transmitted currently. Determine the frame type of the target video frame, wherein the frame type represents the size of the target video frame; The target transmission rate is determined based on the original transmission rate and the frame type of the target video frame; Based on the target transmission rate, the target video frame is transmitted to the receiving end.

2. The data transmission method according to claim 1, characterized in that, The feedback information includes the reception rate of the video frames received by the receiving end; Determining the frame type of the target video frame includes: Determine the bottleneck link bandwidth based on the received rate; The classification threshold is obtained based on the bottleneck link bandwidth, the maximum data packet size in the historical transmission process, and the target transmission time; The frame type of the target video frame is obtained based on the number of data packets in the target video frame and the classification threshold.

3. The data transmission method according to claim 1, characterized in that, Determining the target transmission rate based on the original transmission rate and the frame type of the target video frame includes: Determine the rate correction parameters based on the frame type of the target video frame; The target transmission rate is determined based on the original transmission rate and the rate correction parameter.

4. The data transmission method according to claim 3, characterized in that, Determining the rate correction parameter based on the frame type of the target video frame includes: If the frame type of the target video frame is a first type, the rate correction parameter is determined to be a first correction parameter, where the first type indicates that the number of data packets in the target video frame is greater than or equal to a classification threshold; or, If the target video frame is of type two, the rate correction parameter is determined to be a second correction parameter. The second type indicates that the number of data packets in the target video frame is less than a classification threshold, and the first correction parameter is less than the second correction parameter.

5. The data transmission method according to any one of claims 1-4, characterized in that, The feedback information includes at least one of the queuing delay of the historical video frames during transmission and the playback buffer size in the receiving end.

6. The data transmission method according to any one of claims 1-4, characterized in that, The feedback information includes the queuing delay of multiple transmitted video frames during transmission and the size of the playback buffer in the receiving end; Determining the original transmission rate based on feedback information sent by the receiving end includes: The first transmission rate is determined based on the queuing delay; The second transmission rate is determined based on the playback buffer size; The original transmission rate is determined based on the first transmission rate and the second transmission rate.

7. The data transmission method according to claim 6, characterized in that, Determining the first transmission rate based on the queuing delay includes: Based on the queuing delay, determine the queuing delay difference of multiple adjacent video frames that have been transmitted; The first transmission rate is determined based on the queuing delay difference of the plurality of adjacent video frames and the transmission rate of the historical video frames.

8. The data transmission method according to claim 6, characterized in that, Determining the second transmission rate based on the playback buffer size includes: The second transmission rate is determined based on the playback buffer size, the maximum round-trip time of the historical video frames, and the frame size of the target video frame.

9. The data transmission method according to claim 6, characterized in that, Determining the original transmission rate based on the first transmission rate and the second transmission rate includes: The larger of the first transmission rate and the second transmission rate is determined as the original transmission rate.

10. A data transmission device, characterized in that, include: The first determining module is configured to determine the original transmission rate based on feedback information sent by the receiving end. The feedback information is feedback information sent by the receiving end based on historical video frames received before the target video frame, where the target video frame is the video frame to be transmitted currently. The second determining module is configured to determine the frame type of the target video frame, wherein the frame type represents the size of the target video frame; The third determining module is configured to determine the target transmission rate based on the original transmission rate and the frame type of the target video frame; The transmission module is configured to transmit the target video frame to the receiving end based on the target transmission rate.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processing device, it implements the steps of the method according to any one of claims 1-9.

12. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-9.

Images