Media data transmission method, system and device and storage medium

Abstract

The invention provides a media data transmission method, system and device, and a storage medium. The method comprises the following steps: a sending end obtains a current packet loss rate and a current single round-trip delay; the sending end adopts a first function to calculate a single redundancy multiple; wherein the first function comprises a total sending multiple part and a maximum round-trip time part; for each to-be-sent data packet, the sending end sends an original packet of the to-be-sent data packet and a first redundancy packet corresponding to the single redundancy multiple to a receiving end; and when the sending end receives a retransmission request for the target data packet from the receiving end, the sending end sends a retransmission packet, corresponding to the single retransmission multiple, of the target data packet to the receiving end. Through adoption of the method and the device, packet loss and delay are resisted by adopting a multi-time sending and packet loss retransmission technology for the media data, and a multi-time redundant packet is dispersed to be sent every time by adopting a successive multi-time sending technology, so that the requirement on the network bandwidth is reduced, and the video quality stability is ensured.

Description

Background Art

[0002] In the process of real-time audio and video communication such as online video conferencing, when there is packet loss or delay in the network (hereinafter referred to as a weak network environment), abnormal phenomena such as tremor, freeze, and screen distortion may occur on the receiving end. In order to avoid this problem, technical methods need to be adopted to combat this weak network environment.

[0003] Conventional methods include, for example, packet loss retransmission (ARQ, Automatic Repeat reQuest), forward error correction (FEC, Forward Error Correction) and packet loss compensation (PLC, Packet Loss Concealment). These methods are introduced in the article RFC2354 Options for Repair of Streaming Media. Among them, packet loss retransmission is the simplest and most efficient method, but when the single round-trip delay (generally expressed as RTT, Round Trip Time) is relatively large (for example, 100ms), multiple retransmissions will cause serious delays and jams (for example, 5 retransmissions will result in a 500ms network delay). In addition, the packet loss compensation method cannot solve the situation of large packet loss, and is difficult to implement for videos with large data packet sizes. Therefore, in practice, it is often necessary to use forward error correction methods to combat network packet loss and delay, that is, to send audio and video redundant packets so that the receiving end can rely on redundant packets to recover the lost original packets in the case of packet loss without the need for retransmission requests.

[0004] However, the network error correction capability of the forward error correction method is limited. In the scenario of combating a high network packet loss rate, the redundancy multiple of the forward error correction method is also set to a large value. A large number of redundant packets will occupy a high redundant bit rate when sent, occupying a large amount of bandwidth. When the total transmission bit rate and total bandwidth remain unchanged, the media bit rate will be greatly reduced accordingly, resulting in too little media bit rate and poor image clarity, making the video picture blurry. When the network packet loss rate fluctuates greatly, the image quality will instantly jump. Summary of the invention

[0005] In response to the problems in the prior art, the purpose of this application is to provide a media data transmission method, system, device and storage medium, which adopts multiple sending and packet loss retransmission technology for media data to combat packet loss and delay, and adopts successive multiple sending technology to disperse multiple redundant packets to each sending, thereby reducing the requirements for network bandwidth and ensuring the stability of video quality.

[0006] The present invention provides a method for transmitting media data, including the following steps:

[0007] The sender obtains the current packet loss rate and the current single round-trip delay;

[0008] The transmitting end calculates the single redundancy multiple based on the first function; wherein the first function includes a total sending multiple part and a maximum round trip number part, the total sending multiple part is obtained according to the current packet loss rate and the maximum allowable residual packet loss rate, and the maximum round trip number is obtained according to the current single round trip delay and the maximum allowable network delay;

[0009] For each data packet to be sent, the sending end sends the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end;

[0010] When the sender receives a retransmission request for a target data packet from the receiver, it sends a retransmission packet of the target data packet corresponding to the single retransmission multiple to the receiver. The single retransmission multiple satisfies: the sum of the single redundancy multiple and the number of target data packets is equal to the product of the maximum number of retransmissions within a current single round-trip delay and the single retransmission multiple.

[0011] In some embodiments, the total sending multiple part of the first function satisfies: under the network environment of the current packet loss rate, the remaining packet loss rate when the sending multiple of each data packet to be sent reaches the total sending multiple is equal to the maximum allowed remaining packet loss rate;

[0012] The maximum round trip number part of the first function satisfies: based on the packet loss retransmission mechanism, for a data packet to be sent, the total network delay when the number of consecutive round trips between the sender and the receiver reaches the maximum number of round trips is equal to the maximum allowed network delay.

[0013] In some embodiments, when the sending end receives a retransmission request for a target data packet from the receiving end, the sending end sends a retransmission packet of the target data packet corresponding to a single retransmission multiple to the receiving end, including:

[0014] When the retransmission request interval is equal to the current single round trip delay, the sender determines the single retransmission multiple according to the sum of the single redundancy multiple and the number of target data packets; and / or,

[0015] When the retransmission request interval is less than the current single round-trip delay, the sender calculates the maximum number of retransmissions based on the current single round-trip delay and the retransmission request interval, and calculates the single retransmission multiple based on the maximum number of retransmissions and the single redundancy multiple.

[0016] In some embodiments, the following steps are also included:

[0017] The sender receives the reverse code stream from the receiver and determines whether there is reverse packet loss based on the reverse code stream;

[0018] When the sender does not detect reverse packet loss, the retransmission request interval is set equal to the single round-trip delay;

[0019] When the sender detects reverse packet loss, it shortens the retransmission request interval to less than a single round-trip delay.

[0020] In some embodiments, after the sending end sends the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end, the following steps are also included:

[0021] When the receiving end determines that the target data packet is lost, it calculates the retransmission request interval based on the maximum allowed network delay, the current packet loss rate, and the maximum allowed remaining packet loss rate;

[0022] The receiving end sends a retransmission request corresponding to the target data packet to the sending end based on the retransmission request interval;

[0023] After receiving the retransmission request, the sender sends a retransmission packet corresponding to the target data packet to the receiver.

[0024] In some embodiments, the receiving end calculates the retransmission request interval according to the maximum allowed network delay, the current packet loss rate, and the maximum allowed remaining packet loss rate using the following steps:

[0025] The retransmission request interval is calculated using a second function, the second function includes a maximum allowable network delay part and a total sending multiple part, the total sending multiple part is obtained based on the maximum allowable residual packet loss rate and the current packet loss rate, and the retransmission request interval is equal to the quotient of the maximum allowable network delay part and the total sending multiple part.

[0026] In some embodiments, when the sending end sends the retransmission packet of the target data packet corresponding to the single retransmission multiple to the receiving end, the following steps are also included:

[0027] The sending end calculates the time difference between a first time of receiving the current retransmission request and a second time of receiving the previous retransmission request for the same target data packet;

[0028] The sender calculates the compensation multiple based on the time difference and the retransmission request interval of the receiver;

[0029] When the compensation multiple is a positive number, the sending end sends the second redundant packet of the target data packet corresponding to the compensation multiple to the receiving end.

[0030] In some embodiments, the sending end uses the following steps to send the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end:

[0031] Taking the integer part of the single redundancy multiple as the first redundancy multiple;

[0032] Take the decimal part of the single redundancy multiple, and determine the number of interval packets n according to the decimal part;

[0033] An original packet to be sent at intervals of n packets in the data to be sent is used as an adjusted original packet, the first redundancy multiple plus 1 is used as the current redundancy multiple of the adjusted original packet, and the first redundancy multiple is used as the current redundancy multiple of the non-adjusted original packet;

[0034] The sending end sends the original packet of the data packet to be sent and the first redundant packet whose number is equal to the current redundant multiple to the receiving end.

[0035] The embodiment of the present application further provides a media data transmission system, which is applied to the above-mentioned media data transmission method, and the system includes:

[0036] The network status acquisition module is used to obtain the current packet loss rate and the current single round-trip delay;

[0037] A redundancy multiple calculation module, used to calculate a single redundancy multiple based on a first function; wherein the first function includes a total sending multiple part and a maximum round trip number part, the total sending multiple part is obtained according to a current packet loss rate and a maximum allowable residual packet loss rate, and the maximum round trip number is obtained according to a current single round trip delay and a maximum allowable network delay;

[0038] A data packet sending module, used for sending, for each data packet to be sent, an original packet of the data packet to be sent and a first redundant packet corresponding to a single redundant multiple to a receiving end;

[0039] The data packet retransmission module is used to send a retransmission packet of the target data packet corresponding to the single retransmission multiple to the receiving end when receiving a retransmission request for the target data packet from the receiving end. The single retransmission multiple satisfies: the sum of the single redundancy multiple and the number of target data packets is equal to the product of the maximum number of retransmissions within a current single round-trip delay and the single retransmission multiple.

[0040] The present application also provides a media data transmission device, including:

[0041] processor;

[0042] a memory storing executable instructions of the processor;

[0043] The processor is configured to execute the steps of the media data transmission method by executing the executable instructions.

[0044] The embodiment of the present application also provides a computer-readable storage medium for storing a program, and when the program is executed by a processor, the steps of the media data transmission method are implemented.

[0045] By adopting the media data transmission method, system, device and storage medium of the present application, the following beneficial effects are achieved:

[0046] By adopting the present application, after obtaining the current packet loss rate and the current single round-trip delay, the first function is used to calculate the single redundancy multiple based on the current packet loss rate, the current single round-trip delay, the maximum allowable residual packet loss rate, and the maximum allowable network delay, and multiple sending and packet loss retransmission technology are used for media data to combat packet loss and delay. In the scenario of the current packet loss rate, for each data packet to be sent, based on the packet loss retransmission mechanism, when the number of redundant packets sent by the sender within the time range of each single round-trip delay is equal to the single redundancy multiple, the requirements of the maximum allowable residual packet loss rate and the maximum allowable network delay are met at the same time, thereby being able to meet the needs of normal decoding and smooth video playback at the receiving end, and meeting the requirements of low network delay and reduced jamming of the video received by the receiving end. In addition, the present application adopts the technology of multiple transmissions in succession to disperse multiple redundant packets into each transmission, thereby reducing the requirements for network bandwidth. By distributing the total transmission multiples into multiple retransmissions, the number of redundant packets in a single transmission of a data packet to be sent can be reduced, and redundant packets are prevented from occupying a large amount of bandwidth each time they are sent, thus affecting the media bit rate, ensuring the image quality transmitted to the receiving end, and improving the stability of the video quality, thereby effectively improving the video picture display effect. In addition, when performing redundant calculations and packet loss retransmissions, the present application is performed on a single data packet. For the receiving end, it only needs to determine whether packet loss has occurred based on the order of the received data packets. The receiving end does not need to detect packet loss through decoding, thereby reducing the redundant decoding of the receiving end, and will not increase additional delays, with better real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings.

[0048] Figure 1 is a flow chart of a method for transmitting media data according to an embodiment of the present application;

[0049] Figure 2 is a schematic diagram of a media data transmission system according to an embodiment of the present application;

[0050] Figure 3 is a structural schematic diagram of a media data transmission device according to an embodiment of the present application;

[0051] Figure 4 It is a schematic diagram of the structure of a computer storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted. Although "first" or "second" etc. are used in this specification to represent certain features, they are only used to represent the function and are not used as a limitation on the number and importance of specific features.

[0053] like Figure 1 As shown, in one embodiment, the present application provides a method for transmitting media data, including the following steps S100 to S400. Each step is specifically introduced below:

[0054] S100: The sender obtains the current packet loss rate and the current single round-trip delay.

[0055] The packet loss rate refers to the ratio of the number of lost data packets updated in real time to the number of sent data packets. The round trip time (RTT) refers to the time of a single round trip between the sender and the receiver, that is, the total delay from the start of sending data from the sender to the receiver to the sender receiving the confirmation from the receiver. The packet loss rate and round trip time reflect the current network status. When the packet loss rate and round trip time are large, the current network is a weak network environment.

[0056] The sender updates the current packet loss rate and the current single round trip delay (Round Trip Time, RTT) at a fixed interval (such as 1s, 5s, etc.). After the sender updates the current packet loss rate and the current single round trip delay, step S200 is used to update and calculate the single redundancy multiple;

[0057] S200: The sending end calculates a single redundancy multiple based on a first function; wherein the first function includes a total sending multiple part and a maximum round trip number part, the total sending multiple part is obtained according to a current packet loss rate and a maximum allowable remaining packet loss rate, and the maximum round trip number is obtained according to a current single round trip delay and a maximum allowable network delay.

[0058] In this embodiment, the maximum allowable residual packet loss rate is a preset maximum residual packet loss rate required to ensure that the receiving end can play the video smoothly without any freezes; the maximum allowable network delay is a preset maximum network delay that can ensure that the freezes at the receiving end are not too severe to affect the viewing.

[0059] The total sending multiple is equal to the sum of the number of original packets and redundant packets sent for the first time and the number of retransmitted packets subsequently retransmitted multiple times; in the first function, the total sending multiple of the total sending multiple refers to the minimum sending multiple for a data packet to be sent, under the current packet loss rate environment, which can make the remaining packet loss rate meet the requirement of the maximum allowable remaining packet loss rate (less than or equal to the maximum allowable remaining packet loss rate).

[0060] The maximum number of round trips refers to the maximum number of round trips between the sender and the receiver corresponding to multiple retransmissions under the current single round-trip delay environment, provided that the total network delay caused by a data packet after multiple retransmissions meets the maximum allowed network delay (less than or equal to the maximum allowed network delay).

[0061] Therefore, by adopting the first function, the single redundancy multiple is calculated based on the current packet loss rate, the current single round-trip delay, the maximum allowable residual packet loss rate, and the maximum allowable network delay. The total sending multiples that meet the maximum allowable residual packet loss rate can be allocated to the first sending and multiple retransmissions, thereby meeting the requirement of the maximum residual packet loss rate, ensuring that the receiving end can decode normally, ensuring smooth playback of the video at the receiving end, and avoiding too many redundant packets occupying a large amount of bandwidth when sending data packets for the first time. The maximum number of round trips experienced by multiple retransmissions can meet the requirement of the maximum allowable network delay, avoiding the jamming of the video playback at the receiving end due to excessive network delay.

[0062] S300: For each data packet to be sent, the sending end sends the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end.

[0063] The data packet to be sent refers to each data packet in the video code stream sent by the transmitter to the receiver, and specifically refers to the data packet sent to the receiver for the first time. The concepts of the original packet and the first redundant packet both correspond to the data packet to be sent. For a data packet to be sent, the data content of the original packet and the first redundant packet is the same, the original packet specifically refers to the data packet itself when the data packet to be sent is sent to the receiver for the first time, the first redundant packet is a redundant backup of the data packet to be sent, and in this application specifically refers to the redundant backup of the data packet to be sent when it is sent to the receiver for the first time; the number of first redundant packets is determined according to the single redundant multiple, wherein the single redundant multiple can be directly used as the number of first redundant packets, or the current redundant multiple close to the single redundant multiple obtained after adaptively adjusting the single redundant multiple can be used as the number of first redundant packets.

[0064] S400: When the sending end receives a retransmission request for a target data packet from the receiving end, the sending end sends a retransmission packet of the target data packet corresponding to a single retransmission multiple to the receiving end, and the single retransmission multiple satisfies: the sum of the single redundancy multiple and the number of target data packets is equal to the product of the maximum number of retransmissions within a current single round-trip delay and the single retransmission multiple.

[0065] The concept of retransmission packet corresponds to the target data packet. The target data packet refers to the data packet that is missing when the receiving end finds packet loss. For example, the receiving end determines based on the sequence number of the received data packet that the data packet with the previous sequence number has not been received when the data packet with the later sequence number is received, that is, it is considered that the data packet with the previous sequence number has been lost, and then the data packet with the previous sequence number is used as the target data packet, and a retransmission request is sent to the sending end. After receiving the retransmission request, the sending end sends the retransmission packet corresponding to the target data packet to the receiving end. Therefore, the target data packet specifically refers to the data packet that needs to be retransmitted due to packet loss, and the retransmission packet specifically refers to the retransmission packet corresponding to the target data packet determined according to the number of single retransmission packets, and the data content of the retransmission packet is the same as the target data packet. For each target data packet, the number of target data packets is 1, that is, the single retransmission multiple satisfies: the maximum number of retransmission packets that can be retransmitted within a current single round-trip delay (equal to the product of the maximum number of retransmissions within a current single round-trip delay and the single retransmission multiple) is equal to the single redundancy multiple plus 1.

[0066] The maximum number of retransmissions in step S400 and the maximum number of round trips in step S200 have different meanings. The maximum number of round trips is the number of single round trip delays experienced by a data packet during its first transmission and retransmission process when the maximum allowed network delay is met. The maximum number of retransmissions is the maximum number of times a data packet can be retransmitted within a single round trip delay. The maximum number of retransmissions is equal to the single round trip delay / retransmission request interval. When the retransmission request interval is equal to the single round trip delay, the maximum number of retransmissions is 1. When the retransmission request interval is less than the single round trip delay, the maximum number of retransmissions is greater than 1.

[0067] For the target data packets lost by the receiving end, the sending end can send one retransmission packet to the receiving end each time, but the retransmission request interval is less than the single round-trip delay, so that the maximum number of retransmission packets that can be retransmitted within a single round-trip delay is equal to the sum of the single redundancy multiple and the number of target data packets, or the sending end can use the single round-trip delay as the retransmission request interval, then the number of retransmission packets sent during retransmission is equal to the sum of the single redundancy multiple and the number of target data packets.

[0068] The above steps S100 to S400 are deployed at the sending end of media data transmission, and the sending end is, for example, a user terminal, or a platform or server. The method can be used to control the transmission of audio and video data between the sending end and the receiving end, and because the amount of video data is larger, the bandwidth requirement for transmission is higher than that for audio. When the media data transmission method is applied to video data transmission, the poor video playback quality and video freeze at the receiving end in the prior art can be significantly improved. The corresponding receiving end is, for example, a user terminal, or a platform or server.

[0069] By adopting the media data transmission method of the present application, after obtaining the current packet loss rate and the current single round-trip delay, the first function is used to calculate the single redundancy multiple based on the current packet loss rate, the current single round-trip delay, the maximum allowable residual packet loss rate, and the maximum allowable network delay, and multiple sending and packet loss retransmission technology are used for media data to combat packet loss and delay. In the scenario of the current packet loss rate, for each data packet to be sent, based on the packet loss retransmission mechanism, when the number of redundant packets sent by the sender within the time range of each single round-trip delay is equal to the single redundancy multiple, the requirements of the maximum allowable residual packet loss rate and the maximum allowable network delay are met at the same time, thereby being able to meet the needs of normal decoding and smooth video playback at the receiving end, and meeting the requirements of low network delay and reduced jamming of the video received by the receiving end. In addition, the present application adopts a successive multiple transmission technology to disperse multiple redundant packets into each transmission (first transmission and retransmission), thereby reducing the requirements for network bandwidth. By distributing the total transmission multiples into multiple retransmissions, the number of redundant packets in a single transmission of data packets to be sent can be reduced, thereby avoiding redundant packets occupying a large amount of bandwidth each time sending and affecting the media bit rate, ensuring the image quality transmitted to the receiving end, and improving the stability of video quality, thereby effectively improving the video picture display effect.

[0070] In addition, in some existing packet loss processing methods, when sending redundant packets, the sender comprehensively calculates the number of redundant packets based on the combination of multiple data packets in a code stream, and sends the combination of multiple data packets together with the redundant packets to the receiver. For the receiver, it is necessary to receive multiple data packets for decoding to find out whether packet loss has occurred. Once packet loss is found, decoding will fail and a retry request will need to be initiated, which will cause repeated redundant decoding at the receiver. Since the receiver needs to decode each time to determine packet loss, additional delays are caused, affecting the real-time transmission of media data. In the present application, when performing redundant calculations and packet loss retransmissions, they are all performed for a single data packet. For the receiver, it is only necessary to determine whether packet loss has occurred based on the order of the received data packets. The receiver does not need to detect packet loss through decoding, thereby reducing redundant decoding at the receiver, not adding additional delays, and having better real-time performance.

[0071] This application combines the packet loss retransmission method and the forward error correction method to combat the weak network environment. Common forward error correction method encodings in the prior art include the XOR algorithm and the RS algorithm (Reed-Solomon encoding). The XOR algorithm is relatively simple and has low performance requirements, but its error correction capability is limited, and it is difficult to achieve redundancy of more than 1 times. The RS algorithm is complex to calculate and has high performance requirements, but its error correction capability is very strong and can achieve any multiple of redundancy.

[0072] The present application adopts a multiple transmission method, which is simpler to implement than the XOR algorithm and the RS algorithm, and has lower performance requirements for the device implementing the method (the terminal, platform or server deploying the media data transmission method). The multiple transmission method includes: when the transmitting end sends the data packet to be sent to the receiving end, the same multiple redundant packets are repeatedly sent for an original packet as the simplest forward error correction coding (copy). For example, for an original packet, 4 identical packets are repeatedly sent as redundant packets, so that the receiving end can recover as long as it receives any one of the 5 packets. In the case of a high network packet loss rate, in order to meet the requirements of the maximum residual packet loss rate, a large multiple of redundancy needs to be set. For example, when sending video data, without considering retransmission, 13 times of redundant packets are required to resist a 50% network packet loss rate. If the 13 times redundant packets are sent to the receiving end synchronously when the original packet is sent for the first time, the 13 times redundant packets will occupy a large amount of redundant code rate. When the transmission bandwidth and transmission code rate remain constant, the corresponding media code rate will be reduced, which will result in too little media code rate and poor image clarity. In addition, when the network packet loss rate changes, causing the redundancy multiple to change, for example, from no redundancy to 1x redundancy, the redundant packets instantly occupy half of the network bandwidth. In order to ensure that the total bandwidth does not exceed the conference bit rate, the media bit rate can only be reduced to half of the original, causing the image quality to change instantly.

[0073] Based on this, the present application adopts a forward error correction mechanism of multiple transmissions, which is simpler in calculation and has low requirements on equipment performance. In addition, on the basis of combining forward error correction technology and packet loss retransmission technology, the present application further adopts a successive multiple transmission technology to disperse multiple redundant packets to each transmission (first transmission and retransmission), thereby reducing the requirements on network bandwidth. By distributing the total transmission multiples to multiple retransmissions, the number of redundant packets in a single transmission of a data packet to be sent can be reduced, thereby avoiding redundant packets occupying a large amount of bandwidth each time they are transmitted and affecting the media bit rate. For example, in a network environment with a packet loss rate of 50% without considering the packet loss retransmission mechanism, in order to meet the maximum allowable residual packet loss rate, 13 times redundant packets are required. After adding the retransmission mechanism, the 13 times redundant packets can be distributed to the first transmission and retransmission. Taking the retransmission twice as an example, the sender sends 1 original packet and 4 first redundant packets (that is, 5 identical data packets) when sending for the first time. If the receiver does not receive these 5 data packets, a retransmission request is triggered, and the sender sends 5 retransmission packets (equivalent to 4 redundancies during retransmission). Then, 2 retransmissions (3 transmissions in total) can achieve a total protection multiple of 14 times, and the total transmission multiple reaches 15 times, which meets the maximum allowable residual packet loss rate required by the receiver. Therefore, the bit rate occupied by redundancy is reduced each time a data packet is sent, and more bandwidth can be left for the media bit rate, thereby ensuring the quality of video transmission.

[0074] In step S200, the transmitting end uses the first function to calculate the single redundancy multiple based on the current packet loss rate loss, the current single round-trip delay RTT, the maximum allowable remaining packet loss rate max_remind_rate, and the maximum allowable network delay max_delay. When calculating, the transmitting end can first calculate the total sending multiple total_send_times by the current packet loss rate loss and the maximum allowable remaining packet loss rate remind_rate, calculate the maximum round-trip times max_round_times by the maximum allowable network delay max_delay and the current single round-trip delay RTT, and then calculate the single redundancy multiple red_times according to the total sending multiple total_send_times and the maximum round-trip times max_round_times. Alternatively, the transmitting end pre-constructs a function based on the first function, and the transmitting end directly substitutes the current packet loss rate loss, the current single round-trip delay RTT, the maximum allowable remaining packet loss rate remind_rate, and the maximum allowable network delay max_delay into the first function to calculate the single redundancy multiple red_times.

[0075] In this embodiment, the first function is implemented by, for example, the following formula 1:

[0076] red_times = log(max_remind_rate,loss)*RTT / max_dely-1 (Formula 1)

[0077] Among them, log(max_remind_rate,loss) corresponds to the total sending multiple part, and max_dely / RTT corresponds to the maximum round trip number part. The quotient obtained by dividing the total sending multiple part by the maximum round trip number part is the number of data packets that need to be sent within the time range of each single round trip delay. The quotient obtained by dividing the total sending multiple part by the maximum round trip number part minus 1 (minus the original packet) is the single redundancy multiple red_times.

[0078] Therefore, the single redundancy multiple during video transmission is calculated based on the current packet loss rate and the current single round-trip delay. When the current single round-trip delay is less than 20ms, the redundancy is 0, that is, it can be fully recovered by retransmitting the packet loss, and there is no need to send multiple redundancies, because even if it is retransmitted 13 times, it will only increase the network delay by less than 260ms, which will not affect real-time communication. When the current single round-trip delay increases, multiple retransmissions will cause the network delay to increase. For example, when RTT = 100ms, 3 retransmissions will increase the delay by 300ms. The redundancy of each transmission must be increased to recover the packet loss within a reasonable delay range.

[0079] It can also be seen from formula 1 that when other parameters remain unchanged, the larger the RTT value, the smaller the maximum round trip number, the larger the calculated red_times, that is, the larger the single redundancy multiple, that is, the greater the required redundancy. On the contrary, when the RTT value is smaller, the larger the maximum round trip number, the smaller the calculated red_times, that is, the smaller the single redundancy multiple. When the result of max / RTT is less than the total sending multiple, the total sending multiple is greater than 1, that is, the red_times calculated according to formula 1 is greater than 0, and redundancy needs to be added according to red_times. When the result of max / RTT is less than or equal to the total sending multiple, the method of multiple retransmissions of packet loss can adapt to the current network environment, and the calculated red_times is limited to the minimum value of 0, so there is no need to add redundancy. When other parameters remain unchanged, the higher the packet loss rate, the larger the corresponding total sending multiple, the larger the required redundancy, and the larger the corresponding single redundancy multiple. Therefore, the larger the packet loss rate and the single round trip delay, the higher the required redundancy, and the larger the calculated single redundancy multiple.

[0080] Take the video conferencing scenario where the maximum allowable residual packet loss rate max_remind_rate is 0.008% (i.e. 80ppm, ppm means parts per million) as an example. In the media data transmission scenario, the received packet recovery rate can reach 99.992%, which can basically ensure smooth video playback without lag, that is, the residual packet loss rate that cannot be recovered after the receiving end tries to recover the data packet is less than 0.008%. The value of the maximum allowable residual packet loss rate is only an example. In actual applications, the maximum allowable residual packet loss rate will vary according to the characteristics of different senders, receivers and network environments. The appropriate maximum allowable residual packet loss rate can be obtained through experiments and measurements. In the process of media data transmission, the residual packet loss rate remind_rate is related to the total protection multiple (first redundant packet multiple + retransmission packet multiple) total_protect_times. The relationship between the total protection multiple and the total sending multiple is: total_send_times = total_protect_times+1.

[0081] The total sending multiple satisfies the following formula 2:

[0082] loss^(total_protect_times+1)=remind_rate (formula 2)

[0083] Formula 2 is equivalent to total_protect_times+1=log(remind_rate) / log(loss), or, total_protect_times+1=log(remind_rate, loss).

[0084] That is, (total_protect_times+1=total_send_times) is the logarithmic value with loss as the base and remind_rate as the real number. When the maximum allowable residual packet loss rate needs to be met, substitute max_remind_rate into the remind_rate in formula 2 to obtain the total protection multiple total_protect_times=log(max_remind_rate,loss)-1. Therefore, the total sending multiple can be obtained to meet the following formula 3:

[0085] total_send_times=log(max_remind_rate,loss) (Formula 3)

[0086] Formula 3 corresponds to the total sending multiple part in Formula 1. It can be seen from Formula 3 that the total sending multiple part satisfies: in the network environment with the current packet loss rate loss, when the sending multiple of each data packet to be sent reaches the total sending multiple total_send_times, the remaining packet loss rate remind_rate is equal to the maximum allowable remaining packet loss rate max_remind_rate. The total sending multiple total_send_times obtained by Formula 3 is the minimum sending multiple required for a data packet when the remaining packet loss rate remind_rate is not greater than the maximum allowable remaining packet loss rate max_remind_rate.

[0087] The maximum round trip number of the first function satisfies the following: Based on the packet loss retransmission mechanism, for a data packet to be sent, when the number of consecutive round trips between the sender and the receiver reaches the maximum round trip number max_round_times, the total network delay delay is equal to the maximum allowed network delay max_delay. The maximum round trip number max_round_times is equal to the maximum allowed retransmission number max_nack_times plus 1, where the addition of 1 corresponds to the addition of the first transmission, that is, max_round_times = max_nack_times+1.

[0088] For a data packet to be sent, when the number of consecutive round trips between the sender and the receiver reaches the maximum number of round trips, the total network delay generated satisfies the following formula 4:

[0089] delay=max_round_times*RTT (Formula 4)

[0090] In order to satisfy the requirement that the total network delay is equal to the maximum allowed network delay, it is necessary to make the delay in Formula 4 equal to max_delay, that is, to obtain the following Formula 5:

[0091] max_round_times=max_delay / RTT (Formula 5)

[0092] Therefore, the maximum allowed network delay requirement can be met when the number of round trips caused by sending and retransmission within one RTT is less than or equal to max_round_times.

[0093] The single redundancy multiple red_times is the additional redundancy multiple that needs to be added within the time range of each single round-trip delay during the first transmission and each retransmission (during retransmission, all packets except the first packet in a single round-trip delay are regarded as redundant packets during retransmission), that is, the sending multiple within each single round-trip delay is red_times+1. When the maximum number of round-trip times max_round_times is reached, the total sending multiple total_send_times requirement must be met to meet the maximum allowable residual packet loss rate requirement.

[0094] Therefore, the single redundancy multiple red_times needs to satisfy the following formula 6 in order to meet the requirement of the total sending multiple total_send_times under the maximum round trip number max_round_times:

[0095] (red_times+1)*max_round_times=total_send_times (Formula 6)

[0096] Substituting the above formula 3 and formula 5 into the above formula 6, we can get formula 7:

[0097] (red_times+1)*(max_delay / RTT)=log(max_remind_rate,loss)(Formula 7)

[0098] The above formula 1 can be obtained by converting formula 6.

[0099] When applying, there is no need to calculate the values ​​of Formula 3 and Formula 5 respectively. The maximum allowable residual packet loss rate max_remind_rate, the maximum allowable network delay max_delay, the current packet loss rate loss and the current single round-trip delay RTT are directly substituted into Formula 1 to obtain the value of the single redundancy multiple red_times. The calculation process is simple and fast.

[0100] Taking the maximum allowable residual packet loss rate max_remind_rate as 0.008%, the maximum allowable network delay max_delay as 300ms, the current packet loss rate loss as 50%, and the current single round-trip delay RTT as 100ms as an example, the single redundancy multiple can be obtained as:

[0101] red_times=log(0.00008,0.5) / (300 / 100)-1=3.54.

[0102] The values ​​of the parameters listed here are only examples. In actual applications, the values ​​of the maximum allowable residual packet loss rate max_remind_rate and the maximum allowable network delay max_delay can be set as needed, and the values ​​of the current packet loss rate loss and the current single round-trip delay RTT are updated in real time according to changes in the network environment.

[0103] The retransmission request interval refers to the period at which the receiving end sends a retransmission request when a packet loss is detected. As mentioned above, the retransmission request interval can be equal to the current single round-trip delay RTT, that is, after the receiving end sends a retransmission request to the sending end at time t10, if the sending end sends a retransmission packet and the retransmission packet can successfully reach the receiving end, the time when the receiving end receives the retransmission packet is t20, and the time difference (t20-t10) is equal to a retransmission request interval. If the retransmission request interval is equal to the duration of the single round-trip delay, after the receiving end sends a retransmission request at time t10, if the retransmission packet is still not received after the RTT time, it will resend the next retransmission request to the sending end at time t20. In step S400, when the transmitting end receives the retransmission request for the target data packet from the receiving end, the retransmission packet of the target data packet corresponding to the single retransmission multiple is sent to the receiving end, including: when the retransmission request interval is equal to the current single round trip delay, the transmitting end determines the single retransmission multiple according to the sum of the single redundancy multiple and the number of target data packets, that is, the number of retransmission packets sent at each retransmission (single retransmission multiple) is determined according to (red_times+1). In this case, the single retransmission multiple can satisfy: the sum of the single redundancy multiple and the number of target data packets (i.e., red_times+1) is equal to the product of the maximum number of retransmissions within the current single round trip delay RTT (i.e., 1) and the single retransmission multiple (equal to red_times+1).

[0104] However, in a bidirectional packet loss scenario, the reverse retransmission request may also be lost, that is, the receiving end sends a retransmission request, but the sending end does not receive it, resulting in a prolonged recovery time for some data packets, thereby causing video frame jitter and causing video playback to freeze. In order to solve the problem of reverse retransmission requests in a bidirectional packet loss scenario, it is necessary to perform multiple redundancy on the retransmission requests. If the method of redundantly sending each retransmission request is adopted, it is difficult to implement that the receiving end sends multiple retransmission requests at one time, and it is also difficult for the sending end to distinguish whether the received retransmission request is a normal retransmission request or a repeated retransmission request. Therefore, in this embodiment, the method of shortening the retransmission request interval is adopted to indirectly achieve the effect of multiple retransmission requests. After shortening the retransmission request interval, the duration of the retransmission request interval nack_interval is less than the duration of a single round-trip delay RTT, and there is a multiple relationship between RTT and nack_interval, that is, RTT = m*nack_interval, m>1. Within the time range of a single round-trip delay, the maximum number of retransmission requests that can be sent is m, which is equivalent to achieving m-1 redundancy of retransmission requests within one RTT, that is, achieving m times of sending retransmission requests. For the sender, when the receiver sends m retransmission requests, if there is no loss of retransmission requests, the sender will correspondingly send m retransmission packets, that is, the maximum number of retransmissions of the sender within one RTT is m. Since the time interval for sending retransmission packets within one RTT will be shortened, the maximum number of retransmissions m for sending retransmission packets will increase, and the number of retransmission packets sent each time will decrease accordingly. In step S400, when the transmitting end receives the retransmission request for the target data packet from the receiving end, the transmitting end sends the retransmission packet of the target data packet corresponding to the single retransmission multiple to the receiving end, including: when the retransmission request interval nack_interval is less than the current single round trip delay RTT, the transmitting end calculates the maximum number of retransmissions m=RTT / nack_interval according to the current single round trip delay RTT and the retransmission request interval nack_interval, and calculates the single retransmission multiple (red_times+1) / m according to the maximum number of retransmissions m and the single redundancy multiple red_times. The single retransmission multiple thus obtained can also satisfy: the sum of the single redundancy multiple red_times and the number of target data packets (i.e., red_times+1) is equal to the product of the maximum number of retransmissions m within the current single round trip delay RTT and the single retransmission multiple (equal to (red_times+1) / m).

[0105] For example, the calculated single redundancy multiple is 3. When the retransmission request interval is equal to RTT, the maximum number of retransmissions in each RTT is 1, and the sender sends 4 retransmission packets each time to achieve 3 times redundancy in one RTT. When the retransmission request interval is equal to (RTT / 2), the maximum number of retransmissions in each RTT is 2, and the sender sends 4 / 2=2 retransmission packets each time. A total of 4 retransmission packets are sent in two retransmissions, which can also achieve 3 times redundancy in one RTT.

[0106] In one implementation, it is assumed that each time the sender receives a retransmission request, it only sends one retransmission packet to the receiver, that is, the single retransmission multiple is 1. Then the maximum number of retransmissions m within an RTT is equal to red_times+1, and the retransmission request interval nack_interval satisfies the following formula 8:

[0107] nack_interval*(red_times+1)=RTT (Formula 8)

[0108] Substituting the above formula 1 into formula 8, the following derivation process of the calculation formula of the retransmission request interval can be obtained:

[0109] nack_interval=RTT / (red_times+1)

[0110] =RTT / (log(max_remind_rate,loss)*RTT / max_delay)

[0111] =max_delay*log(loss,max_rremind_rate)

[0112] =max_delay*log(loss) / log(max_rremind_rate)

[0113] That is, the calculation formula of the retransmission request interval is obtained as follows9:

[0114] nack_interval = max_delay *log(loss) / log(max_remind_rate) (Formula 9)

[0115] As can be seen from Formula 9, based on the maximum allowed network delay, the current packet loss rate and the maximum allowed remaining packet loss rate, the retransmission request interval nack_interval can be obtained when the single retransmission multiple is 1 (the sender only sends one retransmission packet each time). For example, taking the current packet loss rate loss as 50%, the maximum allowed remaining packet loss rate max_remind_rate as 0.008%, and the maximum allowed network delay as 300ms as an example, the calculated retransmission request interval nack_interval = 300*log(0.5) / log(0.00008) = 22ms. The values ​​of the various parameters here are only examples and will be different in actual applications.

[0116] In this embodiment, step S300: after the transmitting end sends the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end, the following steps are also included:

[0117] When the receiving end determines that the target data packet is lost, it calculates the retransmission request interval nack_interval based on the maximum allowed network delay max_delay, the current packet loss rate loss and the maximum allowed residual packet loss rate max_rremind_rate;

[0118] The receiving end sends a retransmission request corresponding to the target data packet to the sending end based on the retransmission request interval nack_interval; that is, after the receiving end sends a retransmission request to the sending end, if the retransmission packet corresponding to the target data packet is not received after the retransmission request interval nack_interval, the receiving end sends a retransmission request to the sending end again;

[0119] Corresponding to step S400, after receiving the retransmission request, the sending end sends a retransmission packet corresponding to the target data packet to the receiving end.

[0120] Specifically, the receiving end calculates the retransmission request interval according to the maximum allowed network delay, the current packet loss rate, and the maximum allowed remaining packet loss rate using the following steps:

[0121] The second function is used to calculate the retransmission request interval. The second function includes a maximum allowable network delay part and a total transmission multiple part. The total transmission multiple part is obtained based on the maximum allowable residual packet loss rate and the current packet loss rate. The retransmission request interval is equal to the quotient of the maximum allowable network delay part and the total transmission multiple part. Taking the second function in the form of formula 9 as an example, log(max_remind_rate,loss) corresponds to the total transmission multiple part, and max_delay corresponds to the maximum allowable network delay part. In actual calculations, there is no need to calculate the total transmission multiple first. The maximum allowable network delay, the current packet loss rate, and the maximum allowable residual packet loss rate can be directly substituted into the second function to obtain the retransmission request interval.

[0122] Therefore, in this implementation, redundant transmission of retransmission requests within one RTT is achieved by reducing the retransmission request interval. When the sender and the receiver perform media data transmission, for a data packet, the sender only sends the original packet and the redundant packet corresponding to the single redundancy multiple when sending it for the first time, and sends (red_times+1) / m retransmission packets each time a retransmission request is received subsequently. When m is equal to (red_times+1), the sender only needs to send one retransmission packet each time.

[0123] In this embodiment, in order to combat a weak network environment, the retransmission request interval can be directly set to a value smaller than the RTT. For example, the retransmission request interval can be directly set to max_delay*log(loss) / log(max_rremind_rate). Then, regardless of whether the reverse data packet sent by the current receiving end to the sending end is lost, the retransmission request interval is a fixed value. The sending end will determine the number of retransmission packets to be sent each time based on this shortened retransmission request interval, and the receiving end will send a retransmission request based on this shortened retransmission request interval.

[0124] In another embodiment, the retransmission request interval can be set to a switchable value, that is, when there is no reverse packet loss, the retransmission request interval is maintained at an RTT, and when reverse packet loss occurs, the retransmission request interval is shortened to a value less than the RTT. The switchable retransmission request interval is applicable to the case where there is bidirectional media stream transmission between the sender and the receiver, that is, the sender sends an audio and video stream to the receiver, and at the same time, the receiver also sends a reverse audio and video stream (hereinafter referred to as reverse stream) to the sender. The sender can determine whether there is reverse packet loss by detecting the reverse stream. The media data transmission method also includes the following steps:

[0125] Both the sender and the receiver set the initial retransmission request interval equal to the single round trip delay. If the receiver finds that the data packet sent by the sender is lost, it will send a retransmission request to the sender according to the initial retransmission request interval.

[0126] The sending end receives a reverse code stream from the receiving end, and determines whether there is reverse packet loss according to the reverse code stream; when the sending end receives a reverse data packet of the reverse code stream from the receiving end, it determines whether there is reverse packet loss according to the sequence number of the reverse data packet. For example, when the sending end receives a reverse data packet with a later sequence number but has not yet received a reverse data packet with an earlier sequence number, it is considered that there is reverse packet loss;

[0127] When the sender does not detect reverse packet loss, it determines that the retransmission request interval is equal to the single round trip delay, that is, the retransmission request interval is not shortened at this time, and the retransmission request interval is kept equal to one RTT; that is, both the sender and the receiver keep the retransmission request interval equal to one RTT;

[0128] When the sender detects reverse packet loss, it shortens the retransmission request interval to less than a single round trip delay, for example, shortening the retransmission request interval to 1 / 2, 1 / 3 of an RTT, or shortening the retransmission request interval to max_delay*log(loss) / log(max_rremind_rate). After the sender detects reverse packet loss, it notifies the receiver of the reverse packet loss, and the receiver will simultaneously shorten the retransmission request interval to less than a single round trip delay. Therefore, after shortening the retransmission request interval, the retransmission request interval known by the sender is consistent with the retransmission request interval actually used by the receiver, which can be a pre-agreed reduced value to ensure that the sender can determine the number of retransmission packets to send each time based on the actual retransmission request interval of the receiver.

[0129] When the sending end detects that the retransmission request is lost, the number of times the sending end sends retransmission packets within one RTT is reduced. The reduced number is the number of lost retransmission requests, resulting in the number of retransmission packets sent within one RTT not being equal to the single transmission multiple. Therefore, in this embodiment, the redundancy multiple is further compensated by time deviation, that is, when sending the retransmission packet, the time deviation is compensated according to the compensation multiple. When sending the retransmission packet, the sending end also sends the second redundant packet, and the second redundant packet is used to compensate for the reduction in the number of retransmission packets sent. In step S400, when the sending end sends the retransmission packet corresponding to the single retransmission multiple of the target data packet to the receiving end, it also includes the following steps:

[0130] The sender calculates the time difference (t2-t1) between the first time now_time when the current retransmission request is received and the second time last_send_time when the previous retransmission request for the same target data packet is received;

[0131] The sending end calculates the compensation multiple according to the time difference (t2-t1) and the retransmission request interval nack_interval of the receiving end; the compensation multiple is calculated, for example, using the following formula 10:

[0132] compensate_times=(now_time-last_send_time) / nack_interval-1(Formula 10)

[0133] When the calculation result of compensate_times according to Formula 10 is negative, it is limited to the minimum value of 0;

[0134] When the compensation multiple compensate_times is 0, it means that the time difference between two consecutive retransmission requests received by the sender is consistent with a retransmission request interval, which means that no retransmission request is lost, so there is no need to perform time deviation compensation;

[0135] When the compensation multiple compensate_times is a positive number (i.e., the compensation multiple is greater than 0), it means that the time difference between two consecutive retransmission requests received by the sender is greater than a retransmission request interval, and at least one retransmission request loss occurred between the first time and the second time, and time deviation compensation is required. The sender sends the second redundant packet of the target data packet corresponding to the compensation multiple to the receiving end. That is, when the sender sends the retransmission packet, it also sends a number of second redundant packets equal to the compensation multiple to the receiving end. Here, the second redundant packet specifically refers to the compensated redundant packet determined according to the compensation multiple for time deviation compensation. Therefore, in the event of a retransmission request loss, for a target data packet, when the sender receives a new retransmission request, the number of data packets sent to the receiving end is equal to the number of retransmission packets + the number of second redundant packets.

[0136] Since the time difference between two adjacent retransmission requests sent by the receiving end is equal to the retransmission request interval nack_interval, when there is a retransmission request loss, the time difference between two adjacent retransmission requests received by the sending end should also be equal to a multiple of the retransmission request interval. For example, when one retransmission request is lost between the first time and the second time, (now_time-last_send_time) is twice the retransmission request interval, and when two retransmission requests are lost, (now_time-last_send_time) is three times the retransmission request interval. Therefore, (now_time-last_send_time) is an integer multiple of the retransmission request interval, so that the calculated compensate_times is also an integer. The present application effectively solves the problem of retransmission request loss by adopting a time deviation compensation method. This is equivalent to the sending end receiving fewer retransmission requests and sending fewer retransmission packets to the receiving end, so the less sent retransmission packets are compensated by the second redundant packet. For example, each time the sender sends a retransmission packet to the receiver, if a retransmission request is lost between the first time and the second time, the sender sends one less retransmission packet. At the second time, the compensation multiple calculated by formula 10 is 1. At the second time, the sender sends one retransmission packet + a second redundant packet. The second redundant packet can compensate for the retransmission packet that is less sent, thereby still meeting the requirement of being able to send (red_times+1) times of retransmission packets within one RTT.

[0137] After the single redundancy multiple is calculated by the first function, if the single redundancy multiple is an integer, then in step S300, the transmitting end can directly send the original packet and the redundant packets whose number is equal to the single redundancy multiple to the receiving end. If the single redundancy multiple is a decimal, it needs to be rounded up when actually sending the packets. However, the integer multiple will cause the total network traffic or video quality to jump instantly when the single redundancy multiple changes. For example, if the single redundancy multiple calculated at the previous moment is 4, then 4 redundant packets are sent at the same time when sending the data packet. At the next moment, the network packet loss rate and the single round-trip delay increase, resulting in the calculated single redundancy multiple of 4.5, which is rounded up to 5. When sending the data packet, 5 redundant packets are sent at the same time. The number of redundant packets at two adjacent moments is directly added by 1, which may cause the total network traffic or video quality to jump instantly.

[0138] In order to solve the problem of instantaneous jump, the decimal multiple packet sending method using decimal multiples to replace integer multiples is adopted in this embodiment. For example, the single redundancy multiple of the previous moment is 4, and the single redundancy multiple calculated at the next moment is 4.5. It is possible to control every two data packets as a group, the single redundancy multiple of the previous data packet is still maintained at 4, the single redundancy multiple of the next data packet is changed to 5, and the single redundancy multiple of the next data packet is still maintained at 4. Then, the influence of the single redundancy multiple on the total network flow or video quality can be halved compared with the direct rounding method, that is, the situation of instantaneous jump of the total network flow and video quality can be alleviated. Therefore, by adopting the decimal multiple packet sending method, a current redundancy multiple corresponding to each data packet is obtained according to the single redundancy multiple. When each data packet is sent, the first redundant packet is obtained according to its corresponding current redundancy multiple.

[0139] In step S300, the transmitting end uses the following steps to send the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end:

[0140] Taking the integer part of the single redundancy multiple as the first redundancy multiple;

[0141] Take the decimal part of the single redundancy multiple, and determine the number of interval packages n according to the decimal part. For example, if the decimal part is x, the number of interval packages n=1 / x;

[0142] An original packet to be sent that is separated by n packets in the data to be sent is used as the adjusted original packet, the first redundancy multiple plus 1 is used as the current redundancy multiple of the adjusted original packet, and the first redundancy multiple is used as the current redundancy multiple of the non-adjusted original packet; for example, when the single redundancy multiple is 4.1, in order to avoid directly jumping the current redundancy multiples of all data packets to 5, a decimal multiple packet sending technology is adopted, and the integer part 4 is taken as the first redundancy multiple, the decimal part is 0.1, and the number of interval packets is 10. Then, among 10 consecutive data packets to be sent, there is 1 data packet to be sent with a current redundancy multiple of 5, and the other 9 data packets to be sent have current redundancy multiples of 4. In the next group of 10 consecutive data packets to be sent, there is also 1 data packet to be sent with a current redundancy multiple of 5, and the other 9 data packets to be sent have current redundancy multiples of 4. This is equivalent to reducing the impact of the original integer multiple single redundancy multiple change to 0.1 times the impact of the single redundancy multiple change, which can avoid instantaneous jumps in total network traffic and video quality;

[0143] The sending end sends the original packet of the data packet to be sent and the first redundant packet whose number is equal to the current redundant multiple to the receiving end.

[0144] Therefore, when the packet loss rate increases, for the video bitstream, the redundancy multiple will not jump from 0 to 1 or higher compared to the previous moment, but will continuously change by multiples such as 0.1, 0.3, 0.5, and 0.8. For example, if the single redundancy multiple is 0.1, it means that each data packet has a 10% probability of sending a 1x redundant packet, or in other words, in a video bitstream, on average, 1 out of every 10 consecutive data packets will send a redundant packet, and the other 9 will not send a redundant packet. If the single redundancy multiple is 1.5, then the first data packet will send 1 redundant packet, the second data packet will send 2 redundant packets, the third data packet will send 1 redundant packet, and so on. In this way, the bandwidth occupied by the redundant code rate will not change significantly, and there will be no sudden jump in the video quality.

[0145] The fractional multiple packet sending method can be calculated by accumulating the remaining redundancy multiple. In step S300, for each data packet to be sent in a video code stream, the sender uses the following steps to determine the number of first redundant packets:

[0146] Taking the integer part of the single redundancy multiple as the first redundancy multiple;

[0147] Take the decimal part of the single redundancy multiple, add the decimal part accumulation value corresponding to the previous data packet to be sent to the decimal part, and obtain the decimal part accumulation value of the current data packet to be sent;

[0148] If the accumulated value of the decimal part of the current data packet to be sent is greater than or equal to 1, the first redundancy multiple is added by 1 as the current redundancy multiple of the current data packet to be sent, and the accumulated value of the decimal part of the current data packet to be sent is subtracted by 1, and the current redundancy multiple is the number of the first redundant packets of the current data packet to be sent;

[0149] If the accumulated value of the fractional part is less than 1, the first redundancy multiple is used as the current redundancy multiple of the data packet to be sent, and the current redundancy multiple is the number of the first redundant packets of the data packet to be sent.

[0150] Red_times represents the calculated single redundancy multiple in decimal form, red_times_int represents the current redundancy multiple corresponding to each data packet to be sent, and red_remainder represents the accumulated value of the decimal part. The calculation method of this redundancy multiple can be expressed in C language as follows (calc_red_times_int function is called when calculating the current redundancy multiple for each data packet to be sent):

[0151] static float red_remainder=0; / / Remaining redundancy multiple, initialized once at the beginning of video code stream sending

[0152] int calc_red_times_int(float red_times){

[0153] int red_times_int=(int)red_times; / / get the integer part of single redundancy multiple

[0154] float red_times_frac = red_times-red_times_int; / / Get the fractional part of the single redundancy multiple

[0155] if(red_times_frac>0){

[0156] red_remainder+=red_times_frac; / / Accumulate the decimal part to get the accumulated value of the decimal part

[0157] if(red_remainder>=1){ / / When the accumulated value of the decimal part exceeds 1

[0158] red_times_int++; / / Single redundancy multiple plus 1 to get this redundancy multiple

[0159] red_remainder-=1; / / decimal part accumulated value minus 1

[0160] }

[0161] }

[0162] return red_times_int; / / Return the redundancy multiple for this time, used as the number of the first redundant packets

[0163] }

[0164] The implementation method expressed in the above C language is taken as an example. In other implementations, other types of languages ​​or other logical sequences may also be used to implement the decimal multiple sending method.

[0165] like Figure 2 As shown, the embodiment of the present application also provides a media data transmission system, which is applied to the above-mentioned media data transmission method, and the system includes:

[0166] The network status acquisition module M100 is used to obtain the current packet loss rate and the current single round-trip delay;

[0167] The redundancy multiple calculation module M200 is used to calculate a single redundancy multiple based on a first function; wherein the first function includes a total sending multiple part and a maximum round trip number part, the total sending multiple part is obtained according to the current packet loss rate and the maximum allowable residual packet loss rate, and the maximum round trip number is obtained according to the current single round trip delay and the maximum allowable network delay;

[0168] The data packet sending module M300 is used to send, for each data packet to be sent, an original packet of the data packet to be sent and a first redundant packet corresponding to a single redundant multiple to a receiving end;

[0169] The data packet retransmission module M400 is used to send a retransmission packet of the target data packet corresponding to the single retransmission multiple to the receiving end when receiving a retransmission request for the target data packet from the receiving end. The single retransmission multiple satisfies: the sum of the single redundancy multiple and the number of target data packets is equal to the product of the maximum number of retransmissions within a current single round-trip delay and the single retransmission multiple.

[0170] By adopting the media data transmission system of the present application, the functions of each module in the media data transmission system can be implemented by adopting the specific implementation methods of the corresponding steps in the above-mentioned media data transmission method, and the media data transmission system can obtain the beneficial technical effects of the above-mentioned media data transmission method, which will not be repeated here.

[0171] An embodiment of the present application also provides a media data transmission device, comprising a processor; a memory, in which executable instructions of the processor are stored; wherein the processor is configured to perform the steps of the media data transmission method by executing the executable instructions.

[0172] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0173] Refer to the following Figure 3 The electronic device 600 according to this embodiment of the present application is described. Figure 3 The electronic device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0174] like Figure 3 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0175] The storage unit stores a program code, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps of various exemplary embodiments of the present application described in the above media data transmission method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0176] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0177] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0178] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0179] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0180] By adopting the media data transmission device provided in the present application, the processor executes the media data transmission method when executing the executable instructions, thereby obtaining the beneficial effects of the above media data transmission method.

[0181] The embodiment of the present application also provides a computer-readable storage medium for storing a program, and when the program is executed by a processor, the steps of the media data transmission method are implemented. In some possible implementations, various aspects of the present application can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present application described in the above media data transmission method section of this specification.

[0182] refer to Figure 4 As shown, a program product 800 for implementing the above method according to an embodiment of the present application is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0183] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0184] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0185] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or cluster. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0186] By adopting the computer-readable storage medium provided in the present application, the program stored therein implements the steps of the media data transmission method when being executed, thereby obtaining the beneficial effects of the above-mentioned media data transmission method.

[0187] The above content is a further detailed description of the present application in combination with specific preferred implementation methods, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. For ordinary technicians in the technical field to which the present application belongs, several simple deductions or substitutions can be made without departing from the concept of the present application, which should be deemed to fall within the scope of protection of the present application.

Claims

1. A method for transmitting media data, characterized in that: The steps include: The sender obtains the current packet loss rate and the current single round-trip delay; The transmitting end calculates a single redundancy multiple based on a first function; wherein the first function includes a total transmission multiple part and a maximum round trip number part, the total transmission multiple part is obtained according to the current packet loss rate and the maximum allowable residual packet loss rate, and the maximum round trip number is obtained according to the current single round trip delay and the maximum allowable network delay; For each data packet to be sent, the sending end sends the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end; When the sending end receives the retransmission request for the target data packet from the receiving end, the sending end sends a retransmission packet of the target data packet corresponding to the single retransmission multiple to the receiving end, and the single retransmission multiple satisfies: the sum of the single redundancy multiple and the number of target data packets is equal to the product of the maximum number of retransmissions within a current single round-trip delay and the single retransmission multiple.

2. The media data transmission method according to claim 1, characterized in that: The total sending multiple part of the first function satisfies: under the network environment of the current packet loss rate, the remaining packet loss rate when the sending multiple of each data packet to be sent reaches the total sending multiple is equal to the maximum allowed remaining packet loss rate; The maximum round trip number part of the first function satisfies: based on the packet loss retransmission mechanism, for a data packet to be sent, the total network delay when the number of consecutive round trips between the sending end and the receiving end reaches the maximum round trip number is equal to the maximum allowed network delay.

3. The media data transmission method according to claim 1, characterized in that: When the sending end receives a retransmission request for a target data packet from the receiving end, the sending end sends a retransmission packet of the target data packet corresponding to a single retransmission multiple to the receiving end, including: When the retransmission request interval is equal to the current single round trip delay, the transmitting end determines the number of the retransmission packets according to the sum of the single redundancy multiple and the number of the target data packets; and / or, When the retransmission request interval is less than the current single round-trip delay, the sending end calculates the maximum number of retransmissions based on the current single round-trip delay and the retransmission request interval, and calculates the single retransmission multiple based on the maximum number of retransmissions and the single redundancy multiple.

4. The media data transmission method according to claim 3, characterized in that: The following steps are also included: The transmitting end receives a reverse code stream from the receiving end, and determines whether there is reverse packet loss according to the reverse code stream; When the transmitting end does not detect the presence of reverse packet loss, determining that the retransmission request interval is equal to the single round trip delay; When the sending end detects reverse packet loss, the retransmission request interval is shortened to be less than the single round-trip delay.

5. The method for transmitting media data according to claim 1, characterized in that: After the transmitting end sends the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end, the following steps are also included: When the receiving end determines that the target data packet is lost, the retransmission request interval is calculated according to the maximum allowed network delay, the current packet loss rate and the maximum allowed remaining packet loss rate; The receiving end sends a retransmission request corresponding to the target data packet to the sending end based on the retransmission request interval; After receiving the retransmission request, the sending end sends a retransmission packet corresponding to the target data packet to the receiving end.

6. The media data transmission method according to claim 5, characterized in that: The receiving end calculates the retransmission request interval according to the maximum allowed network delay, the current packet loss rate, and the maximum allowed remaining packet loss rate by using the following steps: The retransmission request interval is calculated using a second function, wherein the second function includes a maximum allowed network delay part and a total sending multiple part, wherein the total sending multiple part is obtained based on the maximum allowed residual packet loss rate and the current packet loss rate, and the retransmission request interval is equal to the quotient of the maximum allowed network delay part and the total sending multiple part.

7. The media data transmission method according to claim 1, characterized in that: When the transmitting end sends the retransmission packet of the target data packet corresponding to the single retransmission multiple to the receiving end, the following steps are also included: The transmitting end calculates the time difference between a first time of receiving the current retransmission request and a second time of receiving the previous retransmission request for the same target data packet; The sending end calculates a compensation multiple according to the time difference and the retransmission request interval of the receiving end; When the compensation multiple is a positive number, the sending end sends a second redundant packet of the target data packet corresponding to the compensation multiple to the receiving end.

8. The media data transmission method according to claim 2, characterized in that: The transmitting end sends the original packet of the data packet to be sent and the first redundant packet corresponding to the single redundant multiple to the receiving end by adopting the following steps: Taking the integer part of the single redundancy multiple as the first redundancy multiple; Taking the decimal part of the single redundancy multiple, and determining the number of interval packets n according to the decimal part; Using an original packet to be sent at intervals of n data packets in the data to be sent as an adjusted original packet, adding 1 to the first redundancy multiple as the current redundancy multiple of the adjusted original packet, and using the first redundancy multiple as the current redundancy multiple of the non-adjusted original packet; The sending end sends the original packet to be sent and the first redundant packets whose number is equal to the current redundant multiple to the receiving end.

9. A media data transmission device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to execute the steps of the media data transmission method according to any one of claims 1 to 8 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the media data transmission method according to any one of claims 1 to 8 are implemented.

Images