Media data anti-weak network transmission method and system

Abstract

The invention relates to the technical field of media data transmission and financial science and technology, and discloses a weak network resistant transmission method and system for media data, and the method comprises the steps: calculating a first packet loss rate of a media packet transmitted between a first user side and a media gateway, and calculating a second packet loss rate of the media packet transmitted between the media gateway and a second user side; the first packet loss rate and the second packet loss rate are fed back to the first user side, so that the first user side dynamically and reasonably adjusts the FEC redundancy proportion according to the received packet loss rate, and the media packet is completely transmitted to the second user side from the first user side to the maximum extent.

Description

Technical Field

[0001] The present invention relates to the fields of media data transmission and financial technology, and in particular to a method and system for media data transmission against weak networks. Background Art

[0002] Currently, in order for a first user terminal and a second user terminal to communicate, media data needs to be transmitted between the first user terminal and the second user terminal. Currently, various harsh network environments may be encountered, such as elevators, restrooms with weak signals and insufficient coverage, subway stations, etc., at which communication requirements are often not met, and media data is lost. Therefore, how to ensure that media data is fully transmitted from the first user terminal to the second user terminal under weak network conditions has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0003] The present invention provides a method and system for transmitting media data in a weak network environment, so as to solve the technical problem that media data cannot be completely transmitted from a first user terminal to a second user terminal under a weak network environment.

[0004] In a first aspect, a method for transmitting media data against a weak network is provided, comprising:

[0005] The first user terminal generates a first FEC packet from the first media packet according to the FEC redundancy ratio, and sends the first media packet and the first FEC packet to the media gateway;

[0006] The media gateway generates a third media packet according to the received second media packet and the second FEC packet; calculates a first packet loss rate according to the number of the third media packets, the number of the second media packets and the number of the first media packets and sends the first packet loss rate to the first user terminal; sends the second media packet, the third media packet and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets;

[0007] The second user terminal generates a fifth media packet according to the received fourth media packet and the third FEC packet; calculates a second packet loss rate according to the number of the fifth media packet, the number of the fourth media packet, the number of the third media packet, and the number of the second media packet, and sends the second packet loss rate to the first user terminal through the media gateway; wherein the number of the fourth media packet is less than or equal to the sum of the number of the second media packet and the third media packet, and the number of the third FEC packet is less than or equal to the number of the second FEC packet;

[0008] The first user terminal adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0009] In a second aspect, a media data weak network resistant transmission system is provided, including a first user terminal, a media gateway, and a second user terminal;

[0010] A first user terminal, configured to generate a first FEC packet from a first media packet according to an FEC redundancy ratio, and send the first media packet and the first FEC packet to a media gateway;

[0011] The media gateway is configured to generate a third media packet according to the received second media packet and the second FEC packet; calculate a first packet loss rate according to the number of the third media packets, the number of the second media packets and the number of the first media packets and send the first packet loss rate to the first user terminal; and send the second media packet, the third media packet and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets;

[0012] The second user terminal is used to generate a fifth media packet according to the received fourth media packet and the third FEC packet; calculate a second packet loss rate according to the number of the fifth media packet, the number of the fourth media packet, the number of the third media packet, and the number of the second media packet, and send the second packet loss rate to the first user terminal through the media gateway; wherein the number of the fourth media packet is less than or equal to the sum of the number of the second media packet and the third media packet, and the number of the third FEC packet is less than or equal to the number of the second FEC packet;

[0013] The first user terminal is used to adjust the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0014] In the scheme implemented by the above-mentioned media data weak network resistant transmission method and system, the first user terminal can generate a first FEC packet from the first media packet according to the FEC redundancy ratio, and send the first media packet and the first FEC packet to the media gateway;

[0015] The media gateway generates a third media packet according to the received second media packet and the second FEC packet; calculates a first packet loss rate according to the number of the third media packets, the number of the second media packets and the number of the first media packets and sends it to the first user end; sends the second media packet, the third media packet and the second FEC packet to the second user end; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; the second user end generates a fifth media packet according to the received fourth media packet and the third FEC packet; calculates a second packet loss rate according to the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets and the number of the second media packets and sends it to the first user end through the first FEC packet; The media gateway sends the fourth media packet to the first user end; wherein the number of the fourth media packet is less than or equal to the sum of the number of the second media packet and the third media packet, and the number of the third FEC packet is less than or equal to the number of the second FEC packet; the first user end adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate. In the present invention, the first packet loss rate of the media packet transmitted between the first user end and the media gateway is calculated, and the second packet loss rate of the media packet transmitted between the media gateway and the second user end is calculated, and the first packet loss rate and the second packet loss rate are fed back to the first user end, so that the first user end dynamically and reasonably adjusts the FEC redundancy ratio according to the received packet loss rate, so that the media packet is transmitted from the first user end to the second user end in its entirety to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0017] Figure 1 This is a schematic diagram of an application environment of a method for transmitting media data against weak networks in an embodiment of the present invention;

[0018] Figure 2 It is a flow chart of a method for transmitting media data in a weak network in accordance with an embodiment of the present invention;

[0019] Figure 3 It is another flowchart of the method for transmitting media data in a weak network in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] At present, the first user terminal and the second user terminal need to transmit media data between the first user terminal and the second user terminal to realize the call. In the related technology, WebRTC (Web Real-Time Communications) is a real-time communication technology that allows network applications or sites to establish a peer-to-peer connection to realize the transmission of video streams and / or audio streams or any other data. The real-time communication system based on WebRTC uses UDP for transmission, and the underlying media packet adopts the form of RTP packet. However, UDP is an unreliable transmission link. The WebRTC technology stack provides a complete set of components with weak network resistance, mainly including accelerated playback, preemptive expand, packet loss compensation (PLC), FEC (Forward Error Correction) and retransmission technology, which are very professional algorithms. It can also exert considerable weak network resistance without any optimization adjustment. However, various harsh network environments may still be encountered, such as elevators, toilets with weak signals and insufficient coverage, subway stations, etc., which often cannot meet the communication requirements. Therefore, how to ensure the reliability of media data transmission under weak network conditions has become a technical problem that needs to be solved urgently by technicians in this field.

[0022] The media data weak network resistant transmission method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, a first user terminal and a second user terminal communicate through a media gateway. The first user terminal generates a first FEC packet from a first media packet according to an FEC redundancy ratio, and sends the first media packet and the first FEC packet to the media gateway; the media gateway generates a third media packet according to the received second media packet and the second FEC packet; calculates a first packet loss rate according to the number of the third media packets, the number of the second media packets, and the number of the first media packets, and sends it to the first user terminal; sends the second media packet, the third media packet, and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; the first The second user end generates a fifth media packet according to the received fourth media packet and the third FEC packet; calculates the second packet loss rate according to the number of the fifth media packet, the number of the fourth media packet, the number of the third media packet, and the number of the second media packet, and sends it to the first user end through the media gateway; wherein the number of the fourth media packet is less than or equal to the sum of the number of the second media packet and the third media packet, and the number of the third FEC packet is less than or equal to the number of the second FEC packet; the first user end adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate. In the present invention, the FEC technology is used to perform forward error correction protection on the media packet to form transmission protection. When the media gateway and the second user end receive the media packet and the FEC packet, the corresponding packet loss rate will be calculated and fed back to the first user end according to the packet loss rate, so that the first user end can dynamically and reasonably adjust the FEC redundancy ratio according to the packet loss rate, provide data transmission reliability, and improve transmission efficiency, so that the media packet is fully transmitted from the first user end to the second user end to the greatest extent.

[0023] The user terminal may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The present invention is described in detail below through specific embodiments.

[0024] See also Figure 2 As shown, Figure 2 A flowchart of a method for transmitting media data against weak networks provided by an embodiment of the present invention includes the following steps S100-S1000.

[0025] S100: The first user terminal generates Ns first FEC packets from Ks first media packets according to the FEC redundancy ratio. Exemplarily, the first user terminal may implement this step using an FEC encoder.

[0026] In the field of Ping An Finance and Insurance, audio and video interaction is required during the double recording of insurance contracts, and the method in the embodiment of this application can be used. In order to ensure the compliance of the insurance process, remote double recording, service terms personnel identity verification, etc. are required, and the whole process requires audio and video interaction.

[0027] S200: Send Ks first media packets and Ns first FEC packets to a media gateway.

[0028] In one example, taking audio Opus encoding as an example, the first media packet is an RTP data packet, 50 RTP data packets are sent per second, and each RTP data packet includes audio of 20 ms duration.

[0029] In the embodiment of the present application, the FEC redundancy ratio N refers to N media packets generating one FEC packet. In one example, the redundancy ratio is calculated as follows: redundancy ratio = (number of FEC packets) / (number of FEC packets + number of media packets).

[0030] It is very important to set the FEC redundancy ratio reasonably. On the one hand, a higher FEC redundancy ratio can provide stronger error correction capability, thereby increasing the reliability of data transmission. In an environment with high noise or strong interference, a higher FEC redundancy ratio can more effectively detect and correct transmission errors and reduce the risk of data loss. In addition, a higher redundancy ratio can also reduce the demand for retransmission, thereby reducing the delay and bandwidth consumption caused by retransmission. On the other hand, a higher FEC redundancy ratio will also increase the overhead of data transmission and increase the amount of data transmitted, thereby occupying more bandwidth resources. In the case of limited bandwidth, an excessively high FEC redundancy ratio may reduce transmission efficiency. Therefore, the embodiment of the present application sets a suitable FEC redundancy ratio in a dynamically adjusted manner, which provides reliability of data transmission while improving transmission efficiency.

[0031] S300, the media gateway generates M third media packets according to the received Kr second media packets and Nr second FEC packets. The number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets. Exemplarily, the media gateway can use an FEC decoder to complete step S300.

[0032] In a weak network environment, packet loss may occur when the first user terminal sends Ks first media packets and Ns first FEC packets to the media gateway. At this time, the media gateway may not be able to receive all the media packets in the Ks first media packets, and may only receive Kr second media packets, Kr <= Ks. Similarly, the media gateway may only receive Nr second FEC packets, Nr <= Ns.

[0033] In the embodiment of the present application, each media packet carries a corresponding RTPSequence sequence, which indicates the order of the media packets. The FEC packet also includes an RTPSequence sequence, and the lost media packet can be determined through the RTPSequence sequences in the media packet and the FEC packet.

[0034] S400 , calculating a first packet loss rate according to the number (M) of the third media packets, the number (Kr) of the second media packets, and the number (Nr) of the first media packets.

[0035] S500: Send a first packet loss rate to the first user terminal. In one example, the first packet loss rate = (Ks-M-Kr) ÷ Ks.

[0036] S600: The media gateway sends the second media packet, the third media packet and the second FEC packet to the second user terminal. In the embodiment of the present application, the first user terminal sends the media packet and the FEC packet to the second user terminal through the media gateway.

[0037] S700, the second user terminal generates M2 fifth media packets according to the received (Kr2+Mr2) fourth media packets and Nr2 third FEC packets. The number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets. Exemplarily, the second user terminal completes S700 through an FEC decoder.

[0038] In a weak network environment, packet loss may occur when the media gateway sends Kr second media packets, M third media packets, and Nr second FEC packets to the second user end. At this time, the second user end may not receive all the media packets in the Kr second media packets and M third media packets, and may only receive (Kr2+Mr2) fourth media packets, where Kr2 fourth media packets are partial media packets received from the Kr second media packets, Kr2<=Kr, and Mr2 fourth media packets are partial media packets received from the M third media packets, Mr2<=M. Similarly, the second user end may only receive Nr2 third FEC packets, Nr2<=Nr.

[0039] In the embodiment of the present application, the process of generating the fifth media package is the same as the process of generating the third media package, which will not be repeated here.

[0040] S800. Calculate the second packet loss rate according to the number of the fifth media packets (M2), the number of the fourth media packets ((Kr2+Mr2)), the number of the third media packets (M) and the number of the second media packets (Kr). S900. Send the second packet loss rate to the first user terminal through the media gateway. Exemplarily, the second packet loss rate = (M+Kr-Kr2-Mr2-M2)÷(M+Kr).

[0041] S1000. The first user terminal adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0042] In the embodiment of the present application, the packet loss rate can reflect information such as the current network situation, and then according to the first packet loss rate between the first user terminal and the media gateway, and the second packet loss rate between the media gateway and the second user terminal, the FEC redundancy ratio is reasonably adjusted to provide reliability of data transmission while improving transmission efficiency. When the first user terminal subsequently sends a media packet, the adjusted FEC redundancy ratio is used to generate an FEC packet.

[0043] In some embodiments, the step of adjusting the FEC redundancy ratio by the first user terminal according to the first packet loss rate and the second packet loss rate includes:

[0044] The first user terminal selects the maximum packet loss rate from the first packet loss rate and the second packet loss rate received within the preset period according to a preset period.

[0045] In the embodiment of the present application, since it takes time for data to be transmitted between the first user terminal, the media gateway, and the second user terminal, the setting of the preset period needs to ensure that the first user terminal receives at least one first packet loss rate and one second packet loss rate. Exemplarily, the preset period can be every 1 second, and the first user terminal selects the maximum packet loss rate from the first packet loss rate and the second packet loss rate received within 1 second. The above operation is performed for the first packet loss rate and the second packet loss rate received within every 1 second.

[0046] The FEC redundancy ratio is determined according to the maximum packet loss rate.

[0047] In one example, if the maximum packet loss rate selected from the received first packet loss rate and the second packet loss rate is 20%, the FEC redundancy ratio is determined to be 2. If the maximum packet loss rate is 40%, the FEC redundancy ratio is determined to be 4. If the maximum packet loss rate is greater than 40%, the FEC redundancy ratio is determined to be 10.

[0048] In some embodiments, in order to enable the media gateway to receive all the contents of the Ks first media packets and the Ns first FEC packets sent by the first user end, the method further includes:

[0049] The media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal. Exemplarily, the media gateway may generate the first packet replenishment request through an uplink retransmission packet generator.

[0050] In an embodiment of the present application, the second media packet and the third media packet are saved as a circular list RList(Kr+M) according to the RTPSequence sequence, and the lost media packet can be determined through the circular list. A first packet replacement request can be generated based on the lost media packet. Exemplarily, the media gateway should receive media packets of RTPSequence sequence 1-5, but the current first media packet is a media packet of RTPSequence sequence 1-2, and the third media packet is a media packet of RTPSequence sequence 3, then the media packets of RTPSequence sequence 4 and 5 are lost. At this time, a first packet replacement request is generated based on the media packets of RTPSequence sequence 4 and 5.

[0051] The first user terminal sends a first retransmission packet Rr to the media gateway according to the first packet replenishment request.

[0052] In the embodiment of the present application, the first user terminal receives the first packet replenishment request to determine the first retransmission packet Rr corresponding to the first packet replenishment request, and sends the first retransmission packet to the media gateway. It should be noted that the actual data in the first retransmission packet in the embodiment of the present application is still a media packet, but in order to distinguish that the retransmission packet is obtained using the packet replenishment request, the distinction is only made in name.

[0053] In some embodiments, when there is a step in the method where the media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal, the step where the media gateway sends the second media packet, the third media packet, and the second FEC packet to the second user terminal includes:

[0054] The media gateway sends the Rr2 second retransmission packets, Kr second media packets, M third media packets and Nr second FEC packets to the second user end, and the number of the fourth media packets is less than or equal to the sum of the number of the second retransmission packets, the second media packets and the third media packets.

[0055] In the embodiment of the present application, in addition to the second media packet, the third media packet and the second FEC packet, the second retransmission packet obtained according to the first packet replenishment request also needs to be transmitted from the media gateway to the second user terminal. Since the fourth media packet is a media packet sent from the media gateway to the second user terminal, during the transmission process from the media gateway to the second user terminal, the media packets in the Rr2 second retransmission packets, Kr second media packets and M third media packets may be lost. Therefore, the number of Kr2+Mr2+Rr3 fourth media packets composed of the second media packet, the third media packet and the second retransmission packet transmitted to the data packet is less than or equal to the sum of the number of Rr2 second retransmission packets, Kr second media packets and M third media packets, where Kr2≤Kr, Mr2≤M, Rr3≤Rr2.

[0056] In some embodiments, Figure 3 This is another flow chart of the method for transmitting media data against weak network in one embodiment of the present invention. The first packet replenishment request includes a missing packet replenishment request, and the second retransmission packet includes a third retransmission packet; the media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal, including the following steps:

[0057] S301. The media gateway determines the lost media packet according to Kr the second media packet and M the third media packet.

[0058] S302: Generate a missing data packet list according to the lost media packet, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal, and start timing using a clock driver.

[0059] Exemplarily, the lost media packets may be media packets of RTPSequence sequences 4 and 5. A missing data packet list is generated according to the media packets of RTPSequence sequences 4 and 5, and a missing packet replenishment request corresponding to the missing data packet list is sent to the first user terminal.

[0060] S303: Determine a first preset packet replenishment interval and a first number of transmission times according to the round-trip time required for the media packet to travel between the media gateway and the first user terminal.

[0061] In the embodiment of the present application, in order to avoid the situation where the missing packet replenishment request cannot be successfully sent to the first user end after being sent only once, or the third retransmission packet obtained by the first user end according to the missing packet replenishment request cannot be successfully transmitted to the media gateway, and thus the lost media packet cannot be supplemented by the third retransmission packet, the missing packet replenishment request is sent a first number of times according to the first preset packet replenishment interval, that is, the missing packet replenishment request is sent multiple times.

[0062] Exemplarily, the round-trip time between the media gateway and the first user terminal is 200ms, the first preset packet replenishment interval may be 40ms, and the first number of transmissions may be obtained by dividing the required time by the first preset packet replenishment interval, so the first number of transmissions is 5 times.

[0063] S304: When the first timing time of the clock driver reaches the first preset packet replenishment interval, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal once until the number of times the missing packet replenishment request is sent reaches a first sending number.

[0064] In this way, sending the missing packet replenishment request multiple times can make the first user end send the corresponding first retransmission packet multiple times according to the missing packet replenishment request, thereby avoiding the loss of the missing packet replenishment request or the failure of the first retransmission packet corresponding to the missing packet replenishment request to be sent once to be smoothly sent from the first user end to the media gateway. Exemplarily, when the first timing time of the clock driver reaches 40ms, the missing packet replenishment request is sent to the first user end once until it is sent 5 times.

[0065] In the embodiment of the present application, when the media gateway finds that there is a lost media packet, it immediately generates a corresponding missing packet replenishment request according to the lost media packet. In addition, a clock drive is added to continuously send the missing packet replenishment request to the first user terminal.

[0066] The method further includes: the media gateway receiving a third retransmission packet sent by the first user terminal according to a missing packet replenishment request.

[0067] In an embodiment of the present application, when the first user terminal receives a missing packet replenishment request, it determines the first retransmission packet corresponding to the missing packet replenishment request, and sends the first retransmission packet to the media gateway. In a weak network environment, the first retransmission packet may not be fully transmitted to the media gateway, and the media gateway may receive part or all of the first retransmission packet, that is, the third retransmission packet.

[0068] In some embodiments, the first packet replenishment request further includes a first estimated packet loss replenishment request, and the second retransmission packet further includes a fourth retransmission packet; the media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and the step of sending the first packet replenishment request to the first user terminal further includes:

[0069] The estimated lost data packet is determined according to the third retransmitted packet and the missing data packet list.

[0070] In the embodiment of the present application, the missing data packet list includes missing media packets, and the third retransmission packet is a retransmission packet sent to the media gateway by the first user terminal according to the missing packet replenishment request. However, due to a weak network condition, the third retransmission packet may only be part of the content of the missing media packet. Therefore, based on the third retransmission packet and the missing data packet list, an estimated lost data packet is determined, and the estimated lost data packet includes the media packets that have not yet been received in the missing data packet list.

[0071] A first estimated lost packet replenishment request is generated according to the preset lost data packet, and the first estimated lost packet replenishment request is sent to the first user terminal according to a second preset replenishment interval until the second timing time of the clock driver reaches the preset time.

[0072] Exemplarily, the missing data packet list includes packet No. 5 and packet No. 7, and the third retransmission packet includes packet No. 5. At this time, the estimated lost data packet is packet No. 7. A first estimated lost packet replenishment request is generated according to packet No. 7, and the first estimated lost packet replenishment request is continuously sent to the first user terminal according to the second preset replenishment interval time until the second timing time of the clock driver reaches the preset time. Exemplarily, the preset time is 2s, and the second preset replenishment time event is 200ms.

[0073] In the embodiment of the present application, the step of generating a first estimated packet loss replenishment request according to a preset lost data packet may include generating an estimated packet loss data packet list according to the preset lost data packet, and generating the first estimated packet loss replenishment request according to the estimated packet loss data packet list.

[0074] In the embodiment of the present application, a packet loss prediction strategy is added to generate a first estimated packet loss replenishment request, thereby increasing the probability that other media packets in the missing data packet list except the third retransmission packet are successfully sent to the media gateway.

[0075] In some embodiments, the first packet replenishment request further includes a second estimated packet loss replenishment request, the second retransmission packet further includes a fifth retransmission packet, and the method further includes:

[0076] The media gateway determines, at a preset detection interval, whether a sixth media packet sent by the first user terminal is received.

[0077] In the embodiment of the present application, if the first user terminal continuously generates media packets, the first user terminal should continuously send media packets to the media gateway. For example, the round-trip time between the media gateway and the first user terminal is 200ms, and a media packet is sent every 20ms. It can be determined whether the media gateway has received 5 media packets within half of the round-trip time, that is, within 100ms. However, if the sixth media packet sent by the first user terminal is still not received within the preset detection time, for example within 100ms, for example, the sixth media packet that should be received includes media packets of RTPSequence sequence 6-10, it is possible that the sixth media packet is lost during the transmission process. At this time, a second preset packet loss replenishment request is generated based on the sixth media packet.

[0078] If the sixth media packet is not received, a second estimated packet loss replenishment request corresponding to the sixth media packet is generated and sent to the first user terminal.

[0079] The method further includes: the media gateway receives a fifth retransmission packet sent by the first user terminal according to the second estimated packet loss replenishment request.

[0080] In the embodiment of the present application, after receiving the second estimated packet loss compensation request, the first user terminal determines the corresponding first retransmission packet according to the second estimated packet loss compensation request, and sends the first retransmission packet corresponding to the second estimated packet loss compensation request to the media gateway. Since the first retransmission packet may be lost during the transmission process, the media gateway may only receive the fifth retransmission packet. The fifth retransmission packet may be part or all of the first retransmission packet corresponding to the second estimated packet loss compensation request.

[0081] In the embodiment of the present application, the media gateway helps to retransmit and recover known lost media packets through missing packet replenishment requests, first estimated packet loss replenishment requests and second estimated packet loss replenishment requests, and predicts possible lost media packets that have not yet been reached to trigger retransmission and recovery in advance, thereby improving the real-time performance and data integrity of the data, and improving the user experience.

[0082] In some embodiments, before the media gateway sends the second retransmission packet, the second media packet, the third media packet and the second FEC packet to the second user end, it also includes: deleting duplicate data packets in the second retransmission packet to update the second retransmission packet.

[0083] Since the second retransmission packet may include multiple identical media packets obtained by the first packet replenishment request, the duplicate data packets in the second retransmission packet are deleted.

[0084] In some embodiments, the method further comprises:

[0085] The second user terminal generates a second packet replenishment request according to the fourth media packet and the fifth media packet, and sends the second packet replenishment request to the media gateway.

[0086] In the embodiment of the present application, the process of the second user terminal generating a second packet replenishment request based on the fourth media packet and the fifth media packet is the same as the process of the media gateway generating a first packet replenishment request based on the second media packet and the third media packet in the above text, and will not be repeated here.

[0087] The media gateway sends the sixth retransmission packet to the second user terminal according to the second packet replenishment request, so that the second user terminal receives the seventh retransmission packet.

[0088] In the embodiment of the present application, when the media gateway receives the second packet replenishment request, it passively sends the sixth retransmission packet to the second user end.

[0089] The media gateway delays sending the sixth retransmission packet with a preset number of redundancy times to the second user terminal based on the current FEC redundancy multiple and the number of retransmission packets corresponding to the second packet replenishment request, so that the second user terminal receives the eighth retransmission packet, and the total number of the seventh retransmission packet and the eighth retransmission packet is less than or equal to the total number of the sixth retransmission packet sent by the media gateway.

[0090] In the embodiment of the present application, after sending the sixth retransmission packet for the first time, the media gateway also actively sends the sixth retransmission packet with a preset number of redundancy times to the second user end. Due to the possible weak network situation, the media gateway may not successfully reach the destination if it only sends the sixth retransmission packet once to the second user end. Therefore, in the embodiment of the present application, the current FEC redundancy multiple and the retransmission packet data corresponding to the second packet replenishment request are used to determine the current network situation, and the sixth retransmission packet is sent to the second user end multiple times with a delay, so that the probability of the second user end receiving all the contents in the sixth retransmission packet can be increased.

[0091] In an embodiment of the present application, a scheme for increasing the delay preset redundant times to send the sixth retransmission packet can shorten the network time for multiple back and forth requests for packet replenishment in a weak network condition, effectively reducing the delay in the call and increasing data integrity.

[0092] In one example, the FEC redundancy multiple is greater than 4 times, the number of retransmission packets corresponding to the second packet replenishment request is less than 10, and the number of consecutive retransmission packets is less than 6, which triggers a 3-fold delayed transmission, that is, the preset redundancy number is 3 times, and a total of 4 sixth retransmission packets are sent to the second user end. In addition, a data storm prevention strategy is also provided in this example. Other conditions remain unchanged. If the number of retransmission packets corresponding to the second packet replenishment request is greater than 15, a 1-fold delayed transmission is triggered, that is, the preset redundancy number is 1 time, and a total of 2 sixth retransmission packets are sent to the second user end. This is because there are too many retransmission packets, and multiple delayed transmissions may occupy excessive network resources.

[0093] In another example, if the FEC redundancy multiple is greater than or equal to 2 times, the number of retransmission packets corresponding to the second packet replenishment request is less than 12, and the number of consecutive retransmission packets is less than 8, then a 2-fold delay is triggered, i.e., the preset redundancy number is 2 times, and a total of 3 sixth retransmission packets are sent to the second user end. In addition, in this example, a data storm prevention strategy is also provided. If other conditions remain unchanged, the number of retransmission packets corresponding to the second packet replenishment request is greater than 20, which will trigger a 1-fold delay, i.e., the preset redundancy number is 1 time, and a total of 2 sixth retransmission packets are sent to the second user end.

[0094] In another example, if the number of retransmitted packets corresponding to the second packet replenishment request is greater than 25, it will trigger a 1x delayed transmission, that is, the preset redundancy number is 1, and a total of 2 sixth retransmitted packets are sent to the second user end.

[0095] In a laboratory environment, when a first user terminal and a second user terminal conduct a one-to-one call, in a scenario where the network randomly loses 60% of packets, the call was difficult to conduct before the improvement, but the method in the embodiment of the present application allows a clear and normal call.

[0096] It can be seen that in the above scheme, the first packet loss rate of the media packet transmitted between the first user end and the media gateway is calculated, and the second packet loss rate of the media packet transmitted between the media gateway and the second user end is calculated, and the first packet loss rate and the second packet loss rate are fed back to the first user end, so that the first user end can dynamically and reasonably adjust the FEC redundancy ratio according to the received packet loss rate, so that the media packet can be transmitted completely from the first user end to the second user end to the greatest extent.

[0097] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0098] In one embodiment, a media data anti-weak network transmission system is provided, and the media data anti-weak network transmission system corresponds one-to-one to the media data anti-weak network transmission method in the above embodiment. The system includes a first user terminal, a media gateway, and a second user terminal;

[0099] A first user terminal, configured to generate a first FEC packet from a first media packet according to an FEC redundancy ratio, and send the first media packet and the first FEC packet to a media gateway;

[0100] The media gateway is configured to generate a third media packet according to the received second media packet and the second FEC packet; calculate a first packet loss rate according to the number of the third media packets, the number of the second media packets and the number of the first media packets and send the first packet loss rate to the first user terminal; and send the second media packet, the third media packet and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets;

[0101] The second user terminal is used to generate a fifth media packet according to the received fourth media packet and the third FEC packet; calculate a second packet loss rate according to the number of the fifth media packet, the number of the fourth media packet, the number of the third media packet, and the number of the second media packet, and send the second packet loss rate to the first user terminal through the media gateway; wherein the number of the fourth media packet is less than or equal to the sum of the number of the second media packet and the third media packet, and the number of the third FEC packet is less than or equal to the number of the second FEC packet;

[0102] The first user terminal is used to adjust the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0103] In some embodiments, the first user terminal is specifically used to filter out the maximum packet loss rate from the first packet loss rate and the second packet loss rate received within the preset period according to a preset period; and determine the FEC redundancy ratio according to the maximum packet loss rate.

[0104] In some embodiments, the media gateway is further configured to generate a first packet replenishment request according to the second media packet and the third media packet, and send the first packet replenishment request to the first user terminal;

[0105] The first user terminal is further configured to send a first retransmission packet to the media gateway according to the first packet replenishment request.

[0106] In some embodiments, the media gateway is specifically used to send the received second retransmission packet, second media packet, third media packet and second FEC packet to the second user end, and the number of the fourth media packet is less than or equal to the sum of the number of the second retransmission packet, the second media packet and the third media packet, and the number of the second retransmission packet is less than or equal to the first retransmission packet.

[0107] In some embodiments, the first packet replenishment request includes a missing packet replenishment request, and the second retransmission packet includes a third retransmission packet; and the media gateway is specifically configured to:

[0108] The media gateway determines the lost media packet according to the second media packet and the third media packet;

[0109] Generate a missing data packet list according to the lost media packet, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal, and start timing by using a clock driver;

[0110] Determine a first preset packet replenishment interval and a first number of transmission times according to the round-trip time required for the media packet between the media gateway and the first user terminal;

[0111] When the first timing time of the clock driver reaches the first preset packet replenishment interval, a missing packet replenishment request corresponding to the missing data packet list is sent to the first user end once until the number of times the missing packet replenishment request is sent reaches a first sending number.

[0112] The media gateway is further configured to receive a third retransmission packet sent by the first user terminal according to the missing packet replenishment request.

[0113] In some embodiments, the first packet replenishment request further includes a first estimated packet loss replenishment request, and the second retransmission packet further includes a fourth retransmission packet; the media gateway is specifically configured to determine the estimated lost data packet according to the third retransmission packet and the missing data packet list;

[0114] Generate a first estimated lost packet replenishment request according to the preset lost data packet, and send the first estimated lost packet replenishment request to the first user terminal according to a second preset replenishment interval until the second timing time of the clock driver reaches the preset time;

[0115] The media gateway is further configured to receive a fourth retransmitted packet sent by the first user terminal according to the first estimated packet loss replenishment request.

[0116] In some embodiments, the first packet replenishment request further includes a second estimated packet loss replenishment request, the second retransmission packet further includes a fifth retransmission packet, and the media gateway is further configured to determine whether a sixth media packet sent by the first user terminal is received at a preset detection time interval;

[0117] If the sixth media packet is not received, generating a second estimated packet loss replenishment request corresponding to the sixth media packet and sending the request to the first user terminal;

[0118] The media gateway is further configured to receive a fifth retransmitted packet sent by the first user terminal according to the second estimated packet loss replenishment request.

[0119] In some embodiments, the media gateway is further configured to delete duplicate data packets in the second retransmission packet to update the second retransmission packet.

[0120] In some embodiments, the second user terminal is further used to generate a second packet replenishment request according to the fourth media packet and the fifth media packet, and send the second packet replenishment request to the media gateway;

[0121] The media gateway sends the sixth retransmission packet to the second user terminal according to the second packet replenishment request, so that the second user terminal receives the seventh retransmission packet;

[0122] The media gateway delays sending the sixth retransmission packet with a preset number of redundancy times to the second user terminal based on the current FEC redundancy multiple and the number of retransmission packets corresponding to the second packet replenishment request, so that the second user terminal receives the eighth retransmission packet, and the total number of the seventh retransmission packet and the eighth retransmission packet is less than or equal to the total number of the sixth retransmission packet sent by the media gateway.

[0123] The present invention provides a media data weak network resistant transmission system, in which a first packet loss rate of a media packet transmitted between a first user end and a media gateway is calculated, and a second packet loss rate of a media packet transmitted between a media gateway and a second user end is calculated, and the first packet loss rate and the second packet loss rate are fed back to the first user end, so that the first user end dynamically and reasonably adjusts the FEC redundancy ratio according to the received packet loss rate, so that the media packet is completely transmitted from the first user end to the second user end to the greatest extent.

[0124] For the specific limitations of the media data anti-weak network transmission system, please refer to the limitations of the media data anti-weak network transmission method above, which will not be repeated here. Each module in the above-mentioned media data anti-weak network transmission system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0125] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0126] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0127] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for transmitting media data against weak networks, characterized in that: include: The first user terminal generates a first FEC packet from the first media packet according to the FEC redundancy ratio, and sends the first media packet and the first FEC packet to the media gateway; The media gateway generates a third media packet according to the received second media packet and the second FEC packet, wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; calculates a first packet loss rate according to the number of the third media packets, the number of the second media packets and the number of the first media packets, and sends the first packet loss rate to the first user terminal; and sends the second media packet, the third media packet and the second FEC packet to the second user terminal; The second user terminal generates a fifth media packet according to the received fourth media packet and the third FEC packet, wherein the number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets; calculates a second packet loss rate according to the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets, and the number of the second media packets, and sends the second packet loss rate to the first user terminal through the media gateway; The first user terminal adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

2. The method according to claim 1, characterized in that The step of adjusting the FEC redundancy ratio by the first user end according to the first packet loss rate and the second packet loss rate includes: The first user terminal selects a maximum packet loss rate from the first packet loss rate and the second packet loss rate received within the preset period according to a preset period; and determines an FEC redundancy ratio according to the maximum packet loss rate.

3. The method according to claim 1, characterized in that Also includes: The media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal; The first user terminal sends a first retransmission packet to the media gateway according to the first packet replenishment request.

4. The method according to claim 3, characterized in that The step of the media gateway sending the second media packet, the third media packet and the second FEC packet to the second user terminal comprises: The media gateway sends the received second retransmission packet, second media packet, third media packet and second FEC packet to the second user end, and the number of the fourth media packet is less than or equal to the sum of the number of the second retransmission packet, the second media packet and the third media packet, and the number of the second retransmission packet is less than or equal to the first retransmission packet.

5. The method according to claim 4, characterized in that The first packet replenishment request includes a missing packet replenishment request, and the second retransmission packet includes a third retransmission packet; the media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal, comprising: The media gateway determines the lost media packet according to the second media packet and the third media packet; Generate a missing data packet list according to the lost media packet, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal, and start timing by using a clock driver; Determine a first preset packet replenishment interval and a first number of transmission times according to the round-trip time required for the media packet between the media gateway and the first user terminal; When the first timing time of the clock driver reaches the first preset packet replenishment interval, a missing packet replenishment request corresponding to the missing data packet list is sent to the first user end once until the number of times the missing packet replenishment request is sent reaches a first sending number. The method further includes: the media gateway receiving a third retransmission packet sent by the first user terminal according to the missing packet replenishment request.

6. The method according to claim 5, characterized in that The first packet replenishment request also includes a first estimated packet loss replenishment request, and the second retransmission packet also includes a fourth retransmission packet; the media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal, further comprising: Determine an estimated lost data packet according to the third retransmitted packet and the missing data packet list; Generate a first estimated lost packet replenishment request according to the preset lost data packet, and send the first estimated lost packet replenishment request to the first user terminal according to a second preset replenishment interval until the second timing time of the clock driver reaches the preset time; The method further includes: the media gateway receiving a fourth retransmission packet sent by the first user terminal according to the first estimated packet loss replenishment request.

7. The method according to claim 5, characterized in that The first packet replenishment request further includes a second estimated packet loss replenishment request, the second retransmission packet further includes a fifth retransmission packet, and the method further includes: The media gateway determines at a preset detection time interval whether a sixth media packet sent by the first user terminal is received; If the sixth media packet is not received, generating a second estimated packet loss replenishment request corresponding to the sixth media packet and sending the request to the first user terminal; The method further includes: the media gateway receives a fifth retransmission packet sent by the first user terminal according to the second estimated packet loss replenishment request.

8. The method according to claim 6, characterized in that Before the media gateway sends the second retransmission packet, the second media packet, the third media packet and the second FEC packet to the second user terminal, the method further includes: deleting duplicate data packets in the second retransmission packet to update the second retransmission packet.

9. The method according to claim 4, characterized in that Also includes: The second user terminal generates a second packet replenishment request according to the fourth media packet and the fifth media packet, and sends the second packet replenishment request to the media gateway; The media gateway sends the sixth retransmission packet to the second user terminal according to the second packet replenishment request, so that the second user terminal receives the seventh retransmission packet; The media gateway delays sending the sixth retransmission packet with a preset number of redundancy times to the second user terminal based on the current FEC redundancy multiple and the number of retransmission packets corresponding to the second packet replenishment request, so that the second user terminal receives the eighth retransmission packet, and the total number of the seventh retransmission packet and the eighth retransmission packet is less than or equal to the total number of the sixth retransmission packet sent by the media gateway.

10. A media data weak network transmission system, characterized in that: comprising a first user terminal, a media gateway and a second user terminal; A first user terminal, configured to generate a first FEC packet from a first media packet according to an FEC redundancy ratio, and send the first media packet and the first FEC packet to a media gateway; The media gateway is configured to generate a third media packet according to the received second media packet and the second FEC packet; calculate a first packet loss rate according to the number of the third media packets, the number of the second media packets and the number of the first media packets, and send the first packet loss rate to the first user terminal; Sending the second media packet, the third media packet and the second FEC packet to the second user end; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; The second user terminal is used to generate a fifth media packet according to the received fourth media packet and the third FEC packet; calculate a second packet loss rate according to the number of the fifth media packet, the number of the fourth media packet, the number of the third media packet, and the number of the second media packet, and send the second packet loss rate to the first user terminal through the media gateway; wherein the number of the fourth media packet is less than or equal to the sum of the number of the second media packet and the third media packet, and the number of the third FEC packet is less than or equal to the number of the second FEC packet; The first user terminal is used to adjust the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

Images