Media data transmission method and system against weak network

Abstract

The present invention relates to the fields of media data transmission and financial technology, and discloses a method and system for media data transmission against weak networks. In the method, a first packet loss rate of a media packet transmitted between a first user terminal and a media gateway is calculated, and a second packet loss rate of the media packet transmitted between the media gateway and a second user terminal is calculated. The first packet loss rate and the second packet loss rate are fed back to the first user terminal, so that the first user terminal dynamically and reasonably adjusts the FEC redundancy ratio according to the received packet loss rate, so that the media packet is transmitted completely from the first user terminal to the second user terminal to the greatest extent possible.

Description

Technical Field

[0001] The present invention relates to the fields of media data transmission and financial technology, and in particular to a method and system for media data transmission against weak networks. Background Art

[0002] Currently, a call between a first user terminal and a second user terminal requires the transmission of media data between the first and second user terminals. This can occur in various harsh network environments, such as elevators, restrooms with weak signals and insufficient coverage, and subway stations. These conditions often fail to meet communication requirements, leading to media data loss. Therefore, ensuring the complete transmission of media data from the first user terminal to the second user terminal in weak network conditions has become a pressing technical challenge for those skilled in the art. Summary of the Invention

[0003] The present invention provides a method and system for transmitting media data in a weak network environment, so as to solve the technical problem that media data cannot be completely transmitted from a first user terminal to a second user terminal in a weak network environment.

[0004] In a first aspect, a method for transmitting media data in a weak network is provided, comprising:

[0005] The first user terminal generates a first FEC packet from the first media packet according to the FEC redundancy ratio, and sends the first media packet and the first FEC packet to the media gateway;

[0006] The media gateway generates a third media packet based on the received second media packet and the second FEC packet; calculates a first packet loss rate based on the number of the third media packets, the number of the second media packets, and the number of the first media packets, and sends the first packet loss rate to the first user terminal; and sends the second media packet, the third media packet, and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets.

[0007] The second user terminal generates a fifth media packet based on the received fourth media packet and the third FEC packet; calculates a second packet loss rate based on the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets, and the number of the second media packets; and sends the second packet loss rate to the first user terminal via the media gateway; wherein the number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets;

[0008] The first user terminal adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0009] In a second aspect, a media data weak network resistant transmission system is provided, comprising a first user terminal, a media gateway, and a second user terminal;

[0010] The first user terminal is configured to generate a first FEC packet from the first media packet according to the FEC redundancy ratio, and send the first media packet and the first FEC packet to a media gateway;

[0011] The media gateway is configured to generate a third media packet based on the received second media packet and the second FEC packet; calculate a first packet loss rate based on the number of the third media packets, the number of the second media packets, and the number of the first media packets, and transmit the first packet loss rate to the first user terminal; and transmit the second media packet, the third media packet, and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets;

[0012] The second user terminal is configured to generate a fifth media packet based on the received fourth media packet and the third FEC packet; calculate a second packet loss rate based on the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets, and the number of the second media packets; and send the second packet loss rate to the first user terminal via the media gateway; wherein the number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets;

[0013] The first user terminal is configured to adjust the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0014] In the solution implemented by the above-mentioned method and system for transmitting media data in a weak network, the first user terminal may generate a first FEC packet based on the FEC redundancy ratio from the first media packet, and send the first media packet and the first FEC packet to the media gateway;

[0015] The media gateway generates a third media packet based on the received second media packet and the second FEC packet; calculates a first packet loss rate based on the number of the third media packets, the number of the second media packets and the number of the first media packets and sends the first packet loss rate to the first user terminal; sends the second media packet, the third media packet and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; the second user terminal generates a fifth media packet based on the received fourth media packet and the third FEC packet; calculates a second packet loss rate based on the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets and the number of the second media packets and sends the first packet loss rate to the first user terminal; The media gateway sends the fourth media packet to the first user terminal; wherein the number of the fourth media packet is less than or equal to the sum of the number of the second media packet and the third media packet, and the number of the third FEC packet is less than or equal to the number of the second FEC packet; the first user terminal adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate. In the present invention, a first packet loss rate of the media packet transmitted between the first user terminal and the media gateway is calculated, and a second packet loss rate of the media packet transmitted between the media gateway and the second user terminal is calculated, and the first packet loss rate and the second packet loss rate are fed back to the first user terminal, so that the first user terminal dynamically and reasonably adjusts the FEC redundancy ratio according to the received packet loss rate, so that the media packet is transmitted from the first user terminal to the second user terminal in its entirety to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 This is a schematic diagram of an application environment of a method for transmitting media data over a weak network in accordance with an embodiment of the present invention;

[0018] Figure 2 This is a flow chart of a method for transmitting media data over a weak network in accordance with an embodiment of the present invention;

[0019] Figure 3 This is another flowchart of the method for transmitting media data in a weak network in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0021] Currently, a call between a first user terminal and a second user terminal requires the transmission of media data between the first and second user terminals. WebRTC (Web Real-Time Communications) is a real-time communication technology that allows peer-to-peer connections between network applications or sites, enabling the transmission of video and / or audio streams, or any other arbitrary data. WebRTC-based real-time communication systems use UDP for transmission, with underlying media packets in the form of RTP packets. However, UDP is an unreliable transmission link. The WebRTC technology stack provides a comprehensive suite of components for robustness against weak network conditions, including accelerated playback, preemptive expansion, packet loss compensation (PLC), forward error correction (FEC), and retransmission technologies. These are highly specialized algorithms. Even without any optimization adjustments, they can demonstrate considerable robustness against weak network conditions. However, various harsh network environments can still be encountered, such as in elevators, restrooms with weak signals and insufficient coverage, and subway stations, where communication requirements are often unsatisfactory. Therefore, ensuring the reliability of media data transmission in weak network conditions has become a pressing technical challenge for those skilled in the art.

[0022] The method for transmitting media data in a weak network environment provided by the embodiment of the present invention can be applied in the following situations: Figure 1In an application environment, a first user terminal and a second user terminal communicate through a media gateway. The first user terminal generates a first FEC packet from the first media packet according to the FEC redundancy ratio, and sends the first media packet and the first FEC packet to the media gateway; the media gateway generates a third media packet according to the received second media packet and the second FEC packet; calculates a first packet loss rate according to the number of the third media packets, the number of the second media packets and the number of the first media packets, and sends the first packet loss rate to the first user terminal; sends the second media packet, the third media packet and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; the third media packet and the second FEC packet are calculated based on the number of the third media packets, the number of the second media packets and the number of the first media packets. The second user terminal generates a fifth media packet based on the received fourth media packet and third FEC packet; calculates a second packet loss rate based on the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets, and the number of the second media packets, and sends the result to the first user terminal via the media gateway; wherein the number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets; the first user terminal adjusts the FEC redundancy ratio based on the first and second packet loss rates. In the present invention, FEC technology is used to perform forward error correction protection on the media packets to form transmission protection. When the media gateway and the second user terminal receive the media packet and FEC packet, they calculate the corresponding packet loss rate and feed it back to the first user terminal based on the packet loss rate, so that the first user terminal can dynamically and reasonably adjust the FEC redundancy ratio based on the packet loss rate, thereby improving data transmission reliability and transmission efficiency, and ensuring that the media packet is transmitted completely from the first user terminal to the second user terminal to the greatest extent possible.

[0023] The user terminal may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The present invention will be described in detail below through specific embodiments.

[0024] See also Figure 2 As shown, Figure 2 A flowchart of a method for transmitting media data in a weak network is provided in an embodiment of the present invention, including the following steps S100-S1000.

[0025] S100: The first user terminal generates Ns first FEC packets from Ks first media packets according to an FEC redundancy ratio. Exemplarily, the first user terminal may implement this step using an FEC encoder.

[0026] In the field of Ping An Finance and Insurance, audio and video interaction is required during the dual recording of insurance contracts, and the method in the embodiment of this application can be used. In order to ensure the compliance of the insurance process, remote dual recording, service terms and personnel identity verification, etc., the entire process requires audio and video interaction.

[0027] S200: Send Ks first media packets and Ns first FEC packets to a media gateway.

[0028] In one example, taking audio Opus encoding as an example, the first media packet is an RTP data packet, 50 RTP data packets are sent per second, and each RTP data packet includes audio of 20 ms duration.

[0029] In the embodiment of the present application, the FEC redundancy ratio N refers to the number of media packets that generate one FEC packet. In one example, the redundancy ratio is calculated as follows: redundancy ratio = (number of FEC packets) / (number of FEC packets + number of media packets).

[0030] Properly setting the FEC redundancy ratio is crucial. On the one hand, a higher FEC redundancy ratio provides stronger error correction capabilities, thereby increasing data transmission reliability. In noisy or interference-prone environments, a higher FEC redundancy ratio can more effectively detect and correct transmission errors, reducing the risk of data loss. Furthermore, a higher redundancy ratio can reduce the need for retransmissions, thereby reducing delays and bandwidth consumption caused by retransmissions. On the other hand, a higher FEC redundancy ratio can also increase data transmission overhead and the amount of data transmitted, thereby consuming more bandwidth resources. In situations where bandwidth is limited, an excessively high FEC redundancy ratio can reduce transmission efficiency. Therefore, the embodiments of the present application dynamically adjust the FEC redundancy ratio to set an appropriate ratio, thereby improving data transmission reliability while also enhancing transmission efficiency.

[0031] S300: The media gateway generates M third media packets based on the received Kr second media packets and Nr second FEC packets. The number of second media packets is less than or equal to the number of first media packets, and the number of second FEC packets is less than or equal to the number of first FEC packets. Exemplarily, the media gateway may utilize an FEC decoder to complete step S300.

[0032] In a weak network environment, packet loss may occur when the first user terminal sends Ks first media packets and Ns first FEC packets to the media gateway. In this case, the media gateway may not receive all of the Ks first media packets and may only receive Kr second media packets, where Kr <= Ks. Similarly, the media gateway may only receive Nr second FEC packets, where Nr <= Ns.

[0033] In the embodiment of the present application, each media packet carries a corresponding RTPSequence sequence, which indicates the order of the media packets. The FEC packet also includes an RTPSequence sequence. The lost media packets can be determined by the RTPSequence sequences in the media packets and the FEC packets.

[0034] S400 : Calculate a first packet loss rate according to the number (M) of the third media packets, the number (Kr) of the second media packets, and the number (Nr) of the first media packets.

[0035] S500: Send a first packet loss rate to the first user terminal. In one example, the first packet loss rate = (Ks-M-Kr) ÷ Ks.

[0036] S600: The media gateway sends the second media packet, the third media packet, and the second FEC packet to the second user terminal. In this embodiment of the present application, the first user terminal sends the media packet and the FEC packet to the second user terminal via the media gateway.

[0037] S700: The second user terminal generates M2 fifth media packets based on the received (Kr2 + Mr2) fourth media packets and Nr2 third FEC packets. The number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets. Exemplarily, the second user terminal completes S700 using an FEC decoder.

[0038] In a weak network environment, packet loss may occur while the media gateway is sending Kr second media packets, M third media packets, and Nr second FEC packets to the second client. In this case, the second client may not receive all of the Kr second media packets and the M third media packets, and may only receive (Kr2 + Mr2) fourth media packets, where the Kr2 fourth media packets are partial media packets received from the Kr second media packets, where Kr2 <= Kr, and the Mr2 fourth media packets are partial media packets received from the M third media packets, where Mr2 <= M. Similarly, the second client may only receive Nr2 third FEC packets, where Nr2 <= Nr.

[0039] In the embodiment of the present application, the process of generating the fifth media package is the same as the process of generating the third media package, and will not be repeated here.

[0040] S800. Calculate a second packet loss rate based on the number of the fifth media packets (M2), the number of the fourth media packets ((Kr2 + Mr2)), the number of the third media packets (M), and the number of the second media packets (Kr). S900. Send the second packet loss rate to the first user terminal via the media gateway. Exemplarily, the second packet loss rate = (M + Kr - Kr2 - Mr2 - M2) ÷ (M + Kr).

[0041] S1000. The first user terminal adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0042] In this embodiment of the present application, the packet loss rate can reflect information such as the current network status. Based on the first packet loss rate between the first client and the media gateway, and the second packet loss rate between the media gateway and the second client, the FEC redundancy ratio is appropriately adjusted to improve data transmission reliability and efficiency. When the first client subsequently sends media packets, it uses the adjusted FEC redundancy ratio to generate FEC packets.

[0043] In some embodiments, the step of adjusting the FEC redundancy ratio by the first user terminal according to the first packet loss rate and the second packet loss rate includes:

[0044] The first user terminal selects the maximum packet loss rate from the first packet loss rate and the second packet loss rate received within the preset period according to a preset period.

[0045] In this embodiment of the present application, because data transmission between the first user terminal, the media gateway, and the second user terminal takes time, the preset period must be set to ensure that the first user terminal receives at least one first packet loss rate and one second packet loss rate. For example, the preset period may be every 1 second, and the first user terminal selects the maximum packet loss rate from the first and second packet loss rates received within that 1 second. The above operation is performed for each of the first and second packet loss rates received within that 1 second.

[0046] An FEC redundancy ratio is determined according to the maximum packet loss rate.

[0047] In one example, if the maximum packet loss rate selected from the received first packet loss rate and the second packet loss rate is 20%, the FEC redundancy ratio is determined to be 2. If the maximum packet loss rate is 40%, the FEC redundancy ratio is determined to be 4. If the maximum packet loss rate is greater than 40%, the FEC redundancy ratio is determined to be 10.

[0048] In some embodiments, in order to enable the media gateway to receive all the contents of the Ks first media packets and Ns first FEC packets sent by the first user terminal, the method further includes:

[0049] The media gateway generates a first packet replenishment request based on the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal. Exemplarily, the media gateway may generate the first packet replenishment request through an uplink retransmission packet generator.

[0050] In an embodiment of the present application, the second media packet and the third media packet are saved as a circular list RList(Kr+M) according to the RTPSequence sequence, and the lost media packet can be determined through the circular list. A first packet replenishment request can be generated based on the lost media packet. For example, the media gateway should receive media packets of RTPSequence sequence 1-5, but the current first media packet is a media packet of RTPSequence sequence 1-2, and the third media packet is a media packet of RTPSequence sequence 3, then the media packets of RTPSequence sequence 4 and 5 are lost. At this time, a first packet replenishment request is generated based on the media packets of RTPSequence sequence 4 and 5.

[0051] The first user terminal sends a first retransmission packet Rr to the media gateway according to the first packet replenishment request.

[0052] In the embodiment of the present application, the first user terminal receives the first packet replenishment request, determines the first retransmission packet Rr corresponding to the first packet replenishment request, and sends the first retransmission packet to the media gateway. It should be noted that the actual data in the first retransmission packet in the embodiment of the present application is still a media packet. The distinction is made in name only to distinguish that the retransmission packet is obtained using the packet replenishment request.

[0053] In some embodiments, when the method includes the step of the media gateway generating a first packet replenishment request based on the second media packet and the third media packet, and sending the first packet replenishment request to the first user terminal, the step of the media gateway sending the second media packet, the third media packet, and the second FEC packet to the second user terminal includes:

[0054] The media gateway sends the Rr2 second retransmission packets, Kr second media packets, M third media packets and Nr second FEC packets to the second user end, and the number of the fourth media packets is less than or equal to the sum of the number of the second retransmission packets, the second media packets and the third media packets.

[0055] In the embodiment of the present application, in addition to the second media packet, the third media packet, and the second FEC packet, the second retransmission packet obtained according to the first packet replenishment request also needs to be transmitted from the media gateway to the second user terminal. Since the fourth media packet is a media packet sent from the media gateway to the second user terminal, during the transmission process from the media gateway to the second user terminal, the media packets in the Rr2 second retransmission packets, Kr second media packets, and M third media packets may be lost. Therefore, the number of Kr2+Mr2+Rr3 fourth media packets composed of the second media packet, the third media packet, and the second retransmission packet transmitted to the data packet is less than or equal to the total number of Rr2 second retransmission packets, Kr second media packets, and M third media packets, where Kr2≤Kr, Mr2≤M, and Rr3≤Rr2.

[0056] In some embodiments, Figure 3 This is another flowchart of a method for transmitting media data over a weak network in accordance with an embodiment of the present invention. The first packet replenishment request includes a missing packet replenishment request, and the second retransmission packet includes a third retransmission packet. The steps of the media gateway generating a first packet replenishment request based on the second and third media packets and sending the first packet replenishment request to the first user terminal include:

[0057] S301. The media gateway determines the lost media packet based on Kr second media packet and M third media packet.

[0058] S302: Generate a missing data packet list according to the lost media packet, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal, and start timing using a clock driver.

[0059] Exemplarily, the lost media packets may be media packets of RTPSequence sequences 4 and 5. A missing data packet list is generated based on the media packets of RTPSequence sequences 4 and 5, and a missing packet replenishment request corresponding to the missing data packet list is sent to the first user terminal.

[0060] S303: Determine a first preset packet replenishment interval and a first number of transmission times according to the round-trip time required for the media packet to travel between the media gateway and the first user terminal.

[0061] In an embodiment of the present application, in order to avoid the situation where a missing packet replenishment request cannot be successfully sent to the first user terminal after being sent only once, or the third retransmission packet obtained by the first user terminal according to the missing packet replenishment request cannot be successfully transmitted to the media gateway, and thus the lost media packet cannot be supplemented by the third retransmission packet, the missing packet replenishment request is sent a first number of times according to a first preset packet replenishment interval, that is, the missing packet replenishment request is sent multiple times.

[0062] Exemplarily, the round-trip time between the media gateway and the first user terminal is 200ms, the first preset packet replenishment interval can be 40ms, and the first number of transmissions can be obtained by dividing the required time by the first preset packet replenishment interval, so the first number of transmissions is 5 times.

[0063] S304: When the first timing time of the clock driver reaches the first preset packet replenishment interval, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal until the number of times the missing packet replenishment request is sent reaches a first sending number.

[0064] In this way, by sending the missing packet replenishment request multiple times, the first user terminal can send the corresponding first retransmitted packet multiple times according to the missing packet replenishment request, thereby preventing the missing packet replenishment request from being lost or the first retransmitted packet corresponding to a single missing packet replenishment request from being successfully sent from the first user terminal to the media gateway. For example, when the first timing time of the clock driver reaches 40ms, the missing packet replenishment request is sent to the first user terminal once, up to five times.

[0065] In the embodiment of the present application, when the media gateway finds that there is a lost media packet, it immediately generates a corresponding missing packet replenishment request based on the lost media packet. In addition, a clock drive is added to continuously send the missing packet replenishment request to the first user terminal.

[0066] The method further includes: the media gateway receiving a third retransmission packet sent by the first user terminal according to the missing packet replenishment request.

[0067] In an embodiment of the present application, after the first user terminal receives a missing packet replenishment request, it determines a first retransmission packet corresponding to the missing packet replenishment request and sends the first retransmission packet to the media gateway. In a weak network environment, the first retransmission packet may not be fully transmitted to the media gateway, and the media gateway may receive part or all of the first retransmission packet, that is, the third retransmission packet.

[0068] In some embodiments, the first packet replenishment request further includes a first estimated packet loss replenishment request, and the second retransmission packet further includes a fourth retransmission packet; and the step of the media gateway generating the first packet replenishment request based on the second media packet and the third media packet, and sending the first packet replenishment request to the first user terminal further includes:

[0069] An estimated lost data packet is determined according to the third retransmitted packet and the missing data packet list.

[0070] In an embodiment of the present application, the missing data packet list includes missing media packets, and the third retransmission packet is a retransmission packet sent to the media gateway by the first user terminal based on the missing packet replenishment request. However, due to a weak network condition, the third retransmission packet may only be part of the content of the missing media packet. Therefore, based on the third retransmission packet and the missing data packet list, an estimated lost data packet is determined, and the estimated lost data packet includes the media packets that have not yet been received in the missing data packet list.

[0071] A first estimated lost packet replenishment request is generated according to the preset lost data packet, and the first estimated lost packet replenishment request is sent to the first user terminal according to a second preset replenishment interval until the second timing time of the clock driver reaches the preset time.

[0072] Exemplarily, the missing data packet list includes packet number 5 and packet number 7, and the third retransmitted packet includes packet number 5. In this case, the estimated lost data packet is packet number 7. A first estimated lost packet replenishment request is generated based on packet number 7, and the first estimated lost packet replenishment request is continuously sent to the first user terminal at a second preset replenishment interval until the second timing of the clock driver reaches a preset time. Exemplarily, the preset time is 2 seconds, and the second preset replenishment time event is 200 milliseconds.

[0073] In an embodiment of the present application, the step of generating a first estimated packet loss replenishment request based on a preset lost data packet may include generating an estimated packet loss data packet list based on the preset lost data packet, and generating the first estimated packet loss replenishment request based on the estimated packet loss data packet list.

[0074] In the embodiment of the present application, a packet loss prediction strategy is added to generate a first estimated packet loss replenishment request, thereby increasing the probability that other media packets except the third retransmission packet in the missing data packet list are successfully sent to the media gateway.

[0075] In some embodiments, the first packet replenishment request further includes a second estimated packet loss replenishment request, the second retransmission packet further includes a fifth retransmission packet, and the method further includes:

[0076] The media gateway determines, at a preset detection interval, whether a sixth media packet sent by the first user terminal is received.

[0077] In an embodiment of the present application, if the first user terminal continuously generates media packets, the first user terminal should continuously send media packets to the media gateway. For example, the round-trip time between the media gateway and the first user terminal is 200ms, and a media packet is sent every 20ms. It can be determined whether the media gateway has received 5 media packets within half of the round-trip time, that is, within 100ms. However, if the sixth media packet sent by the first user terminal is still not received within the preset detection time, for example within 100ms, for example, the sixth media packet that should be received includes media packets of RTPSequence sequence 6-10, it is possible that the sixth media packet was lost during the transmission process. In this case, a second preset packet loss compensation request is generated based on the sixth media packet.

[0078] If the sixth media packet is not received, a second estimated packet loss replenishment request corresponding to the sixth media packet is generated and sent to the first user terminal.

[0079] The method further includes: the media gateway receiving a fifth retransmission packet sent by the first user terminal according to the second estimated packet loss compensation request.

[0080] In this embodiment of the present application, after receiving the second estimated packet loss compensation request, the first user terminal determines a corresponding first retransmission packet based on the second estimated packet loss compensation request and sends the first retransmission packet corresponding to the second estimated packet loss compensation request to the media gateway. Because the first retransmission packet may be lost during transmission, the media gateway may only receive a fifth retransmission packet. This fifth retransmission packet may be part or all of the first retransmission packet corresponding to the second estimated packet loss compensation request.

[0081] In the embodiment of the present application, the media gateway helps to retransmit and recover known lost media packets through missing packet replenishment requests, first estimated lost packet replenishment requests and second estimated lost packet replenishment requests, and predicts possible lost media packets that have not yet been reached to trigger retransmission and recovery in advance, thereby improving the real-time performance and data integrity of the data and improving the user experience.

[0082] In some embodiments, before the media gateway sends the second retransmission packet, the second media packet, the third media packet and the second FEC packet to the second user end, it also includes: deleting duplicate data packets in the second retransmission packet to update the second retransmission packet.

[0083] Since the second retransmitted packet may include multiple identical media packets obtained by the first packet replenishment request, the duplicate data packets in the second retransmitted packet are deleted.

[0084] In some embodiments, the method further comprises:

[0085] The second user terminal generates a second packet replenishment request according to the fourth media packet and the fifth media packet, and sends the second packet replenishment request to the media gateway.

[0086] In the embodiment of the present application, the process of the second user terminal generating the second packet replenishment request based on the fourth media packet and the fifth media packet is the same as the process of the media gateway generating the first packet replenishment request based on the second media packet and the third media packet in the above text, and will not be repeated here.

[0087] The media gateway sends the sixth retransmission packet to the second user terminal according to the second packet replenishment request, so that the second user terminal receives the seventh retransmission packet.

[0088] In the embodiment of the present application, when the media gateway receives the second packet replenishment request, it passively sends the sixth retransmission packet to the second user terminal.

[0089] The media gateway delays sending the sixth retransmission packet with a preset number of redundancy times to the second user terminal based on the current FEC redundancy multiple and the number of retransmission packets corresponding to the second packet replenishment request, so that the second user terminal receives the eighth retransmission packet, and the total number of the seventh retransmission packet and the eighth retransmission packet is less than or equal to the total number of the sixth retransmission packet sent by the media gateway.

[0090] In this embodiment of the present application, after sending the first sixth retransmission packet, the media gateway also proactively sends a preset number of sixth retransmission packets to the second client. Due to the possibility of a weak network connection, the media gateway may risk failing to successfully reach the destination by sending the sixth retransmission packet only once. Therefore, in this embodiment of the present application, the current network conditions are determined by using the current FEC redundancy factor and the retransmission packet data corresponding to the second packet replenishment request. The delay in sending the sixth retransmission packet multiple times to the second client increases the probability that the second client will receive the entire content of the sixth retransmission packet.

[0091] In an embodiment of the present application, a scheme for increasing the delay to send the sixth retransmission packet by a preset number of redundant times can shorten the network time for multiple requests for packet replenishment in a weak network condition, effectively reduce the delay in the call and increase data integrity.

[0092] In one example, if the FEC redundancy factor is greater than 4, the number of retransmitted packets corresponding to the second supplemental packet request is less than 10, and the number of consecutive retransmitted packets is less than 6, a 3x delay is triggered, meaning the default redundancy count is 3, and the sixth retransmitted packet is sent to the second user end four times. Furthermore, this example also includes a data storm prevention strategy. If the number of retransmitted packets corresponding to the second supplemental packet request is greater than 15, a 1x delay is triggered, meaning the default redundancy count is 1, and the sixth retransmitted packet is sent to the second user end two times. This is because if there are too many retransmitted packets, multiple delayed transmissions could consume excessive network resources.

[0093] In another example, if the FEC redundancy factor is greater than or equal to 2, the number of retransmitted packets corresponding to the second supplemental packet request is less than 12, and the number of consecutive retransmitted packets is less than 8, a 2x delay is triggered, i.e., the preset redundancy count is 2, and a total of 3 retransmitted packets of the sixth retransmitted packet are sent to the second user end. Furthermore, in this example, a data storm prevention strategy is also provided. If all other conditions remain unchanged, the number of retransmitted packets corresponding to the second supplemental packet request is greater than 20, a 1x delay is triggered, i.e., the preset redundancy count is 1, and a total of 2 retransmitted packets of the sixth retransmitted packet are sent to the second user end.

[0094] In another example, if the number of retransmitted packets corresponding to the second packet replenishment request is greater than 25, it will trigger a 1x delayed transmission, that is, the preset redundancy number is 1, and a total of 2 sixth retransmitted packets are sent to the second user end.

[0095] In a laboratory environment, when a first user terminal and a second user terminal conduct a one-to-one call, in a scenario where the network randomly loses 60% of packets, the call was difficult to make before the improvement, but the method in the embodiment of the present application allows a clear and normal call.

[0096] It can be seen that in the above scheme, the first packet loss rate of the media packet transmitted between the first user end and the media gateway is calculated, and the second packet loss rate of the media packet transmitted between the media gateway and the second user end is calculated, and the first packet loss rate and the second packet loss rate are fed back to the first user end, so that the first user end can dynamically and reasonably adjust the FEC redundancy ratio according to the received packet loss rate, so that the media packet can be transmitted completely from the first user end to the second user end to the greatest extent.

[0097] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0098] In one embodiment, a media data anti-weak network transmission system is provided, which corresponds one-to-one to the media data anti-weak network transmission method in the above embodiment. The system includes a first user terminal, a media gateway, and a second user terminal;

[0099] The first user terminal is configured to generate a first FEC packet from the first media packet according to the FEC redundancy ratio, and send the first media packet and the first FEC packet to a media gateway;

[0100] The media gateway is configured to generate a third media packet based on the received second media packet and the second FEC packet; calculate a first packet loss rate based on the number of the third media packets, the number of the second media packets, and the number of the first media packets, and transmit the first packet loss rate to the first user terminal; and transmit the second media packet, the third media packet, and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets;

[0101] The second user terminal is configured to generate a fifth media packet based on the received fourth media packet and the third FEC packet; calculate a second packet loss rate based on the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets, and the number of the second media packets; and send the second packet loss rate to the first user terminal via the media gateway; wherein the number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets;

[0102] The first user terminal is configured to adjust the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

[0103] In some embodiments, the first user terminal is specifically used to filter out the maximum packet loss rate from the first packet loss rate and the second packet loss rate received within the preset period according to a preset period; and determine the FEC redundancy ratio based on the maximum packet loss rate.

[0104] In some embodiments, the media gateway is further configured to generate a first packet replenishment request based on the second media packet and the third media packet, and send the first packet replenishment request to the first user terminal;

[0105] The first user terminal is further configured to send a first retransmission packet to the media gateway according to the first packet replenishment request.

[0106] In some embodiments, the media gateway is specifically used to send the received second retransmission packet, second media packet, third media packet and second FEC packet to the second user end, and the number of the fourth media packet is less than or equal to the sum of the number of the second retransmission packet, the second media packet and the third media packet, and the number of the second retransmission packet is less than or equal to the first retransmission packet.

[0107] In some embodiments, the first packet replenishment request includes a missing packet replenishment request, and the second retransmission packet includes a third retransmission packet; and the media gateway is specifically configured to:

[0108] The media gateway determines the lost media packet based on the second media packet and the third media packet;

[0109] Generate a missing data packet list according to the lost media packet, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal, and start timing using a clock driver;

[0110] Determine a first preset packet replenishment interval and a first number of transmission times according to a round-trip time required for the media packet to travel between the media gateway and the first user terminal;

[0111] When the first timing time of the clock driver reaches the first preset packet replenishment interval, a missing packet replenishment request corresponding to the missing data packet list is sent to the first user terminal once until the number of times the missing packet replenishment request is sent reaches a first sending number.

[0112] The media gateway is further configured to receive a third retransmission packet sent by the first user terminal according to the missing packet replenishment request.

[0113] In some embodiments, the first packet replenishment request further includes a first estimated lost packet replenishment request, and the second retransmission packet further includes a fourth retransmission packet; the media gateway is specifically configured to determine the estimated lost data packet based on the third retransmission packet and the missing data packet list;

[0114] Generate a first estimated lost packet replenishment request according to the preset lost data packet, and send the first estimated lost packet replenishment request to the first user terminal according to a second preset packet replenishment interval until the second timing time of the clock driver reaches a preset time;

[0115] The media gateway is further configured to receive a fourth retransmitted packet sent by the first user terminal according to the first estimated packet loss compensation request.

[0116] In some embodiments, the first packet replenishment request further includes a second estimated packet loss replenishment request, the second retransmission packet further includes a fifth retransmission packet, and the media gateway is further configured to determine whether a sixth media packet sent by the first user terminal is received at a preset detection time interval;

[0117] If the sixth media packet is not received, generating a second estimated packet loss replenishment request corresponding to the sixth media packet and sending the request to the first user terminal;

[0118] The media gateway is further configured to receive a fifth retransmitted packet sent by the first user terminal according to the second estimated packet loss compensation request.

[0119] In some embodiments, the media gateway is further configured to delete duplicate data packets in the second retransmission packet to update the second retransmission packet.

[0120] In some embodiments, the second user terminal is further configured to generate a second packet replenishment request based on the fourth media packet and the fifth media packet, and send the second packet replenishment request to the media gateway;

[0121] The media gateway sends the sixth retransmitted packet to the second user terminal according to the second packet replenishment request, so that the second user terminal receives the seventh retransmitted packet;

[0122] The media gateway delays sending the sixth retransmission packet with a preset number of redundancy times to the second user terminal based on the current FEC redundancy multiple and the number of retransmission packets corresponding to the second packet replenishment request, so that the second user terminal receives the eighth retransmission packet, and the total number of the seventh retransmission packet and the eighth retransmission packet is less than or equal to the total number of the sixth retransmission packet sent by the media gateway.

[0123] The present invention provides a media data weak network resistant transmission system, in which a first packet loss rate of a media packet transmitted between a first user terminal and a media gateway is calculated, and a second packet loss rate of a media packet transmitted between the media gateway and a second user terminal is calculated. The first packet loss rate and the second packet loss rate are fed back to the first user terminal, so that the first user terminal dynamically and reasonably adjusts the FEC redundancy ratio according to the received packet loss rate, so that the media packet is transmitted completely from the first user terminal to the second user terminal to the greatest extent possible.

[0124] For the specific definition of the media data anti-weak network transmission system, please refer to the definition of the media data anti-weak network transmission method above, which will not be repeated here. The various modules in the above-mentioned media data anti-weak network transmission system can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0125] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0126] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0127] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for transmitting media data in a weak network, characterized in that: include: The first user terminal generates a first FEC packet from the first media packet according to the FEC redundancy ratio, and sends the first media packet and the first FEC packet to the media gateway; The media gateway generates a third media packet based on the received second media packet and the second FEC packet, wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; calculates a first packet loss rate based on the number of the third media packets, the number of the second media packets, and the number of the first media packets, and sends the first packet loss rate to the first user terminal; and sends the second media packet, the third media packet, and the second FEC packet to the second user terminal. The second user terminal generates a fifth media packet based on the received fourth media packet and the third FEC packet, wherein the number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets; calculates a second packet loss rate based on the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets, and the number of the second media packets, and sends the second packet loss rate to the first user terminal through the media gateway; The first user terminal adjusts the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

2. The method according to claim 1, characterized in that The step of adjusting the FEC redundancy ratio by the first user end according to the first packet loss rate and the second packet loss rate includes: The first user terminal selects a maximum packet loss rate from the first packet loss rate and the second packet loss rate received within the preset period according to a preset period; and determines an FEC redundancy ratio according to the maximum packet loss rate.

3. The method according to claim 1, characterized in that Also includes: The media gateway generates a first packet replenishment request according to the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal; The first user terminal sends a first retransmission packet to the media gateway according to the first packet replenishment request.

4. The method according to claim 3, characterized in that The step of the media gateway sending the second media packet, the third media packet and the second FEC packet to the second user terminal includes: The media gateway sends the received second retransmission packet, second media packet, third media packet and second FEC packet to the second user end, and the number of the fourth media packet is less than or equal to the sum of the number of the second retransmission packet, the second media packet and the third media packet, and the number of the second retransmission packet is less than or equal to the first retransmission packet.

5. The method according to claim 4, characterized in that The first packet replenishment request includes a missing packet replenishment request, and the second retransmission packet includes a third retransmission packet; and the media gateway generates a first packet replenishment request based on the second media packet and the third media packet, and sends the first packet replenishment request to the first user terminal, including: The media gateway determines the lost media packet based on the second media packet and the third media packet; Generate a missing data packet list according to the lost media packet, send a missing packet replenishment request corresponding to the missing data packet list to the first user terminal, and start timing using a clock driver; Determine a first preset packet replenishment interval and a first number of transmission times according to a round-trip time required for the media packet to travel between the media gateway and the first user terminal; When the first timing time of the clock driver reaches the first preset packet replenishment interval, sending a missing packet replenishment request corresponding to the missing data packet list to the first user terminal until the number of times the missing packet replenishment request is sent reaches a first sending number; The method further includes: the media gateway receiving a third retransmission packet sent by the first user terminal according to the missing packet replenishment request.

6. The method according to claim 5, characterized in that The first packet replenishment request further includes a first estimated packet loss replenishment request, and the second retransmission packet further includes a fourth retransmission packet; and the step of the media gateway generating a first packet replenishment request based on the second media packet and the third media packet, and sending the first packet replenishment request to the first user terminal further includes: Determining an estimated lost data packet based on the third retransmitted packet and the missing data packet list; generating a first estimated lost packet replenishment request according to the estimated lost data packet, and sending the first estimated lost packet replenishment request to the first user terminal according to a second preset packet replenishment interval until the second timing time of the clock driver reaches a preset time; The method further includes: the media gateway receiving a fourth retransmission packet sent by the first user terminal according to the first estimated packet loss compensation request.

7. The method according to claim 5, characterized in that The first packet replenishment request further includes a second estimated packet loss replenishment request, the second retransmission packet further includes a fifth retransmission packet, and the method further includes: The media gateway determines, at a preset detection interval, whether a sixth media packet sent by the first user terminal is received; If the sixth media packet is not received, generating a second estimated packet loss replenishment request corresponding to the sixth media packet and sending the request to the first user terminal; The method further includes: the media gateway receiving a fifth retransmission packet sent by the first user terminal according to the second estimated packet loss compensation request.

8. The method according to claim 6, characterized in that Before the media gateway sends the second retransmission packet, the second media packet, the third media packet and the second FEC packet to the second user terminal, the method further includes: deleting duplicate data packets in the second retransmission packet to update the second retransmission packet.

9. The method according to claim 4, characterized in that Also includes: The second user terminal generates a second packet replenishment request according to the fourth media packet and the fifth media packet, and sends the second packet replenishment request to the media gateway; The media gateway sends the sixth retransmitted packet to the second user terminal according to the second packet replenishment request, so that the second user terminal receives the seventh retransmitted packet; The media gateway delays sending the sixth retransmission packet with a preset number of redundancy times to the second user terminal based on the current FEC redundancy multiple and the number of retransmission packets corresponding to the second packet replenishment request, so that the second user terminal receives the eighth retransmission packet, and the total number of the seventh retransmission packet and the eighth retransmission packet is less than or equal to the total number of the sixth retransmission packet sent by the media gateway.

10. A media data transmission system against weak network, characterized in that: comprising a first user terminal, a media gateway and a second user terminal; The first user terminal is configured to generate a first FEC packet from the first media packet according to the FEC redundancy ratio, and send the first media packet and the first FEC packet to a media gateway; The media gateway is configured to generate a third media packet based on the received second media packet and the second FEC packet; calculate a first packet loss rate based on the number of the third media packets, the number of the second media packets, and the number of the first media packets, and send the first packet loss rate to the first user terminal; Sending the second media packet, the third media packet, and the second FEC packet to the second user terminal; wherein the number of the second media packets is less than or equal to the number of the first media packets, and the number of the second FEC packets is less than or equal to the number of the first FEC packets; The second user terminal is configured to generate a fifth media packet based on the received fourth media packet and the third FEC packet; calculate a second packet loss rate based on the number of the fifth media packets, the number of the fourth media packets, the number of the third media packets, and the number of the second media packets; and send the second packet loss rate to the first user terminal via the media gateway; wherein the number of the fourth media packets is less than or equal to the sum of the number of the second media packets and the third media packets, and the number of the third FEC packets is less than or equal to the number of the second FEC packets; The first user terminal is configured to adjust the FEC redundancy ratio according to the first packet loss rate and the second packet loss rate.

Images