Data jitter buffering method and device

Abstract

The invention provides a data jitter buffering method and device, and the method comprises the steps: receiving a to-be-transmitted data package, obtaining a current data package, judging whether the current data package is a current expected data package or not, and when the current data package is not the current expected data package, carrying out the jitter buffering of the current data package. If the current expected data packet is not received, sending the current data packet to a jitter buffer queue for sorting, caching and waiting for a preset buffer duration, and when the current expected data packet is not received within the preset buffer duration, outputting the sorted continuous data packet at the queue head in the jitter buffer queue; and when the current data packet is the current expected data packet, the current data packet and the data packet which is located at the queue head in the jitter buffer queue and is continuous with the current data packet do not need to be cached and waited in the jitter buffer queue. According to the method, the problems of data loss and delayed accumulation caused by dynamic adjustment of the size of the buffer area in the prior art can be solved through a buffer strategy of double buffer durations.

Description

Technical Field

[0001] The present invention relates to the technical field of audio - video buffering and playing, and particularly to a data jitter buffering method and device. Background Art

[0002] With the continuous development of audio - video communication technology, applications such as real - time voice, video calls, and streaming media playback have become an important part of people's daily lives. In these applications, the stability and smoothness of network transmission play a crucial role in the user experience.

[0003] However, in the actual network environment, the arrival time of data packets is not always uniform. Especially in weak networks and congestion situations, network latency and jitter significantly affect the transmission quality of audio - video data. Currently, through Jitter Buffer technology, users can still obtain a better audio - video experience when the network conditions are poor. Existing Jitter Buffer technologies often rely on a fixed buffering duration or rely on RTT (Round Trip Time, that is, the total delay of data packets from the sender to the receiver and back) and Jitter values to adjust the buffering duration. However, the fixed buffering duration has the problem that the delay will not decrease when the network state improves; in the dynamic adjustment of the buffering duration scheme, in a weak network environment, such as in a mobile network or when the Wi - Fi signal is unstable, due to more serious network delay and packet loss, the jitter buffer usually needs to be dynamically adjusted frequently. The buffer capacity changes with the buffering duration, and reducing the buffer may cause data loss in the buffer, and the playback picture will be mosaic; increasing the buffer may cause problems such as a large delay.

[0004] In summary, the buffering strategies of the existing technologies have poor adaptability to complex network scenarios such as screen mirroring playback, which may lead to problems such as playback stuttering, mosaic, noise, and high delay, affecting the user's viewing experience.

[0005] Therefore, the existing technologies have defects and need to be improved and developed. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a data jitter buffering method and device in view of the above - mentioned defects of the existing technologies, aiming to solve the problems of data loss and delay accumulation caused by the dynamic adjustment of the buffer size in the existing technologies.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] A data jitter buffering method, wherein the method includes:

[0009] Receive the data packet to be transmitted to obtain the current data packet, and determine whether the current data packet is the currently expected data packet;

[0010] When the current data packet is not the currently expected data packet, send the current data packet to the jitter buffer queue for sorting, cache and wait for the preset buffer duration, and when the currently expected data packet is not received within the preset buffer duration, output the consecutive data packets at the head of the sorted jitter buffer queue;

[0011] When the current data packet is the currently expected data packet, there is no need to cache and wait in the jitter buffer queue, and directly output the current data packet and the data packets consecutive to the current data packet at the head of the jitter buffer queue.

[0012] In one implementation, the receiving the data packet to be transmitted to obtain the current data packet includes:

[0013] Receive the RTP data packet of the data channel to obtain the current data packet, and record the reception timestamp corresponding to the current data packet;

[0014] Parse the current data packet to determine the transmission timestamp, payload data, and current data packet sequence number of the current data packet; wherein, the sending the current data packet to the jitter buffer queue includes

[0015] Send the transmission timestamp, the reception timestamp, the payload data, and the current data packet sequence number of the current data packet to the jitter buffer queue.

[0016] In one implementation, the determining whether the current data packet is the currently expected data packet includes:

[0017] Determine the sequence number of the previous output data packet, and determine the currently expected data packet sequence number based on the sequence number of the previous output data packet;

[0018] Judge whether the current data packet sequence number is the same as the currently expected data packet sequence number;

[0019] When the current data packet sequence number is the same as the currently expected data packet sequence number, determine that the current data packet is the currently expected data packet;

[0020] When the current data packet sequence number is not the same as the currently expected data packet sequence number, determine that the current data packet is not the currently expected data packet.

[0021] In one implementation, before sending the current data packet to the jitter buffer queue for sorting and caching and waiting for the preset buffer duration, it further includes:

[0022] Judge whether the current data packet is the first data packet;

[0023] When the current data packet is not the first data packet, it is determined whether the current data packet is an expired data packet;

[0024] When the current data packet is the expired data packet, the expired data packet is discarded, and the step of receiving data to obtain the current data packet is executed again;

[0025] When the current data packet is not the expired data packet, it is determined whether the current data packet is a duplicate data packet;

[0026] When the current data packet is the duplicate data packet, the duplicate data packet is discarded, and the step of receiving data to obtain the current data packet is executed again;

[0027] Among them, sending the current data packet to the jitter buffer queue for sorting includes:

[0028] When the current data packet is not the duplicate data packet, the current data packet is sent to the jitter buffer queue for sorting to obtain the sorted jitter buffer queue.

[0029] In one implementation, determining whether the current data packet is a duplicate data packet includes:

[0030] Determine the sequence numbers of the data packets in the jitter buffer queue, and determine whether the current data packet sequence number is the same as each of the data packet sequence numbers;

[0031] When the current data packet sequence number is the same as any of the data packet sequence numbers, it is determined that the current data packet is a duplicate data packet;

[0032] When the current data packet sequence number is not the same as each of the data packet sequence numbers, it is determined that the current data packet is not a duplicate data packet.

[0033] In one implementation, determining whether the current data packet is an expired data packet includes:

[0034] Determine the transmission timestamp of the previous output data packet, and calculate the difference between the transmission timestamp of the current data packet and the transmission timestamp of the previous output data packet;

[0035] Determine whether the difference is less than the preset timestamp rollover threshold;

[0036] When the difference is less than the preset timestamp rollover threshold, it is determined whether the transmission timestamp of the current data packet is less than or equal to the transmission timestamp of the previous output data packet;

[0037] When the transmission timestamp of the current data packet is less than or equal to the transmission timestamp of the previous output data packet, it is determined that the current data packet is an expired data packet;

[0038] When the transmission timestamp of the current data packet is greater than the transmission timestamp of the previous output data packet, it is determined that the current data packet is a non-expired data packet;

[0039] Or, determine the sequence number of the previous output data packet, determine the current expected data packet sequence number based on the sequence number of the previous output data packet, and calculate the packet interval between the current data packet sequence number and the current expected data packet sequence number;

[0040] Judge whether the packet interval is less than a preset packet sequence number flip threshold;

[0041] When the packet interval is less than the preset packet sequence number flip threshold, judge whether the current data packet sequence number is less than the current expected data packet sequence number;

[0042] When the current data packet sequence number is less than the current expected data packet sequence number, it is determined that the current data packet is an expired data packet;

[0043] When the current data packet sequence number is not less than the current expected data packet sequence number, it is determined that the current data packet is a non-expired data packet.

[0044] In one implementation, the step of sending the current data packet to the jitter buffer queue for sorting to obtain the sorted jitter buffer queue includes:

[0045] Determine the sequence number of the first data packet at the head of the jitter buffer queue, and use the sequence number of the first data packet at the head as the current sorting reference number;

[0046] Judge whether the current data packet sequence number is less than the current sorting reference number;

[0047] When the current data packet sequence number is not less than the current sorting reference number, determine the sequence number of the next data packet after the current sorting reference number, use the next data packet sequence number as the current sorting reference number, and re-execute the step of judging whether the current data packet sequence number is less than the current sorting reference number;

[0048] When the current data packet sequence number is less than the current sorting reference number, insert the current data packet in front of the data packet corresponding to the current sorting reference number to obtain the sorted jitter buffer queue.

[0049] In one implementation, during the process of sending the current data packet to the jitter buffer queue for sorting and caching for a preset buffer duration, it further includes:

[0050] Determine the sequence number of the data packet to be retransmitted based on the current expected data packet sequence number and the sequence numbers of the data packets in the sorted jitter buffer queue, request retransmission of the data packet corresponding to the sequence number of the data packet to be retransmitted, and continue to execute the step of receiving the data packet to be transmitted to obtain the current data packet.

[0051] In one implementation, during the process of sending the current data packet to the jitter buffer queue for sorting and caching while waiting for a preset buffer duration, the following steps are further included:

[0052] Determine whether the cache waiting duration of the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue exceeds the preset buffer duration;

[0053] When the cache waiting duration of the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue does not exceed the preset buffer duration, the current buffer capacity of the jitter buffer queue is determined in real time, and it is judged whether the current buffer capacity is not less than the preset buffer capacity;

[0054] When the current buffer capacity is not less than the preset buffer capacity, directly output the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue;

[0055] When the current buffer capacity is less than the preset buffer capacity, calculate the packet interval between the current expected data packet and the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue, and judge whether the packet interval is less than the preset packet skipping threshold;

[0056] When the packet interval is not less than the preset packet skipping threshold, output the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue.

[0057] The present invention also discloses a data jitter buffering device, wherein the device includes:

[0058] A data receiving module, configured to receive a data packet to be transmitted to obtain the current data packet, and judge whether the current data packet is the current expected data packet;

[0059] A first buffer processing module, configured to, when the current data packet is not the current expected data packet, send the current data packet to the jitter buffer queue for sorting and caching while waiting for a preset buffer duration, and output the consecutive data packets at the head of the queue in the sorted jitter buffer queue when the current expected data packet is not received within the preset buffer duration;

[0060] A second buffer processing module, configured to, when the current data packet is the current expected data packet, directly output the current data packet and the consecutive data packets of the current data packet at the head of the queue in the jitter buffer queue without caching and waiting in the jitter buffer queue.

[0061] The present invention also discloses a terminal, which includes: a memory, a processor, and a data jitter buffer program stored on the memory and executable on the processor. When the data jitter buffer program is executed by the processor, it implements the steps of the data jitter buffer method as described above.

[0062] The present invention also discloses a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the data jitter buffer method as described above.

[0063] A data jitter buffer method and device provided by the present invention. The data jitter buffer method includes: receiving a data packet to be transmitted to obtain a current data packet, and determining whether the current data packet is the currently expected data packet; when the current data packet is not the currently expected data packet, sending the current data packet to the jitter buffer queue for sorting, caching and waiting for a preset buffer duration, and when the currently expected data packet is not received within the preset buffer duration, outputting consecutive data packets at the head of the sorted jitter buffer queue; when the current data packet is the currently expected data packet, there is no need to cache and wait in the jitter buffer queue, and directly output the current data packet and consecutive data packets at the head of the jitter buffer queue that are consecutive to the current data packet. It can be seen that the present invention realizes immediate output when the currently expected data packet is received through a buffer strategy with a double buffer duration, and the buffer duration is 0. When the currently expected data packet is not received, it caches and waits in the jitter buffer queue for the preset buffer duration, and when the currently expected data packet is still not received within the preset buffer duration, outputs consecutive data packets at the head of the sorted jitter buffer queue. That is, through the data jitter buffer scheme based on the double buffer duration of the present application, it can solve the problems of data loss and delay accumulation caused by dynamic adjustment of the buffer size in the prior art, and can reduce delays, freezes, mosaics, noises, etc. caused by network jitter while maintaining the audio and video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flowchart of a preferred embodiment of the data jitter buffer method in the present invention;

[0065] Figure 2 is a schematic diagram of a specific data packet reception disclosed by the present invention;

[0066] Figure 3 is a flowchart of a method for judging duplicate and expired packet losses disclosed by the present invention;

[0067] Figure 4 is a flowchart of a specific method for judging duplicate and expired packets disclosed by the present invention;

[0068] Figure 5It is a flowchart of a method for judging expired packets based on timestamps disclosed by the present invention;

[0069] Figure 6 It is a flowchart of a method for judging expired packets based on packet sequence numbers disclosed by the present invention;

[0070] Figure 7 It is a flowchart of a method for judging duplicate packets disclosed by the present invention;

[0071] Figure 8 It is a flowchart of a specific out-of-order rearrangement method disclosed by the present invention;

[0072] Figure 9 It is a flowchart of a specific data packet retransmission method disclosed by the present invention;

[0073] Figure 10 It is a flowchart of a specific data output method disclosed by the present invention;

[0074] Figure 11 It is a schematic diagram of a data jitter buffer framework disclosed by the present invention;

[0075] Figure 12 It is a functional principle block diagram of a preferred embodiment of a data jitter buffer device in the present invention;

[0076] Figure 13 It is a functional principle block diagram of a preferred embodiment of a terminal in the present invention. Detailed implementation manners

[0077] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further elaborates on the present invention by way of examples with reference to the accompanying drawings. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0078] The basic principle of the existing jitter buffer technology is that due to the instability of network latency, the intervals between audio and video data packets received at the receiving end may be uneven, resulting in problems such as audio interruptions, noise, and video stuttering during playback. The jitter buffer, that is, the shared data area, smooths the data playback and reduces the impact of jitter on the audio and video quality by setting a buffer at the receiving end to temporarily store the arriving data packets and readjusting the playback order and timing of the packets according to a preset time interval.

[0079] In the prior art, the adjustment methods of jitter buffering are generally based on the following three methods:

[0080] First, buffering based on RTT estimation: By calculating the RTT value in real time to estimate network latency and jitter, and then dynamically adjusting the size of the Jitter Buffer. The buffering method based on RTT can adapt to changes in network latency. However, when the network fluctuates greatly or is unstable, the adjustment speed is slow, which may lead to the accumulation of latency and affect the user experience.

[0081] Second, dynamic adjustment based on the Jitter value: The Jitter value refers to the variation in the arrival time intervals of data packets. The Jitter Buffer dynamically adjusts the size of the buffer based on historical Jitter values. When Jitter is large, the buffer will automatically increase to prevent data loss or playback interruption; when Jitter is small and the network is stable, the buffer will automatically decrease to reduce latency. However, the change of Jitter is not completely predictable. Complex filtering algorithms may increase the computational overhead during the calculation process and may not be able to respond quickly to instantaneous network fluctuations, resulting in the accumulation of latency.

[0082] Third, buffering based on a fixed buffering duration: Such as the SRT (Secure Reliable Transpt) protocol. When the buffer receives the expected data packet, it does not immediately output for playback but needs to wait for a fixed buffering duration before outputting for playback, resulting in no reduction in latency when the network state is good.

[0083] In summary, the existing Jitter Buffer technologies often rely on a fixed buffering duration or rely on RTT and Jitter values to adjust the buffering duration. However, there is a problem that the latency will not decrease when the network state improves with a fixed buffering duration; in a weak network environment, such as in a mobile network or when the Wi-Fi signal is unstable, due to more serious network latency and packet loss, the jitter buffer usually needs to be dynamically adjusted frequently. The buffer capacity will change with the buffering duration, and reducing the buffer size may cause data loss in the buffer, and the playback picture will be pixelated; increasing the buffer size may cause a larger latency and other problems. Therefore, this application provides a data jitter buffering method, which can solve the problems of data loss and latency accumulation caused by the dynamic adjustment of the buffer size in the existing technology, and can reduce latency, stuttering, pixelation, and noise caused by network jitter while maintaining the audio and video quality.

[0084] Please refer to Figure 1 , Figure 1 which is the flowchart of the data jitter buffering method in the present invention. As Figure 1 shown, the data jitter buffering method described in the embodiments of the present invention includes:

[0085] Step S11: Receive the packet to be transmitted to obtain the current packet, and determine whether the current packet is the currently expected packet.

[0086] In this embodiment, the data reception operation is performed. Specifically, to receive the packet to be transmitted to obtain the current packet, refer to Figure 2 As shown, before receiving data, the socket of the data channel can be obtained, then the socket buffer size can be configured, the transmission protocol of the data channel can be determined, and then the RTP packet of the data channel can be received to obtain the current packet, and the reception timestamp corresponding to the current packet can be recorded. The current packet is parsed to determine the transmission timestamp, the payload data, and the current packet sequence number of the current packet. When it is subsequently determined that the current packet is not the currently expected packet, specifically, the transmission timestamp, the reception timestamp, the payload data, and the current packet sequence number of the current packet are sent to the jitter buffer queue, and the packet to be transmitted is continuously received until all the data to be transmitted is received.

[0087] It should be noted that the RTP (Real-time Transport Protocol) packet is a network packet used to transmit streaming media data such as audio and video in real time on the Internet. It is widely used in scenarios such as video conferencing, online live broadcast, and VoIP, such as Skype, and supports multimedia communication with low latency and high real-time performance. Moreover, since the RTP packet belongs to real-time streaming media and is sensitive to latency, a certain buffer is required to cope with network jitter. Therefore, the network data transmission performance and stability can be optimized by setting the socket buffer size before receiving data. For example, a larger reception buffer can reduce packet loss under high load, while a larger transmission buffer can reduce the possibility of transmission blocking.

[0088] In this embodiment, after receiving the packet to be transmitted to obtain the current packet, it is determined whether the current packet is the currently expected packet. Specifically, it can be determined whether the current packet is the currently expected packet based on the packet sequence number. First, the sequence number of the previous output packet is determined, and the currently expected packet sequence number is determined based on the sequence number of the previous output packet. Then, it is determined whether the current packet sequence number is the same as the currently expected packet sequence number. When the current packet sequence number is the same as the currently expected packet sequence number, it is determined that the current packet is the currently expected packet; when the current packet sequence number is different from the currently expected packet sequence number, it is determined that the current packet is not the currently expected packet.

[0089] For example, the sequence number of the previous output data packet is 10, and it is expected to receive the next data packet with sequence number 11, that is, the expected data packet sequence number is 11. The currently received data packet sequence number is 12, that is, the current data packet sequence number is 12. Since the expected data packet sequence number is different from the current data packet sequence number, it indicates that the currently received data packet is not the expected data packet, and it is sent to the jitter buffer for caching and waiting.

[0090] Step S12: When the current data packet is not the current expected data packet, send the current data packet to the jitter buffer queue for sorting, and cache and wait for a preset buffer duration. If the current expected data packet is not received within the preset buffer duration, output the consecutive data packets at the head of the sorted jitter buffer queue.

[0091] In this embodiment, when the current data packet is received and it is not the current expected data packet, it is sent to the jitter buffer queue for sorting, and cached and waited for a preset buffer duration. If the current expected data packet is still not received within the preset buffer duration, it times out, and directly outputs the consecutive data packets at the head of the sorted jitter buffer queue. If the current expected data packet is received while caching and waiting in the jitter buffer queue for the preset buffer duration, output the current expected data packet, and output the consecutive data packets at the head of the jitter buffer queue that are consecutive to the current expected data packet. Among them, the data packets in the jitter buffer are the data packets after out-of-order rearrangement, and the preset buffer duration is the maximum buffer duration configured in advance.

[0092] For example, within the maximum buffer time, the expected data packet 11 is still not received, and data packets with sequence numbers 12, 15, 13, and 16 are received in sequence (due to network reasons, they may not arrive in order, and there may be packet loss, etc.). The sequence numbers of the data packets in the sorted jitter buffer queue are 12, 13, 15, and 16 respectively. Then when the maximum buffer time arrives (i.e., waiting for the expected data packet times out), output the consecutive data packets at the head of the jitter buffer queue, that is, the two data packets with sequence numbers 12 and 13, update the output data packet sequence number to 13, the expected data packet sequence number to 14, and the remaining data packet sequence numbers in the jitter buffer are 15 and 16. If the data packet with sequence number 14 is not currently received, then cache and wait for the maximum buffer duration as described above and output. If the data packet with sequence number 14 is currently received, output the data packet with sequence number 14, update the output data packet sequence number to 14, the expected data packet sequence number to 15. There are data packets with sequence number 15 and sequence number 16 that are consecutive to the data packet with sequence number 14 in the jitter buffer, then immediately output the consecutive data packets with sequence numbers 15 and 16.

[0093] It should be noted that if there is already data output, update the previous output data packet and the current expected data packet, and the current expected data packet sequence number = the previous output data packet sequence number + 1.

[0094] In this embodiment, when it is determined that the currently received data packet is not the currently expected data packet and the currently received data packet is sent to the jitter buffer queue to buffer and wait before the preset buffer duration, it can be further determined whether it is a duplicate data packet or an expired data packet. See Figure 3 As shown, the specific determination method may include:

[0095] Step S121: Determine whether the currently received data packet is the first data packet.

[0096] Step S122: When the currently received data packet is not the first data packet, determine whether the currently received data packet is an expired data packet.

[0097] Step S123: When the currently received data packet is the expired data packet, discard the expired data packet and re - execute the step of receiving data to obtain the currently received data packet.

[0098] Step S124: When the currently received data packet is not the expired data packet, determine whether the currently received data packet is a duplicate data packet.

[0099] Step S125: When the currently received data packet is the duplicate data packet, discard the duplicate data packet and re - execute the step of receiving data to obtain the currently received data packet.

[0100] Step S126: When the currently received data packet is not the duplicate data packet, send the currently received data packet to the jitter buffer queue for sorting to obtain the sorted jitter buffer queue.

[0101] It can be understood that due to network reasons, a large number of duplicate packets or expired packets may be received. Therefore, when preparing to cache non-expected packets, it is also necessary to determine whether the non-expected packets are expired packets or duplicate packets, etc. That is, it is determined that the currently received packet is not the currently expected packet, and the current packet is marked as a non-expected packet. Then, it is further determined whether the non-expected packet is an expired packet. If so, it is discarded. If not, it is further determined whether the non-expected packet is a duplicate packet. If so, it is discarded. If not, the non-expected packet is marked as a non-duplicate packet and sent into the jitter buffer queue. The non-expired and non-duplicate packets are cached in the jitter buffer queue for a preset buffering duration and sorted with other non-expired and non-duplicate packets in the jitter buffer queue to obtain the sorted jitter buffer queue. Moreover, when the sequence number of the current packet is the same as the sequence number of any packet in the received jitter buffer, it indicates that a duplicate packet has been received. It should be noted that before further determining whether the non-expected packet is a duplicate packet, it can be first determined whether the non-expected packet is the first packet. When it is not the first packet, it is further determined whether it is an expired packet or a duplicate packet. When it is the first packet, it is directly sent into the jitter buffer queue to be cached for the preset buffering duration. During this process, the next packet is received, and it is also first determined whether it is the expected data, and then the above judgment is repeated.

[0102] Among them, specifically, it can be determined whether the current packet is a duplicate packet based on the packet sequence number, that is: determine the sequence numbers of the packets in the jitter buffer queue, and determine whether the current packet sequence number is the same as each packet sequence number; when the current packet sequence number is the same as any packet sequence number, it is determined that the current packet is a duplicate packet; when the current packet sequence number is different from each packet sequence number, it is determined that the current packet is not a duplicate packet, that is, a non-duplicate packet. For example, the expected packet sequence number is 15, and the packet sequence numbers in the jitter buffer are 16 and 17. When the currently received packet sequence number is 16, and there is already a packet with the same sequence number in the jitter buffer, the duplicate packet is actively discarded to avoid repeatedly outputting and playing the same packet.

[0103] It should be noted that obtain the transmission protocol used by the data channel, and then the current used expired packet loss strategy can be configured according to this transmission protocol. If there is no data output yet, that is, it means that the program has just started and there is no data output yet, then initialize the packet sequence number of the previous output packet, for example, initialize it to 0.

[0104] The first specific expired packet loss strategy can specifically determine whether the current data packet is an expired data packet based on the time stamp, that is: determine the sending time stamp of the previous output data packet, and calculate the difference between the sending time stamp of the current data packet and the sending time stamp of the previous output data packet; determine whether the difference is less than the preset time stamp rollover threshold; when the difference is less than the preset time stamp rollover threshold, determine whether the sending time stamp of the current data packet is less than or equal to the sending time stamp of the previous output data packet; when the sending time stamp of the current data packet is less than or equal to the sending time stamp of the previous output data packet, determine that the current data packet is an expired data packet; when the sending time stamp of the current data packet is greater than the sending time stamp of the previous output data packet, determine that the current data packet is not an expired data packet, and when the difference is not less than the preset time stamp rollover threshold, determine that the current data packet is not an expired data packet.

[0105] The second specific expired packet loss strategy can specifically determine whether the current data packet is an expired data packet based on the data packet sequence number, that is: determine the previous output data packet sequence number, determine the current expected data packet sequence number based on the previous output data packet sequence number, and calculate the packet interval between the current data packet sequence number and the current expected data packet sequence number; determine whether the packet interval is less than the preset packet sequence number rollover threshold; when the packet interval is less than the preset packet sequence number rollover threshold, determine whether the current data packet sequence number is less than the current expected data packet sequence number; when the current data packet sequence number is less than the current expected data packet sequence number, determine that the current data packet is an expired data packet; when the current data packet sequence number is not less than the current expected data packet sequence number, determine that the current data packet is not an expired data packet.

[0106] For example, see Figure 4As shown, determine the transmission protocol used by the data channel, configure the expired packet loss policy according to the transmission protocol, receive data packets, and when the current data packet is an unexpected data packet, record the sequence number S1 of the current data packet and the transmission timestamp T1 of the current data packet, determine the sequence number S2 of the previous output data packet, and then determine whether the current data packet is the first data packet. If so, send it to the jitter buffer queue for buffering and waiting for the preset buffering duration, and sort it with other data packets subsequently sent to the buffer queue. If the current data packet is not the first data packet, determine whether the current data packet is an expired data packet based on the expired packet loss policy. If it is an expired data packet, discard the current data packet. If it is not an expired data packet, determine whether the current data packet is a duplicate data packet, such as by determining whether the sequence number of the current data packet is the same as the sequence numbers of each data packet in the jitter buffer queue. If there is a data packet with the same sequence number as the current data packet, it indicates that the current data packet is a duplicate data packet, discard the duplicate data packet, continue to receive the next data packet to be transmitted, and re - execute the determination; if there is no data packet with the same sequence number as the current data packet, it indicates that the current data packet is not a duplicate data packet, then send it to the jitter buffer queue for buffering and waiting for the preset buffering duration, and sort it, and at the same time continue to receive the next data packet to be transmitted, and re - execute the determination.

[0107] Among them, configure the expired packet loss policy according to the transmission protocol, and this expired packet loss policy is the expired packet loss based on the timestamp, see Figure 5 As shown, that is, determine whether the current data packet is an expired data packet based on the timestamp. Receive the data packet, and when the current data packet is an unexpected data packet, record the sequence number S1 of the current data packet and the transmission timestamp T1 of the current data packet, and then determine the sequence number S2 of the previous output data packet, the transmission timestamp T2 of the previous output data packet, and the preset timestamp rollover threshold T0. First, calculate the difference between the transmission timestamp T1 of the current data packet and the transmission timestamp T2 of the previous output data packet, that is, T3 = T1 - T2, and then determine whether the difference T3 is less than the preset timestamp rollover threshold T0; if T3 < T0, then further determine whether the transmission timestamp T1 of the current data packet is less than or equal to the transmission timestamp T2 of the previous output data packet; if T1 ≤ T2, it indicates that the current data packet is an expired data packet and is discarded, continue to receive the next data packet to be transmitted, and when the next data packet to be transmitted is an unexpected data packet, re - execute the determination; if T3 ≥ T0 or T1 > T2, mark the current data packet as a non - expired data packet, send it to the jitter buffer queue for buffering and waiting for the preset buffering duration, and sort it, continue to receive the next data packet to be transmitted, and when the next data packet to be transmitted is an unexpected data packet, re - execute the determination.

[0108] And configure the expired packet loss policy according to the transmission protocol, and this expired packet loss policy is the expired packet loss based on the packet sequence number, see Figure 6As shown, that is, based on the packet sequence number to determine whether the current data packet is an expired data packet. When receiving a data packet, if the current data packet is an unexpected data packet, record the current data packet sequence number S1 and the sending timestamp T1. Determine the previous output data packet sequence number S2 and the preset packet sequence number flipping threshold S0, and then determine the current expected data packet sequence number S3 based on the previous output data packet sequence number S2, that is: S3 = S2 + 1, and calculate the packet distance Sd between the current data packet and the current expected data packet, that is: Sd = abs(S1 - S3). Furthermore, determine whether the packet distance Sd is less than the preset packet sequence number flipping threshold S0. If Sd < S0, then determine whether the current data packet sequence number S1 is less than the packet sequence number S3 of the current expected data packet; if S1 < S3, it indicates that the current data packet is an expired data packet and is discarded, continue to receive the next data packet to be transmitted, and when the next data packet to be transmitted is an unexpected data packet, re-determine the expected data packet sequence number S3 and perform subsequent judgments; if Sd ≥ S0 or S1 ≥ S3, mark the current packet as a non-expired data packet, send it to the jitter buffer queue for buffering and waiting for the preset buffer duration, and perform sorting, continue to receive the next data packet to be transmitted, and when the next data packet to be transmitted is an unexpected data packet, re-determine the expected data packet sequence number S3 and perform the judgment.

[0109] It should be noted that for packet sequence number flipping (packet sequence number wrapping), when the packet sequence number exceeds the maximum value, it will wrap around to 0. If the packet sequence number is represented by u16 (unsigned integer 16 bits), the value range of the packet sequence number is 0 to 65535, and the maximum value is 65535. When receiving a data packet with the packet sequence number 65535, the next packet sequence number will start from 0 instead of 65536.

[0110] Moreover, in this embodiment, sort the current data packet and each data packet in the jitter buffer queue to obtain the sorted jitter buffer queue, which specifically may include: determining the first data packet sequence number at the head of the jitter buffer queue, and using the first data packet sequence number at the head as the current sorting reference number; determining whether the current data packet sequence number is less than the current sorting reference number; when the current data packet sequence number is not less than the current sorting reference number, determine the next data packet sequence number of the current sorting reference number, use the next data packet sequence number as the current sorting reference number, and re-execute the step of determining whether the current data packet sequence number is less than the current sorting reference number; when the current data packet sequence number is less than the current sorting reference number, insert the current data packet in front of the data packet corresponding to the current sorting reference number to obtain the sorted jitter buffer queue.

[0111] For example, see Figure 7As shown, the current data packet is received, and all the data packet sequence numbers in the jitter buffer queue are obtained. If the sequence number of the currently received data packet is the same as any of the data packet sequence numbers in the jitter buffer queue, it is determined that the current data packet is a duplicate data packet, and the current data packet is directly discarded; if the sequence number of the currently received data packet is different from all the data packet sequence numbers in the jitter buffer queue, it is determined that the current data packet is a non-duplicate data packet, and the current data packet is sent into the jitter buffer queue and sorted. For example, in the audio transmission process of Airplay screen mirroring, redundant packets will be transmitted, and each audio packet will be transmitted 3 times; the receiving end may receive up to 3 identical audio data packets repeatedly. Therefore, it is necessary to remove duplicate data packets in the jitter buffer to avoid repeated playback of the same data.

[0112] For example, refer to Figure 8 As shown, the currently received data packet is a non-expired data packet, and its packet sequence number is denoted as S1. Then, the data packet sequence number of the head node of the jitter buffer queue is determined, that is, the sequence number of the first data packet at the head of the jitter buffer queue is determined; its packet sequence number is denoted as S0. Then, it is judged whether the non-expired data packet S1 is less than the data packet sequence number S0 of the head node; if S1 < S0, the non-expired data packet is inserted in front of the head node data packet, and the step of receiving the current data packet and the subsequent steps when the current data packet is a non-expired data packet are continued to be executed; if S1 ≥ S0, the next node S2 of the packet sequence number S0 is obtained, S0 is updated to S2, and it is re-judged whether the packet sequence number S1 is less than the packet sequence number S2.

[0113] In this embodiment, when determining that the currently received data packet is not the currently expected data packet and sending the current data packet to the jitter buffer queue to buffer and wait for the preset buffer duration, it may specifically further include: determining the sequence number of the data packet to be retransmitted based on the currently expected data packet sequence number and the data packet sequence numbers in the sorted jitter buffer queue, and requesting the retransmission of the data packet corresponding to the sequence number of the data packet to be retransmitted, and continuing to execute the step of receiving the data packet to obtain the current data packet. It can be understood that during the caching process of the non-expected data packet in the jitter buffer queue, the retransmission of the data packet can be requested so as to be able to receive the expected data packet within the preset buffer duration, and output the expected data packet and the data packets buffered in the jitter buffer queue that are consecutive with the expected data packet.

[0114] For example, if the expected packet sequence number is 15, the packet sequence number in the jitter buffer is 17, and the currently received packet sequence number is 16, it will enter the jitter buffer for buffering and waiting, and perform out-of-order rearrangement and retransmission request operations. The packet sequence numbers in the sorted jitter buffer queue are 16 and 17 respectively, and the packet with sequence number 15 is requested to be retransmitted. Another example is that if the lost data is continuous (network packet loss), for example, the numbers in the jitter buffer are 18 and 19, and the expected packet sequence number is 15, then it is determined that the starting packet sequence number of the retransmission request packet is 15 and the quantity is 3, that is, 3 packets with sequence numbers 15, 16, and 17 are requested to be retransmitted; if the lost data is not continuous, for example, the packet sequence numbers in the jitter buffer are 17 and 18 respectively, and the expected packet sequence number is 16, then it is determined that the starting sequence number of the retransmission request packet is 16 and the quantity is 1.

[0115] It should be noted that if retransmitted data is received after timeout output, the expired data is considered to be directly discarded. For example, the expected packet sequence number is 15, the packet sequence numbers in the jitter buffer are 16 and 17 respectively, the packet with sequence number 15 is requested to be retransmitted with a quantity of 1, and the packet at the head of the jitter buffer queue is obtained, whose sequence number is 16. Its reception timestamp is determined. If packet 15 is not received within 60 ms from this timestamp, then 16 is output. Since 17 is consecutive data with 16, 17 will also be output immediately. If packet 15 is received after 60 ms from the reception timestamp of the packet with sequence number 16, since the data of 16 has been output for playback, 15 is already expired data and is actively discarded.

[0116] For example, refer to Figure 9 As shown, determine the sequence number S0 of the previous output packet output from the jitter buffer, calculate the current expected packet sequence number S1 = S0 + 1. The currently received packet is a non-expected packet, and its packet sequence number is S2, and S2 > S1. Store S2 in the jitter buffer for sorting. Then judge whether the current expected packet sequence number S1 is the same as the packet sequence number S3 at the head of the sorted jitter buffer. If they are the same, directly output the packet with the same sequence number as the current expected packet and the packets consecutive to the current expected packet in the sorted jitter buffer. If they are not the same, calculate the packet interval count between the expected packet sequence number S1 and the packet sequence number S3 at the head of the sorted jitter buffer, request retransmission, retransmit count packets starting from the expected packet sequence number S1, receive the retransmitted data, and when the received retransmitted packet is an expected packet, output the retransmitted packet and the packets consecutive to the retransmitted packet in the buffer.

[0117] Among them, the packet spacing represents the distance between the packet sequence numbers of two adjacent data packets. For example, for two data packets with packet sequence numbers S1 and S2 respectively, the packet spacing Sd = abs(S1 - S2), where the abs symbol represents taking the absolute value. Here, both the packet sequence numbers S1 and S2 are represented by signed integer variables in the program to avoid abnormal packet spacing calculation caused by packet sequence number flipping.

[0118] In this embodiment, during the process of determining that the currently received data packet is not the currently expected data packet and sending the current data packet to the jitter buffer queue to buffer and wait for a preset buffer duration, it may specifically further include: determining whether the buffer waiting duration of the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue exceeds the preset buffer duration; when the buffer waiting duration of the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue does not exceed the preset buffer duration, then determining the current buffer capacity of the jitter buffer queue in real time and judging whether the current buffer capacity is not less than the preset buffer capacity; when the current buffer capacity is not less than the preset buffer capacity, directly outputting the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue; when the current buffer capacity is less than the preset buffer capacity and the currently expected data packet is received within the preset buffer duration, calculating the packet spacing between the currently expected data packet and the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue, and judging whether the packet spacing is less than the preset skip packet threshold; when the packet spacing is not less than the preset skip packet threshold, outputting the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue; when the packet spacing is less than the preset skip packet threshold, it indicates that the currently expected data packet and the data packet corresponding to the earliest reception timestamp in the jitter buffer queue are consecutive sequence number data packets, and outputting the currently expected data packet and the data packet corresponding to the earliest reception timestamp.

[0119] Among them, a skipped packet means that the packet spacing between the current data packet and the previously received data packet is greater than 1, that is, it means that the packet sequence numbers of adjacent data packets are not consecutive. Exceeding the preset skip packet threshold indicates that the network environment is not very ideal, the network connection is unstable, and data packet skipping is serious. It is necessary to immediately output and update the expected data packet sequence number, and the preset buffer capacity is the capacity of a preset fixed-size buffer queue, such as the maximum buffer queue capacity.

[0120] For example, see Figure 10As shown, determine the current expected data packet sequence number S0, and the pre-configured maximum buffer duration T0 (pre-set buffer duration), the maximum buffer packet number C0 (pre-set buffer capacity), and the packet loss threshold St; determine the data packet corresponding to the earliest reception timestamp in the jitter buffer, and parse out the packet sequence number S1 and the reception timestamp T1; calculate the packet interval Sd between the packet sequence number S1 corresponding to the earliest reception timestamp and the expected packet sequence number S0; determine whether the packet interval Sd is equal to 0; if Sd = 0, it indicates that the data packet corresponding to the earliest reception timestamp is the current expected data packet S0, and directly send the data packet corresponding to the earliest reception timestamp to the player for playback, and update the current expected data packet sequence number S0 = S1 + 1, and continue to determine the data packet corresponding to the earliest reception timestamp in the jitter buffer, and parse out the packet sequence number S1 and the reception timestamp T1; if Sd ≠ 0, determine the current system timestamp T2, and determine whether the current system timestamp T2 is greater than the sum of the reception timestamp T1 of the data packet corresponding to the earliest reception timestamp and the maximum buffer time T0, that is, whether T2 is greater than T1 + T0; if T2 > T1 + T0, wait for the expected packet S0 to time out and directly output; if T2 ≤ T1 + T0, determine the current buffer capacity C1 in real time, and determine whether the current buffer capacity C1 is not less than the pre-set buffer capacity C0; if C1 ≥ C0, it indicates that the buffer is full during the process of waiting for the current expected data packet S0, and directly output the data packets in the jitter buffer; if C1 < C0, determine whether the packet interval Sd is not less than the packet loss threshold St; if Sd < St, return to continue to determine the data packet corresponding to the earliest reception timestamp in the jitter buffer, and parse out the packet sequence number S1 and the reception timestamp T1; if Sd ≥ St, immediately output the data packet, send it to the player for playback, and update the current expected data packet sequence number S0 = S1 + 1.

[0121] For example, the current expected data packet sequence number is 0, and the packet sequence number of the data packet corresponding to the earliest reception timestamp in the buffer queue is 5000. If the pre-set packet loss threshold is 3000, the packet interval between the expected data packet and the data packet corresponding to the earliest reception timestamp is 5000, which is greater than the packet loss threshold 3000. At this time, it should not wait excessively for the expected data packet with the sequence number 0, but immediately output the data packet with the sequence number 5000, and update the previous output data packet sequence number to 5000, and the current expected data packet sequence number to 5001.

[0122] Step S13, when the current data packet is the current expected data packet, there is no need to cache and wait in the jitter buffer queue, and directly output the current data packet and the data packets consecutive to the current data packet at the head of the jitter buffer queue.

[0123] In this embodiment, if the currently received data packet is the currently expected data packet, there is no need to cache and wait in the jitter buffer queue. At this time, the buffering duration is zero. Moreover, when the currently received data packet is the first data packet, it indicates that there is no data packet waiting to be buffered in the jitter buffer queue, and the currently received data packet can be directly output. If the currently received data packet is not the first data packet, there are data packets waiting to be buffered in the jitter buffer queue. When the data packet at the head of the jitter buffer queue is a consecutive data packet with the currently received data packet, the currently received data packet and the consecutive data packet at the head of the queue that is consecutive with the currently received data packet are output. For example, if the sequence number of the currently received data packet is 14 and the sequence number of the data packet at the head of the jitter buffer queue is 15, it indicates that they are consecutive data packets. If there are data packets waiting to be buffered in the jitter buffer queue, but the data packet at the head of the jitter buffer queue is not consecutive with the sequence number of the currently received data packet, only the currently received data packet is output.

[0124] It can be seen that in the embodiment of the present invention, through the buffering strategy of double buffering duration, the currently expected data packet can be output immediately upon receipt, and the buffering duration is 0. If the currently expected data packet is not received, it is cached and waited in the jitter buffer queue for the preset buffering duration. When the currently expected data packet is still not received within the preset buffering duration, the consecutive data packets at the head of the sorted jitter buffer queue are output. That is, through the data jitter buffering scheme based on double buffering duration in this application, the problems of data loss and delay accumulation caused by dynamic adjustment of the buffer size in the prior art can be solved, and while maintaining the audio and video quality, the delay, stuttering, mosaic, noise and other phenomena caused by network jitter can be reduced.

[0125] It should be noted that the specific application scenarios of the technical solution of data jitter buffering based on double buffering duration in this application can include but are not limited to Android phone Miracast screen mirroring to TV. Among them, Miracast is a wireless display standard designed to directly transmit high-definition audio and video content between devices without relying on traditional cables or Internet connections; iPhone phone Airplay screen mirroring, mirroring or music screen mirroring. Among them, AirPlay is a proprietary wireless screen mirroring and audio streaming protocol mainly used to achieve seamless transmission of audio and video content between Apple ecosystem devices such as iPhone, iPad, Mac, Apple TV, HomePod, etc.; private protocol screen mirroring of iPhone or Android phones to TV, and screen mirroring of TV private protocol to iPhone or Android phones and other scenarios.

[0126] For example, see Figure 11As shown, in the jitter buffer framework of the data jitter buffer technology solution based on double buffer duration in the present application, it includes three parts: data reception, jitter buffering, and data output. That is, when the currently received packet to be transmitted, i.e., the current packet, and after judging duplicate packets and expired packets, the current packet is a non-expected packet, not a duplicate packet, nor an expired packet, it is sent to the jitter buffer for operations such as out-of-order rearrangement and packet retransmission. Then, when the currently received packet is an expected packet, the current packet and the packets consecutive to the current packet sequence number in the jitter buffer are directly output.

[0127] In one embodiment, as Figure 12 shown, based on the above data jitter buffer method, the present invention also correspondingly provides a data jitter buffer device, including:

[0128] A data reception module 11, configured to receive the packet to be transmitted to obtain the current packet, and judge whether the current packet is the currently expected packet;

[0129] A first buffer processing module 12, configured to, when the current packet is not the currently expected packet, send the current packet to the jitter buffer queue for sorting, and cache and wait for the preset buffer duration, and output the consecutive packets at the head of the sorted jitter buffer queue when the currently expected packet is not received within the preset buffer duration;

[0130] A second buffer processing module 13, configured to, when the current packet is the currently expected packet, directly output the current packet and the packets consecutive to the current packet at the head of the jitter buffer queue without caching and waiting in the jitter buffer queue.

[0131] Figure 13 It is a schematic structural diagram of a terminal provided in an embodiment of the present application. The terminal may include:

[0132] A memory 501, a processor 502, and a computer program stored on the memory 501 and executable on the processor 502.

[0133] When the processor 502 executes the program, it implements the data jitter buffer method provided in the above embodiment.

[0134] Further, the terminal further includes:

[0135] A communication interface 503, configured for communication between the memory 501 and the processor 502.

[0136] The memory 501 is used to store a computer program executable on the processor 502.

[0137] The memory 501 may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

[0138] If the memory 501, the processor 502, and the communication interface 503 are implemented independently, the communication interface 503, the memory 501, and the processor 502 can be interconnected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0139] Optionally, in a specific implementation, if the memory 501, the processor 502, and the communication interface 503 are integrated on a single chip, the memory 501, the processor 502, and the communication interface 503 can communicate with each other through an internal interface.

[0140] The processor 502 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0141] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above data jitter buffering method is implemented.

[0142] Those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the claims.

[0143] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or N embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0144] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can read and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices.

[0145] It should be understood that the various parts of this application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGA, Programmable Gate Array), field programmable gate arrays (FPGA, Field-Programmable Gate Array), etc.

[0146] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A data jitter buffering method, characterized in that, The method includes: Receiving a data packet to be transmitted to obtain a current data packet, and determining whether the current data packet is the currently expected data packet; When the current data packet is not the currently expected data packet, sending the current data packet to a jitter buffer queue for sorting, caching and waiting for a preset buffer duration, and outputting consecutive data packets at the head of the sorted jitter buffer queue when the currently expected data packet is not received within the preset buffer duration; When the current data packet is the currently expected data packet, directly outputting the current data packet and consecutive data packets at the head of the jitter buffer queue without caching and waiting in the jitter buffer queue.

2. The data jitter buffering method according to claim 1, wherein The receiving a data packet to be transmitted to obtain a current data packet includes: Receiving an RTP data packet of a data channel to obtain a current data packet, and recording a reception timestamp corresponding to the current data packet; Parsing the current data packet to determine a transmission timestamp, payload data, and a current data packet sequence number of the current data packet; Among them, the sending the current data packet to the jitter buffer queue includes: Sending the transmission timestamp, the reception timestamp, the payload data, and the current data packet sequence number of the current data packet to the jitter buffer queue.

3. The data jitter buffering method according to claim 2, wherein The determining whether the current data packet is the currently expected data packet includes: Determining a previous output data packet sequence number, and determining a currently expected data packet sequence number based on the previous output data packet sequence number; Judging whether the current data packet sequence number is the same as the currently expected data packet sequence number; When the current data packet sequence number is the same as the currently expected data packet sequence number, determining that the current data packet is the currently expected data packet; When the current data packet sequence number is different from the currently expected data packet sequence number, determining that the current data packet is not the currently expected data packet.

4. The data jitter buffering method according to claim 2, wherein Before sending the current data packet to the jitter buffer queue for sorting, caching and waiting for a preset buffer duration, it further includes: Judging whether the current data packet is the first data packet; When the current data packet is not the first data packet, judging whether the current data packet is an expired data packet; When the current data packet is the expired data packet, discarding the expired data packet, and re-executing the step of receiving data to obtain the current data packet; When the current data packet is not the expired data packet, judging whether the current data packet is a duplicate data packet; When the current data packet is the duplicate data packet, discarding the duplicate data packet, and re-executing the step of receiving data to obtain the current data packet; Among them, the sending the current data packet to the jitter buffer queue for sorting includes: When the current data packet is not the duplicate data packet, sending the current data packet to the jitter buffer queue for sorting to obtain the sorted jitter buffer queue.

5. The data jitter buffering method according to claim 4, wherein The judging whether the current data packet is a duplicate data packet includes: Determining data packet sequence numbers of each data packet in the jitter buffer queue, and judging whether the current data packet sequence number is the same as each of the data packet sequence numbers; When the current data packet sequence number is the same as any one of the data packet sequence numbers, determining that the current data packet is a duplicate data packet; When the current data packet sequence number is different from each of the data packet sequence numbers, determining that the current data packet is not a duplicate data packet.

6. The data jitter buffering method according to claim 4, wherein Determining whether the current data packet is an expired data packet includes: Determining the transmission timestamp of the previous output data packet, and calculating the difference between the transmission timestamp of the current data packet and the transmission timestamp of the previous output data packet; Judging whether the difference is less than a preset timestamp rollover threshold; When the difference is less than the preset timestamp rollover threshold, judging whether the transmission timestamp of the current data packet is less than or equal to the transmission timestamp of the previous output data packet; When the transmission timestamp of the current data packet is less than or equal to the transmission timestamp of the previous output data packet, determining that the current data packet is an expired data packet; When the transmission timestamp of the current data packet is greater than the transmission timestamp of the previous output data packet, determining that the current data packet is a non-expired data packet; Or, determining the previous output data packet sequence number, determining the current expected data packet sequence number based on the previous output data packet sequence number, and calculating the packet interval between the current data packet sequence number and the current expected data packet sequence number; Judging whether the packet interval is less than a preset packet sequence number rollover threshold; When the packet interval is less than the preset packet sequence number rollover threshold, judging whether the current data packet sequence number is less than the current expected data packet sequence number; When the current data packet sequence number is less than the current expected data packet sequence number, determining that the current data packet is an expired data packet; When the current data packet sequence number is not less than the current expected data packet sequence number, determining that the current data packet is a non-expired data packet.

7. The data jitter buffering method according to claim 4, wherein Sending the current data packet to the jitter buffer queue for sorting to obtain the sorted jitter buffer queue includes: Determining the first data packet sequence number at the head of the jitter buffer queue, and using the first data packet sequence number at the head as the current sorting reference number; Judging whether the current data packet sequence number is less than the current sorting reference number; When the current data packet sequence number is not less than the current sorting reference number, determining the next data packet sequence number of the current sorting reference number, using the next data packet sequence number as the current sorting reference number, and re-executing the step of judging whether the current data packet sequence number is less than the current sorting reference number; When the current data packet sequence number is less than the current sorting reference number, inserting the current data packet in front of the data packet corresponding to the current sorting reference number to obtain the sorted jitter buffer queue.

8. The data jitter buffering method according to claim 7, wherein During the process of sending the current data packet to the jitter buffer queue for sorting and caching and waiting for a preset buffer duration, it further includes: Determining the data packet sequence number to be retransmitted based on the current expected data packet sequence number and the data packet sequence numbers in the sorted jitter buffer queue, and requesting retransmission of the data packet corresponding to the data packet sequence number to be retransmitted, and continuing to execute the step of receiving the data packet to be transmitted to obtain the current data packet.

9. The data jitter buffering method according to any one of claims 2 to 8, characterized in that During the process of sending the current data packet to the jitter buffer queue for sorting and caching and waiting for a preset buffer duration, it further includes: Determining whether the cache waiting duration of the data packet corresponding to the earliest reception timestamp in the sorted jitter buffer queue exceeds the preset buffer duration; When the cache waiting duration of the packet corresponding to the earliest received timestamp in the sorted jitter buffer queue does not exceed the preset buffer duration, the current buffer capacity of the jitter buffer queue is determined in real time, and it is judged whether the current buffer capacity is not less than the preset buffer capacity; When the current buffer capacity is not less than the preset buffer capacity, the packet corresponding to the earliest received timestamp in the sorted jitter buffer queue is directly output; When the current buffer capacity is less than the preset buffer capacity, the packet interval between the current expected packet and the packet corresponding to the earliest received timestamp in the sorted jitter buffer queue is calculated, and it is judged whether the packet interval is less than the preset packet skipping threshold; When the packet interval is not less than the preset packet skipping threshold, the packet corresponding to the earliest received timestamp in the sorted jitter buffer queue is output.

10. A data jitter buffer device, characterized in that, The device includes: A data receiving module, configured to receive a packet to be transmitted to obtain a current packet, and judge whether the current packet is the current expected packet; A first buffer processing module, configured to, when the current packet is not the current expected packet, send the current packet to the jitter buffer queue for sorting, cache and wait for the preset buffer duration, and output the consecutive packets at the head of the queue in the sorted jitter buffer queue when the current expected packet is not received within the preset buffer duration; A second buffer processing module, configured to, when the current packet is the current expected packet, directly output the current packet and the consecutive packets of the current packet at the head of the queue in the jitter buffer queue without caching and waiting in the jitter buffer queue.

Images