Audio playback method and apparatus, device, and storage medium

Abstract

Embodiments of the present application relate to the technical field of audio, and disclose an audio playback method and apparatus, a device, and a storage medium. The method comprises: in a process of receiving an audio data packet, determining a transmission jitter state and a transmission channel interference state (301); on the basis of the transmission jitter state and the transmission channel interference state, dynamically adjusting a jitter buffer area and an audio playback rate, wherein the jitter buffer area is used for caching the received audio data packet (302); performing playback rate change processing on audio decoded data on the basis of the adjusted audio playback rate, wherein the audio decoded data is acquired by decoding the audio data packet (303); and performing audio playback on the basis of the audio decoded data having undergone the playback rate change processing (304). By using the solution provided by the embodiments of the present application, the smoothness and stability of audio playback can be improved.

Description

Audio playback method, device, equipment and storage medium

[0001] This application claims priority to Chinese patent application number 202410329833.6, filed on March 21, 2024, entitled “Audio playback method, device, equipment and storage medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments of the present application relate to the field of audio technology, and in particular to an audio playback method, apparatus, device, and storage medium. Background Art

[0003] When real-time audio transmission is performed between computer devices, noise or lag may occur due to external environmental influences.

[0004] For example, when a computer device transmits audio via Bluetooth near-field transmission, audio playback may become stuck when the Bluetooth channel is interfered with; when a computer device transmits audio based on WebRTC, audio may become stuck in a weak network environment. Summary of the Invention

[0005] The present invention provides an audio playback method, apparatus, device, and storage medium. The technical solution is as follows:

[0006] In one aspect, an embodiment of the present application provides an audio playback method, the method being executed by a computer device, the method comprising:

[0007] During the reception of the audio data packet, determining the transmission jitter status and the transmission channel interference status;

[0008] Dynamically adjusting a jitter buffer and an audio playback rate based on the transmission jitter state and the transmission channel interference state, the jitter buffer being used to cache the received audio data packets;

[0009] performing speed-varying processing on audio decoded data based on the adjusted audio playback rate, the audio decoded data being obtained by decoding the audio data packet;

[0010] Audio playback is performed based on the audio decoded data after speed change processing.

[0011] On the other hand, an embodiment of the present application provides an audio playback device, the device comprising:

[0012] A state determination module, configured to determine a transmission jitter state and a transmission channel interference state during reception of an audio data packet;

[0013] an adjustment module, configured to dynamically adjust a jitter buffer and an audio playback rate based on the transmission jitter state and the transmission channel interference state, wherein the jitter buffer is configured to cache the received audio data packets;

[0014] a speed change module, configured to perform speed change processing on audio decoded data based on the adjusted audio playback rate, the audio decoded data being obtained by decoding the audio data packet;

[0015] The playing module is used to play audio based on the audio decoding data after speed change processing.

[0016] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the audio playback method described in the above aspect.

[0017] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one instruction is stored. The instruction is loaded and executed by a processor to implement the audio playback method as described in the above aspects.

[0018] In another aspect, an embodiment of the present application provides a computer program product, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio playback method provided in the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0020] FIG2 is a schematic diagram of an audio collection, transmission, and playback process provided by an exemplary embodiment of the present application;

[0021] FIG3 is a flow chart of an audio playback method provided by an exemplary embodiment of the present application;

[0022] FIG4 is a flow chart of a jitter interference monitoring process shown in an exemplary embodiment of the present application;

[0023] FIG5 is a flow chart of a transmission jitter state determination process shown in an exemplary embodiment of the present application;

[0024] FIG6 is a flow chart of a jitter rate calculation process shown in an exemplary embodiment of the present application;

[0025] FIG7 is a flow chart of a process for determining a transmission channel interference state according to an exemplary embodiment of the present application;

[0026] FIG8 is a schematic diagram of a jitter buffer and a process for dynamically adjusting an audio playback rate according to an exemplary embodiment of the present application;

[0027] FIG9 is a flowchart of a jitter buffer and a process for dynamically adjusting an audio playback rate, shown in an exemplary embodiment of the present application;

[0028] FIG10 is a schematic diagram illustrating an implementation of a jitter buffer dynamic adjustment process according to an exemplary embodiment of the present application;

[0029] FIG11 is a schematic diagram of a process for dynamically adjusting an audio playback rate according to an exemplary embodiment of the present application;

[0030] FIG12 is a flowchart of a deceleration decision process shown in an exemplary embodiment of the present application;

[0031] FIG13 is a flow chart of an accelerated decision-making process shown in an exemplary embodiment of the present application;

[0032] FIG14 is a flow chart of a rate fine-tuning and playback process shown in an exemplary embodiment of the present application;

[0033] FIG15 is a structural block diagram of an audio playback device provided by an exemplary embodiment of the present application;

[0034] FIG16 is a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0036] Please refer to FIG1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment may include: an audio transmitter 110 and an audio receiver 120 .

[0037] The audio transmitter 110 is an electronic device used to transmit real-time audio to the audio receiver 120. The electronic device can be a smartphone, a tablet computer, a personal computer, etc. In some embodiments, the real-time audio transmitted by the audio transmitter 110 can be audio captured by a microphone, audio read from an audio file, or recorded system audio, which is not limited in this embodiment of the present application.

[0038] Optionally, the audio transmitting end 110 performs echo cancellation, noise reduction, and automatic gain processing before encoding the audio, and then performs audio encoding on the processed audio through an audio encoder.

[0039] In different audio transmission scenarios, the audio transmitter 110 transmits real-time audio to the audio receiver 120 in different ways. In some embodiments, in a Bluetooth near-field transmission scenario, the audio transmitter 110 uses the encoded audio data as byte data and calls the OutputStream IO interface to send it to the audio receiver 120. In a WebRTC scenario, the audio transmitter 110 adds redundant packets after audio encoding, encapsulates them into RTP packets, and stores them in a Pacing queue. The RTP packets in the Pacing queue are then sent sequentially to the audio receiver 120 over the network by calling the UDP Socket interface.

[0040] The audio receiving terminal 120 is an electronic device for receiving and playing real-time audio. The electronic device can be a smart phone, a tablet computer, a personal computer, or an audio playing device (such as a Bluetooth speaker).

[0041] In order to reduce the audio playback jamming caused by jitter, as shown in Figure 2, the audio receiving end 120 in the embodiment of the present application is provided with a receiving module 121, a jitter buffer 122, a jitter interference monitor 123, an anti-jitter module 124, a decoding module 125, a playback controller 126 and a playback module 127.

[0042] After the audio transmitter 110 completes audio collection through the acquisition module 111, it performs audio encoding through the encoding module 112 and sends the encoded audio data packet to the audio receiver 120 through the sending module 113. After receiving the audio data packet through the receiving module 121, the audio receiver 120 caches the audio data packet in the jitter buffer 122. Furthermore, the jitter interference monitor 123 monitors the transmission jitter and transmission channel interference in real time and provides the monitored transmission jitter status and transmission channel interference status to the anti-jitter module 124, which makes an anti-jitter decision. After making the anti-jitter decision, the anti-jitter module 124 dynamically adjusts the size of the jitter buffer 122 and outputs the audio playback rate to the playback controller 126.

[0043] After decoding the audio data packets in the jitter buffer 122, the decoding module 125 provides the audio decoding data to the playback controller 126, and the playback controller 126 performs audio speed change processing based on the audio decoding data and the audio playback rate, and finally inputs the speed-changed audio data (such as pulse code modulation (PCM) data) into the playback module 127, which then plays the audio.

[0044] By setting up a jitter interference monitor and anti-jitter module, the receiver can dynamically adjust the jitter buffer size and audio playback speed based on real-time jitter interference conditions to adapt to different transmission environments. For example, in transmission environments with high jitter and interference, the jitter buffer can be increased and the audio playback rate can be reduced to avoid audio playback lag; in transmission environments with low jitter and interference, the jitter buffer can be reduced and the audio playback speed can be increased to reduce audio playback latency.

[0045] As shown in FIG1 , the solution provided in the embodiment of the present application can be used in a Bluetooth near-field transmission scenario or a WebRTC transmission scenario.

[0046] For example, the solution provided in the embodiments of this application can be applied to Bluetooth playback devices. When a smartphone plays music through a Bluetooth playback device, the Bluetooth playback device can monitor transmission jitter and Bluetooth channel interference in real time while receiving Bluetooth audio data packets, and adaptively adjust the jitter buffer and audio playback rate to ensure smooth and stable audio playback on the Bluetooth playback device side.

[0047] For another example, the solution provided by the embodiments of the present application can be applied to a terminal that supports network audio and video calling. When the terminal receives Bluetooth audio data packets over the network, it can monitor transmission jitter and WiFi channel interference in real time and adaptively adjust the jitter buffer and audio playback rate to ensure smoothness and stability of the audio on the other end during the audio and video call.

[0048] Of course, in addition to the above application scenarios, the solution provided in the embodiments of the present application can also be applied to other scenarios of real-time audio transmission through transmission media, and the present application is not limited to specific application scenarios.

[0049] 3 is a flow chart of an audio playback method provided by an exemplary embodiment of the present application. This embodiment uses the method applied to the audio receiving terminal shown in FIG1 as an example for explanation. The method may include the following steps.

[0050] Step 301: During the process of receiving an audio data packet, determine the transmission jitter state and the transmission channel interference state.

[0051] In different application scenarios, the audio receiver receives audio data packets in different ways. For example, in the Bluetooth near-field transmission scenario, the audio receiver receives audio data packets through the Bluetooth module, while in the WebRTC transmission scenario, the audio receiver receives audio data packets through a network module (such as a WiFi module).

[0052] In some embodiments, the audio receiving end is provided with a jitter interference monitor. The jitter interference monitor is used to periodically monitor transmission jitter and determine the transmission jitter state based on the monitoring results. The jitter interference monitor is also used to monitor channel interference of the transmission channel and determine the transmission channel interference state based on the monitoring results.

[0053] Optionally, the transmission jitter state is used to characterize at least one of a real-time jitter condition, a jitter rate, and a jitter change trend. The real-time jitter condition may include a real-time jitter interval, the jitter rate may include jitter rates of different jitter levels or a comprehensive jitter rate, and the jitter change trend may include a jitter increase trend and a jitter decrease trend.

[0054] Optionally, the transmission channel interference state is used to characterize changes in interference factors (interference sources) affecting the transmission channel under the current transmission environment. The interference factor may include at least one of an interference device and an interference channel. For example, in a Bluetooth near-field transmission scenario, the interference device may include a Bluetooth device, and the interference channel may include a WiFi channel.

[0055] Step 302: Dynamically adjust the jitter buffer and the audio playback rate based on the transmission jitter state and the transmission channel interference state. The jitter buffer is used to cache received audio data packets.

[0056] In some embodiments, in order to combat jitter, the audio receiving end is provided with a jitter buffer (Packet buffer in the WebRTC framework). After receiving the audio data packet, the audio receiving end first caches the audio data packet in the jitter buffer. Among them, the size of the jitter buffer not only affects the anti-jitter effect, but also affects the audio playback delay. The larger the jitter buffer, the stronger the anti-jitter ability, but the longer the audio playback delay. Conversely, the smaller the jitter buffer, the worse the anti-jitter ability, but the shorter the audio playback delay.

[0057] In addition to using a jitter buffer to combat jitter, in embodiments of the present application, the audio receiving end can also assist in achieving anti-jitter by adjusting the audio playback rate. In some embodiments, the audio playback rate is used as a multiple of the baseline audio playback speed. For example, 1.0 represents 1 times the baseline audio playback speed, i.e., unaccelerated and decelerated playback; 0.8 represents 0.8 times the baseline audio playback speed, i.e., decelerated playback; and 1.1 represents 1.1 times the baseline audio playback speed, i.e., accelerated playback.

[0058] To dynamically adapt to the audio transmission environment and maintain smooth and stable audio playback in various audio transmission environments, the audio receiving end does not use a fixed jitter buffer size and audio playback rate. In one possible implementation, the audio receiving end dynamically adjusts the jitter buffer size and the audio playback rate (i.e., performs audio speed change) based on the monitored transmission jitter status and transmission channel interference status.

[0059] In one possible scenario, when the transmission jitter status indicates severe jitter and severe transmission channel interference, the audio receiver can increase the jitter buffer and reduce the audio playback rate to avoid audio playback freezes caused by severe jitter. In another possible scenario, when the transmission jitter status indicates reduced jitter and weakened transmission channel interference, the audio receiver can reduce the jitter buffer and increase the audio playback rate, thereby reducing audio playback delay and improving the real-time performance of audio playback while ensuring a certain degree of anti-jitter capability.

[0060] The specific strategy for dynamically adjusting the jitter buffer and the audio playback rate will be described in detail in the following embodiments.

[0061] Step 303 : performing speed-variable processing on the audio decoded data based on the adjusted audio playback rate, where the audio decoded data is obtained by decoding the audio data packet.

[0062] After dynamically adjusting the jitter buffer size and the audio playback rate, the audio receiving end decodes the audio data packets cached in the jitter buffer and performs speed-shifting processing on the decoded audio data based on the audio playback rate.

[0063] The speed change process may include acceleration process, deceleration process and non-speed change process.

[0064] Optionally, the audio decoded data after speed change processing is stored in a buffer area to be played.

[0065] In some embodiments, when audio acceleration is required, the audio receiving end calls the acceleration function to accelerate the decoded PCM data, and stores the accelerated PCM data in the buffer zone to be played; when audio deceleration is required, the deceleration function is called to decelerate the PCM data, and stores the decelerated PCM data in the buffer zone to be played; when audio speed change is not required (i.e., playback at the original speed), the decoded PCM data is directly stored in the buffer zone to be played.

[0066] Step 304: Play audio based on the audio decoded data after speed change processing.

[0067] In some embodiments, the audio playback end obtains the audio decoding data after speed change processing from the buffer zone to be played, and performs audio playback.

[0068] To sum up, in the embodiment of the present application, by monitoring the jitter situation and channel interference situation in real time during the reception of audio data packets, and dynamically adjusting the audio playback rate of the jitter buffer according to the transmission jitter status and transmission channel interference status determined after monitoring, the jitter buffer size and the audio playback rate are adapted to the current transmission environment; the decoded audio data is subsequently processed and played at a variable speed based on the adjusted audio playback rate, which can counteract the influence of factors such as large jitter and sudden jitter on the reception delay of audio data packets, reduce the problem of audio playback jamming, and improve the smoothness and stability of audio playback at the receiving end.

[0069] As one of the core modules for implementing anti-jitter, the jitter interference monitor provides a basis for subsequent anti-jitter decisions. As shown in Figure 4, in a schematic application scenario, the jitter monitor calculates the reception interval based on the moment the audio data packet is received, thereby determining the jitter rate based on the reception interval. Furthermore, the monitor monitors interference sources based on changes in WiFi type and Bluetooth device status, and determines the state changes of the interference sources. The jitter monitor provides the jitter rate and interference source state changes to the anti-jitter module for its anti-jitter decision-making.

[0070] The following describes a process in which the interference monitor determines the transmission jitter state and the transmission channel interference state through exemplary embodiments.

[0071] As shown in FIG5 , the process of determining the transmission jitter state may include the following steps.

[0072] Step 501: determining a jitter type of transmission jitter based on a time interval between receiving adjacent audio data packets, where different jitter types correspond to different jitter degrees.

[0073] Each time an audio packet is received, the audio receiver records the time it was received and determines the interval between adjacent audio packets based on the time intervals between them. When jitter is low, the interval between adjacent audio packets is typically stable and low. However, when jitter is high, the interval between adjacent audio packets fluctuates significantly. Therefore, the audio receiver can determine the jitter type of the current transmission jitter based on this interval.

[0074] Optionally, the audio receiver determines the jitter type of the current transmission jitter according to the receiving time interval each time it receives an audio data packet. It may also determine the jitter type of the current transmission jitter according to the receiving time interval periodically, for example, every 100ms.

[0075] In some embodiments, multiple jitter types are obtained according to the jitter degree. For example, the jitter types may include normal jitter, small jitter, and large jitter, and the corresponding jitter degrees increase in sequence.

[0076] In a possible implementation, different jitter types may correspond to different receiving time interval ranges. Accordingly, the audio receiving end determines the jitter type of the transmission jitter based on the receiving time interval range to which the receiving time interval belongs.

[0077] In one possible design, the audio receiving end identifies the jitter type based on a first interval threshold and a second interval threshold, wherein the first interval threshold is less than the second interval threshold and is greater than an ideal receiving time interval, which can be determined based on the audio sending end's packetization duration and the audio data packet sending interval.

[0078] Optionally, the first interval threshold and the second interval threshold may be fixed values ​​or dynamic values.

[0079] Optionally, when the receiving time interval is less than the first interval threshold, the audio receiving end determines that the transmission jitter is normal jitter; when the receiving time interval is greater than the first interval threshold and less than the second interval threshold, the audio receiving end determines that the transmission jitter is small jitter; when the receiving time interval is greater than the second interval threshold, the audio receiving end determines that the transmission jitter is large jitter.

[0080] Of course, in addition to determining the jitter type based on the first interval threshold and the second interval threshold, the jitter type can also be more finely divided based on more interval thresholds, and the embodiments of the present application are not limited to this.

[0081] Step 502 : Determine a transmission jitter state based on the jitter occurrence frequencies of transmission jitters of different jitter types.

[0082] Furthermore, the audio receiving end may determine the occurrence frequencies of various types of transmission jitters according to the jitter types of the monitored transmission jitters, and further determine the transmission jitter state according to the jitter occurrence frequencies.

[0083] In some embodiments, the transmission jitter state may include a jitter rate, which may include at least one of jitter rates corresponding to different jitter types and a comprehensive jitter rate of different jitter types.

[0084] In one possible implementation, when the jitter types include small jitter and large jitter, the audio receiving end determines the small jitter rate based on the frequency of occurrence of small jitter, and determines the large jitter rate based on the frequency of occurrence of large jitter, thereby determining the transmission jitter state based on the large jitter rate and the small jitter rate.

[0085] Among them, the small jitter rate = the frequency of small jitter occurrence / the total number of received audio data packets, and the large jitter rate = the frequency of large jitter occurrence / the total number of received audio data packets.

[0086] Because different types of jitter have different impacts on audio playback stability, when determining the comprehensive jitter rate, it is necessary to set separate jitter impact weights for different types of jitter.

[0087] In one possible implementation, when the jitter types include small jitter and large jitter, the jitter impact weight of small jitter is greater than the jitter impact weight of large jitter. For example, the jitter impact weight of small jitter is 1, and the jitter impact weight of large jitter is 2. This embodiment does not limit the specific values ​​set for the jitter impact weights.

[0088] Accordingly, the audio receiving end determines the comprehensive jitter rate based on the occurrence frequency of small jitter, the occurrence frequency of large jitter, and the jitter influence weights corresponding to small jitter and large jitter, and thus determines the transmission jitter state based on the comprehensive jitter rate.

[0089] The comprehensive jitter rate = [(small jitter occurrence frequency * small jitter impact weight) + (large jitter occurrence frequency * large jitter impact weight)] / total number of received audio data packets.

[0090] Taking the example that the jitter types include large jitter and small jitter, and the transmission jitter status includes small jitter rate, large jitter rate and comprehensive jitter rate, as shown in FIG6 , determining the transmission jitter status may include the following steps.

[0091] 1. Receive audio data packets.

[0092] 2. Record the time of receipt.

[0093] 3. Is this the first time the audio data packet is received? If so, proceed to step 4; if not, proceed to step 5.

[0094] 4. Wait for the next reception.

[0095] 5. Calculate the time interval between receiving adjacent audio data packets.

[0096] 6. Update the total number of received messages.

[0097] 7. Determine whether the jitter is large based on the receiving time interval. If so, proceed to step 8; if not, proceed to step 9.

[0098] 8. Calculate the maximum jitter rate.

[0099] 9. Determine whether the jitter is small. If so, proceed to step 10.

[0100] 10. Calculate the small jitter rate.

[0101] 11. Update the comprehensive jitter value according to the jitter impact weights corresponding to the large jitter and the small jitter.

[0102] 12. Calculate the comprehensive jitter rate based on the comprehensive jitter number.

[0103] In addition to determining the above-mentioned jitter rate as the transmission jitter state, in other possible implementations, the audio receiving end can also determine the real-time jitter (i.e., the real-time receiving time interval) as the transmission jitter state, and can also perform transmission quality evaluation based on the real-time jitter rate, and determine the transmission quality evaluation as the transmission jitter state, and can also perform trend prediction based on the change of the jitter rate, and determine the jitter change trend as the transmission jitter state. This embodiment does not limit this.

[0104] Of course, the jitter interference monitor can also directly provide data such as jitter rate and real-time jitter to the subsequent anti-jitter module, which will further process the data to obtain the above-mentioned transmission quality evaluation and jitter change trend information. This embodiment does not limit this.

[0105] As shown in FIG7 , the process of determining the transmission channel interference state may include the following steps.

[0106] Step 701: Determine the interference source state change of the interference source.

[0107] In a possible implementation, the audio receiving end monitors the state of the interference source to obtain a state change of the interference source, wherein the interference source refers to an object that interferes with the transmission channel (a channel for transmitting audio data packets).

[0108] Optionally, the interference source state change may include a newly added interference source, a disappeared interference source, etc. Wherein, when a newly added interference source occurs, the interference to the transmission channel will be increased, and when the interference source occurs, the interference to the transmission channel will be reduced.

[0109] Since the frequency bands of the WiFi channel and the Bluetooth channel overlap, they may interfere with each other. Therefore, in some embodiments, when audio data packets are transmitted through the Bluetooth channel (Bluetooth near-field transmission scenario) or when audio data packets are transmitted through the WiFi channel (WebRTC scenario), determining the change in the interference source state may include at least one of the following.

[0110] 1. Determine the change of the WiFi type. The change of the WiFi type is used to represent the change of the frequency band of the WiFi channel.

[0111] In one possible implementation, the audio receiving end monitors whether the frequency band of the WiFi channel has changed to obtain a WiFi type change. Optionally, the WiFi type change may include a change from 2.4 GHz to another frequency band (e.g., 5 GHz, 6 GHz), or a change from another frequency band to 2.4 GHz. When WiFi changes from 2.4 GHz to another frequency band, interference to the transmission channel is reduced, and when WiFi changes from another frequency band to 2.4 GHz, interference to the transmission channel is increased.

[0112] 2. Determine the device access and removal status of the Bluetooth device. The device access and removal status is used to represent the connection establishment and disconnection status between the receiving end and other Bluetooth devices.

[0113] In one possible implementation, the audio receiving end detects the connection status with other Bluetooth devices to obtain the device access and removal status. Optionally, the device access and removal status includes establishing a connection with the other Bluetooth device or disconnecting a connection with the other Bluetooth device. When establishing a connection with the other Bluetooth device, interference to the transmission channel increases, and when disconnecting from the other Bluetooth device, interference to the transmission channel decreases.

[0114] In some embodiments, the above determination of WiFi type change and device access removal can be performed periodically, for example, the audio receiving end performs the above process every 5 seconds.

[0115] Step 702: Determine the interference state of the transmission channel based on the change in the interference source state.

[0116] When a change in the interference source state is detected, the audio receiving end determines the transmission channel interference state based on the change in the interference source state, and the transmission channel interference state is subsequently provided to the anti-jitter module for anti-jitter decision-making.

[0117] It should be noted that when the interference source state does not change, the audio receiving end will not transmit the channel interference state to the subsequent anti-jitter module.

[0118] In an illustrative example, the transmission channel interference state determined by the audio receiving end includes: WiFi is converted to 2.4 GHz, and a connection is established with Bluetooth device A.

[0119] In this embodiment, the jitter interval during the transmission of audio data packets is monitored in real time to determine the jitter state of the transmission channel, and the interference source is monitored in real time to determine the state change of the interference source, thereby providing a basis for the subsequent quality evaluation of the transmission channel in terms of both jitter and interference conditions. This is conducive to the subsequent dynamic adjustment of the anti-jitter strategy based on the quality of the transmission channel, thereby realizing adaptive anti-jitter in the real-time audio transmission process.

[0120] As one of the core modules for implementing anti-jitter, the anti-jitter module can dynamically determine the size of the jitter buffer and the audio playback rate suitable for the current transmission environment based on the monitoring results of the jitter interference monitor. As shown in Figure 8, the anti-jitter module of the audio receiving end can determine the jitter buffer reference value suitable for the current transmission channel interference environment based on the expected anti-jitter target and the transmission channel interference status. Based on the jitter buffer reference value, the anti-jitter module can determine the dynamic jitter buffer (that is, the size of the jitter buffer changes dynamically based on the jitter status) based on the jitter buffer reference value according to the transmission interference status, thereby realizing adaptive adjustment of the jitter buffer size. Furthermore, the anti-jitter module dynamically determines the audio playback rate based on the real-time transmission jitter status and the current dynamic jitter buffer, taking into account the stability and delay of the audio playback.

[0121] FIG9 is a flowchart of a jitter buffer and a process for dynamically adjusting the audio playback rate, shown in an exemplary embodiment of the present application. The process may include the following steps:

[0122] Step 901: Determine a jitter buffer reference value based on a transmission channel interference state.

[0123] The jitter buffer reference value in the embodiment of the present application is used to indicate the upper limit of the jitter buffer's anti-jitter capability, and can have the effect of limiting the upper limit of the jitter buffer size. That is, when the jitter buffer is subsequently dynamically adjusted, it is necessary to ensure that the anti-jitter capability of the adjusted jitter buffer is within the anti-jitter capability represented by the jitter buffer reference value, thereby avoiding the problem of excessive latency caused by unlimited increase of the jitter buffer.

[0124] In some embodiments, the jitter buffer baseline value is related to the jitter resistance requirement of the current scenario and the state of interference sources in the current environment. Specifically, the higher the jitter resistance requirement of the current scenario, the higher the jitter buffer baseline value; and the more interference sources there are in the current environment, the higher the jitter buffer baseline value.

[0125] In one possible implementation, the audio receiver sets an expected jitter resistance target for the current audio transmission scenario. The expected jitter resistance target refers to the degree of jitter resistance expected to be achieved in the scenario. For example, an expected jitter resistance target of 200ms indicates that the receiver is expected to be able to withstand approximately 200ms of transmission jitter in the scenario.

[0126] In some embodiments, the audio receiving end stores a correspondence between different audio transmission scenarios and expected anti-jitter targets, wherein the expected anti-jitter targets can be determined based on the anti-jitter capability requirements of the scenarios.

[0127] In an illustrative example, in a Bluetooth audio playback scenario, the determined expected anti-jitter target is 100ms; in an audio and video call scenario, the determined expected anti-jitter target is 200ms.

[0128] In a possible implementation, determining the jitter buffer reference value may include the following sub-steps:

[0129] Step 901A: Adjust the expected anti-jitter target based on the transmission channel interference status.

[0130] In some embodiments, the audio receiving end adjusts the expected anti-jitter target based on the change in the interference source state represented by the interference state of the transmission channel. If the change in the interference source state represents a decrease in interference, the expected anti-jitter target may be adjusted by lowering the expected anti-jitter target; if the change in the interference source state represents an increase in interference, the expected anti-jitter target may be adjusted by raising the expected anti-jitter target.

[0131] Since there may be multiple interference sources and the changes in the states of different interference sources are different, in one possible implementation, when the transmission channel interference state includes at least two sub-interference states, the audio receiving end adjusts the expected anti-jitter target based on the jitter impact amount corresponding to each of the at least two sub-interference states, where different sub-interference states correspond to different interference factors.

[0132] Optionally, a mapping relationship between different sub-interference states and jitter impact amounts is provided. Wherein, when the sub-interference state indicates an increase in interference, the corresponding jitter impact amount is a positive value and is positively correlated with the jitter impact amount and the interference increase amount; and when the sub-interference state indicates a decrease in interference, the corresponding jitter impact amount is a negative value and is positively correlated with the jitter impact amount and the interference decrease amount.

[0133] Optionally, when there are at least two sub-interference states, the audio receiving end can determine a comprehensive jitter influence amount based on the jitter influence amounts corresponding to the at least two sub-interference states, and then adjust the expected anti-jitter target based on the comprehensive jitter influence amount.

[0134] Optionally, the comprehensive jitter influence amount can be the sum of different jitter influence amounts, the weighted sum of different jitter influence amounts, the maximum value among different jitter influence amounts, or the median value among different jitter influence amounts, etc. The embodiment of the present application does not limit the specific method for determining the comprehensive jitter influence amount.

[0135] In an illustrative example, in a Bluetooth near-field transmission or WebRTC transmission scenario, the corresponding relationship between different sub-interference states and jitter impact amounts is shown in Table 1.

[0136] Table 1

[0137] When the transmission channel interference state includes the sub-interference states "WiFi changes to 2.4GHz" and "Connect Bluetooth device," and the expected anti-jitter target is 200ms, the audio receiver determines the comprehensive jitter impact (the maximum value among the different jitter impacts) as +50ms, and then adjusts the expected anti-jitter target to 250ms.

[0138] Step 901B: Determine a jitter buffer reference value based on the adjusted expected anti-jitter target.

[0139] In a possible implementation, the audio receiving end determines the expected anti-jitter target as the jitter buffer reference value, or performs a preset offset (i.e., appropriately increases or decreases) based on the expected jitter target to obtain the jitter buffer reference value.

[0140] In order to take into account the delay requirements, in one possible implementation, the audio receiving end determines the jitter buffer reference value based on the delay requirements of the current audio transmission scenario and the adjusted expected anti-jitter target. Optionally, the jitter buffer reference value is not greater than the upper limit of the delay of the current scenario. For example, when the adjusted expected anti-jitter target is less than or equal to the upper limit of the delay of the current audio transmission scenario, the audio receiving end determines the adjusted expected anti-jitter target as the jitter buffer reference value; when the adjusted expected anti-jitter target is greater than the upper limit of the delay of the current audio transmission scenario, the audio receiving end determines the upper limit of the delay as the jitter buffer reference value.

[0141] In an illustrative example, in a Bluetooth audio playback scenario (with lower latency requirements), when the adjusted expected anti-jitter target is 200ms and the upper latency limit for Bluetooth audio playback is 300ms, the audio receiver determines the jitter buffer base value to be 200ms. In an audio and video call scenario (with higher latency requirements), when the adjusted expected anti-jitter target is 250ms and the upper latency limit for Bluetooth audio playback is 200ms, the audio receiver determines the jitter buffer base value to be 200ms.

[0142] Step 902: Based on the jitter buffer reference value, dynamically adjust the jitter buffer according to the transmission jitter state.

[0143] Because jitter variations in actual scenarios are complex and diverse, to avoid the inability of a fixed-size jitter buffer to adapt to different jitter scenarios, the audio receiver dynamically adjusts the jitter buffer size based on the transmission jitter status based on the jitter buffer baseline value.

[0144] To avoid buffer adjustment lag, the audio receiver can predict the jitter situation according to the transmission jitter state, obtain the jitter change trend, and then adjust the jitter buffer size in advance based on the jitter change trend.

[0145] In a possible implementation, dynamically adjusting the dynamic buffer may include the following sub-steps.

[0146] Step 902A: Determine a jitter buffer range based on the jitter buffer reference value, wherein the maximum value of the jitter buffer range is not greater than the jitter buffer reference value.

[0147] To prevent the jitter buffer from being oversized or undersized, in some embodiments, the audio receiver determines a jitter buffer range based on a jitter buffer baseline value and subsequently dynamically adjusts the dynamic buffer size within this range. The maximum value of the determined jitter buffer range must be less than or equal to the jitter buffer baseline value to avoid excessive latency caused by an oversized jitter buffer.

[0148] Regarding the method of determining the jitter buffer range, in one possible implementation, the audio receiving end uses the jitter buffer reference value as the maximum value of the jitter buffer range, and sets the minimum value of the jitter buffer range based on the minimum anti-jitter capability requirement of the current audio transmission scenario.

[0149] Of course, in other possible implementations, the audio receiving end may also decrement the jitter buffer base value twice (for example, by 5% and 50% respectively) to obtain the maximum and minimum values ​​of the jitter buffer range, respectively. The embodiments of the present application do not limit the specific method of setting the jitter buffer range.

[0150] Step 902B: dynamically adjust the jitter buffer within the jitter buffer range based on the jitter variation trend represented by the transmission jitter state.

[0151] In order to adjust the dynamic buffer in advance to cope with the upcoming sudden jitter, the audio receiver adjusts the jitter buffer size in advance based on the jitter change trend represented by the transmission jitter state.

[0152] The jitter variation trend can be predicted by the anti-jitter module based on the jitter rate provided by the jitter interference monitor, or can be directly obtained from the transmission jitter state provided by the jitter interference monitor (ie, predicted by the jitter interference monitor based on the jitter rate).

[0153] Optionally, the jitter variation trend can be predicted based on continuous historical jitter rates. For example, the jitter variation trend can be predicted based on changes in historical jitter rates over 10 consecutive monitoring periods.

[0154] Optionally, the jitter change trend may include improvement (i.e., jitter reduction), deterioration (i.e., jitter increase), and no change. Of course, the jitter change trend may further include the degree of change (which may be an amplitude or level), such as the jitter increase amplitude or level, the jitter reduction amplitude or level, etc., to provide a more detailed basis for subsequent jitter buffer adjustment, which is not limited in this embodiment.

[0155] In a possible implementation, the audio receiving end determines an adjustment amount based on a jitter change trend, and dynamically adjusts the size of the jitter buffer based on the adjustment amount on the basis of a current jitter buffer.

[0156] Optionally, a correspondence between a jitter change trend and an adjustment amount is provided in the audio receiving end, so that the adjustment amount is determined based on the correspondence. Wherein, when the jitter change trend indicates jitter reduction, the corresponding adjustment amount is a negative value (i.e., reducing the size of the jitter buffer), and when the jitter degree is included, the adjustment amount is positively correlated with the jitter reduction degree (i.e., the higher the jitter reduction degree, the greater the reduction in the jitter buffer); when the jitter change trend indicates jitter increase, the corresponding adjustment amount is a positive value (i.e., increasing the size of the jitter buffer), and when the jitter degree is included, the adjustment amount is positively correlated with the jitter increase degree (i.e., the higher the jitter increase degree, the greater the increase in the jitter buffer).

[0157] In an illustrative example, when the jitter variation trend indicates that the jitter is weakening, the audio receiving end reduces the jitter buffer from 100ms to 90ms; when the jitter variation trend indicates that the jitter is increasing, the audio receiving end increases the jitter buffer from 100ms to 110ms.

[0158] In another illustrative example, when the jitter change trend indicates jitter reduction and the reduction level is level 1, the audio receiving end reduces the jitter buffer from 100ms to 90ms; when the jitter change trend indicates jitter reduction and the reduction level is level 2, the audio receiving end reduces the jitter buffer from 100ms to 80ms.

[0159] In conjunction with the description of steps 901 to 902 above, FIG10 illustrates a schematic diagram of the implementation of the dynamic jitter buffer adjustment process. The audio receiver monitors changes in WiFi type and Bluetooth device status. When the WiFi frequency band changes to 2.4 GHz, the jitter impact amount 1 is added to the expected anti-jitter target. When a Bluetooth device is connected, the jitter impact amount 2 is added to the expected anti-jitter target. The maximum adjusted expected jitter target is then used as the jitter buffer baseline value.

[0160] The audio receiver determines the combined jitter rate based on the monitored large and small jitter rates, and predicts the jitter trend based on historical combined jitter rates. If the jitter trend indicates improved transmission quality (i.e., reduced jitter), the jitter buffer is reduced; if the jitter trend indicates deterioration (i.e., increased jitter), the jitter buffer is increased (to a value not exceeding the jitter buffer baseline); if the jitter trend indicates unchanged transmission quality (i.e., stable jitter), the jitter buffer is maintained, enabling dynamic jitter buffer adjustment.

[0161] Step 903 : Dynamically adjust the audio playback rate based on the transmission jitter status, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played.

[0162] In some scenarios, when transmission quality improves from poor to good, the audio data packets stored in the previously large jitter buffer can increase playback delay. Furthermore, when sudden high jitter occurs, the audio data packets stored in the jitter buffer may not be sufficient to compensate for the jitter, resulting in playback lag. To further balance latency and improve resilience to sudden high jitter, the audio receiver dynamically adjusts the audio playback rate rather than using a fixed rate for audio playback. This variable-speed playback reduces latency and mitigates sudden high jitter.

[0163] The audio playback rate can be adjusted in various ways, including constant speed, acceleration, and deceleration. Furthermore, to avoid audio distortion caused by excessive speed changes, the audio receiving end pre-sets several candidate audio playback rates and dynamically adjusts the audio playback rate between the candidate audio playback rates.

[0164] In an illustrative example, the candidate audio playback rates set by the audio receiving end include: super fast deceleration (TOP_DECELERATE) = 0.7, fast deceleration (FAST_DECELERATE) = 0.8, deceleration (DECELERATE) = 0.9, normal rate = 1.0, acceleration (ACCELERATE) = 1.1, and fast acceleration (FAST_ACCELERATE) = 1.2.

[0165] In a possible implementation, the audio receiving end dynamically adjusts the audio playback rate based on the transmission jitter status, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played.

[0166] The transmission jitter state used to determine the audio playback rate can include real-time jitter and jitter rate (comprehensive jitter rate). The undecoded audio duration is the audio duration corresponding to the audio data packets to be decoded cached in the jitter buffer, and the to-be-played audio duration is the audio duration of the audio data decoded after speed change in the to-be-played buffer.

[0167] Because sudden jitter significantly impacts playback stability, appropriate deceleration can be used to extend the playback duration of buffered audio in the presence of sudden jitter. Accelerated playback is typically used to reduce latency when jitter is stable. Therefore, to prioritize audio playback stability, in one possible implementation, the audio receiver prioritizes deceleration over acceleration.

[0168] As shown in Figure 11, after the audio receiving end dynamically adjusts the jitter buffer, it is determined whether to play audio at a fixed rate. wherein, in order to ensure the audio playback quality in the initial stage, a fixed rate is usually adopted in the initial stage for audio playback (for example, audio is played at a fixed rate of 0.88). when a fixed rate is not adopted, it is further determined whether to decelerate by the deceleration decision module, and a deceleration strategy is output when deceleration is needed. if deceleration is not performed (i.e., the deceleration decision module has no strategy output), it is further determined whether to accelerate, and an acceleration strategy is output when acceleration is needed. the audio playback rate indicated by the variable speed strategy obtained will be input into the playback control module, which realizes variable speed playback based on the audio playback rate.

[0169] Of course, in other possible ways, the audio receiving end may also make acceleration and deceleration decisions in parallel, which is not limited in this embodiment.

[0170] In a possible implementation, dynamically adjusting the audio playback rate may include the following sub-steps.

[0171] Step 903A: Make a deceleration decision based on the transmission jitter status, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played.

[0172] In some scenarios, when sudden high jitter occurs, the time interval between receiving adjacent audio packets is too long, which may cause audio playback to freeze. In other scenarios, when the undecoded audio duration corresponding to the undecoded audio packets in the jitter buffer is too short, audio playback may also freeze. To address these scenarios where audio playback freezes, the audio receiver can reduce the playback speed by appropriately extending the playable audio duration to reduce the occurrence of freezes in these scenarios.

[0173] Therefore, in some embodiments, the audio receiving end can determine a deceleration strategy suitable for the current scenario based on the real-time jitter represented by the transmission jitter state, the size of the current jitter buffer, the length of the undecoded audio corresponding to the undecoded audio data packet, and the length of the audio to be played in the buffer to be played after decoding and speed change processing.

[0174] In one possible real-time approach, the deceleration decision process may include the following sub-steps.

[0175] 1. Based on the jitter interval level of the real-time jitter represented by the transmission jitter state, the undecoded audio duration level of the undecoded audio duration, and the jitter buffer, a deceleration decision is made to obtain an initial deceleration strategy.

[0176] In order to improve the ability to resist sudden jitter, in some embodiments, the audio receiving end is set with several jitter interval levels (corresponding to different jitter degrees) and determines the jitter interval level to which the real-time jitter belongs, that is, determines the real-time jitter degree.

[0177] For example, five jitter interval levels are set, namely SUPER_HIGH=200ms, HIGH_INTERVAL=120ms, MEDIUM_INTERVAL=80ms, RISKY_INTERVAL=60ms, and NORMAL_INTERVAL=40ms.

[0178] When the real-time jitter interval level exceeds the level threshold (i.e., the real-time jitter is severe), the audio receiver makes a deceleration decision based on the real-time jitter interval level, resulting in an initial deceleration strategy. When the real-time jitter interval level is less than the level threshold, the audio receiver further makes a deceleration decision based on the duration of the undecoded audio and the size of the jitter buffer (to avoid lag caused by short undecoded audio duration), resulting in an initial deceleration strategy.

[0179] In some embodiments, the audio receiving end is provided with a plurality of undecoded audio duration levels, and determines the undecoded audio duration level to which the current undecoded audio duration belongs.

[0180] For example, six undecoded audio duration levels are set, namely TOP_HIGH_LEVEL=300, SUPER_HIGH_LEVEL=200, HIGHER_LEVEL=160, HIGH_LEVEL=120, MEDIUM_LEVEL=80, and LOW_LEVEL=40.

[0181] When the real-time undecoded audio duration level is greater than the first level threshold (i.e., the undecoded audio duration is too long), the audio receiving end determines that an acceleration decision is required. When the real-time undecoded audio duration level is less than the first level threshold and greater than the second level threshold (i.e., the undecoded audio duration is moderate), the audio receiving end makes a deceleration decision based on the real-time undecoded audio duration level and the jitter buffer (determining whether the undecoded audio duration matches the jitter buffer size), obtaining an initial deceleration decision. When the real-time undecoded audio duration level is less than the second level threshold (i.e., the undecoded audio duration is too short), a deceleration decision is made based on the real-time undecoded audio duration level.

[0182] Optionally, the initial deceleration strategy includes an initial deceleration rate, and the initial deceleration rate belongs to a preset candidate deceleration rate, such as TOP_DECELERATE=0.7, FAST_DECELERATE=0.8, and DECELERATE=0.9.

[0183] 2. Based on the duration of the undecoded audio and the duration of the audio to be played, determine the duration of the playable audio under the initial deceleration strategy.

[0184] After determining the initial deceleration strategy, in order to evaluate whether the initial deceleration strategy is effective, the audio receiving end determines the playable audio duration under the initial deceleration strategy based on the undecoded audio duration, the initial deceleration rate included in the initial deceleration strategy, and the duration of the audio to be played.

[0185] In some embodiments, the playable audio duration under the initial deceleration strategy = (undecoded audio duration / initial deceleration rate) + to-be-played audio duration.

[0186] For example, when the undecoded audio duration is 200ms, the audio to be played duration is 5ms, and the initial deceleration rate is 0.8, the playable audio duration under the initial deceleration strategy is 255ms.

[0187] 3. Based on the playable audio duration, real-time jitter, and jitter buffer, a deceleration decision is made to obtain a target deceleration strategy. The target deceleration strategy may be the same as or different from the initial deceleration strategy, and different deceleration strategies correspond to different deceleration rates.

[0188] In the embodiment of the present application, to avoid excessive deceleration, the audio receiving end uses a two-stage deceleration decision. During the initial deceleration decision, the audio receiving end determines a larger initial deceleration rate, and then makes a second-stage deceleration decision based on the expected deceleration effect of the initial deceleration rate. If the expected deceleration effect of the initial deceleration decision is poor, a smaller deceleration rate is output during the second-stage deceleration decision. If the expected deceleration effect of the initial deceleration decision is good, the initial deceleration rate is used during the second-stage deceleration decision.

[0189] In one possible implementation, when making a two-stage deceleration decision, the audio receiving end makes a deceleration decision based on the relationship between the playable audio duration and the real-time jitter, and the relationship between the playable audio duration and the size of the jitter buffer.

[0190] Combining the examples in the above steps, the deceleration decision process is shown in Figure 12.

[0191] Step 1201: Check whether the real-time jitter is greater than or equal to the SUPER_HIGH interval.

[0192] Step 1202: If the real-time jitter is greater than or equal to the SUPER_HIGH interval, determine whether the playable audio duration after fast deceleration is less than the jitter buffer, or whether the playable audio duration is less than the real-time jitter. If the playable audio duration is less than the jitter buffer, or the playable audio duration is less than the real-time jitter, then perform super-fast deceleration. If the playable audio duration is greater than the jitter buffer, and the playable audio duration is greater than the real-time jitter, then perform fast deceleration.

[0193] Step 1203: Check whether the real-time jitter is greater than or equal to the HIGH_INTERVAL interval.

[0194] Step 1204: If the real-time jitter is greater than or equal to the HIGH_INTERVAL interval, determine whether the duration of the playable audio after fast deceleration is less than the jitter buffer. If the duration of the playable audio is less than the jitter buffer, perform ultra-fast deceleration. If the duration of the playable audio is greater than the jitter buffer, perform fast deceleration.

[0195] Step 1205: Check whether the real-time jitter is greater than or equal to the MEDIUM_INTERVAL interval.

[0196] Step 1206: If the real-time jitter is greater than or equal to the MEDIUM_INTERVAL interval, determine whether the playable audio duration after deceleration is less than the jitter buffer or less than the real-time jitter. If the playable audio duration is less than the jitter buffer or less than the real-time jitter, then perform a fast deceleration. If the playable audio duration is greater than the jitter buffer and greater than the real-time jitter, then perform a deceleration.

[0197] Step 1207 : Check whether the real-time jitter is greater than or equal to the RISKY_INTERVAL interval, and whether the duration of the undecoded audio is greater than HIGH_LEVEL.

[0198] Step 1208: If the real-time jitter is greater than or equal to the RISKY_INTERVAL interval and the undecoded audio duration is greater than HIGH_LEVEL, determine that after deceleration, the playable audio duration is less than the jitter buffer or less than the real-time jitter. If the playable audio duration is less than the jitter buffer or less than the real-time jitter, then perform a fast deceleration. If the playable audio duration is greater than the jitter buffer and greater than the real-time jitter, then perform a deceleration.

[0199] Step 1209: Check whether the duration of the undecoded audio is less than LOW_LEVEL.

[0200] Step 1210: If the undecoded audio duration is less than LOW_LEVEL, determine whether the playable audio duration after fast deceleration is less than the jitter buffer. If the playable audio duration is less than the jitter buffer, perform ultra-fast deceleration. If the playable audio duration is not less than the jitter buffer, perform fast deceleration.

[0201] Step 1211, detect (the duration of undecoded audio < MEDIUM_LEVEL) || (the duration of undecoded audio < jitter buffer && the duration of undecoded audio < HIGH_LEVEL).

[0202] Step 1212, when (the duration of undecoded audio < MEDIUM_LEVEL) || (the duration of undecoded audio < jitter buffer && the duration of undecoded audio < HIGH_LEVEL), determine whether the duration of undecoded audio / fast deceleration is less than the jitter buffer. If the duration of undecoded audio / fast deceleration is less than the jitter buffer, perform fast deceleration.

[0203] Step 1213, if the duration of undecoded audio / fast deceleration is not less than the jitter buffer, determine whether the duration of undecoded audio / deceleration is less than the jitter buffer. If the duration of undecoded audio / deceleration is less than the jitter buffer, perform fast deceleration; if the duration of undecoded audio / deceleration is less than the jitter buffer, do not perform speed change.

[0204] Step 903B, when the target deceleration strategy is obtained, dynamically adjust the audio playback rate based on the target deceleration strategy.

[0205] In a possible implementation manner, when the target deceleration strategy indicates deceleration, the audio receiving end adjusts the audio playback rate to the target deceleration rate indicated by the target deceleration strategy. When the target deceleration strategy indicates no speed change, the audio receiving end adjusts the audio playback rate to the default audio playback rate.

[0206] Step 903C, when the target deceleration strategy is not obtained, make an acceleration decision based on the transmission jitter state, the jitter buffer, and the duration of undecoded audio.

[0207] When the target deceleration strategy is not obtained (that is, there is no output after making a deceleration decision, and it is considered that there is an output for the deceleration decision if the deceleration rate or no speed change is output), the audio receiving end makes an acceleration decision. For example, when the duration of undecoded audio is too long, since there is no output of the deceleration strategy, the audio receiving end needs to make an acceleration decision.

[0208] In some scenarios, when the transmission jitter state indicates slight jitter but the duration of undecoded audio is too long, if the default audio playback rate is maintained for audio playback, it will cause too high latency. At this time, appropriately increasing the audio playback rate can reduce the latency without affecting the playback stability.

[0209] In some other scenarios, when the duration of undecoded audio corresponding to the undecoded audio packet is greater than the real-time jitter buffer, the audio receiving end can appropriately increase the audio playback rate to make the duration of undecoded audio match the real-time jitter buffer.

[0210] In order to reduce latency and match the undecoded audio duration with the jitter buffer, in one possible implementation, the audio receiving end makes an acceleration decision based on the undecoded audio duration level to which the undecoded audio duration belongs, the jitter level to which the jitter rate represented by the transmission jitter state belongs, the jitter interval level to which the real-time jitter belongs, and the jitter buffer, and obtains a target acceleration strategy, where different acceleration strategies correspond to different acceleration rates.

[0211] Optionally, the target acceleration strategy includes a target acceleration rate, and the target acceleration rate belongs to a preset candidate acceleration rate, for example, the preset candidate acceleration rates include acceleration (ACCELERATE) = 1.1 and fast acceleration (FAST_ACCELERATE) = 1.2.

[0212] Optionally, the audio receiving end is set with several jitter levels, and determines the jitter level to which the current jitter rate belongs.

[0213] For example, six jitter levels are set, namely NORMAL_JITTER_RATIO=0.008, LOW_JITTER_RATIO=0.01, MEDIUM_JITTER_RATIO=0.02, HIGH_JITTER_RATIO=0.04, SUPER_JITTER_RATIO=0.1, and BIG_JITTER_RATIO=0.05.

[0214] Combining the examples in the above steps, the accelerated decision-making process is shown in Figure 13.

[0215] Step 1301: When the duration of the undecoded audio is greater than or equal to TOP_HIGH_LEVEL, perform fast acceleration.

[0216] Step 1302: Check whether the undecoded audio duration ≥ SUPER_HIGH_LEVEL is satisfied.

[0217] Step 1303 : If the undecoded audio duration is ≥ SUPER_HIGH_LEVEL, it is detected whether the real-time jitter ≤ NORMAL_INTERVAL interval and the undecoded audio duration is greater than the jitter buffer.

[0218] Step 1304: If the real-time jitter does not meet the NORMAL_INTERVAL interval and the undecoded audio duration is greater than the jitter buffer, check whether the jitter rate < MEDIUM_RATIO || real-time jitter interval ≤ MEDIUM_INTERVEL. If not, maintain the previous audio playback rate. If so, accelerate.

[0219] Step 1305, detect whether the duration of the undecoded audio is greater than the jitter buffer. If not, maintain the previous audio playback rate.

[0220] Step 1306, when the duration of the undecoded audio is greater than the jitter buffer, detect whether the duration of the undecoded audio is greater than or equal to HIGHER_LEVEL.

[0221] Step 1307, if the duration of the undecoded audio is greater than or equal to HIGHER_LEVEL, detect whether the real-time jitter is less than or equal to NORMAL_INTERVEL.

[0222] Step 1308, if the real-time jitter is less than or equal to NORMAL_INTERVEL, detect whether the jitter rate is less than or equal to LOW_RATIO. If so, perform fast acceleration.

[0223] Step 1309, if the jitter rate is not less than or equal to LOW_RATIO, detect whether the duration of the undecoded audio is greater than the jitter buffer * acceleration. If so, perform acceleration.

[0224] Step 1310, if the duration of the undecoded audio is not greater than or equal to HIGHER_LEVEL, detect whether the duration of the undecoded audio is greater than MEDIUM_LEVEL.

[0225] Step 1311, if the duration of the undecoded audio is not greater than MEDIUM_LEVEL, detect whether the duration of the undecoded audio is less than the jitter buffer * acceleration && the previous time was acceleration. If so, maintain the previous audio playback rate. If not, do not change the speed.

[0226] Step 1312, if the duration of the undecoded audio is greater than MEDIUM_LEVEL, detect whether the real-time jitter < NORMAL_INTERVEL && the jitter rate < MEDIUM_RATIO. If so, perform acceleration.

[0227] Step 1313, if the real-time jitter < NORMAL_INTERVEL && the jitter rate < MEDIUM_RATIO is not satisfied, detect whether the duration of the undecoded audio is greater than the jitter buffer && the real-time jitter is less than or equal to NORMAL_INTERVEL. If so, perform acceleration.

[0228] Step 1314, if the duration of the undecoded audio is greater than the jitter buffer && the real-time jitter is less than or equal to NORMAL_INTERVEL is not satisfied, detect whether the duration of the undecoded audio is greater than the jitter buffer * fast acceleration. If so, perform fast acceleration.

[0229] Step 1315: If the condition that the undecoded audio duration is greater than the jitter buffer * fast acceleration is not satisfied, then a check is performed to determine whether the condition that the real-time jitter is less than the medium_interval and the jitter rate is less than the low_rate is satisfied. If so, fast acceleration is performed.

[0230] In step 1316, if the real-time jitter < MEDIUM_INTERVEL && jitter rate < LOW_RATIO is not satisfied, check whether the last time was a deceleration. If not, maintain the last audio playback rate; if deceleration was the last time, do not change the speed.

[0231] Step 1317: If the real-time jitter ≤ NORMAL_INTERVEL is not satisfied, then check whether the real-time jitter ≤ MEDIUM_INTERVEL is satisfied.

[0232] Step 1318: If the real-time jitter ≤ MEDIUM_INTERVEL is not satisfied, then check whether the jitter rate ≤ LOW_RATIO is satisfied. If so, perform rapid acceleration.

[0233] Step 1319: If the jitter rate ≤ LOW_RATIO is not satisfied, then check whether the jitter rate ≤ HIGH_RATIO || undecoded audio duration ≥ HIGH_LEVEL is satisfied. If so, speed up the playback. If not, maintain the previous audio playback rate.

[0234] Step 1320: If the real-time jitter ≤ MEDIUM_INTERVEL is not satisfied, then check whether the jitter rate < LOW_RATIO is satisfied. If not, then maintain the last audio playback rate.

[0235] Step 903D: When the target acceleration strategy is obtained, the audio playback rate is dynamically adjusted based on the target acceleration strategy.

[0236] In one possible implementation, when the target acceleration policy indicates acceleration, the audio receiving end adjusts the audio playback rate to the target acceleration rate indicated by the target acceleration policy. When the target acceleration policy indicates no speed change, the audio receiving end adjusts the audio playback rate to the default audio playback rate. When the target acceleration policy indicates retention, the audio receiving end maintains the last audio playback rate.

[0237] In this embodiment, the audio receiving end dynamically adjusts the jitter buffer size while also dynamically determining whether to change the speed. In the presence of sudden large jitter, the audio playback rate can be reduced to avoid sudden large jitter jams. When the undecoded audio corresponding to the undecoded audio data packet is too long, the audio playback rate can be increased to speed up the audio playback and reduce audio playback delay.

[0238] In some embodiments, in order to avoid the jitter buffer being too small, resulting in an inability to cope with sudden large jitter, the audio receiving end can set the expected playback duration of the audio. Before performing speed-changing processing based on the adjusted audio playback rate, the playable audio duration can be determined based on the adjusted audio playback rate, the undecoded audio duration, and the audio duration to be played, thereby fine-tuning the audio playback rate based on the playable audio duration and the expected playback duration. Accordingly, the audio receiving end performs speed-changing processing on the audio decoded data based on the fine-tuned audio playback rate.

[0239] Among them, the playable audio duration = (undecoded audio duration / audio playback rate) + audio duration to be played.

[0240] Regarding the method for determining the expected playback time, in one possible method, the expected playback time is determined based on a jitter rate represented by a transmission jitter state, and the expected playback time is positively correlated with the jitter rate.

[0241] In an illustrative example, when the jitter rate is greater than 0.04, the expected playback duration is 70 ms; and when the jitter rate is greater than 0.02, the expected playback duration is 40 ms.

[0242] Furthermore, when the playable audio duration is less than the expected playback duration, the audio receiving end fine-tunes the audio playback rate, wherein the playable audio duration after the fine-tuned audio playback rate is not less than the expected playback duration, thereby maintaining a certain size of jitter buffer in a low jitter rate state to cope with sudden large jitter.

[0243] In an illustrative example, based on an audio acceleration rate of 1.2, an undecoded audio duration of 40ms, and an audio duration of 5ms to be played, it is determined that the playable audio duration is 38.3ms. Since the playable audio duration is less than the expected playback duration of 40ms, the audio receiving end appropriately reduces the audio playback rate based on the audio acceleration rate of 1.2, such as reducing the audio acceleration rate to 1.14, so that the fine-tuned playable audio duration is greater than 40ms.

[0244] After the speed change processing, the audio receiving end stores the speed-changed audio decoded data into the buffer to be played. During playback, if the audio duration of the audio decoded data in the buffer to be played is not less than the duration of a single frame, the audio receiving end extracts the single-frame audio decoded data from the buffer to be played and plays it. If the audio duration of the audio decoded data in the buffer to be played is not less than the duration of a single frame, the audio receiving end obtains an audio data packet from the jitter buffer and performs speed change processing on the audio data packet, thereby writing the speed-changed audio decoded data into the buffer to be played.

[0245] Schematically, as shown in FIG14 , the audio receiving end performs speed calculation on the undecoded audio duration based on the audio playback rate output by the anti-jitter module, and determines the playable audio duration based on the undecoded audio duration after speed change and the duration of the audio to be played. If the playable audio duration is longer than the expected playback duration, there is no need to fine-tune the audio playback rate; if the playable audio duration is shorter than the expected playback duration, the audio playback rate needs to be fine-tuned. The audio receiving end takes out the audio data packet from the jitter buffer for decoding, and performs speed change processing according to the determined audio playback rate, and stores the audio decoded data after speed change processing into the buffer to be played. When the audio duration of the audio to be played in the buffer to be played is less than the duration of a single frame, the audio receiving end continues to take out the audio data packet from the jitter buffer for decoding and speed change processing until the audio duration of the audio to be played in the buffer to be played is greater than the duration of a single frame, and then takes out the audio decoded data of the single frame duration for audio playback.

[0246] In this embodiment, the audio receiving end sets the expected playback time of the audio based on the real-time jitter rate, and when the playable audio time after speed change processing is less than the expected playback time, the audio playback rate is fine-tuned so that the playable audio time after fine-tuning is not less than the expected playback time. This avoids the problem of being unable to cope with sudden large jitter when the playable audio time is too short, and further ensures the stability of audio playback.

[0247] Referring to FIG15 , FIG15 is a block diagram of an audio playback device provided by an exemplary embodiment of the present application. The device includes:

[0248] The state determination module 1501 is used to determine the transmission jitter state and the transmission channel interference state during the process of receiving the audio data packet;

[0249] An adjustment module 1502 is configured to dynamically adjust a jitter buffer and an audio playback rate based on the transmission jitter state and the transmission channel interference state, wherein the jitter buffer is configured to cache the received audio data packets;

[0250] a speed changing module 1503, configured to perform speed changing processing on the audio decoded data based on the adjusted audio playback rate, the audio decoded data being obtained by decoding the audio data packet;

[0251] The playing module 1504 is configured to play audio based on the audio decoded data after speed change processing.

[0252] Optionally, the state determination module 1501 includes a first state determination unit configured to:

[0253] Determine the jitter type of transmission jitter based on the time interval between the reception of adjacent audio data packets, where different jitter types correspond to different jitter degrees;

[0254] The transmission jitter state is determined based on jitter occurrence frequencies of transmission jitters of different jitter types.

[0255] Optionally, the first state determining unit is configured to:

[0256] When the receiving time interval is less than the first interval threshold, determining that the transmission jitter is normal jitter;

[0257] When the receiving time interval is greater than the first interval threshold and less than the second interval threshold, determining that the transmission jitter is small jitter;

[0258] When the receiving time interval is greater than the second interval threshold, it is determined that the transmission jitter is large jitter.

[0259] Optionally, the first state determining unit is configured to:

[0260] Determining a small jitter rate based on a frequency of occurrence of small jitter; determining a large jitter rate based on a frequency of occurrence of large jitter; and determining the transmission jitter state based on the large jitter rate and the small jitter rate;

[0261] Based on the jitter occurrence frequency of small jitter, the jitter occurrence frequency of large jitter, and the jitter impact weights corresponding to small jitter and large jitter, the comprehensive jitter rate is determined, wherein the jitter impact weight of small jitter is greater than the jitter impact weight of large jitter; and the transmission jitter state is determined based on the comprehensive jitter rate.

[0262] Optionally, the state determination module 1501 includes a second state determination unit configured to:

[0263] Determine the change in interference source status of the interference source;

[0264] The transmission channel interference state is determined based on the change in the interference source state.

[0265] Optionally, when the audio data packet is transmitted through a WiFi channel or a Bluetooth channel, the second state determining unit is configured to:

[0266] Determine a change in the WiFi type, where the change in the WiFi type is used to characterize a change in the frequency band of the WiFi channel;

[0267] Determine the device access and removal status of the Bluetooth device, where the device access and removal status is used to represent the connection establishment and disconnection status between the receiving end and other Bluetooth devices.

[0268] Optionally, the adjustment module 1502 is configured to:

[0269] Determining a jitter buffer reference value based on the transmission channel interference state;

[0270] Dynamically adjusting the jitter buffer based on the transmission jitter state on the basis of the jitter buffer reference value;

[0271] The audio playback rate is dynamically adjusted based on the transmission jitter state, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played.

[0272] Optionally, the adjustment module 1502 is configured to:

[0273] adjusting an expected anti-jitter target based on the transmission channel interference state;

[0274] The jitter buffer reference value is determined based on the adjusted expected anti-jitter target.

[0275] Optionally, the adjustment module 1502 is configured to:

[0276] In a case where the transmission channel interference state includes at least two sub-interference states, the expected anti-jitter target is adjusted based on jitter impact amounts corresponding to the at least two sub-interference states, wherein different sub-interference states correspond to different interference factors.

[0277] Optionally, the adjustment module 1502 is configured to:

[0278] determining a jitter buffer range based on the jitter buffer reference value, wherein a maximum value of the jitter buffer range is not greater than the jitter buffer reference value;

[0279] The jitter buffer area is dynamically adjusted within the jitter buffer area based on a jitter variation trend represented by the transmission jitter state.

[0280] Optionally, the adjustment module 1502 is configured to:

[0281] Making a deceleration decision based on the transmission jitter state, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played;

[0282] When a target deceleration strategy is obtained, dynamically adjusting the audio playback rate based on the target deceleration strategy;

[0283] If no target deceleration strategy is obtained, making an acceleration decision based on the transmission jitter state, the jitter buffer, and the duration of the undecoded audio;

[0284] When a target acceleration strategy is obtained, the audio playback rate is dynamically adjusted based on the target acceleration strategy.

[0285] Optionally, the adjustment module 1502 is configured to:

[0286] Based on the jitter interval level to which the real-time jitter represented by the transmission jitter state belongs, the undecoded audio duration level to which the undecoded audio duration belongs, and the jitter buffer, a deceleration decision is made to obtain an initial deceleration strategy;

[0287] Determining a playable audio duration under the initial deceleration strategy based on the undecoded audio duration and the to-be-played audio duration;

[0288] A deceleration decision is made based on the playable audio duration, the real-time jitter, and the jitter buffer to obtain the target deceleration strategy. The target deceleration strategy is the same as or different from the initial deceleration strategy, and different deceleration strategies correspond to different deceleration rates.

[0289] Optionally, the adjustment module 1502 is configured to:

[0290] Based on the undecoded audio duration level to which the undecoded audio duration belongs, the jitter rate represented by the transmission jitter state, the jitter interval level to which the real-time jitter belongs, and the jitter buffer, an acceleration decision is made to obtain a target acceleration strategy, where different acceleration strategies correspond to different acceleration rates.

[0291] Optionally, the device further includes a fine-tuning module, configured to:

[0292] Determining a playable audio duration based on the adjusted audio playback rate, the undecoded audio duration, and the to-be-played audio duration;

[0293] Fine-tuning the audio playback rate based on the playable audio duration and the expected playback duration;

[0294] The speed change module 1503 is used to:

[0295] The audio decoding data is subjected to speed change processing based on the finely adjusted audio playback rate.

[0296] Optionally, the fine-tuning module is used to:

[0297] determining the expected playback duration based on a jitter rate represented by the transmission jitter state, wherein the expected playback duration is positively correlated with the jitter rate;

[0298] When the playable audio duration is less than the expected playback duration, fine-tune the audio playback rate, wherein after fine-tuning the audio playback rate, the playable audio duration is not less than the expected playback duration.

[0299] Optionally, the playing module 1504 is configured to:

[0300] Storing the audio decoded data after speed change processing into a buffer area to be played;

[0301] When the audio duration of the audio decoded data in the buffer to be played is not less than the duration of a single frame, a single frame of audio decoded data is extracted from the buffer to be played and played.

[0302] To sum up, in the embodiment of the present application, by monitoring the jitter situation and channel interference situation in real time during the reception of audio data packets, and dynamically adjusting the audio playback rate of the jitter buffer according to the transmission jitter status and transmission channel interference status determined after monitoring, so that the jitter buffer size and the audio playback rate are adapted to the current transmission environment; subsequently, the decoded audio data is processed and played at a variable speed based on the adjusted audio playback rate, which can counteract the influence of factors such as large jitter and sudden jitter on the reception delay of audio data packets, reduce the problem of audio playback jamming, and improve the smoothness and stability of audio playback at the receiving end.

[0303] It should be noted that the division of modules in the above-mentioned device is for illustrative purposes only. The device may be divided into more or fewer modules, or some modules may be further subdivided, depending on the functional implementation. This embodiment of the present application does not constitute a limitation on this. In addition, the process of implementing audio playback by the above-mentioned device can refer to the above-mentioned method embodiment, and this embodiment will not be repeated here.

[0304] Referring to Figure 16 , Figure 16 is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of the present application. This computer device can be implemented as the audio receiving end in the aforementioned embodiments. This computer device can be a smartphone, tablet computer, smart wearable device, etc. This computer device can also include one or more of the following components: a processor 1610 and a memory 1620.

[0305] Optionally, the processor 1610 utilizes various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1620, and calling data stored in the memory 1620. Optionally, the processor 1610 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1610 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), and a baseband chip. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; the NPU is used to implement artificial intelligence (AI) functions; and the baseband chip is used to handle wireless communication. It is understandable that the above-mentioned baseband chip may not be integrated into the processor 1610, but may be implemented by a separate chip.

[0306] The memory 1620 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1620 includes a non-transitory computer-readable storage medium. The memory 1620 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, a phone book), etc.

[0307] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal, and the terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0308] For example, the computer device may also be provided with a Bluetooth component to implement Bluetooth near-field audio transmission, and may be provided with a WiFi component to implement WebRTC transmission. In addition, the computer device may also have a built-in speaker or an external speaker to implement audio playback.

[0309] The present application also provides a computer-readable storage medium having at least one instruction stored therein, which is loaded and executed by a processor to implement the method described in the above embodiment. Optionally, the computer-readable storage medium may include: ROM, RAM, solid-state drive (SSD) or optical disk, etc. Among them, RAM may include resistance random access memory (ReRAM) and dynamic random access memory (DRAM).

[0310] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio playback method provided in various optional implementations of the above aspects.

[0311] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An audio playback method, the method being executed by a computer device, the method comprising: During the reception of the audio data packet, determining the transmission jitter status and the transmission channel interference status; Dynamically adjusting a jitter buffer and an audio playback rate based on the transmission jitter state and the transmission channel interference state, the jitter buffer being used to cache the received audio data packets; performing speed-varying processing on audio decoded data based on the adjusted audio playback rate, the audio decoded data being obtained by decoding the audio data packet; Audio playback is performed based on the audio decoded data after speed change processing.

2. The method according to claim 1, wherein The determining of the transmission jitter state includes: Determine the jitter type of transmission jitter based on the time interval between the reception of adjacent audio data packets, where different jitter types correspond to different jitter degrees; The transmission jitter state is determined based on jitter occurrence frequencies of transmission jitters of different jitter types.

3. The method according to claim 2, wherein: The determining the jitter type of transmission jitter based on the reception time interval of adjacent audio data packets includes: When the receiving time interval is less than the first interval threshold, determining that the transmission jitter is normal jitter; When the receiving time interval is greater than the first interval threshold and less than the second interval threshold, determining that the transmission jitter is small jitter; When the receiving time interval is greater than the second interval threshold, it is determined that the transmission jitter is large jitter.

4. The method according to claim 3, wherein: The determining of the transmission jitter state based on the jitter occurrence frequency of transmission jitters of different jitter types includes at least one of the following: Determining a small jitter rate based on a frequency of occurrence of small jitter; determining a large jitter rate based on a frequency of occurrence of large jitter; and determining the transmission jitter state based on the large jitter rate and the small jitter rate; Determining a comprehensive jitter rate based on the jitter occurrence frequency of small jitter, the jitter occurrence frequency of large jitter, and the jitter impact weights corresponding to the small jitter and the large jitter, wherein the jitter impact weight of small jitter is greater than the jitter impact weight of large jitter; The transmission jitter state is determined based on the comprehensive jitter rate.

5. The method according to any one of claims 1 to 4, wherein: The determining of the transmission channel interference state includes: Determine the change in interference source status of the interference source; The transmission channel interference state is determined based on the change in the interference source state.

6. The method according to claim 5, wherein: When the audio data packet is transmitted through a WiFi channel or a Bluetooth channel, the interference source state change of determining the interference source includes at least one of the following: Determine a change in the WiFi type, where the change in the WiFi type is used to characterize a change in the frequency band of the WiFi channel; Determine the device access and removal status of the Bluetooth device, where the device access and removal status is used to represent the connection establishment and disconnection status between the receiving end and other Bluetooth devices.

7. The method according to any one of claims 1 to 6, wherein: The dynamically adjusting the jitter buffer and the audio playback rate based on the transmission jitter state and the transmission channel interference state includes: Determining a jitter buffer reference value based on the transmission channel interference state; Dynamically adjusting the jitter buffer based on the transmission jitter state on the basis of the jitter buffer reference value; The audio playback rate is dynamically adjusted based on the transmission jitter state, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played.

8. The method according to claim 7, wherein: The determining a jitter buffer reference value based on the transmission channel interference state includes: adjusting an expected anti-jitter target based on the transmission channel interference state; The jitter buffer reference value is determined based on the adjusted expected anti-jitter target.

9. The method according to claim 8, wherein The adjusting the expected anti-jitter target based on the transmission channel interference state includes: In a case where the transmission channel interference state includes at least two sub-interference states, the expected anti-jitter target is adjusted based on jitter impact amounts corresponding to the at least two sub-interference states, wherein different sub-interference states correspond to different interference factors.

10. The method according to claim 7, wherein: The dynamically adjusting the jitter buffer based on the transmission jitter state on the basis of the jitter buffer reference value includes: determining a jitter buffer range based on the jitter buffer reference value, wherein a maximum value of the jitter buffer range is not greater than the jitter buffer reference value; The jitter buffer area is dynamically adjusted within the jitter buffer area based on a jitter variation trend represented by the transmission jitter state.

11. The method according to claim 7, wherein: The dynamically adjusting the audio playback rate based on the transmission jitter state, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played includes: Making a deceleration decision based on the transmission jitter state, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played; When a target deceleration strategy is obtained, dynamically adjusting the audio playback rate based on the target deceleration strategy; If no target deceleration strategy is obtained, making an acceleration decision based on the transmission jitter state, the jitter buffer, and the duration of the undecoded audio; When a target acceleration strategy is obtained, the audio playback rate is dynamically adjusted based on the target acceleration strategy.

12. The method according to claim 11, wherein The making a deceleration decision based on the transmission jitter state, the jitter buffer, the duration of the undecoded audio, and the duration of the audio to be played includes: Based on the jitter interval level to which the real-time jitter represented by the transmission jitter state belongs, the undecoded audio duration level to which the undecoded audio duration belongs, and the jitter buffer, a deceleration decision is made to obtain an initial deceleration strategy; Determining a playable audio duration under the initial deceleration strategy based on the undecoded audio duration and the to-be-played audio duration; A deceleration decision is made based on the playable audio duration, the real-time jitter, and the jitter buffer to obtain the target deceleration strategy. The target deceleration strategy is the same as or different from the initial deceleration strategy, and different deceleration strategies correspond to different deceleration rates.

13. The method according to claim 11, wherein The making an acceleration decision based on the transmission jitter state, the jitter buffer, and the undecoded audio duration includes: Based on the undecoded audio duration level to which the undecoded audio duration belongs, the jitter rate represented by the transmission jitter state, the jitter interval level to which the real-time jitter belongs, and the jitter buffer, an acceleration decision is made to obtain a target acceleration strategy, where different acceleration strategies correspond to different acceleration rates.

14. The method according to claim 7, wherein: The method further comprises: Determining a playable audio duration based on the adjusted audio playback rate, the undecoded audio duration, and the to-be-played audio duration; Fine-tuning the audio playback rate based on the playable audio duration and the expected playback duration; The variable speed processing of the audio decoded data based on the adjusted audio playback rate includes: The audio decoding data is subjected to speed change processing based on the finely adjusted audio playback rate.

15. The method according to claim 14, wherein The method further comprises: determining the expected playback duration based on a jitter rate represented by the transmission jitter state, wherein the expected playback duration is positively correlated with the jitter rate; The fine-tuning of the audio playback rate based on the playable audio duration and the expected playback duration includes: When the playable audio duration is less than the expected playback duration, fine-tune the audio playback rate, wherein after fine-tuning the audio playback rate, the playable audio duration is not less than the expected playback duration.

16. The method according to any one of claims 1 to 15, wherein: The audio playback based on the audio decoded data after the speed change processing includes: Storing the audio decoded data after speed change processing into a buffer area to be played; When the audio duration of the audio decoded data in the buffer to be played is not less than the duration of a single frame, a single frame of audio decoded data is extracted from the buffer to be played and played.

17. An audio playback device, comprising: A state determination module, configured to determine a transmission jitter state and a transmission channel interference state during reception of an audio data packet; an adjustment module, configured to dynamically adjust a jitter buffer and an audio playback rate based on the transmission jitter state and the transmission channel interference state, wherein the jitter buffer is configured to cache the received audio data packets; a speed change module, configured to perform speed change processing on audio decoded data based on the adjusted audio playback rate, the audio decoded data being obtained by decoding the audio data packet; The playing module is used to play audio based on the audio decoding data after speed change processing.

18. A computer device comprising a processor and a memory; the memory storing at least one instruction, wherein the at least one instruction is configured to be executed by the processor to implement the audio playback method according to any one of claims 1 to 16.

19. A computer-readable storage medium, wherein the storage medium stores at least one instruction, wherein the at least one instruction is configured to be executed by a processor to implement the audio playback method according to any one of claims 1 to 16.

20. A computer program product, comprising computer instructions, wherein the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device implements the audio playback method as described in any one of claims 1 to 16.

Images