Audio synchronous playing method and system, audio playing device and audio system

Abstract

The present disclosure relates to an audio synchronous playing method and system, an audio playing device and an audio system, and relates to the technical field of communication. The method determines a target offset duration, and determines a target playing time of a target audio packet according to a preset playing delay, the target offset duration, a receiving time and an audio duration of the target audio packet, and then plays the target audio packet at the target playing time, thereby completing synchronous audio playing of one or more audio playing devices. The method can not only ensure accurate synchronous playing of the same audio by multiple audio playing devices, but also does not need to use additional bandwidth for time service, thereby reducing bandwidth demand and improving utilization of communication bandwidth.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and more specifically, to an audio synchronous playback method, system, audio playback device, and audio system. Background Technology

[0002] Generally, to achieve better audio playback, multiple audio playback devices can be connected to a single terminal device, i.e., a one-to-many connection. In related technologies, to achieve synchronized audio playback across multiple audio playback devices, the audio playback devices and the terminal device need to use a unified timing method. The audio playback devices need to reference the precise time synchronization of the terminal device, and the multiple audio playback devices need to continuously communicate in real-time to agree on the clock information for synchronized playback, ensuring that the time difference between the synchronized playback of the multiple audio playback devices remains within a preset range. However, this synchronization method requires establishing communication between multiple audio playback devices, consuming additional communication bandwidth and reducing the effective bandwidth utilization of the entire communication system. Summary of the Invention

[0003] The purpose of this disclosure is to provide an audio synchronous playback method, system, audio playback device, and audio system, so that the audio playback device can achieve multi-device audio synchronous playback without utilizing the timing of the audio transmission device.

[0004] In a first aspect, this disclosure provides an audio synchronous playback method, applied to an audio playback device, the method comprising:

[0005] Determine the target offset duration, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmission device and the start point of the transmission window of the audio transmission device transmitting the target audio packet;

[0006] The target playback time for playing the target audio packet is determined based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet. The preset playback delay represents the time required for at least one audio playback device connected to the audio transmission device to complete receiving the target audio packet and synchronously playing the target audio packet.

[0007] The target audio package is played at the target playback time.

[0008] Optionally, playing the target audio package at the target playback time includes:

[0009] When the target audio packet is the first audio packet of the audio stream, the waiting time is determined based on the decoding time of the target audio packet, the target playback time, and the reception time.

[0010] After the audio playback device has completed decoding the target audio packet and the waiting time has elapsed, the target playback time is determined, and the target audio packet is played.

[0011] Optionally, the method further includes:

[0012] During the playback of an audio packet, the actual playback delay is determined based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration of the currently played audio packet. The remaining buffer duration represents the remaining audio duration of the currently played audio packet when the decoding of the next target audio packet is completed.

[0013] Based on the actual playback delay and the preset playback delay, a target playback rate is determined for playing the audio packet with the remaining duration of the buffer, wherein the target playback rate ensures that the difference between the preset playback delay and the actual playback delay determined based on the target playback rate remains within a preset range.

[0014] Optionally, determining the actual playback delay based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration of the currently played audio packet includes:

[0015] The actual playback delay is determined based on the sum of the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration.

[0016] Optionally, determining the target playback rate for playing the audio packet with the remaining buffer duration based on the actual playback delay and the preset playback delay includes:

[0017] If the actual playback delay is less than the preset playback delay, reduce the playback rate of the audio playback device to obtain the target playback rate;

[0018] If the actual playback delay is greater than the preset playback delay, the playback rate of the audio playback device is increased to obtain the target playback rate.

[0019] Optionally, determining the target offset duration includes:

[0020] Determine the window offset duration and the reception offset duration, wherein the window offset duration represents the time interval between the start time of the audio transmitting device transmitting the target audio packet to the audio playback device and the start point of the transmission window, and the reception offset duration represents the time interval between the time when the audio playback module of the audio playback device receives the target audio packet and the start time;

[0021] The target offset duration is obtained by summing the window offset duration and the receiving offset duration.

[0022] Optionally, determining the target playback time for playing the target audio packet based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet includes:

[0023] The first target duration is determined based on the sum of the target offset duration and the audio duration of the target audio packet;

[0024] The second target duration is determined based on the difference between the preset playback delay and the first target duration;

[0025] The time interval after the receiving time and the second target duration is determined as the target playback time.

[0026] Secondly, this disclosure provides an audio synchronous playback system applied to an audio playback device, the system comprising:

[0027] The first determining module is configured to determine the target offset duration, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmitting device and the starting point of the transmission window of the audio transmitting device transmitting the target audio packet;

[0028] The second determining module is configured to determine the target playback time for playing the target audio packet based on the preset playback delay, the target offset duration, the receiving time, and the audio duration of the target audio packet. The preset playback delay represents the duration required for at least one audio playback device connected to the audio transmitting device to complete receiving the target audio packet and synchronously playing the target audio packet.

[0029] The playback module is configured to play the target audio package at the target playback time.

[0030] Thirdly, this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect.

[0031] Fourthly, this disclosure provides an audio playback device, including:

[0032] A memory on which computer programs are stored;

[0033] A controller for executing the computer program in the memory to implement the steps of the method described in the first aspect.

[0034] Fifthly, this disclosure provides an audio system, including an audio transmitting device and at least one audio playback device, wherein:

[0035] The audio transmitting device is configured to send a target audio packet to the at least one audio playback device;

[0036] The at least one audio playback device is configured as follows:

[0037] Determine the target offset duration, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmission device and the start point of the transmission window of the audio transmission device transmitting the target audio packet;

[0038] The target playback time for playing the target audio packet is determined based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet. The preset playback delay represents the time required for at least one audio playback device connected to the audio transmission device to complete receiving the target audio packet and synchronously playing the target audio packet.

[0039] The target audio package is played at the target playback time.

[0040] With the above technical solution, when an audio transmitting device sends an audio packet to an audio playback device, other audio playback devices connected to the audio transmitting device do not need to exchange clock information, meaning that different audio playback devices do not need to perform clock alignment operations. Each audio playback device calculates the corresponding target playback time based on its own target offset duration, the reception time of the target audio packet, and the audio duration of the target audio packet, combined with a preset playback delay, thus completing the synchronous audio playback of one or more audio playback devices. This not only ensures precise synchronous playback of the same audio by multiple audio playback devices but also eliminates the need for additional bandwidth to exchange clock information, reducing bandwidth requirements and improving the utilization rate of communication bandwidth. Experiments have shown that, compared to the same starting point for the audio transmitting devices, the latency error of multiple audio playback devices can be controlled within 6µs or even less.

[0041] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0042] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0043] Figure 1 This is a schematic diagram illustrating an application scenario of an audio synchronization playback method according to an exemplary embodiment.

[0044] Figure 2 This is a flowchart illustrating an audio synchronization playback method according to an exemplary embodiment.

[0045] Figure 3 This is a schematic diagram illustrating the transmission of a target audio packet according to an exemplary embodiment.

[0046] Figure 4 This is a schematic diagram of a transmission window according to an exemplary embodiment.

[0047] Figure 5 This is a schematic diagram illustrating clock transition according to an exemplary embodiment.

[0048] Figure 6 This is a schematic diagram illustrating the waiting time according to an exemplary embodiment.

[0049] Figure 7 This is a flowchart illustrating an audio synchronization playback method according to another exemplary embodiment.

[0050] Figure 8 This is a schematic diagram illustrating the principle of audio playback according to an exemplary embodiment.

[0051] Figure 9 This is a schematic diagram illustrating the actual playback delay according to an exemplary embodiment.

[0052] Figure 10 This is a schematic diagram of the module connections of an audio synchronization playback system according to an exemplary embodiment.

[0053] Figure 11 This is a block diagram illustrating an audio playback device according to an exemplary embodiment. Detailed Implementation

[0054] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0055] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0056] This disclosure proposes an audio system including an audio transmitting device and one or more audio playback devices. The audio transmitting device and the audio playback devices can be connected via Bluetooth. For example, the audio transmitting device can broadcast audio to one or more audio playback devices based on the CIS and BIS channels in LeAudio (Bluetooth Low Energy Audio). Alternatively, they can be connected via other networks, such as WiFi, cellular networks, or wired communication. The audio transmitting device can refer to a mobile terminal, in-vehicle terminal, or other terminal device, while the audio playback device can refer to a mobile terminal, headphones, speakers, or other devices with audio playback functionality. Figure 1 This is a schematic diagram illustrating an application scenario of an audio synchronization playback method according to an exemplary embodiment, such as... Figure 1 As shown, an audio transmitting device can be connected to a first audio playback device, a second audio playback device, ..., an Nth audio playback device, respectively.

[0057] Figure 2 This is a flowchart illustrating an audio synchronization playback method according to an exemplary embodiment. Figure 2 As shown in the embodiments of this disclosure, an audio synchronization playback method is proposed, which can be applied to, for example... Figure 1 The audio playback device shown can be specifically executed through an audio synchronization playback system configured within the audio playback device. This system can be implemented in software and / or hardware and configured within the audio playback device. For example... Figure 2 As shown, the method may include the following steps.

[0058] In step 210, the target offset duration is determined, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmission device and the start point of the transmission window of the audio transmission device transmitting the target audio packet.

[0059] here, Figure 3 This is a schematic diagram illustrating the transmission of a target audio packet according to an exemplary embodiment, such as... Figure 3 As shown, the audio transmitting device 30 communicates with the audio playback device 31. The audio playback device 31 receives the target audio packet sent by the audio transmitting device 30 through the data transmission module 311, and transmits the received target audio packet to the audio playback module 312. The audio playback module 312 decodes the target audio packet, converts the digital signal into analog information, and controls the audio output module (…). Figure 3 (Not shown) Play sound.

[0060] According to the synchronous transmission protocol between the audio transmitting device and the audio playback device, the audio transmitting device transmits the target audio packet to one or more audio playback devices connected to the audio transmitting device within a transmission window. Figure 4 This is a schematic diagram of a transmission window according to an exemplary embodiment, such as... Figure 4 As shown, within the transmission window of the audio transmitting device, the audio transmitting device sends the target audio packet to the first audio playback device within a first sub-transmission window, which is offset by a first window time from the start point of the transmission window. The audio transmitting device then sends the target audio packet to the second audio playback device within a second sub-transmission window, which is offset by a second window time from the start point of the transmission window. Finally, the audio transmitting device sends the target audio packet to the Nth audio playback device within an Nth sub-transmission window, which is offset by an Nth window time from the start point of the transmission window. Within each transmission window, the audio transmitting device sends the target audio packet to the audio playback device according to a different timing sequence. It is worth noting that within the same transmission window, the transmitted target audio packet can be the same, or it can be different channel formats of the same audio, such as left channel audio or right channel audio.

[0061] It should be understood that the window offset duration refers to the time interval between the start time of the audio transmitting device transmitting the target audio packet to the audio playback device and the start point of the transmission window of the audio transmitting device. This window offset duration can be negotiated when the audio transmitting device and the audio playback device establish a communication connection. After this window offset duration, the audio transmitting device transmits the target audio packet to the audio playback device. Within this transmission window, the target audio packet is allowed to be transmitted once or multiple times so that it can be retransmitted if the transmission is interfered with, thereby increasing the probability of successful transmission of the target audio packet.

[0062] Furthermore, the transmission of a single target audio packet takes time. Therefore, when the audio playback device receives the target audio packet, there is a delay relative to the start of the transmission window. Adding the processing delay of the audio playback device's data transmission system, this is the reception offset duration. The reception offset duration represents the time interval between the moment the audio playback module of the audio playback device receives the target audio packet and the start time. It is worth noting that if the target audio packet fails to be transmitted successfully within the entire transmission window, an empty packet will be returned to the audio playback module of the audio playback device after the last transmission failure.

[0063] In some embodiments, the audio playback device can determine its corresponding window offset duration and receive offset duration, and obtain the target offset duration based on the sum of the window offset duration and the receive offset duration.

[0064] For example, if the transmission window of the audio transmitting device is 10,000 µs, and the starting point of the transmission window when transmitting the target audio packet A is 100,000 µs, the window offset of the audio playback device B is 3,000 µs, and the receiving offset is 1,200 µs, then at 103,000 µs, the audio transmitting device starts transmitting the target audio packet A to the data transmission module of the audio playback device B, and at 104,200 µs, the audio playback module of the audio playback device B receives the target audio packet A, with a corresponding target offset of 4,200 µs.

[0065] It's worth noting that the window offset duration for each audio playback device is fixed, and the reception offset duration is limited to the range of the sub-transmission window. The window offset duration and the size of the sub-transmission window are agreed upon when the audio playback device and the audio transmission device establish a communication connection. The values ​​of the window offset duration and the sub-transmission window depend on the synchronization transmission protocol. For example, as... Figure 4 As shown, each of the first, second, and Nth sub-transmission windows contains a corresponding reception offset duration. Within the time range corresponding to the sub-transmission window, the transmission and reception of the target audio packet can be completed.

[0066] Furthermore, the aforementioned target offset duration is based on the clock of the audio playback device. The audio playback module of the audio playback device has a clock that starts counting from 0µs after startup. During the data transmission of the target audio packet, the start point of the transmission window, the receive offset duration, and the window offset duration are timed using the clock of the audio transmitting device. When the audio playback device plays back the audio, it needs to convert the start point of the transmission window, the receive offset duration, and the window offset duration, which are timed using the clock of the audio transmitting device, to be timed using the clock of the audio playback module.

[0067] Figure 5 This is a schematic diagram illustrating clock transitions according to an exemplary embodiment, such as... Figure 5 As shown, the starting point of the transmission window of the audio transmitting device is 100,000 µs on the audio transmitting device's clock. The window offset time of the audio playback device B is 3,000 µs. Therefore, at 103,000 µs, the audio transmitting device begins transmitting the target audio packet A to the data transmission system of the audio playback device B. The receiving offset time of the audio playback device B is 1,200 µs. Therefore, at 104,200 µs, the audio playback module of the audio playback device B receives the target audio packet A, with a corresponding target offset time of 4,200 µs.

[0068] Corresponding to the clock of the audio playback device, if the audio playback module of audio playback device B receives the target audio packet A at 210000us, then the corresponding starting point is 205800us.

[0069] In step 220, the target playback time for playing the target audio packet is determined based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet. The preset playback delay represents the time required for at least one audio playback device connected to the audio transmitting device to complete receiving the target audio packet and synchronously playing the target audio packet.

[0070] Here, the preset playback delay is a time window within which one or more audio playback devices can complete the actions of receiving the target audio packet, decoding the target audio packet, synchronously starting to play the target audio packet, and synchronously completing the playback of the target audio packet. Therefore, the preset playback delay can be the maximum time for one or more audio playback devices to complete the actions of receiving, decoding, and synchronously completing the playback of the target audio packet. Moreover, different audio playback devices can have the same preset playback delay, which can be negotiated between the audio transmitting device and the audio playback devices. For example, the preset playback delay can be 20,000 µs. Within this 20,000 µs duration, all audio playback devices connected to the audio transmitting device can complete the actions of receiving, decoding, playing, and completing the playback of the target audio packet simultaneously.

[0071] It is worth noting that the preset playback delay can be determined by the audio transmitting device acquiring the decoding time, target offset time, and communication time of each audio playback device when the audio playback device and the audio transmitting device are connected.

[0072] The audio duration of a target audio packet refers to the length of the audio data used in encoding the target audio packet. For example, for audio with a 48kHz sampling rate, each target audio packet requires 480 samples for encoding, so the audio duration of the target audio packet is 10000µs. Alternatively, the audio duration of a target audio packet can also refer to the time required to play the entire target audio packet at a preset playback speed. For example, the preset playback speed can be the same as the sampling rate of the target audio packet, in which case the actual time required to play the entire target audio packet is equal to the length of the audio data used in encoding the audio packet.

[0073] The target playback time refers to the moment when the audio playback module of the audio playback device begins playing the target audio packet. Each target audio packet has a corresponding target playback time. On the timeline, the end time of the previous audio packet in the audio stream is the playback time of the next audio packet.

[0074] In some embodiments, a first target duration can be determined based on the sum of the target offset duration and the audio duration of the target audio packet, and then a second target duration can be determined based on the difference between the preset playback delay and the first target duration, and the time interval between the second target duration after the receiving time can be determined as the target playback time.

[0075] For example, the first target duration = target offset duration + audio duration; the second target duration = preset playback delay - first target duration; and the target playback time = reception time + second target duration.

[0076] It should be understood that although the target offset durations may differ for different audio playback devices, the target playback time for the same target audio package is consistent across different audio playback devices.

[0077] In step 230, the target audio package is played at the target playback time.

[0078] Here, the target playback time is referenced to the clock in the audio playback module of the audio playback device. When the target playback time is reached, the audio playback module of the audio playback device starts playing the target audio packet so that the audio playback devices connected to the audio transmission device can play the same target audio packet synchronously.

[0079] Therefore, when an audio transmitting device sends an audio packet to an audio playback device, other audio playback devices connected to the audio transmitting device do not need to exchange clock information; that is, different audio playback devices do not need to perform clock alignment operations. The audio playback device calculates the corresponding target playback time based on its own target offset duration, the reception time of the target audio packet, and the audio duration of the target audio packet, combined with a preset playback delay, thus completing the synchronous audio playback of one or more audio playback devices. This not only ensures precise synchronous playback of the same audio by multiple audio playback devices but also eliminates the need for additional bandwidth to exchange clock information, reducing bandwidth requirements and improving the utilization rate of communication bandwidth. Experiments have shown that, compared to the same starting point for the audio transmitting device, the latency error of the audio playback device can be controlled within 6µs or even less, and the time difference between synchronous audio playback of multiple audio playback devices can be controlled within 12µs or even less.

[0080] In some feasible implementations, when the target audio packet is the first audio packet of the audio stream, the waiting time can be determined based on the decoding time of the target audio packet, the target playback time, and the reception time. After the audio playback device has completed decoding the target audio packet and waited for the specified time interval, the target playback time is determined to have been reached, and the target audio packet is played.

[0081] Here, the target audio packet is an audio packet segmented from the audio stream. These segmented target audio packets are arranged sequentially in chronological order to form the audio stream. Because different audio playback devices have different target offset durations, the waiting times for different audio playback devices connected to the audio sending device will also differ. After their respective waiting times have elapsed, all audio playback devices connected to the audio sending device will synchronously play the same target audio packet.

[0082] The decoding time of the target audio packet refers to the time required for the audio playback device to decode the target audio packet.

[0083] The waiting time refers to the time that an audio playback device needs to wait after decoding the target audio packet. After decoding the target audio packet and waiting for the specified time, the target playback time is determined, and the audio playback module of the audio playback device begins playing the target audio packet. Different audio playback devices have the same preset playback delay, and for each audio playback device, the corresponding target offset time, decoding time, and audio duration can be determined. Therefore, the waiting time of that audio playback device is also deterministic.

[0084] It should be understood that the target playback time and reception time of the target audio packet are both referenced to the clock in the audio playback module.

[0085] Within the time window between the target playback time and the reception time, the audio playback module decodes the target audio packet. Normally, after decoding the target audio packet, the audio playback module directly reaches the target playback time. However, since the preset playback delay is the time required for one or more audio playback devices to complete the actions of receiving the target audio packet, decoding the target audio packet, synchronously starting playback of the target audio packet, and synchronously completing playback of the target audio packet, the decoding times of one or more audio playback devices differ, thus the corresponding waiting times also differ.

[0086] Figure 6 This is a schematic diagram illustrating the waiting time according to an exemplary embodiment, such as... Figure 6As shown, assuming the target audio packet has a duration of 10000µs and a preset playback delay of 20000µs, the audio transmitting device creates a transmission window for transmitting the target audio packet A at 100000µs. After a window offset of 3000µs, at 103000µs, the audio transmitting device begins transmitting the target audio packet A to the audio playback device B. After a reception offset of 1200µs, the audio playback module of the audio playback device B receives the target audio packet A. Converted to the clock of the audio playback module, this means that the audio playback module of the audio playback device B receives the target audio packet A at the reception time of 210000µs. The starting point of the transmission window is 205800µs. At time 210000us, the audio playback module begins decoding the target audio packet A. After decoding for 2500us, it completes decoding of the target audio packet A at time 212500us. Since the target playback time for the target audio packet A is 215800us, the corresponding waiting time is 3300us. That is, after the audio playback module completes decoding the target audio packet A, a time interval of 3300us is required before the target playback time is reached.

[0087] It's worth noting that different audio playback devices, after waiting for different durations, will play the same target audio packet at the same time. Playing the same target audio packet at the same time means playing it at the same time according to the world clock. For example, the first audio playback device can play target audio packet A at 215800us on its corresponding audio playback module's clock, and the second audio playback device can play target audio packet A at 216800us on its corresponding audio playback module's clock. However, under the world clock, the first and second audio playback devices are playing target audio packet A synchronously.

[0088] In other embodiments, if the audio packet chain list for audio playback includes multiple audio packets, the target duration can be determined based on the target offset duration, the decoding duration of the target audio packet, the audio duration of the target audio packet, and the sum of the audio durations of other audio packets. Then, the waiting duration can be determined based on the difference between the preset playback delay and the target duration.

[0089] Of course, in the above implementation of determining the waiting time, since it is assumed that the data transmission module of all audio playback devices takes the same amount of time to send audio packets to the audio playback module, the impact of the sending time on synchronous playback is not considered when calculating the waiting time.

[0090] In some feasible implementations, the waiting time can be calculated by combining the transmission time taken by the data transmission module to send audio packets to the audio playback module. For example, assuming the preset playback delay is 20000us, the target offset time is 4200us, the decoding time is 2500us, the audio duration is 10000us, and the transmission time is 10us, then the corresponding waiting time = 20000us - 10000us - 4200us - 2500us - 10us, which is 3290us.

[0091] like Figure 6 As shown, the target playback time is 215800us after a waiting period of 3300us. The audio playback module plays the target audio package A at the target playback time of 215800us.

[0092] After the audio playback device completes decoding of the target audio packet and waits for the specified time interval, the target playback time is reached, and the audio playback module of the audio playback device begins to play the target audio packet.

[0093] It is worth noting that by calculating the waiting time for the first audio packet in the audio stream, multiple audio playback devices can start playing the audio stream synchronously.

[0094] Figure 7 This is a flowchart illustrating an audio synchronization playback method according to another exemplary embodiment. For example... Figure 7 As shown, the audio synchronization playback method may also include the following steps.

[0095] In step 410, during the playback of the audio packet, the actual playback delay is determined based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration of the currently played audio packet. The remaining buffer duration represents the remaining audio duration of the currently played audio packet when the decoding of the next target audio packet is completed.

[0096] Here, the remaining buffer duration refers to the remaining audio duration of the currently playing audio packet when decoding the next target audio packet is complete. Of course, this remaining buffer duration can also be the duration required to play the remaining audio at the default playback rate. Figure 8 This is a schematic diagram illustrating the principle of audio playback according to an exemplary embodiment, such as... Figure 8 As shown, there are multiple audio packets in the audio packet linked list. Audio packet N is the most recently received audio packet. When the audio playback module plays back the sound, it retrieves audio packet N-2 from the audio packet linked list and plays it back to form sound. Therefore, when calculating the actual playback delay, if the next target audio packet is audio packet N-1, it is necessary to consider the remaining buffer time of the currently playing audio packet N-2 in the audio stream.

[0097] In some embodiments, the actual playback delay can be determined based on the sum of the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration.

[0098] Figure 9 This is a schematic diagram illustrating the actual playback delay according to an exemplary embodiment. For example... Figure 9 As shown, assuming a preset playback delay of 20000µs and a target audio packet duration of 10000µs, the transmission window for the target audio packet from the audio transmitting device starts at 205800µs on the audio playback device's clock. After the target offset duration, the audio playback module receives the target audio packet at 210000µs. After the decoding duration, the audio playback module completes decoding the next target audio packet at 212500µs. Playback of the next target audio packet begins at 215800µs.

[0099] Assuming the remaining buffer duration is 3298us, the actual playback delay is the target offset duration + decoding duration + remaining buffer duration = 19998us.

[0100] In step 420, a target playback rate for playing the audio packet with the remaining duration of the buffer is determined based on the actual playback delay and the preset playback delay, wherein the target playback rate ensures that the difference between the preset playback delay and the actual playback delay determined based on the target playback rate is maintained within a preset range.

[0101] Here, during the playback of the audio package, the audio playback module calculates the actual playback delay in real time or periodically. Based on the actual playback delay and the preset playback delay, it calculates the target playback rate for the audio package with the remaining buffer duration, and adjusts the playback rate of the audio playback module to the target playback rate.

[0102] like Figure 9 As shown, time 215800us is the target playback time corresponding to the next target audio packet. When the audio playback device finishes playing the currently playing audio packet at the first playback rate, the target playback time of the next target audio packet should be reached.

[0103] Here, the first playback rate refers to the current playback rate of the audio playback module of the audio playback device. For example, if the playback rate of the currently playing audio packet in the audio playback module is A, then when playing the next target audio packet, if the rate A is not adjusted, then rate A is used as the first playback rate. If the target audio packet is the first audio packet of the audio stream, then the playback rate currently set by the audio playback module is used as the first playback rate.

[0104] Since the actual playback delay is the sum of the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration, and the remaining buffer duration can be changed by the playback rate, by adjusting the first playback rate of the audio playback module to the target playback rate, the difference between the preset playback delay and the actual playback delay determined according to the target playback rate can be maintained within a preset range.

[0105] The preset range can be set according to the actual situation; it can be 0µs or -6 to 6µs. When the preset range is 0µs, the time when the currently played audio packet finishes playing is the same as the target playback time of the next target audio packet.

[0106] For example, assuming the preset range is 0µs and the preset playback delay is 20000µs, such as Figure 9 As shown, the actual playback delay is 19998us, indicating that the audio playback module finished playing the currently playing audio packet at time 215798us. The target time for playing the next target audio packet is 215800us. Therefore, if the playback rate of the audio playback module is not adjusted, the playback time of the next target audio packet will be advanced by 2us, or there will be a 2us audio interruption because the next target audio packet still needs 2us to complete decoding.

[0107] By slowing down the initial playback rate to the target playback rate, the time required to play the remaining 3298µs buffer duration becomes 3300µs. That is, at 215800µs, the audio playback module finishes playing one audio packet and begins playing the next target audio packet, achieving precise delayed playback of the next target audio packet.

[0108] In some embodiments, if the actual playback delay is less than the preset playback delay, the playback rate of the audio playback device is reduced to obtain the target playback rate.

[0109] In some embodiments, if the actual playback delay is greater than the preset playback delay, the playback rate of the audio playback device is increased to obtain the target playback rate.

[0110] It should be understood that after the first playback rate is adjusted to the target playback rate, the audio playback module continues to play the audio package at the target playback rate until the playback rate is adjusted again.

[0111] For example, the playback rate is set to a first playback rate when the first audio package begins playing. Midway through playback, the first playback rate is adjusted to a second playback rate. If the playback rate is not adjusted when playing the second audio package, then playback continues at the second playback rate.

[0112] Therefore, by adjusting the playback rate of the audio packet during playback, one or more audio playback devices connected to the audio transmitting device can precisely synchronize the playback of the same audio packet.

[0113] Figure 10 This is a schematic diagram of the module connections of an audio synchronization playback system according to an exemplary embodiment. Figure 10 As shown, this disclosure proposes an audio synchronous playback system 1000, which is applied to an audio playback device. The system 1000 includes:

[0114] The first determining module 1001 is configured to determine the target offset duration, wherein the target offset duration represents the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmitting device and the starting point of the transmission window of the audio transmitting device transmitting the target audio packet;

[0115] The second determining module 1002 is configured to determine the target playback time for playing the target audio packet based on the preset playback delay, the target offset duration, the receiving time, and the audio duration of the target audio packet. The preset playback delay represents the duration required for at least one audio playback device connected to the audio transmitting device to complete receiving the target audio packet and synchronously playing the target audio packet.

[0116] The playback module 1003 is configured to play the target audio package at the target playback time.

[0117] Optionally, the playback module 1003 includes:

[0118] The first determining subunit is configured to determine a waiting time based on the decoding time of the target audio packet, the target playback time, and the receiving time when the target audio packet is the first audio packet of the audio stream;

[0119] The playback subunit is configured to determine the arrival of the target playback time and play the target audio package after the audio playback device has completed decoding the target audio package and the waiting time has elapsed.

[0120] Optionally, the system 1000 further includes:

[0121] The first calculation unit is configured to determine the actual playback delay during the playback of an audio packet based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration of the currently played audio packet, wherein the remaining buffer duration represents the remaining audio duration of the currently played audio packet when the decoding of the next target audio packet is completed.

[0122] The second calculation unit is configured to determine a target playback rate for playing the audio packet with the remaining duration of the buffer based on the actual playback delay and the preset playback delay, wherein the target playback rate ensures that the difference between the preset playback delay and the actual playback delay determined based on the target playback rate is maintained within a preset range.

[0123] Optionally, the first computing unit is specifically configured as follows:

[0124] The actual playback delay is determined based on the sum of the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration.

[0125] Optionally, the second computing unit is specifically configured as follows:

[0126] If the actual playback delay is less than the preset playback delay, reduce the playback rate of the audio playback device to obtain the target playback rate;

[0127] If the actual playback delay is greater than the preset playback delay, the playback rate of the audio playback device is increased to obtain the target playback rate.

[0128] Optionally, the first determining module 1001 includes:

[0129] The second determining subunit is configured to determine the window offset duration and the receiving offset duration, wherein the window offset duration represents the time interval between the start time of the audio transmitting device transmitting the target audio packet to the audio playback device and the start point of the transmission window, and the receiving offset duration represents the time interval between the time when the audio playback module of the audio playback device receives the target audio packet and the start time.

[0130] The third determining subunit is configured to obtain the target offset duration based on the sum of the window offset duration and the receiving offset duration.

[0131] Optionally, the second determining module 1002 includes:

[0132] The fourth determining subunit is configured to determine the first target duration based on the sum of the target offset duration and the audio duration of the target audio packet;

[0133] The fifth determining subunit is configured to determine the second target duration based on the difference between the preset playback delay and the first target duration;

[0134] The sixth determining subunit is configured to determine the target playback time as the time after the receiving time, at an interval of the second target duration.

[0135] Regarding the system in the above embodiments, the specific ways in which each functional module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0136] Figure 11 This is a block diagram illustrating an audio playback device according to an exemplary embodiment. Figure 11 As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0137] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the aforementioned audio synchronization playback method. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0138] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the audio synchronization playback method described above.

[0139] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the audio synchronization playback method described above. For example, the computer-readable storage medium may be the memory 702 including the program instructions described above, which may be executed by the processor 701 of the electronic device 700 to complete the audio synchronization playback method described above.

[0140] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described audio synchronization playback method when executed by the programmable device.

[0141] This disclosure also provides an audio system, including an audio transmitting device and at least one audio playback device, wherein:

[0142] The audio transmitting device is configured to send a target audio packet to the at least one audio playback device;

[0143] The at least one audio playback device is configured as follows:

[0144] Determine the target offset duration, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmission device and the start point of the transmission window of the audio transmission device transmitting the target audio packet;

[0145] The target playback time for playing the target audio packet is determined based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet. The preset playback delay represents the time required for at least one audio playback device connected to the audio transmission device to complete receiving the target audio packet and synchronously playing the target audio packet.

[0146] The target audio package is played at the target playback time.

[0147] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0148] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0149] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for synchronized audio playback, characterized in that, Applied to an audio playback device, the method includes: Determine the target offset duration, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmission device and the start point of the transmission window of the audio transmission device transmitting the target audio packet; The target playback time for playing the target audio packet is determined based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet. The preset playback delay represents the time required for at least one audio playback device connected to the audio transmission device to complete receiving the target audio packet and synchronously playing the target audio packet. The target audio packet is played at the target playback time. During the playback of the audio packet, the actual playback delay is determined based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the buffer remaining duration of the currently played audio packet. The buffer remaining duration represents the remaining audio duration of the currently played audio packet when the decoding of the next target audio packet is completed. Based on the actual playback delay and the preset playback delay, a target playback rate is determined for playing the audio packet with the remaining duration of the buffer, wherein the target playback rate ensures that the difference between the preset playback delay and the actual playback delay determined based on the target playback rate remains within a preset range.

2. The method according to claim 1, characterized in that, Playing the target audio package at the target playback time includes: When the target audio packet is the first audio packet of the audio stream, the waiting time is determined based on the decoding time of the target audio packet, the target playback time, and the reception time. After the audio playback device has completed decoding the target audio packet and the waiting time has elapsed, the target playback time is determined, and the target audio packet is played.

3. The method according to claim 1, characterized in that, The step of determining the actual playback delay based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration of the currently played audio packet includes: The actual playback delay is determined based on the sum of the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration.

4. The method according to claim 1, characterized in that, The step of determining the target playback rate for playing the audio packet with the remaining buffer duration based on the actual playback delay and the preset playback delay includes: If the actual playback delay is less than the preset playback delay, reduce the playback rate of the audio playback device to obtain the target playback rate; If the actual playback delay is greater than the preset playback delay, the playback rate of the audio playback device is increased to obtain the target playback rate.

5. The method according to claim 1, characterized in that, The determination of the target offset duration includes: Determine the window offset duration and the reception offset duration, wherein the window offset duration represents the time interval between the start time of the audio transmitting device transmitting the target audio packet to the audio playback device and the start point of the transmission window, and the reception offset duration represents the time interval between the time when the audio playback module of the audio playback device receives the target audio packet and the start time; The target offset duration is obtained by summing the window offset duration and the receiving offset duration.

6. The method according to claim 1, characterized in that, The step of determining the target playback time for playing the target audio packet based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet includes: The first target duration is determined based on the sum of the target offset duration and the audio duration of the target audio packet; The second target duration is determined based on the difference between the preset playback delay and the first target duration; The time interval after the receiving time and the second target duration is determined as the target playback time.

7. An audio synchronous playback system, characterized in that, The system, applied to an audio playback device, includes: The first determining module is configured to determine the target offset duration, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmitting device and the starting point of the transmission window of the audio transmitting device transmitting the target audio packet; The second determining module is configured to determine the target playback time for playing the target audio packet based on the preset playback delay, the target offset duration, the receiving time, and the audio duration of the target audio packet. The preset playback delay represents the duration required for at least one audio playback device connected to the audio transmitting device to complete receiving the target audio packet and synchronously playing the target audio packet. The playback module is configured to play the target audio packet at the target playback time. During the playback of the audio packet, the actual playback delay is determined based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration of the currently played audio packet. The remaining buffer duration represents the remaining audio duration of the currently played audio packet when the decoding of the next target audio packet is completed. Based on the actual playback delay and the preset playback delay, a target playback rate is determined for playing the audio packet with the remaining buffer duration. The target playback rate ensures that the difference between the preset playback delay and the actual playback delay determined based on the target playback rate remains within a preset range.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1-6.

9. An audio playback device, characterized in that, include: A memory on which computer programs are stored; A controller for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-6.

10. An audio system, characterized in that, It includes an audio transmitting device and at least one audio playback device, wherein: The audio transmitting device is configured to send a target audio packet to the at least one audio playback device; The at least one audio playback device is configured as follows: Determine the target offset duration, wherein the target offset duration characterizes the time interval between the time when the audio playback module of the audio playback device receives the target audio packet sent by the audio transmission device and the start point of the transmission window of the audio transmission device transmitting the target audio packet; The target playback time for playing the target audio packet is determined based on the preset playback delay, the target offset duration, the reception time, and the audio duration of the target audio packet. The preset playback delay represents the time required for at least one audio playback device connected to the audio transmission device to complete receiving the target audio packet and synchronously playing the target audio packet. The target audio packet is played at the target playback time. During the playback of the audio packet, the actual playback delay is determined based on the target offset duration, the decoding duration of the next target audio packet, the audio duration of the next target audio packet, and the remaining buffer duration of the currently played audio packet. The remaining buffer duration represents the remaining audio duration of the currently played audio packet when the decoding of the next target audio packet is completed. Based on the actual playback delay and the preset playback delay, a target playback rate is determined for playing the audio packet with the remaining buffer duration. The target playback rate ensures that the difference between the preset playback delay and the actual playback delay determined based on the target playback rate remains within a preset range.

Images