Audio synchronization method, audio system, and computer-readable storage medium

Abstract

The present application provides an audio synchronization method, an audio system, and a computer-readable storage medium. The audio synchronization method is applied to an audio synchronization unit. The audio synchronization method comprises: acquiring an initial transmission timestamp of an audio signal transmitted by a transmitting end; acquiring received audio signals of a plurality of receiving ends, the received audio signal of each receiving end comprising one initial reception timestamp and one initial loopback timestamp; on the basis of the initial reception timestamp and the initial loopback timestamp of the received audio signal of each receiving end, determining a link transmission latency at the receiving end; on the basis of the link transmission latencies in one-to-one correspondence with the plurality of receiving ends, determining a target latency and a target receiving end corresponding to the target latency; determining an audio synchronization latency on the basis of the target latency, the initial transmission timestamp, and the initial reception timestamp of the target receiving end; and performing latency compensation on a transmission link between the transmitting end and each receiving end on the basis of the audio synchronization latency. The method can effectively improve audio synchronization performance of the audio system, thereby effectively improving user experience.

Description

Audio synchronization methods, audio systems, and computer-readable storage media

[0001] This application claims priority to Chinese Patent Application No. 202510494423.1, filed on April 18, 2025, entitled "Audio Synchronization Method, Audio System and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference.

[0002] [Technical Field]

[0003] This application relates to the field of audio synchronization technology for audio systems, and particularly to an audio synchronization method, an audio system, and a computer-readable storage medium.

[0004] [Background Technology]

[0005] Wireless / wired communication technology, audio signal processing technology, and digital signal processing technology are crucial technological fields in modern information society. With the continuous development of wireless communication and digital audio technologies, audio systems that transmit audio signals wirelessly / wiredly have been widely used in home entertainment, public address systems, online music systems, and conferencing systems. These systems typically consist of one transmitter and multiple receivers. The transmitter transmits the audio signal to multiple receivers via a wireless or wired network for user listening. However, existing audio systems suffer from poor audio synchronization, which is a key indicator affecting user experience. Therefore, improving audio synchronization is a pressing technical problem that needs to be solved.

[0006] [Summary of the Invention]

[0007] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows: This application provides an audio synchronization method, which is applied to an audio synchronization unit. The audio synchronization method includes: obtaining an initial transmission timestamp of an audio signal transmitted by a transmitting end; obtaining received audio signals from several receiving ends, each receiving end's received audio signal including an initial reception timestamp and an initial retrieval timestamp; determining the link transmission delay of the receiving end based on the initial reception timestamp and the initial retrieval timestamp of the received audio signal of the receiving end; determining a target delay and a target receiving end corresponding to the target delay according to the link transmission delay corresponding to each of the several receiving ends; determining an audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp of the target receiving end; and performing delay compensation on the transmission links between the transmitting end and each receiving end according to the audio synchronization delay.

[0008] In some embodiments, determining the target delay according to the link transmission delay corresponding to a plurality of receivers includes: setting the maximum value among the link transmission delays corresponding to a plurality of receivers as the target delay.

[0009] In some embodiments, delay compensation is performed on the transmission links between the transmitter and each receiver according to the audio synchronization delay, including: obtaining the current transmission timestamp of the transmission link between the transmitter and each receiver, and the current retrieval timestamp of the transmission link of each receiver; obtaining the current transmission link duration of each receiver based on the current transmission timestamp and the current retrieval timestamp of each receiver; obtaining the current transmission link duration of each receiver and the delay error of the audio synchronization delay; and when the delay error is greater than a preset error value, performing delay compensation on each receiver according to the audio synchronization delay.

[0010] In some embodiments, delay compensation is performed on the transmission links between the transmitter and each receiver based on the audio synchronization delay, including: obtaining the initial transmission link duration of the receiver based on the initial transmission timestamp and the initial retrieval timestamp of the receiver; determining the delay compensation value of the receiver based on the audio synchronization delay and the initial transmission link duration; and adjusting the timestamp of the transmitter sending the audio signal to the receiver based on the delay compensation value.

[0011] In some embodiments, delay compensation is performed on the transmission links between the transmitter and each receiver based on the audio synchronization delay, including: obtaining a first delay compensation step size; determining a first delay compensation value according to the first delay compensation step size; adjusting the timestamp of the audio signal sent from the transmitter to the receiver according to the first delay compensation value, and obtaining the current transmission link duration; if the delay error between the current transmission link duration and the audio synchronization delay is not within a preset error range, determining a second delay compensation value according to the first delay compensation step size, until the delay error between the compensated transmission link duration and the audio synchronization delay is within a preset error range.

[0012] In some embodiments, determining a second delay compensation value according to a first delay compensation step size until the delay errors of the compensated transmission link duration and audio synchronization delay are within a preset error range includes: determining a second delay compensation value according to a first delay compensation step size until the delay errors of the compensated transmission link duration and audio synchronization delay are within a preset error range, or ending the current iteration and entering the next iteration after compensating a preset number of times; when entering the next iteration, determining a second delay compensation step size according to the first delay compensation step size; determining a third delay compensation value according to the second delay compensation step size until the delay errors of the compensated transmission link duration and audio synchronization delay are within a preset error range, or starting the next iteration; wherein, the second delay compensation step size is greater than the first delay compensation step size.

[0013] In some embodiments, obtaining the first delay compensation step size includes: determining the first delay compensation step size according to the audio sampling frequency.

[0014] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows: This application provides an audio synchronization method, which is applied to an audio synchronization system, wherein the audio synchronization system includes a transmitting end and a plurality of receiving ends; the audio synchronization method includes: the transmitting end sending an audio signal to the plurality of receiving ends and recording an initial transmission timestamp; the receiving ends receiving the audio signal and performing hardware retrieval, recording the corresponding initial reception timestamp and initial retrieval timestamp; the transmitting end determining the link transmission delay corresponding to each of the plurality of receiving ends based on the initial reception timestamp and initial retrieval timestamp of the plurality of receiving ends; the transmitting end determining a target delay and a target receiving end corresponding to the target delay based on the link transmission delay corresponding to each of the plurality of receiving ends; the transmitting end determining an audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp of the target receiving end; and the transmitting end performing delay compensation on the transmission link between the transmitting end and each receiving end according to the audio synchronization delay.

[0015] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows: This application provides an audio system, which includes: a transmitting end and several receiving ends. The transmitting end includes an audio transmitting unit and an audio correction unit. The receiving end includes a control unit, an audio processing unit, and a hardware retrieval unit. The audio transmitting unit transmits audio signals to the several receiving ends. The audio correction unit records an initial transmission timestamp. The control unit receives audio signals and records the corresponding initial reception timestamp. The audio processing unit performs audio processing operations on the audio signals. The hardware retrieval unit retrieves the processed audio signals back to the control unit. The control unit also records an initial retrieval timestamp. The control unit determines the link transmission delay of the receiving end based on the initial reception timestamp and the initial retrieval timestamp. The audio correction unit determines a target delay based on the link transmission delays of the several receiving ends. The audio correction unit determines an audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp. The audio correction unit compensates for the delay of the transmission link between the audio transmitting unit and each receiving end according to the audio synchronization delay.

[0016] To solve the above-mentioned technical problems, the technical solution adopted in this application is as follows: This application provides a computer-readable storage medium storing program instructions, which can be executed by a processor to implement the audio synchronization method described in any of the above embodiments.

[0017] The beneficial effects of this application's embodiments are as follows: The audio synchronization method of this application includes: obtaining the initial transmission timestamp of the audio signal transmitted by the transmitting end; obtaining the received audio signals of several receiving ends, each receiving end's received audio signal including an initial reception timestamp and an initial retrieval timestamp; determining the link transmission delay of the receiving end based on the initial reception timestamp and initial retrieval timestamp of the received audio signal of the receiving end; determining the target delay and the target receiving end corresponding to the target delay according to the link transmission delay corresponding to each of the several receiving ends; determining the audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp of the target receiving end; and performing delay compensation on the transmission links between the transmitting end and each receiving end according to the audio synchronization delay. After determining the audio synchronization delay, the audio synchronization unit performs delay compensation on the transmission links that need delay compensation based on the audio synchronization delay, so that the transmission link duration after delay compensation of each transmission link is as similar as possible to the audio synchronization delay, that is, the transmission link duration of each transmission link is as equal as possible, thereby effectively improving the audio synchronization of the audio system, and thus effectively improving the user experience. Furthermore, the audio synchronization delay spans between the initial transmission timestamp and the initial retrieval timestamp. In other words, the audio synchronization method of this application comprehensively considers the differences in signal transmission delay and link transmission delay of each transmission link. The audio synchronization unit performs delay compensation on the transmission links between the transmitter and each receiver based on the audio synchronization delay, so as to synchronize the transmission link duration of each transmission link as closely as possible to the audio synchronization delay. Based on this, it can effectively reduce the delay differences caused by the differences in hardware performance and processing capabilities of the receiver, while also effectively reducing the delay differences caused by the unstable network connection between the transmitter and receiver, thereby effectively improving the audio synchronization of the audio system and thus effectively improving the user experience.

[0018] [Attached Image Description]

[0019] Figure 1 is a schematic diagram of the method steps of the first embodiment of the audio synchronization method of this application;

[0020] Figure 2 is a schematic diagram of the method steps of step S600 of the audio synchronization method shown in Figure 1 according to an embodiment;

[0021] Figure 3 is a schematic diagram of the steps of the audio synchronization method shown in Figure 2, specifically step S630.

[0022] Figure 4 is a schematic diagram of the method steps of step S600 of the audio synchronization method shown in Figure 1;

[0023] Figure 5 is a schematic diagram of the steps of step S604 of the audio synchronization method shown in Figure 4;

[0024] Figure 6 is a schematic diagram of the method steps of the second embodiment of the audio synchronization method of this application;

[0025] Figure 7 is a schematic diagram of the structure of an embodiment of the audio system of this application;

[0026] Figure 8 is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application.

[0027]

Detailed Implementation Methods

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] The terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise expressly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0030] This application provides an audio synchronization method applied to an audio synchronization unit, which is a functional unit of an audio system. The audio system includes a transmitter and several receivers, wherein the transmitter and receivers are signal-connected to form multiple transmission links. The transmitter sends audio signals to each receiver, and the receivers perform various audio processing operations on the audio signals, such as equalizer (EQ) adjustment, dynamic range control (DRC), crossover, and bass boost. The receivers then play the audio signals after processing to improve sound quality and enhance the listening experience. The inclusion of several receivers in the audio system can be understood as the audio system including multiple receivers. For example, in some embodiments, the number of transmitters in the audio system is one, while the number of receivers can be two, three, four, or other orders of magnitude. The transmitter is signal-connected to each receiver via wireless and / or wired networks, enabling each receiver to process and play the same type of audio information. Because the hardware performance and processing capabilities of each receiver may differ, even if the signal transmission delay between the transmitter and receiver is synchronized, hardware differences at the receiver will still lead to differences in audio latency. This results in variations in the transmission link duration for each link, affecting the audio playback synchronization of the audio system and consequently impacting the user experience. Signal transmission delay is the time required for the transmitter to send an audio signal to the receiver and for the receiver to receive the audio signal; this time is also referred to as signal transmission delay in this article. Signal transmission delay is determined by the connection network between the transmitter and receiver. Link transmission delay is primarily determined by the hardware performance and processing capabilities of the receiver; in other words, link transmission delay is the time required for the receiver to perform audio processing operations on the audio signal.

[0031] In some embodiments, the entity executing the audio synchronization method is an audio synchronization unit. The audio synchronization method can be implemented through the method steps shown in Figure 1, specifically referring to steps S100 to S600:

[0032] Step S100: Obtain the initial transmission timestamp of the audio signal sent by the transmitting end.

[0033] After the audio system starts up, the transmitter establishes a signal connection with each receiver via a wireless or wired network and activates the audio synchronization unit. The transmitter can send a specific audio signal to each receiver synchronously or sequentially. This specific audio signal is also referred to as the delay correction audio signal (the delay correction audio signal is an audio signal that is imperceptible to the human ear; for example, it can be a square wave or sine wave greater than or equal to 18kHz. The purpose of sending the delay correction audio signal is to obtain the audio synchronization delay). The audio synchronization unit records the initial transmission timestamp of the delay correction audio signal sent by the transmitter to each receiver to prepare for obtaining the audio synchronization delay. It should be noted that if the transmitter sends the delay correction audio signal synchronously to each receiver, the audio synchronization unit only needs to record one initial transmission timestamp. If the transmitter sends the delay correction audio signal sequentially to each receiver, the audio synchronization unit needs to record each initial transmission timestamp when the transmitter sends the delay correction audio signal to each receiver.

[0034] Step S200: Obtain the received audio signals from several receivers. Each receiver's received audio signal includes an initialization receiving timestamp and an initialization retrieval timestamp.

[0035] The initial reception timestamp is the timestamp at which the receiver receives the delay-corrected audio signal. The audio synchronization unit is also used to re-sample the audio signal after the audio processing operation at the receiver, and record the timestamp of the received audio signal after the audio processing operation, so as to use this timestamp as the initial re-sample timestamp.

[0036] It should be noted that obtaining the initial reception timestamp and initial retrieval timestamp of the received audio signals from several receiving ends can be understood as the audio synchronization unit obtaining the initial reception timestamp and initial retrieval timestamp of the received audio signals from each receiving end in the audio system.

[0037] Step S300: Determine the link transmission delay of the receiving end based on the initial reception timestamp and initial retrieval timestamp of the received audio signal at the receiving end.

[0038] The audio synchronization unit determines the link transmission delay corresponding to each receiving end based on the initial reception timestamp and initial retrieval timestamp obtained in step S200. In this embodiment, after obtaining the initial reception timestamp and initial retrieval timestamp, the audio synchronization unit can obtain the link transmission delay by subtracting the initial reception timestamp and the initial retrieval timestamp. In other words, the link transmission delay is the difference between the initial retrieval timestamp and the initial reception timestamp. It should be noted that determining the link transmission delay of several receiving ends can be understood as the audio synchronization unit determining the link transmission delay of each receiving end.

[0039] Step S400: Determine the target delay and the target receiving end corresponding to the target delay according to the link transmission delay corresponding to each of the receiving ends.

[0040] The audio synchronization unit obtains the link transmission delay of each receiving end in the manner described in step S300, and determines the target delay from the link transmission delays of several receiving ends.

[0041] In some embodiments, the audio synchronization unit sets the maximum value among the link transmission delays of a plurality of receiving ends as the target delay.

[0042] Step S500: Determine the audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp of the target receiver.

[0043] The audio synchronization delay is equal to the sum of the signal transmission delay and the target delay. After obtaining the target delay, initializing the transmission timestamp, and initializing the reception timestamp, the audio synchronization unit can calculate the signal transmission delay and then further calculate the audio synchronization delay.

[0044] Step S600: Perform delay compensation on the transmission links between the sending end and each receiving end according to the audio synchronization delay.

[0045] Specifically, after determining the audio synchronization delay, the audio synchronization unit performs delay compensation on the transmission links that require delay compensation based on the audio synchronization delay. This ensures that the transmission link duration after delay compensation for each transmission link is as similar as possible to the audio synchronization delay, thus making the transmission link duration of each transmission link as equal as possible. This effectively improves the audio synchronization of the audio system and consequently enhances the user experience. Furthermore, the audio synchronization delay spans between the initial transmission timestamp and the initial retrieval timestamp. In other words, the audio synchronization method of this application comprehensively considers the differences in signal transmission delay and link transmission delay for each transmission link. The audio synchronization unit performs delay compensation on the transmission links between the transmitter and each receiver based on the audio synchronization delay, synchronizing the transmission link duration of each transmission link as closely as possible to the audio synchronization delay. This effectively reduces delay differences caused by variations in the hardware performance and processing capabilities of the receiver, as well as delay differences caused by unstable network connections between the transmitter and receiver, thereby effectively improving the audio synchronization of the audio system and consequently enhancing the user experience.

[0046] It should be noted that, in this embodiment of the application, before delay compensation, the transmission link duration is equal to the signal transmission delay plus the link transmission delay. After delay compensation, the transmission link duration is equal to the sum of the signal transmission delay, the link transmission delay, and the delay compensation value (e.g., the delay compensation value described in steps S631 to S632 below, and the first delay compensation value and the second delay compensation value described below).

[0047] Optionally, in some embodiments, after the audio synchronization unit completes the delay compensation for each transmission link of the audio system through steps S100 to S600, it will also monitor the transmission link duration of each transmission link of each audio system in real time, so as to dynamically adjust the transmission link duration in a timely manner when the environment or equipment changes, and ensure the long-term audio synchronization of the audio system.

[0048] Optionally, in some embodiments, the specific implementation of step S600 can be achieved by the method steps shown in FIG2, specifically including steps S610 to S630, as follows:

[0049] Step S610: Obtain the current transmission link duration for each transmission link.

[0050] Specifically, obtaining the current transmission link duration of each transmission link can be understood as obtaining the current transmission link duration between the transmitting end and each receiving end. In some embodiments, the audio synchronization unit can re-execute steps S100 to S200 and obtain the current transmission timestamp and current retrieval timestamp of each transmission link in the manner described in steps S100 to S200. After obtaining the current transmission timestamp and current retrieval timestamp, the audio synchronization unit obtains the current transmission link duration based on the current transmission timestamp and current retrieval timestamp. Specifically, the current transmission link duration is the difference between the current retrieval timestamp and the current transmission timestamp, that is, the current transmission link duration is equal to the signal transmission delay plus the link transmission delay.

[0051] In some embodiments, the audio synchronization unit may also directly use the initial transmission timestamp obtained in steps S100 to S200 as the current transmission timestamp and the initial retrieval timestamp as the current retrieval timestamp to obtain the current transmission link duration.

[0052] Step S620: Determine whether delay compensation is needed for the transmission link.

[0053] Step S630: When the delay error is greater than the preset error value, perform delay compensation on the receiving end according to the audio synchronization delay.

[0054] Steps S620 to S630 will be described uniformly. Specifically, the audio synchronization unit compares the current transmission link duration of each transmission link with the audio synchronization delay to determine the transmission links that require delay compensation. Then, it performs delay compensation on these transmission links to ensure that the transmission link duration of each transmission link is as synchronized as possible with the audio synchronization delay. More specifically, the audio synchronization unit obtains the delay error between the current transmission link duration and the audio synchronization delay. The delay error is equal to the absolute value of the difference between the current transmission link duration and the audio synchronization delay. If the delay error is less than a preset error value, the audio synchronization unit determines that the transmission link does not require delay compensation. If the delay error is greater than the preset error value, the audio synchronization unit determines that the transmission link requires delay compensation and executes step S630 to perform delay compensation on the transmission link (it should be noted that in this document, delay compensation on the transmission link is also referred to as delay compensation on the receiving end). In other words, when the delay error is greater than the preset error value, it means that the link transmission delay of the receiving end is less than the target delay. Therefore, delay compensation is needed for the receiving end so that the transmission link duration corresponding to the receiving end matches the audio synchronization delay, thereby ensuring the audio synchronization of the audio system.

[0055] Optionally, in some embodiments, step S630 is implemented through the method steps shown in FIG3, specifically including steps S631 to S632, as follows:

[0056] Step S631: Determine the delay compensation value of the receiving end according to the audio synchronization delay and the current transmission link duration.

[0057] Specifically, the delay compensation value is the absolute value of the difference between the audio synchronization delay and the current transmission link duration. The audio synchronization unit can obtain the delay compensation value by subtracting the acquired audio synchronization delay from the current transmission link duration.

[0058] Step S632: Adjust the timestamp of the audio signal sent from the transmitter to the receiver according to the delay compensation value.

[0059] Specifically, since the link transmission delay at the receiving end is determined by hardware performance and processing capabilities, directly adjusting the link transmission delay at the receiving end requires upgrading or replacing the hardware, which is costly. Therefore, in this embodiment, after obtaining the delay compensation value, the audio synchronization unit adjusts the transmission link duration by adjusting the timestamp of the audio signal sent from the transmitting end to the receiving end. This effectively saves audio synchronization costs in the audio system. More specifically, after obtaining the delay compensation value corresponding to each receiving end, the audio synchronization unit adjusts the timestamp of the audio signal sent from the transmitting end to the receiving end based on the delay compensation value to reduce the difference in transmission link duration between each transmission link, thereby effectively improving the audio synchronization of the audio system. For example, relative to the reference receiver (the receiver whose link transmission delay is the target delay), if the delay compensation value of the receiver to be compensated is 2 milliseconds, then the transmitter needs to start sending the audio signal to the receiver to be compensated 2 milliseconds after it starts sending the audio signal to the reference receiver. This makes the transmission link duration of the transmission link corresponding to the receiver to be compensated the same as the transmission link duration corresponding to the reference receiver, thereby effectively reducing the difference in transmission link duration of each transmission link and thus effectively improving the audio synchronization of the audio system.

[0060] Of course, in some embodiments, the audio synchronization unit may omit steps S620 and S630, thus reducing the computational burden on the audio unit. Specifically, after executing step S610, the audio synchronization unit directly executes steps S631 to S632 to perform delay compensation for each receiving end.

[0061] In some embodiments, the specific implementation of step S600 can also be achieved through the method steps shown in FIG4, specifically including steps S601 to S604, as follows:

[0062] Step S601: Obtain the first delay compensation step size.

[0063] Specifically, the audio synchronization unit can determine the first delay compensation step size according to the audio sampling frequency. The audio sampling frequency is the same as the signal frequency of the audio signal transmitted by the transmitting end; in other words, the audio synchronization unit can determine the first delay step size according to the current signal frequency of the audio signal currently transmitted by the transmitting unit. Specifically, the audio synchronization unit can determine the first delay compensation step size according to the current signal frequency of the audio signal currently transmitted by the transmitting unit, and the first delay compensation step size is not greater than the reciprocal of the current signal frequency. For example, if the current signal frequency is 48kHz, the first delay compensation step size can be set to 20.83µs, that is, the reciprocal of the current signal frequency can be directly used as the first delay compensation step size.

[0064] Step S602: Determine the first delay compensation value according to the first delay compensation step size.

[0065] Specifically, the audio synchronization unit determines the first delay compensation value based on the current iteration count and the first delay compensation step size. In other words, the first delay compensation value is equal to the first delay compensation step size multiplied by the iteration count. For example, if the current iteration count is 1, then the first delay compensation value is equal to the first delay compensation step size. The audio synchronization unit is equipped with an iteration counter, which can start counting from 0. In other words, the first delay compensation value can start from 0 and iterate according to the first delay compensation step size until the required delay compensation value is obtained.

[0066] Step S603: Adjust the timestamp of the audio signal sent from the transmitter to the receiver according to the first delay compensation value, and obtain the current transmission link duration.

[0067] Specifically, after obtaining the first delay compensation value as described above, the audio synchronization unit uses the first delay compensation value as the delay value for the transmitter to send the audio signal to the receiver, and obtains the current transmission link duration. In this embodiment, the current transmission link duration refers to the transmission link duration after delay compensation. In other words, the current transmission link duration is equal to the sum of the first delay compensation value, the signal transmission delay, and the link transmission delay.

[0068] Step S604: When the delay error of the current transmission link duration and audio synchronization delay is not within the preset error range, determine the second delay compensation value according to the first delay compensation step size, until the delay error of the compensated transmission link duration and audio synchronization delay is within the preset error range.

[0069] Specifically, the current transmission link duration described in step S604 refers to the transmission link duration after compensation by the first delay compensation value. After the audio synchronization unit performs delay compensation on the transmission link in step S603, it compares the compensated transmission link duration with the audio synchronization delay and obtains the absolute value of the difference between the current transmission link duration and the audio synchronization delay to obtain the delay error between the current transmission link duration and the audio synchronization duration. If the delay error between the current transmission link duration and the audio synchronization delay is not within the preset error range, the audio synchronization unit executes step S602 again to enter the next iteration to obtain the second delay compensation value. It then adjusts the timestamp of the audio signal sent from the transmitting end to the receiving end using the second delay compensation value in the manner described in step S603 to obtain the transmission link duration after compensation by the second delay compensation value. If the delay error between the compensated transmission link duration and the audio synchronization delay is within the preset error range, the audio synchronization unit stops iterating, and the delay compensation of the transmission link by the audio synchronization unit is completed. If the delay error of the compensated transmission link duration and audio synchronization delay is not within the preset error range, the audio synchronization unit will continue to perform the next iteration until the delay error of the compensated transmission link duration and audio synchronization delay is within the preset error range.

[0070] In some embodiments, the specific implementation of step S604 can be achieved through the method steps shown in FIG5, specifically including steps S6041 to S6043, as follows:

[0071] Step S6041: Determine the second delay compensation value according to the first delay compensation step size until the delay error of the compensated transmission link duration and audio synchronization delay is within the preset error range, or end the current iteration and enter the next iteration after compensating a preset number of times.

[0072] Specifically, if the audio synchronization unit performs iterative compensation according to the first delay compensation step size and after compensating for a preset number of times, the obtained delay compensation value still cannot make the delay error of the compensated transmission link duration and audio synchronization delay within the preset error range, then the audio synchronization unit ends the current iteration (the current iteration refers to the iterative compensation performed by the audio synchronization unit through the first delay compensation step size) and executes steps S6042 to S6043 to enter the next iteration.

[0073] Specifically, after the audio synchronization unit performs delay compensation on the transmission link in step S603, it determines whether the delay error between the compensated transmission link duration and the audio synchronization delay is within a preset error range. If the delay error between the compensated transmission link duration and the audio synchronization delay is within the preset error range, the audio synchronization unit ends the current iteration, thus completing the delay compensation for the transmission link. If the delay error between the compensated transmission link duration and the audio synchronization delay is not within the preset error range, the audio synchronization unit further determines whether the number of iterations performed by the first delay compensation step size is greater than a preset number. If the number of compensations is not greater than the preset number, the audio synchronization unit further iterates and compensates the transmission link duration using the first delay compensation step size. If the number of compensations is greater than the preset number, the audio synchronization unit ends the current iteration and executes steps S6042 to S6043 to enter the next iteration.

[0074] Of course, in some embodiments, after determining that the delay error of the compensated transmission link duration and audio synchronization delay is not within the preset error range, the audio synchronization unit can further determine whether the delay error of the compensated transmission link duration and audio synchronization delay is greater than the preset error value. If the delay error is greater than the preset error value, the audio synchronization unit can directly execute steps S6042 to S6043 to enter the next iteration. Based on this, the compensation efficiency of the audio synchronization unit for delay compensation of the transmission link can be effectively improved.

[0075] Step S6042: When entering the next iteration, determine the second delay compensation step size according to the first delay compensation step size, wherein the second delay compensation step size is greater than the first delay compensation step size.

[0076] Specifically, the second delay compensation step size can be a preset multiple of the first delay compensation step size. For example, the second delay compensation step size can be two times, three times, or other specific multiples of the first delay compensation step size. Of course, in some embodiments, the second delay compensation step size can also be any value greater than the first delay compensation step size. The second delay compensation step size is not greater than the reciprocal of the current signal frequency.

[0077] Step S6043: Determine the third delay compensation value according to the second delay compensation step size until the delay error of the compensated transmission link duration and audio synchronization delay is within the preset error range, or start the next iteration.

[0078] Specifically, after acquiring the second delay compensation step size, the audio synchronization unit begins iterative compensation using the second delay compensation step size until the delay error between the compensated transmission link duration and the audio synchronization delay is within a preset error range. Of course, if the audio synchronization unit fails to bring the delay error between the compensated transmission link duration and the audio synchronization delay within the preset error range after performing iterative compensation using the second delay compensation step size a preset number of times, the audio synchronization unit can initiate the next iteration, that is, acquire a new delay compensation step size again and enter the next iteration with the new delay compensation step size. For details, please refer to the steps described above; they will not be elaborated upon here.

[0079] This application provides a second embodiment of an audio synchronization method, which is applied to an audio synchronization system. The audio synchronization system includes a transmitting end and several receiving ends. This embodiment can be implemented according to the method steps shown in Figure 6, specifically including steps S700 to S1200, as follows:

[0080] Step S700: The transmitting end sends audio signals to several receiving ends and records the initial transmission timestamp.

[0081] The implementation method of step S700 can be found in step S100, and will not be described in detail here.

[0082] Step S800: The receiving end receives the audio signal and performs hardware retrieval, recording the corresponding initial reception timestamp and initial retrieval timestamp.

[0083] The implementation method of step S800 can be found in step S200, and will not be described in detail here.

[0084] Step S900: The transmitting end determines the link transmission delay of several receiving ends based on the initial reception timestamp and initial retrieval timestamp of several receiving ends.

[0085] The implementation method of step S900 can be found in step S300, and will not be described in detail here.

[0086] Step S1000: The sending end determines the target delay based on the link transmission delay of several receiving ends.

[0087] The implementation method of step S1000 can be found in step S400, and will not be described in detail here.

[0088] Step S1100: The sending end determines the audio synchronization delay based on the target delay, the initialized sending timestamp, and the initialized receiving timestamp.

[0089] The implementation method of step S1100 can be found in step S500, and will not be described in detail here.

[0090] Step S1200: The transmitting end performs delay compensation on the transmission link between the transmitting end and each receiving end according to the audio synchronization delay.

[0091] The implementation method of step S1200 can be found in step S600, and will not be described in detail here.

[0092] This application provides an audio system, as shown in Figure 7. The audio system includes a transmitter 100 and several receivers 200. The transmitter 100 includes an audio transmission unit 120 and an audio correction unit 110. The receivers 200 include a control unit 210, an audio processing unit 220, and a hardware retrieval unit 230. The audio correction unit 110, the control units 210 of all receivers 200, and the hardware retrieval units 230 of all receivers 200 together constitute the audio synchronization unit of the audio system.

[0093] Audio transmitting unit 120 is used to transmit audio signals to several receiving ends 200; audio correction unit 110 is used to record the initial transmission timestamp; control unit 210 is used to receive audio signals and record the corresponding initial reception timestamp; audio processing unit 220 is used to perform audio processing operations on the audio signals; hardware retrieval unit 230 is used to retrieve the audio signals after audio processing operations back to control unit 210; control unit 210 is also used to record the initial retrieval timestamp; control unit 210 is used to determine the link transmission delay of receiving ends 200 based on the initial reception timestamp and the initial retrieval timestamp; audio correction unit 110 is used to determine the target delay according to the link transmission delay of several receiving ends 200; audio correction unit 110 is used to determine the audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp; audio correction unit 110 is used to perform delay compensation on the transmission link between audio transmitting unit 120 and each receiving end 200 according to the audio synchronization delay.

[0094] This application discloses a computer-readable storage medium. Please refer to Figure 8, which is a schematic diagram of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium is used to store program instructions, which, when executed by a processor, are used to implement the audio synchronization method of any of the above embodiments.

[0095] The computer-readable storage medium 30 of this application embodiment stores program instructions 31, which are executed to implement the above-described audio synchronization method.

[0096] Specifically, program instructions 31 can be formed into a program file and stored in the aforementioned storage medium in the form of a software product, so that an electronic device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0097] In this embodiment, the computer-readable storage medium 30 may be, but is not limited to, a USB flash drive, SD card, PD optical drive, portable hard drive, large-capacity floppy drive, flash memory, multimedia memory card, server, etc.

[0098] In one embodiment, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the steps described in the audio synchronization method embodiment.

[0099] Furthermore, if the aforementioned functions are implemented as software functions and sold or used as independent products, they can be stored in a mobile terminal-readable storage medium. That is, this application also provides a storage device storing program instructions that can be executed to implement the methods of the above embodiments. This storage device can be, for example, a USB flash drive, an optical disc, or a server. In other words, this application can be embodied in the form of a software product, which includes several instructions to cause a smart terminal to execute all or part of the steps of the methods described in the various embodiments.

[0100] In summary, after determining the audio synchronization delay, the audio synchronization unit performs delay compensation on the transmission links that require delay compensation based on the audio synchronization delay. This ensures that the transmission link duration after delay compensation for each transmission link is as similar as possible to the audio synchronization delay, effectively improving the audio synchronization of the audio system and thus enhancing the user experience. Furthermore, the audio synchronization delay spans between the initial transmission timestamp and the initial retrieval timestamp. In other words, the audio synchronization method of this application comprehensively considers the differences in signal transmission delay and link transmission delay for each transmission link. The audio synchronization unit performs delay compensation on the transmission links between the transmitter and each receiver based on the audio synchronization delay, synchronizing the transmission link duration of each transmission link as closely as possible to the audio synchronization delay. This effectively reduces delay differences caused by variations in the hardware performance and processing capabilities of the receiver, as well as delay differences caused by unstable network connections between the transmitter and receiver, thereby effectively improving the audio synchronization of the audio system and enhancing the user experience.

[0101] It is worth noting that the accompanying drawings are only for illustrating the structural and connection relationships of the product of this invention, and do not limit the specific structural dimensions of the product of this invention.

[0102] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An audio synchronization method, characterized by, The audio synchronization method is applied to an audio synchronization unit, and the audio synchronization method includes: Get the initial transmission timestamp of the audio signal sent by the sending end; Acquire received audio signals from several receivers. Each received audio signal from a receiver includes an initial reception timestamp and an initial retrieval timestamp. Based on the initial reception timestamp and the initial retrieval timestamp of the received audio signal at the receiving end, the link transmission delay of the receiving end is determined; Based on the link transmission delay corresponding to each of the aforementioned receivers, determine the target delay and the target receiver corresponding to the target delay. The audio synchronization delay is determined based on the target delay, the initial transmission timestamp, and the initial reception timestamp of the target receiver. The transmission links between the sending end and each receiving end are compensated for according to the audio synchronization delay.

2. The audio synchronization method according to claim 1, characterized in that, The step of determining the target delay according to the link transmission delay corresponding to each of the plurality of receiving ends includes: The maximum value among the link transmission delays corresponding to each of the plurality of receiving ends is set as the target delay.

3. The audio synchronization method according to claim 1, characterized in that, The delay compensation for the transmission links between the sending end and each receiving end according to the audio synchronization delay includes: Obtain the current transmission timestamp of the transmission link between the sending end and each receiving end, and the current retrieval timestamp of the transmission link of each receiving end; Based on the current transmission timestamp and the current retrieval timestamp of each receiving end, the current transmission link duration of each receiving end is obtained. Obtain the current transmission link duration and the delay error of the audio synchronization delay for each receiving end; When the delay error is greater than the preset error value, delay compensation is performed on each receiving end according to the audio synchronization delay.

4. The audio synchronization method according to claim 1, characterized in that, The delay compensation for the transmission links between the sending end and each receiving end according to the audio synchronization delay includes: Based on the initial transmission timestamp and the initial retrieval timestamp of the receiving end, the initial transmission link duration of the receiving end is obtained; The delay compensation value of the receiving end is determined according to the audio synchronization delay and the initial transmission link duration. The timestamp of the audio signal sent from the transmitting end to the receiving end is adjusted according to the delay compensation value.

5. The audio synchronization method according to claim 1, characterized in that, The delay compensation for the transmission links between the sending end and each receiving end according to the audio synchronization delay includes: Obtain the first delay compensation step size; The first delay compensation value is determined according to the first delay compensation step size; Adjust the timestamp of the audio signal sent from the transmitting end to the receiving end according to the first delay compensation value, and obtain the current transmission link duration; When the delay error of the current transmission link duration and the audio synchronization delay is not within the preset error range, a second delay compensation value is determined according to the first delay compensation step size until the delay error of the compensated transmission link duration and the audio synchronization delay is within the preset error range.

6. The audio synchronization method according to claim 5, characterized in that, The step of determining the second delay compensation value according to the first delay compensation step size until the delay error between the compensated transmission link duration and the audio synchronization delay is within the preset error range includes: The second delay compensation value is determined according to the first delay compensation step size until the delay error between the compensated transmission link duration and the audio synchronization delay is within the preset error range, or the current iteration ends and the next iteration begins after a preset number of compensations. When entering the next iteration, the second delay compensation step size is determined according to the first delay compensation step size; The third delay compensation value is determined according to the second delay compensation step size until the delay error between the compensated transmission link duration and the audio synchronization delay is within the preset error range, or the next iteration is started; Wherein, the second delay compensation step size is greater than the first delay compensation step size.

7. The audio synchronization method according to claim 5 or 6, characterized in that, The step of obtaining the first delay compensation step size includes: The first delay compensation step size is determined according to the audio sampling frequency.

8. An audio synchronization method, characterized by, The audio synchronization method is applied to an audio synchronization system, wherein the audio synchronization system includes a transmitting end and several receiving ends; the audio synchronization method includes: The transmitting end sends audio signals to the plurality of receiving ends and records the initial transmission timestamp; The receiving end receives the audio signal and performs hardware retrieval, recording the corresponding initial reception timestamp and initial retrieval timestamp. The transmitting end determines the link transmission delay corresponding to each of the plurality of receiving ends based on the initial receiving timestamp and initial retrieval timestamp of the plurality of receiving ends; The transmitting end determines the target delay and the target receiving end corresponding to the target delay according to the link transmission delay corresponding to each of the plurality of receiving ends; The transmitting end determines the audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp of the target receiving end; The transmitting end performs delay compensation on the transmission link between the transmitting end and each receiving end according to the audio synchronization delay.

9. An audio system, characterized by The audio system includes a transmitter and several receivers. The transmitter includes an audio transmitting unit and an audio correction unit. The receivers include a control unit, an audio processing unit, and a hardware retrieval unit. The audio transmitting unit transmits audio signals to the receivers. The audio correction unit records an initial transmission timestamp. The control unit receives the audio signals and records the corresponding initial reception timestamp. The audio processing unit performs audio processing operations on the audio signals. The hardware retrieval unit retrieves the processed audio signals back to the control unit. The control unit also records an initial retrieval timestamp. The control unit determines the link transmission delay of the receivers based on the initial reception timestamp and the initial retrieval timestamp. The audio correction unit determines a target delay and a target receiver corresponding to the target delay based on the link transmission delays of the receivers. The audio correction unit determines an audio synchronization delay based on the target delay, the initial transmission timestamp, and the initial reception timestamp of the target receiver. The audio correction unit compensates for the delay in the transmission links between the audio transmitter and each receiver based on the audio synchronization delay.

10. A computer-readable storage medium, characterized in that, The device stores program instructions that, when executed by a processor, implement the audio synchronization method as described in any one of claims 1-8.

Images