Multi-room audio synchronous playback method, system, device and storage medium

Abstract

The present invention relates to the field of audio playback technology, and discloses a multi-room audio synchronous playback method, system, device and storage medium, which are used to solve the problem that different speakers play the same music out of sync when multiple rooms play the same music. The multi-room audio synchronous playback method includes: using multiple audio devices that support a target wireless communication protocol to build a self-organizing network covering multiple rooms through point-to-point connection, and the target wireless communication protocol operates in the 5GHz to 7GHz frequency band; by dynamically analyzing the link quality, selecting the optimal transmission path, and dynamically optimizing the network topology of the self-organizing network through multi-hop relay; transmitting the target audio data between the audio devices in the self-organizing network according to the optimal transmission path, and performing real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters; according to the synchronization parameters, controlling each audio device in the self-organizing network to synchronously play the target audio data.

Description

Technical Field

[0001] The present invention relates to the field of audio playback technology, and in particular to a multi-room audio synchronous playback method, system, device and storage medium. Background Art

[0002] Traditional multi-room audio systems generally use home WiFi routers as data centers, and audio devices are connected to the router through protocols such as the Digital Living Network Alliance (DLNA) and Airplay for audio streaming and synchronous control.

[0003] On the one hand, the traditional networking mode is easily affected by the interference of the network environment, and the connection stability and transmission rate are difficult to guarantee; on the other hand, it is limited by the performance of the router and it is difficult to support the real-time transmission of multi-channel high-resolution audio. There are problems such as large playback delay and obvious sound quality loss, which cannot meet the needs of high-quality music appreciation. Summary of the invention

[0004] The present invention provides a multi-room audio synchronous playback method, system, device and storage medium, which are used to solve the problem that different speakers play the same music asynchronously when the multi-room audio plays the same music.

[0005] The first aspect of the present invention provides a multi-room audio synchronous playback method, comprising: using multiple audio devices that support a target wireless communication protocol to build a self-organizing network covering multiple rooms through point-to-point connection, wherein the target wireless communication protocol operates in the 5GHz to 7GHz frequency band; dynamically analyzing link quality to select the optimal transmission path, and dynamically optimizing the network topology of the self-organizing network through multi-hop relays; transmitting target audio data between audio devices in the self-organizing network according to the optimal transmission path, and performing real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters; and controlling each audio device in the self-organizing network to synchronously play the target audio data according to the synchronization parameters.

[0006] The second aspect of the present invention provides a multi-room audio synchronous playback system, comprising: multiple audio devices supporting a target wireless communication protocol, wherein the multiple audio devices build a self-organizing network covering multiple rooms through point-to-point connections, and the target wireless communication protocol operates in the 5GHz to 7GHz frequency band; each audio device comprises: a network module, which is used to select the optimal transmission path by dynamically analyzing the link quality, and dynamically optimize the network topology of the self-organizing network through multi-hop relays; a synchronization module, which is used to transmit target audio data between audio devices in the self-organizing network according to the optimal transmission path, and perform real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters; a playback module, which is used to synchronously play the target audio data according to the playback timestamp carried by the synchronization signal.

[0007] A third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned multi-room audio synchronous playback method when executed by the processor.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned multi-room audio synchronous playback method.

[0009] In the technical solution provided by the present invention, a plurality of audio devices supporting the target wireless communication protocol are used to build a self-organizing network covering multiple rooms through point-to-point connection. The audio devices dynamically analyze the link quality, select the transmission path with the lowest delay, and dynamically optimize the network topology through multi-hop relay, thereby reducing the transmission delay and realizing high-resolution audio data transmission between audio devices to meet the requirements of lossless sound quality transmission. The audio devices improve the accuracy of synchronous playback through real-time interaction and dynamic compensation of synchronization signals to meet the requirements of sound and picture synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of an embodiment of the multi-room audio synchronous playback method in this embodiment;

[0011] Figure 2 This is a schematic diagram of another embodiment of the multi-room audio synchronous playback method in this embodiment;

[0012] Figure 3 A schematic diagram of an embodiment of a multi-room audio synchronous playback system in this embodiment;

[0013] Figure 4 This is a schematic diagram of another embodiment of the multi-room audio synchronous playback system in this embodiment;

[0014] Figure 5 Schematic diagram of an embodiment of a computer device in this embodiment. DETAILED DESCRIPTION

[0015] The present invention provides a multi-room audio synchronous playback method, system, device and storage medium, which are used to solve the problem that different speakers play the same music asynchronously when the multi-room audio plays the same music.

[0016] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0017] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 In this embodiment, an embodiment of the multi-room audio synchronous playback method includes:

[0018] 101. Utilize multiple audio devices that support a target wireless communication protocol to build a multi-room ad hoc network through point-to-point connection. The target wireless communication protocol operates in the 5 GHz to 7 GHz frequency band.

[0019] It is understandable that the execution subject of the present invention may be a multi-room audio synchronous playback system, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a multi-room audio synchronous playback system as an example.

[0020] The multi-room audio synchronous playback system of this embodiment includes multiple audio devices, and the multiple audio devices can be distributed in multiple rooms. For example, in a multi-room audio playback system installed in a residence, one audio device can be located in the living room, while another audio device is located in the kitchen, and other audio devices can be located in various bedrooms of the house.

[0021] The multiple audio devices include one or more of a television, a projector, a surround sound box, a bass sound box, and a sound bar. The audio device may also include a left channel sound box and a right channel sound box.

[0022] Each audio device has a built-in network module that supports target wireless communication protocols running in the 5GHz to 7GHz frequency band, such as WiFi 6E, Wi-Fi 5, Wi-Fi 7 and other wireless communication protocol standards. It does not rely on ordinary home routers. Audio devices can independently establish a high-speed and stable peer-to-peer network, using optimized network topology and routing mechanisms, which greatly reduces data transmission delays.

[0023] By supporting the target wireless communication protocol in the 5-7GHz frequency band, it reduces co-channel interference and increases data transmission rate compared to the conventional 2.4GHz frequency band. It also utilizes the high-bandwidth characteristics of the target wireless communication protocol to achieve high-resolution audio data transmission between audio devices, ensuring sound quality.

[0024] Optionally, the audio device supports Direct Stream Digital (DSD) to provide high-resolution encoding requirements.

[0025] Optionally, the audio device supports the I2S transmission protocol for transmitting digital audio data between internal devices of the audio device. In the I2S transmission protocol, data signals, clock signals and control signals are transmitted through data signal lines, clock signal lines and left and right channel selection signal lines, respectively.

[0026] Optionally, the audio device integrates dual-core CPU and DSP chips, adopts high-performance processing chips such as BES2610, and is equipped with 1MB SRAM and 8MB Flash storage resources.

[0027] Specifically, a device is selected as a main audio device from a plurality of audio devices supporting a target wireless communication protocol, and a group establishment request is sent to other audio devices within a target playback range through the main audio device; if there is at least one occupied device, a priority comparison is performed based on a first scene mode corresponding to the multi-room audio playback request and a second scene mode corresponding to each occupied device; when the priority of the first scene mode is higher than the priority of the second scene mode, the occupied device is controlled to release the occupied state, and joins the synchronous audio group corresponding to the main audio device, an ad hoc network corresponding to the synchronous audio group is established, and a delay compensation corresponding to the main audio device and each slave audio device in the ad hoc network is determined.

[0028] The occupied state is used to indicate the audio device, for example, the slave audio device is executing other audio tasks, or has joined other synchronous audio groups.

[0029] In this embodiment, the first scene mode and the second scene mode are used to indicate the audio playback scenes corresponding to two audio groups or two audio devices. The audio playback scenes include interactive intercom scenes such as meetings and telephones, and can also be background music scenes, home theater scenes, etc. The priority of the audio playback scene can be set according to user habits. For example, the priority order from high to low is: interactive intercom scene, home theater scene and background music scene. When the first scene mode corresponding to the multi-room audio playback request is the interactive intercom scene, and the second scene mode corresponding to the occupied device is the background music scene, the occupied device is controlled to release the occupied state and join the self-organizing network corresponding to the multi-room audio playback request for control, so as to adapt to the intelligent allocation of audio playback resources, ensure that scenes with higher importance are given priority, and improve the flexibility of the multi-room audio system. Users do not need to manually operate each audio device one by one, which greatly improves the user experience.

[0030] The master audio device of this embodiment serves as the control center of the room audio synchronous playback system and is responsible for clock synchronization, audio data reception, processing and forwarding in the ad hoc network. Therefore, the master audio device is often the audio device with the best performance within the target playback range.

[0031] 102. By dynamically analyzing the link quality, the optimal transmission path is selected, and the network topology of the self-organizing network is dynamically optimized through multi-hop relay.

[0032] In this embodiment, the link quality is a performance evaluation index of the communication link between the master audio device and any slave audio device in the ad hoc network. The optimal transmission path between the master audio device and any slave audio device can be determined through the network information of each audio device. The optimal transmission path includes at least one audio device as a relay node to achieve relay forwarding. It is suitable for multi-room audio synchronous playback scenarios. The network topology is dynamically optimized by monitoring the network information of each audio device and the exit and entry of audio devices in the ad hoc network.

[0033] The audio device of this embodiment dynamically analyzes the link quality, selects the transmission path with the lowest delay, and dynamically optimizes the network topology through multi-hop relay, so that the end-to-end transmission delay is controlled within 10ms, thereby achieving real-time audio synchronization.

[0034] 103. Transmit target audio data between audio devices in the ad hoc network according to an optimal transmission path, and perform real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters.

[0035] The ad hoc network includes a master audio device and multiple slave audio devices; the master audio device receives the audio data to be played sent by the user terminal or wireless router, and encodes and decodes the audio data to be played to obtain the target audio data. The master audio device distributes the target audio data to the slave audio devices according to the optimal transmission path.

[0036] The format of the above target audio data is 24bit / 192kHz pulse code modulation PCM format.

[0037] It can be understood that the delay compensation between the master audio device and each slave audio device in the ad hoc network refers to compensating for the delay in audio data transmission to achieve synchronous playback based on the network topology and the characteristics of each audio device, and the synchronization accuracy of the ad hoc network is guaranteed by determining the delay compensation between the master audio device and each slave audio device.

[0038] The synchronization parameters of this embodiment include a target playback timestamp corresponding to each audio device, and are used to control the audio devices in the ad hoc network to play synchronously.

[0039] Specifically, any audio device in the ad hoc network is determined as a reference clock device, and a synchronization signal is broadcast according to a preset clock synchronization period, wherein the synchronization signal includes a reference playback timestamp; the target audio device and the reference clock device are synchronized according to a preset target filtering algorithm, and the target audio device corresponding to the target playback timestamp is determined according to the reference playback timestamp, and the target audio device is an audio device in the ad hoc network that receives the synchronization signal.

[0040] 104. Control each audio device in the ad hoc network to synchronously play the target audio data according to the synchronization parameter.

[0041] Specifically, the play timestamp corresponding to each audio device in the ad hoc network is determined according to the synchronization parameter; and the target audio data is played by each audio device in the ad hoc network according to the synchronized local clock and the play timestamp corresponding to each audio device.

[0042] The technical solution provided by the present invention utilizes multiple audio devices that support the target wireless communication protocol to build a self-organizing network covering multiple rooms through point-to-point connection. The audio devices dynamically analyze the link quality, select the transmission path with the lowest delay, and dynamically optimize the network topology through multi-hop relay, thereby reducing the transmission delay and realizing high-resolution audio data transmission between audio devices to meet the demand for lossless sound quality transmission; the audio devices improve the accuracy of synchronous playback through real-time interaction and dynamic compensation of synchronization signals to meet the demand for sound and picture synchronization.

[0043] See also Figure 2 Another embodiment of the multi-room audio synchronous playback method in this embodiment includes:

[0044] 201. Using multiple audio devices that support a target wireless communication protocol, an ad hoc network covering multiple rooms is constructed through point-to-point connections. The target wireless communication protocol operates in the 5 GHz to 7 GHz frequency band.

[0045] Step 201 can be performed with reference to step 101 and will not be described in detail here.

[0046] 202. By dynamically analyzing the link quality, the optimal transmission path is selected, and the network topology of the self-organizing network is dynamically optimized through multi-hop relay.

[0047] This embodiment proposes an optimal path selection algorithm based on link quality weighting when selecting the optimal transmission path by dynamically analyzing the link quality. The algorithm constructs a weighted connectivity graph with each audio device as a node, comprehensively considers the two key indicators of link signal strength and delay, and can quantitatively evaluate the transmission quality of different paths, thereby selecting the optimal transmission path with the lowest delay and the strongest anti-interference ability to each slave audio device.

[0048] Specifically, any audio device in the ad hoc network is determined as the main audio device; the main audio device detects surrounding audio devices according to a first preset period to obtain network information corresponding to each slave audio device, and broadcasts the network information corresponding to the main audio device to the surrounding audio devices; the link quality between any two audio devices is determined according to the network information corresponding to each audio device ,in, Indicates audio device and audio equipment The link quality score between express and The signal strength of the link between Indicates the link delay, and is a weighting coefficient; a weighted connectivity graph is constructed based on the link quality, and a transmission path with the largest sum of link qualities from the master audio device to each slave audio device is determined as the optimal transmission path; when the network topology corresponding to the ad hoc network changes, the optimal transmission path corresponding to the master audio device and each slave audio device is updated.

[0049] In this embodiment, the network information of each audio device includes the signal strength and link delay of the link between the audio device and other audio devices. and In order to be able to set according to the actual situation, . Optionally, and The normalized values ​​are mapped to the same scale to improve the rationality of the score.

[0050] Specifically, each audio device periodically exchanges beacon frames to collect the received signal strength (RSSI) and round-trip time (RTT) data of the link and share them with the main audio device; the main audio device calculates the quality score of the link between devices based on the collected link data through the link quality evaluation function and constructs a weighted connectivity graph; starting from the main audio device, run the shortest path algorithm such as Dijkstra to find the transmission path with the largest sum of the quality scores of each slave audio device as the optimal transmission path; when a new audio device enters or leaves the network or the link status changes, the weighted connectivity graph and the optimal transmission path are updated in real time. This embodiment and this algorithm comprehensively consider the channel status and delay factors, overcome the limitations of relying on a single indicator, such as the Ad hoc On-Demand Distance Vector Routing (AODV), and ensure efficient and reliable transmission of audio data in harsh wireless transmission environments, with end-to-end delay controlled at 5ms-10ms.

[0051] In this embodiment, the network topology corresponding to the self-organizing network changes, which may include the withdrawal of audio devices from the group and the entry of audio devices into the group, as well as the change of network information of each audio device resulting in the need to redetermine the optimal transmission path. The update of the optimal transmission path can be performed with reference to the aforementioned method for determining the optimal transmission path, that is, broadcasting and path determination through the main audio device.

[0052] 203. Encode and decode the audio to be played to obtain target audio data.

[0053] The audio data to be played is obtained through the main audio device corresponding to the self-organizing network, and the audio type corresponding to the audio data to be played is obtained by parsing; the target encoding algorithm is determined according to the audio type of the audio data to be played and the network bandwidth information; the audio data to be played is divided into multiple original audio segments according to the preset frame period of the preset I2S interface through the main audio device, and each original audio segment is encoded according to the target encoding algorithm to obtain the target audio data.

[0054] In this embodiment, the frame period preset by the I2S interface is the LR clock period corresponding to the left and right channel selection signals, and the original audio clip in each I2S LR clock period is mapped into one network packet to reduce the packet header overhead and improve the transmission efficiency.

[0055] In this embodiment, the encoding algorithm can adopt lossless audio encoding formats such as Direct Stream Digital (DSD) to adapt to the loss of high-resolution sound quality, and audio compression encoding based on Bluetooth A2DP (Advanced Audio Distribution Profile) to reduce the amount of transmitted data, such as SBC encoding algorithm and apt-X compression algorithm, which can reach 70%. Through efficient audio encoding methods, while ensuring relatively high-quality audio, the delay of audio data transmission is reduced, and the audio synchronization performance between master and slave audio devices is improved.

[0056] 204. Transmit the target audio data to a next audio device according to the optimal transmission path, and the next audio device determines whether to forward the target audio data according to the current hop number corresponding to the target audio data, until the target audio data is transmitted to the end audio device of the optimal transmission path.

[0057] The target audio data is transmitted to the next audio device according to the optimal transmission path through the main audio device; the receiving target audio data is determined as the first audio device, and the first audio device is used to determine whether the current hop number corresponding to the target audio data is less than the set hop number of the optimal transmission path; if so, the target audio data is sent to the second audio device according to the optimal transmission path through the first audio device, and the current hop number corresponding to the target audio data is increased by one, and the second audio device is used to determine whether to forward the target audio data; if not, the first audio device is determined as the end point audio device of the optimal transmission path.

[0058] 205. Determine any audio device in the ad hoc network as a reference clock device, and broadcast a synchronization signal according to a preset clock synchronization period, where the synchronization signal includes a reference playback timestamp.

[0059] In this embodiment, the reference clock device can be a master audio device or a slave audio device. The reference clock device broadcasts a synchronization signal according to a preset clock synchronization period. The synchronization signal includes a reference playback timestamp and local clock information.

[0060] Among them, the local clock information of the reference clock device is used as a benchmark to achieve audio synchronization of audio devices in the ad hoc network, and the reference playback timestamp is used as a benchmark to confirm the playback timestamps of other audio devices to overcome the delay problem caused by the hardware gap between different audio devices.

[0061] 206. Perform clock synchronization on the target audio device and the reference clock device according to a preset target filtering algorithm, and determine the target audio device corresponding to the target playback timestamp according to the reference playback timestamp, and the target audio device is an audio device that receives the synchronization signal in the ad hoc network.

[0062] The clock deviation between the target audio device and the reference clock device is filtered according to a preset target filtering algorithm to obtain an optimal clock deviation estimate; the local clock of the target audio device is compensated according to the optimal clock deviation estimate; the transmission time value between the sound card and the speaker in the target audio device is determined to obtain a delay calibration value corresponding to the target audio device; and the target playback timestamp corresponding to the target audio device is determined according to the reference playback timestamp and the delay calibration value corresponding to the target audio device.

[0063] In this embodiment, the target playback timestamp is determined based on the delay calibration value and the reference playback timestamp, which can eliminate the hardware parameter differences between the playback devices, so that the time base of the target audio device and the reference clock device to output audio remains consistent. Combined with the synchronized local clock, the accuracy of self-organizing network synchronous playback is improved, even to the microsecond level.

[0064] The clock deviation is the deviation value between the local clock of the target audio device and the local clock information of the reference clock device. Specifically, the receiving time of the synchronization signal received by each target audio device and the sending time of the synchronization signal sent by the reference clock device are subtracted to obtain the first delay time corresponding to each target audio device; the feedback receiving time and the corresponding feedback sending time of the feedback signal received by the reference clock device from each target audio device are subtracted to obtain the second delay time corresponding to each target audio device; the first delay time and the second delay time corresponding to each slave audio device are half-differenced to obtain the clock deviation corresponding to each slave audio device, that is, the clock deviation of each target audio device and the reference clock device is half of the difference between the first delay time and the second delay time.

[0065] In actual application, the reference clock device periodically sends a synchronization signal. Since the device conditions and clock drift rates of each audio device are different, the clock deviation calculated in each clock synchronization cycle may change. This embodiment estimates the next possible clock deviation through a target filtering algorithm to improve synchronization accuracy.

[0066] It can be understood that when the reference clock device is the master audio device and the target audio device is the slave audio device in the ad hoc network, when the master audio device is the slave audio device, the target audio device includes the master audio device and other slave audio devices, which improves the flexibility of clock synchronization. When the reference clock device is the slave audio device, the master audio device can allocate more computing resources to the processing of audio data and the maintenance of network topology, while the slave audio device performs clock synchronization of the ad hoc network, which reduces the performance requirements for the master audio device.

[0067] In this embodiment, the target filtering algorithm can be an improved Kalman filtering algorithm. Specifically, a state space model of the clock deviation is established, the clock deviation and the drift rate are used as state variables, a state transfer equation and an observation equation are constructed, and the actually observed clock deviation is calculated according to the timestamp data exchanged between the master audio device and the slave audio device; the state transfer equation is predicted by using the Kalman filter to obtain a priori estimates of the predicted clock deviation and drift rate; the Kalman gain is calculated, and the a priori estimate is updated by using the observation equation to obtain a posteriori estimate, and the local clock of the slave audio device is compensated by using the posteriori estimate to achieve synchronization with the clock of the master audio device; Kalman filtering is performed according to the actually observed clock deviation to achieve continuous dynamic clock synchronization.

[0068] Optionally, the clock deviation, clock drift rate, network delay time and audio buffer occupancy rate between each target audio device and the reference clock device are used as state variables to construct a state transfer equation and an observation equation; the reference clock device is controlled to exchange timestamp data with each target audio device according to the clock synchronization cycle to obtain a clock deviation observation value corresponding to each target audio device; the clock deviation of the next clock synchronization cycle is predicted according to the current state variable matrix and the state transfer equation to obtain a clock deviation prediction value corresponding to each target audio device; the state variable matrix is ​​updated through the observation equation, the clock deviation prediction value corresponding to each target audio device and the clock deviation prediction value corresponding to each target audio device, and prediction is performed based on the updated state variable matrix to obtain an optimal clock deviation estimate value corresponding to each target audio device; clock synchronization is performed on each target audio device in the ad hoc network according to the optimal clock deviation estimate value corresponding to each target audio device.

[0069] The specific process of the improved Kalman filter algorithm is described below:

[0070] Establish the state space model of clock deviation, assuming The state vector of a synchronization cycle is ,in, is the clock deviation, is the clock drift rate, is the network delay time, is the audio buffer occupancy rate;

[0071] The above state transfer equation is: ,in, No. The state vector of the synchronization cycle, is the state transfer matrix, is the control input matrix, To control the input vector (such as audio parameter adjustment, etc.), is process noise, which obeys a multivariate Gaussian distribution with a mean of 0. and The specific form is determined according to the prior statistical model of parameters such as the audio coding format and sampling rate, and sparse representation is used to reduce the computational complexity.

[0072] Set up The observation vector of the synchronization period ,in, The clock deviation observed by exchanging timestamps is called clock deviation observation. Actual measured audio buffer occupancy

[0073] The observation equation can be expressed as: , among which, among which, is the observation matrix, The observation noise follows a multivariate Gaussian distribution with a mean of 0. The specific form of is determined according to the network topology and takes into account the spatial correlation of observation noise among different devices.

[0074] According to the clock deviation observation value corresponding to each target audio device, the state variable at the next moment is predicted by the state transfer equation, and the prediction result is corrected by using the observation equation to obtain the optimal estimation value of the clock deviation and drift rate.

[0075] The prediction process of the target filtering algorithm, ,in, Indicates The predicted estimate of the state of the synchronization cycle, Indicates The predicted estimate of the state for the synchronization period.

[0076] ,in, Indicates The synchronization period is the predicted value of the covariance, Indicates The synchronization period is the predicted value of the covariance, Represents the covariance matrix of the process noise.

[0077] The update process of the target filtering algorithm:

[0078]

[0079]

[0080]

[0081] in, Indicates The Kalman gain matrix of the synchronization period is optimized in real time by the gradient descent algorithm. Indicates The covariance matrix of the observation noise for synchronization periods.

[0082] Through the iterative prediction and update process, the optimal estimated trajectory of each state variable can be obtained. The estimated clock deviation is applied to the local clock compensation of the slave audio device to achieve synchronization with the master audio device clock.

[0083] To facilitate understanding, an embodiment is provided in which the above improved Kalman filter algorithm is used for clock synchronization in a wireless audio system consisting of one master audio device and four slave devices. The sampling rate of the system is 44.1kHz, and the audio format is 16-bit PCM.

[0084] At the initial stage of the algorithm, the process noise covariance matrix can be set to ;

[0085] The observation noise covariance matrix is ​​set as ;

[0086] Kalman gain matrix .

[0087] During the operation of the system, a synchronization cycle is triggered every 10 seconds to exchange timestamp data. Using the observed data, the state vector of each target audio device is dynamically estimated through the improved Kalman filter algorithm mentioned above. Based on the estimated clock deviation, the local clock is compensated from the device.

[0088] This embodiment introduces targeted improvements under the traditional Kalman filter framework, establishes a clock deviation dynamic model containing multiple key state variables, and comprehensively considers the influence of multiple factors such as the clock system and network conditions; fully utilizes the prior knowledge and statistical characteristics of the multi-room audio system, introduces a control input mechanism, and provides a more flexible adjustment method for clock synchronization; fully utilizes multiple observables to improve the applicability and robustness of the algorithm. Compared with traditional least squares estimation, PI control and other methods, this algorithm can adapt to complex and changeable network environments, dynamically track changes in system states, and can significantly improve estimation accuracy and reduce random errors. Experiments show that the algorithm can control the synchronization error at the sub-microsecond level, laying a solid foundation for achieving a highly synchronized immersive audio experience. While reducing the computational complexity, the synchronization accuracy and robustness are improved. At the same time, the clock synchronization and audio scheduling are organically combined to further optimize the overall performance of the system, thereby effectively improving the synchronization accuracy and providing a solid technical guarantee for high-quality multi-room audio playback.

[0089] 207. Control each audio device in the ad hoc network to synchronously play the target audio data according to the synchronization parameter.

[0090] Determine the playback timestamp corresponding to each audio device in the ad hoc network according to the synchronization parameters; and play the target audio data through each audio device in the ad hoc network according to the synchronized local clock and the playback timestamp corresponding to each audio device.

[0091] The technical solution provided by the present invention utilizes multiple audio devices that support the target wireless communication protocol to build a self-organizing network covering multiple rooms through point-to-point connection. The audio devices dynamically analyze the link quality, select the transmission path with the lowest delay, and dynamically optimize the network topology through multi-hop relay, thereby reducing the transmission delay and realizing high-resolution audio data transmission between audio devices to meet the demand for lossless sound quality transmission; the audio devices improve the accuracy of synchronous playback through real-time interaction and dynamic compensation of synchronization signals to meet the demand for sound and picture synchronization.

[0092] The above describes the multi-room audio synchronous playback method in this embodiment. The following describes the multi-room audio synchronous playback system in this embodiment. Figure 3 In this embodiment, an embodiment of a multi-room audio synchronous playback system includes:

[0093] Support multiple audio devices of the target wireless communication protocol, where multiple audio devices build a multi-room ad hoc network through point-to-point connections, and the target wireless communication protocol operates in the 5 GHz to 7 GHz frequency band;

[0094] Each audio device includes:

[0095] The network module 301 is used to select the optimal transmission path by dynamically analyzing the link quality and dynamically optimize the network topology of the ad hoc network through multi-hop relay;

[0096] The synchronization module 302 is used to transmit the target audio data between the audio devices in the ad hoc network according to the optimal transmission path, and to perform real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters;

[0097] The playing module 303 is used to synchronously play the target audio data according to the playing timestamp carried by the synchronization signal.

[0098] The technical solution provided by the present invention utilizes multiple audio devices that support the target wireless communication protocol to build a self-organizing network covering multiple rooms through point-to-point connection. The audio devices dynamically analyze the link quality, select the transmission path with the lowest delay, and dynamically optimize the network topology through multi-hop relay, thereby reducing the transmission delay and realizing high-resolution audio data transmission between audio devices to meet the demand for lossless sound quality transmission; the audio devices improve the accuracy of synchronous playback through real-time interaction and dynamic compensation of synchronization signals to meet the demand for sound and picture synchronization.

[0099] See also Figure 4 Another embodiment of the multi-room audio synchronous playback system in this embodiment includes:

[0100] Support multiple audio devices of the target wireless communication protocol, where multiple audio devices build a multi-room ad hoc network through point-to-point connections, and the target wireless communication protocol operates in the 5 GHz to 7 GHz frequency band;

[0101] Each audio device includes:

[0102] The network module 301 is used to select the optimal transmission path by dynamically analyzing the link quality and dynamically optimize the network topology of the ad hoc network through multi-hop relay;

[0103] The synchronization module 302 is used to transmit the target audio data between the audio devices in the ad hoc network according to the optimal transmission path, and to perform real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters;

[0104] The playing module 303 is used to synchronously play the target audio data according to the playing timestamp carried by the synchronization signal.

[0105] Optionally, the network module 301 is used to determine any audio device in the ad hoc network as the main audio device; the main audio device detects the surrounding audio devices according to the first preset period to obtain the network information corresponding to each slave audio device, and broadcasts the network information corresponding to the main audio device to the surrounding audio devices; determines the link quality between any two audio devices according to the network information corresponding to each audio device ,in, Indicates audio device and audio equipment The link quality between express and The signal strength of the link between Indicates the link delay, and is a weighting coefficient; a weighted connectivity graph is constructed based on the link quality, and a transmission path with the largest sum of link qualities from the master audio device to each slave audio device is determined as the optimal transmission path; when the network topology corresponding to the ad hoc network changes, the optimal transmission path corresponding to the master audio device and each slave audio device is updated.

[0106] Optionally, the synchronization module 302 includes:

[0107] The audio codec unit 3021 is used to encode and decode the audio to be played to obtain target audio data;

[0108] The transmission unit 3022 is used to transmit the target audio data to the next audio device according to the optimal transmission path, and the next audio device determines whether to forward the target audio data according to the current hop number corresponding to the target audio data, until the target audio data is transmitted to the end audio device of the optimal transmission path;

[0109] The broadcast unit 3023 is used to determine any audio device in the ad hoc network as a reference clock device, and broadcast a synchronization signal according to a preset clock synchronization period, wherein the synchronization signal includes a reference playback timestamp;

[0110] The synchronization unit 3024 is used to synchronize the clock of the target audio device with the reference clock device according to a preset target filtering algorithm, and determine the target audio device corresponding to the target playback timestamp according to the reference playback timestamp, and the target audio device is an audio device that receives the synchronization signal in the self-organizing network.

[0111] Optionally, the synchronization unit 3024 is specifically configured to: filter the clock deviation between the target audio device and the reference clock device according to a preset target filtering algorithm to obtain an optimal clock deviation estimation value;

[0112] Compensating a local clock of a target audio device according to an optimal clock deviation estimate;

[0113] Determine the transmission time value between the sound card and the speaker in the target audio device, and obtain the delay calibration value corresponding to the target audio device;

[0114] A target playback timestamp corresponding to the target audio device is determined according to the reference playback timestamp and the delay calibration value corresponding to the target audio device.

[0115] Optionally, the transmission unit 3022 is specifically configured to: transmit the target audio data to the next audio device through the main audio device according to the optimal transmission path;

[0116] Determine the first audio device as the receiving target audio data, and determine through the first audio device whether the current hop number corresponding to the target audio data is less than the set hop number of the optimal transmission path;

[0117] If yes, the target audio data is sent to the second audio device according to the optimal transmission path through the first audio device, and the current hop number corresponding to the target audio data is increased by one, and the second audio device is used to determine whether to forward the target audio data;

[0118] If not, the first audio device is determined as the end point audio device of the optimal transmission path.

[0119] Optionally, the audio codec unit 3021 is specifically used to: obtain the audio data to be played through the main audio device corresponding to the ad hoc network, and parse to obtain the audio type corresponding to the audio data to be played;

[0120] Determine a target encoding algorithm according to the audio type of the audio data to be played and the network bandwidth information;

[0121] The audio data to be played is divided into a plurality of original audio segments according to a preset frame period of a preset I2S interface through a main audio device, and each original audio segment is encoded according to a target encoding algorithm to obtain target audio data.

[0122] Optionally, the playing module 303 is specifically used to determine the playing timestamp corresponding to each audio device in the ad hoc network according to the synchronization parameter;

[0123] Each audio device in the ad hoc network plays the target audio data according to the synchronized local clock and the playback timestamp corresponding to each audio device.

[0124] The technical solution provided by the present invention utilizes multiple audio devices that support the target wireless communication protocol to build a self-organizing network covering multiple rooms through point-to-point connection. The audio devices dynamically analyze the link quality, select the transmission path with the lowest delay, and dynamically optimize the network topology through multi-hop relay, thereby reducing the transmission delay and realizing high-resolution audio data transmission between audio devices to meet the demand for lossless sound quality transmission; the audio devices improve the accuracy of synchronous playback through real-time interaction and dynamic compensation of synchronization signals to meet the demand for sound and picture synchronization.

[0125] above Figure 3 and Figure 4 The multi-room audio synchronous playback system in this embodiment is described in detail from the perspective of modular functional entities, and the computer device in this embodiment is described in detail from the perspective of hardware processing.

[0126] See also Figure 5 As shown, the computer device includes a processor 500 and a memory 501, wherein the memory 501 stores machine executable instructions that can be executed by the processor 500, and the processor 500 executes the machine executable instructions to implement the above-mentioned multi-room audio synchronous playback method.

[0127] further, Figure 5 The computer device shown further includes a bus 502 and a communication interface 503 , and the processor 500 , the communication interface 503 and the memory 501 are connected via the bus 502 .

[0128] Among them, the memory 501 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), for example, at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 503 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 502 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0129] The processor 500 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 500. The above processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 501 , and the processor 500 reads the information in the memory 501 and completes the method steps of the above-mentioned embodiment in combination with its hardware.

[0130] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the multi-room audio synchronous playback method.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0132] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0133] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-room audio synchronous playback method, characterized in that: The multi-room audio synchronous playback method comprises: Using multiple audio devices supporting a target wireless communication protocol to build a multi-room ad hoc network through point-to-point connections, the target wireless communication protocol operating in the 5 GHz to 7 GHz frequency band; By dynamically analyzing link quality, selecting the optimal transmission path, and dynamically optimizing the network topology of the ad hoc network through multi-hop relay; Transmitting target audio data between audio devices in the ad hoc network according to the optimal transmission path, and performing real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters; Control each audio device in the ad hoc network to synchronously play the target audio data according to the synchronization parameter; The method of transmitting the target audio data between the audio devices in the ad hoc network according to the optimal transmission path, and performing real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters includes: Encode and decode the audio to be played to obtain target audio data; Transmitting the target audio data to a next audio device according to the optimal transmission path, and determining whether to forward the target audio data by the next audio device according to the current hop number corresponding to the target audio data, until the target audio data is transmitted to the end audio device of the optimal transmission path; Determine any audio device in the ad hoc network as a reference clock device, broadcast a synchronization signal according to a preset clock synchronization period, wherein the synchronization signal includes a reference playback timestamp; Performing clock synchronization on a target audio device and the reference clock device according to a preset target filtering algorithm, and determining a target playback timestamp corresponding to the target audio device according to the reference playback timestamp, wherein the target audio device is an audio device in the ad hoc network that receives the synchronization signal; The clock synchronization of the target audio device with the reference clock device according to the preset target filtering algorithm includes: constructing a state transfer equation and an observation equation by taking the clock deviation, clock drift rate, network delay time and audio buffer occupancy rate between each target audio device and the reference clock device as state variables; controlling the reference clock device to exchange timestamp data with each target audio device according to the clock synchronization cycle to obtain a clock deviation observation value corresponding to each target audio device; predicting the clock deviation of the next clock synchronization cycle according to the current state variable matrix and the state transfer equation to obtain a clock deviation prediction value corresponding to each target audio device; updating the state variable matrix through the observation equation, the clock deviation prediction value corresponding to each target audio device and the clock deviation prediction value corresponding to each target audio device, and predicting based on the updated state variable matrix to obtain an optimal clock deviation estimation value corresponding to each target audio device; and clock synchronization of each target audio device in the ad hoc network according to the optimal clock deviation estimation value corresponding to each target audio device.

2. The multi-room audio synchronous playback method according to claim 1, characterized in that: The target audio data is in a 24-bit / 192kHz pulse code modulation (PCM) format.

3. The multi-room audio synchronous playback method according to claim 1, characterized in that: The ad hoc network includes a master audio device and a plurality of slave audio devices; The target audio data and synchronization signal are distributed to the slave audio devices through the master audio device, and the network topology is dynamically maintained.

4. The multi-room audio synchronous playback method according to claim 1, characterized in that: The target wireless communication protocol is the WIFI-6E protocol.

5. The multi-room audio synchronous playback method according to claim 1, characterized in that: The method of dynamically analyzing link quality, selecting an optimal transmission path, and dynamically optimizing the network topology of the ad hoc network through multi-hop relays includes: Determine any one audio device in the ad hoc network as a main audio device; The master audio device detects surrounding audio devices according to a first preset period to obtain network information corresponding to each slave audio device, and broadcasts the network information corresponding to the master audio device to the surrounding audio devices; Determine the link quality score between any two audio devices based on the network information corresponding to each audio device ,in, Indicates audio device and audio equipment The link quality score between express and The signal strength of the link between Indicates the link delay, and is the weighting coefficient; constructing a weighted connectivity graph based on the link qualities, and determining a transmission path having the largest sum of link qualities from the master audio device to each slave audio device as an optimal transmission path; When the network topology corresponding to the ad hoc network changes, the optimal transmission path corresponding to the master audio device and each slave audio device is updated.

6. The multi-room audio synchronous playback method according to claim 1, characterized in that: The step of determining a target playback timestamp corresponding to a target audio device according to the reference playback timestamp includes: Determine a transmission time value between a sound card and a speaker in the target audio device, and obtain a delay calibration value corresponding to the target audio device; A target playback timestamp corresponding to the target audio device is determined according to the reference playback timestamp and the delay calibration value corresponding to the target audio device.

7. The multi-room audio synchronous playback method according to claim 1, characterized in that: The method of transmitting the target audio data to a next audio device according to the optimal transmission path, and determining whether to forward the target audio data according to the current hop number corresponding to the target audio data by the next audio device until the target audio data is transmitted to the end point audio device of the optimal transmission path, includes: transmitting the target audio data to the next audio device according to the optimal transmission path through the main audio device; Determine that the target audio data is received as a first audio device, and determine, through the first audio device, whether a current hop number corresponding to the target audio data is less than a set hop number of the optimal transmission path; If yes, the target audio data is sent to the second audio device according to the optimal transmission path through the first audio device, the current hop number corresponding to the target audio data is increased by one, and the second audio device is used to determine whether to forward the target audio data; If not, the first audio device is determined as the endpoint audio device of the optimal transmission path.

8. The multi-room audio synchronous playback method according to claim 1, characterized in that: The encoding and decoding of the audio to be played to obtain target audio data includes: Acquire the audio data to be played through the main audio device corresponding to the ad hoc network, and parse to obtain the audio type corresponding to the audio data to be played; Determining a target encoding algorithm according to the audio type and network bandwidth information of the audio data to be played; The main audio device divides the audio data to be played into a plurality of original audio segments according to a preset frame period of a preset I2S interface, and encodes each original audio segment according to the target encoding algorithm to obtain target audio data.

9. The multi-room audio synchronous playback method according to claim 1, characterized in that: The step of controlling each audio device in the ad hoc network to synchronously play the target audio data according to the synchronization parameter includes: Determine a playback timestamp corresponding to each audio device in the ad hoc network according to the synchronization parameter; The target audio data is played by each audio device in the ad hoc network according to the synchronized local clock and the play timestamp corresponding to each audio device.

10. A multi-room audio synchronous playback system, characterized in that: include: A plurality of audio devices supporting a target wireless communication protocol, wherein the plurality of audio devices form an ad hoc network covering multiple rooms through point-to-point connections, and the target wireless communication protocol operates in a frequency band of 5 GHz to 7 GHz; Each audio device includes: A network module, used to select an optimal transmission path by dynamically analyzing link quality, and dynamically optimize the network topology of the ad hoc network through multi-hop relay; A synchronization module, used to transmit target audio data between audio devices in the ad hoc network according to the optimal transmission path, and to perform real-time interaction and dynamic compensation through synchronization signals to obtain synchronization parameters; A playing module, configured to synchronously play the target audio data according to a playing timestamp carried by the synchronization signal; The synchronization module is specifically used to: encode and decode the audio to be played to obtain target audio data; Transmitting the target audio data to a next audio device according to the optimal transmission path, and determining whether to forward the target audio data by the next audio device according to the current hop number corresponding to the target audio data, until the target audio data is transmitted to the end audio device of the optimal transmission path; Determine any audio device in the ad hoc network as a reference clock device, broadcast a synchronization signal according to a preset clock synchronization period, wherein the synchronization signal includes a reference playback timestamp; Performing clock synchronization on a target audio device and the reference clock device according to a preset target filtering algorithm, and determining a target playback timestamp corresponding to the target audio device according to the reference playback timestamp, wherein the target audio device is an audio device in the ad hoc network that receives the synchronization signal; The clock synchronization of the target audio device with the reference clock device according to the preset target filtering algorithm includes: constructing a state transfer equation and an observation equation by taking the clock deviation, clock drift rate, network delay time and audio buffer occupancy rate between each target audio device and the reference clock device as state variables; controlling the reference clock device to exchange timestamp data with each target audio device according to the clock synchronization cycle to obtain a clock deviation observation value corresponding to each target audio device; predicting the clock deviation of the next clock synchronization cycle according to the current state variable matrix and the state transfer equation to obtain a clock deviation prediction value corresponding to each target audio device; updating the state variable matrix through the observation equation, the clock deviation prediction value corresponding to each target audio device and the clock deviation prediction value corresponding to each target audio device, and predicting based on the updated state variable matrix to obtain an optimal clock deviation estimation value corresponding to each target audio device; and clock synchronization of each target audio device in the ad hoc network according to the optimal clock deviation estimation value corresponding to each target audio device.

11. The multi-room audio synchronous playback system according to claim 10, characterized in that: The plurality of audio devices include one or more of a television, a projector, a surround sound box, a bass speaker box, and a sound bar.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the multi-room audio synchronous playback method as described in any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-room audio synchronous playback method as described in any one of claims 1 to 9 are implemented.

Images