Method and system for delay synchronization and mixing processing between acoustic device cascades

Abstract

The application discloses a kind of acoustic equipment cascade between delay synchronization and mixing processing method and system, including one main device and several slave devices, comprising: the main device and slave device establish communication network, the main device sends information to all slave devices, records the communication delay of main device from sending to receiving slave device reply;The main device sends start instruction to all slave devices, receives the slave device audio data sent by each slave device, and the main device is synchronized with all slave devices according to the communication delay T (n) with preset time interval;The main device mixes all slave device audio data sent by slave device after synchronization, and outputs mixed audio data.By confirming the delay of slave device sending instruction, the channel data is shifted according to the delay synchronization, the delay is confirmed in real time during data transmission process, and the mixed audio data is mixed according to the channel weight, to ensure the balance and quality of mixing, while ensuring the synchronization rate of multi-channel audio data.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to a method and system for delay synchronization and mixing between cascaded acoustic devices. Background Technology

[0002] Audio mixing involves integrating audio from multiple sources into a single stereo or mono track. These mixed audio signals may originate from different instruments, vocals, or orchestral instruments, and often involve mixing multiple audio channels. Therefore, synchronizing and balancing multi-channel audio is crucial.

[0003] Wired cascaded acoustic equipment is one of the application scenarios for audio mixing. However, existing PoE cascades or other wired cascaded acoustic devices lack methods for synchronizing and mixing data during recording and playback. Current technologies often use ultrasonic signals as audio timestamps for synchronization, but this method is unusable when the speaker amplifier supports low frequencies; or it requires hardware optimization but cannot be easily applied to software. Furthermore, general mixing algorithms often experience data overflow during data addition, causing sound distortion, or they fail to adapt well to data from each channel. Therefore, a method is needed to improve the mixing quality of wired cascaded acoustic devices while maintaining synchronization rates. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method and system for delay synchronization and mixing between cascaded acoustic devices. By collecting communication delays, the method synchronizes multi-channel audio and performs mixing based on channel weights, thus resolving the problems of existing wired cascaded mixing that cannot achieve smooth data synchronization and mixing quality.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for delay synchronization and mixing processing between cascaded acoustic devices, comprising a master device and several slave devices, including:

[0007] The master device establishes a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending the information to receiving the reply from the slave device. The communication delay between the master device and the nth slave device is denoted as T(n).

[0008] Furthermore, the establishment of a communication network between the master device and the slave device specifically includes: establishing a communication network via SOCKET.

[0009] Furthermore, the master device synchronizes with all slave devices at a preset time interval based on the communication delay T(n), and also includes: obtaining the local delay L(n) of each slave device from receiving the start command to starting to send audio data.

[0010] Furthermore, the master device synchronizes with all slave devices at a preset time interval based on the communication delay T(n). The specific implementation method includes: the master device sends a synchronization command to all slave devices at a preset time interval, requiring all slave devices to report the number of frames of slave device audio data that have been sent, updating the communication delay T(n), and performing frame calibration and alignment of all slave device audio data based on the number of audio data frames sent by each slave device, combined with the communication delay T(n) and the local delay L(n).

[0011] Furthermore, the specific implementation of the frame calibration is as follows: For each slave device, the master device adds the communication delay and local delay of the corresponding slave device, and then performs calibration based on the number of frames of audio data sent by the corresponding slave device reported by the slave device. When the number of audio data frames received by the master device does not match the number of audio data frames sent by the slave device, the missing audio data frames are filled with zero values, and the excess audio data frames are discarded.

[0012] Furthermore, the specific implementation of the alignment is as follows: based on the communication delay T(n), the audio data sent by each slave device is calibrated, and the master device delays or advances the audio data of each slave device until the order of the slave device audio data frames received by the master device at the same time is the same.

[0013] Furthermore, the master device will mix the audio data sent by all slave devices after synchronization. Specifically, each slave device corresponds to one channel, the mixing weight of each channel is calculated, and the mixing process is performed according to the mixing weight.

[0014] Furthermore, the calculation method for the mixing weights is as follows: For each channel, calculate the average amplitude A of all sampling points of that channel within one second:

[0015]

[0016] Where N represents the sampling frequency, X i This represents the amplitude of the channel at the i-th sampling point;

[0017] Calculate the amplitude ratio of each channel within one second:

[0018]

[0019] Where j represents the j-th channel, B represents the sampling bit depth, and M represents the maximum value of the data type;

[0020] Calculate the weight quantization factor for each channel:

[0021]

[0022] Among them, Qj This represents the weight quantization factor for the j-th channel.

[0023] Furthermore, the mixing process based on mixing weights is specifically implemented as follows: the value of each channel at the current sampling point is multiplied by the corresponding channel's weight quantization factor, and the results are summed to obtain the mixing result Y. i :

[0024]

[0025] Among them, X j,i Q represents the amplitude of the j-th channel at the i-th sampling point. j This represents the weight quantization factor for the j-th channel.

[0026] Through the above technical solution, the present invention has the following beneficial effects: the master device confirms the delay by sending instructions to the slave device, shifts and synchronizes the channel data according to the delay, confirms the delay in real time during data transmission, and mixes the audio according to the channel weight, so as to ensure the balance and quality of the mixing, and at the same time ensure the synchronization rate of multi-channel audio data. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall process of a method for delay synchronization and mixing between cascaded acoustic devices according to the present invention.

[0028] Figure 2 This is a schematic diagram of the structure of a delay synchronization and mixing system between cascaded acoustic devices in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1

[0032] See Figure 1 A method for delay synchronization and mixing between cascaded acoustic devices, comprising a master device and several slave devices, including:

[0033] The master device establishes a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending the information to receiving the reply from the slave device. The communication delay between the master device and the nth slave device is denoted as T(n).

[0034] The master device sends a start command to all slave devices, and the master device begins to receive audio data sent by each slave device. The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n) and monitors the data transmission status.

[0035] The master device mixes all the audio data sent by the slave devices after synchronization and outputs the mixed audio data.

[0036] In an optional embodiment, the master device and the slave device establish a communication network, which specifically includes: establishing a communication network via SOCKET.

[0037] When multiple audio devices are cascaded to a master device via PoE, a communication network is established using sockets. Socket communication provides real-time, bidirectional data transmission, which is crucial for synchronization and coordination between audio devices. Furthermore, sockets are typically based on the TCP / IP protocol, which provides reliable, connection-oriented communication services, ensuring the stability and integrity of data transmission.

[0038] In an optional embodiment, the master device synchronizes with all slave devices at a preset time interval based on the communication delay T(n), and further includes: obtaining the local delay L(n) of each slave device from receiving the start command to starting to send audio data.

[0039] Local latency is the delay between a slave device processing instructions and returning audio recording results. By obtaining the local latency, the processing speed of each slave device can be determined, and the audio data returned by each slave device can be adjusted according to the processing speed to ensure that the audio between multiple slave devices can be synchronized and to avoid audio misalignment problems.

[0040] In an optional embodiment, the master device synchronizes with all slave devices at a preset time interval based on the communication delay T(n). The specific implementation includes: the master device sends a synchronization command to all slave devices at a preset time interval, requiring all slave devices to report the number of frames of slave device audio data that have been sent, updating the communication delay T(n), and performing frame calibration and alignment of the audio data of all slave devices based on the number of audio data frames sent by each slave device, combined with the communication delay T(n) and the local delay L(n).

[0041] The master device synchronizes at preset time intervals to confirm the data transmission status of all slave devices and update the communication latency of each slave device, thereby updating the communication status of the cascaded network in real time. The duration of each audio frame is fixed; therefore, the frame count is calibrated based on the latency to avoid synchronization errors caused by transmission delays. When the master device receives the first frame of data from a slave device, it discards the data from the time interval T(n) prior to receiving the audio data, synchronizing and aligning the data.

[0042] In an optional embodiment, the frame calibration is specifically implemented as follows: for each slave device, the master device adds the communication delay and local delay of the corresponding slave device, and then performs calibration based on the number of frames of audio data sent by the corresponding slave device reported by the slave device. When the number of audio data frames received by the master device does not match the number of audio data frames sent by the slave device, the missing audio data frames are padded with zero values, and the excess audio data frames are discarded.

[0043] By combining local latency and communication latency, it is possible to confirm whether the currently received audio data frame is the same as the total data frames sent by the device, and to supplement or discard any extra or missing data frames.

[0044] In an optional embodiment, the alignment is specifically implemented as follows: based on the communication delay T(n), the audio data sent by each slave device is calibrated, and the master device delays or advances the audio data of each slave device until the order of the slave device audio data frames received by the master device at the same time is the same.

[0045] By adjusting the order of data, all audio data are ensured to be in the same position at the same time, thus avoiding audio misalignment during mixing. A data buffer is established on the master device to store excess data frames sent by slave devices. Audio data is output in a first-in, first-out (FIFO) order, allowing audio data from faster-sending slave devices to align with that from slower-sending slave devices. Data synchronization between multiple nodes is achieved by delaying or advancing data output by a corresponding amount of time.

[0046] In an optional embodiment, the master device mixes the audio data sent by all slave devices after synchronization. Specifically, each slave device corresponds to one channel, the mixing weight of each channel is calculated, and the mixing process is performed according to the mixing weight.

[0047] In a multi-channel audio system, each slave device typically corresponds to one audio channel. In order to achieve mixing, we need to calculate a mixing weight for each channel, which determines the contribution of that channel in the final mix.

[0048] In an optional embodiment, the mixing weights are calculated as follows: for each channel, the average amplitude A of all sample points within one second of that channel is calculated.

[0049]

[0050] Where N represents the sampling frequency, X i This represents the amplitude of the channel at the i-th sampling point;

[0051] Calculate the amplitude ratio of each channel within one second:

[0052]

[0053] Where j represents the j-th channel, B represents the sampling bit depth, and M represents the maximum value of the data type;

[0054] Calculate the weight quantization factor for each channel:

[0055]

[0056] Among them, Q j This represents the weight quantization factor for the j-th channel.

[0057] Audio amplitude represents the intensity or volume of an audio signal. It reflects the magnitude of sound wave energy. Audio data with greater intensity has a greater weight and plays a larger role in mixing.

[0058] In an optional embodiment, the mixing process based on mixing weights is specifically implemented as follows: multiplying the value of each channel at the current sampling point by the corresponding channel's weight quantization factor and summing the results, yields the mixing result Y. i :

[0059]

[0060] Among them, X j,i Q represents the amplitude of the j-th channel at the i-th sampling point. j This represents the weight quantization factor for the j-th channel.

[0061] Example 2

[0062] See Figure 2 An acoustic device cascade delay synchronization and mixing system, comprising a master device and several slave devices, including:

[0063] The communication establishment module is used to enable the master device to establish a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending to receiving the slave device's reply. The communication delay between the master device and the nth slave device is denoted as T(n).

[0064] The data synchronization module is used to enable the master device to send a start command to all slave devices. The master device starts to receive audio data sent by each slave device. The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n) and monitors the data transmission status.

[0065] The audio mixing module is used to enable the master device to mix all the synchronized audio data sent by the slave devices and output the mixed audio data.

[0066] The embodiments disclosed in this specification are merely illustrative of one aspect of the invention, and the scope of protection of the invention is not limited to these embodiments. Any other functionally equivalent embodiments fall within the scope of protection of the invention. Those skilled in the art can make various other corresponding changes and modifications based on the technical solutions and concepts described above, and all such changes and modifications should fall within the scope of protection of the claims of this invention.

Claims

1. A method for delay synchronization and mixing processing between cascaded acoustic devices, comprising a master device and several slave devices, characterized in that, include: The master device establishes a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending the information to receiving the reply from the slave device. The communication delay between the master device and the nth slave device is denoted as T(n). The master device sends a start command to all slave devices, and the master device begins to receive audio data sent by each slave device. The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n) and monitors the data transmission status. The master device mixes all the synchronized audio data sent by the slave devices and outputs the mixed audio data. The master device synchronizes with all slave devices at preset time intervals based on the communication delay T(n). Specifically, this synchronization includes: acquiring the local delay L(n) of each slave device from receiving the start command to starting to send audio data; the master device sending synchronization commands to all slave devices at preset time intervals, requesting all slave devices to report the number of frames of audio data they have sent; updating the communication delay T(n); and performing frame calibration and alignment of the audio data of all slave devices based on the number of audio data frames sent by each slave device, combined with the communication delay T(n) and the local delay L(n). The specific implementation of the frame calibration is as follows: For each slave device, the master device adds the communication delay and local delay of the corresponding slave device, and then performs calibration based on the number of audio data frames reported by the corresponding slave device. When the number of audio data frames received by the master device does not match the number of audio data frames sent by the slave device, the missing audio data frames are padded with zero values, and the excess audio data frames are discarded. The master device mixes the audio data sent by all slave devices after synchronization. Specifically, each slave device corresponds to one channel; the mixing weights for each channel are calculated; and mixing is performed based on these weights. The mixing weights are calculated as follows: For each channel, the average amplitude A of all sample points within one second of that channel is calculated. Where N represents the sampling frequency, This represents the amplitude of the channel at the i-th sampling point; Calculate the amplitude ratio of each channel within one second: Where j represents the j-th channel, B represents the sampling bit depth, and M represents the maximum value of the data type; Calculate the weight quantization factor for each channel: in, This represents the weight quantization factor of the j-th channel. The mixing process based on mixing weights is specifically implemented as follows: multiply the value of each channel at the current sampling point by the corresponding channel's weight quantization factor and sum them to obtain the mixing result. : in, This represents the amplitude of the j-th channel at the i-th sampling point. This represents the weight quantization factor for the j-th channel.

2. The method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 1, characterized in that, The establishment of a communication network between the master device and the slave device specifically includes: establishing a communication network through SOCKET.

3. The method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 1, characterized in that, The specific implementation of the alignment is as follows: based on the communication delay T(n), the audio data sent by each slave device is calibrated, and the master device delays or advances the audio data of each slave device until the order of the slave device audio data frames received by the master device at the same time is the same.

4. A delay synchronization and mixing system for cascaded acoustic devices, comprising a master device and several slave devices, characterized in that, include: The communication establishment module is used to enable the master device to establish a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending to receiving the slave device's reply. The communication delay between the master device and the nth slave device is denoted as T(n). The data synchronization module enables the master device to send a start command to all slave devices. The master device then begins receiving audio data from each slave device. The master device synchronizes with all slave devices at preset time intervals based on the communication delay T(n), and monitors the data transmission status. The master device synchronizes with all slave devices at preset time intervals based on the communication delay T(n). Specifically, this synchronization includes: acquiring the local delay L(n) of each slave device from receiving the start command to starting to send audio data; the master device sending synchronization commands to all slave devices at preset time intervals, requesting all slave devices to report the number of frames of audio data they have sent; updating the communication delay T(n); and performing frame calibration and alignment of the audio data of all slave devices based on the number of audio data frames sent by each slave device, combined with the communication delay T(n) and the local delay L(n). The specific implementation of the frame calibration is as follows: For each slave device, the master device adds the communication delay and local delay of the corresponding slave device, and then performs calibration based on the number of frames of audio data sent by the corresponding slave device reported by the slave device. When the number of audio data frames received by the master device does not match the number of audio data frames sent by the slave device, the missing audio data frames are filled with zero values, and the excess audio data frames are discarded. The audio mixing module enables the master device to mix all synchronized audio data sent from slave devices and output the mixed audio data. The master device mixes the audio data sent by all slave devices after synchronization. Specifically, each slave device corresponds to one channel; the mixing weights for each channel are calculated; and mixing is performed based on these weights. The mixing weights are calculated as follows: For each channel, the average amplitude A of all sampling points within one second of that channel is calculated. Where N represents the sampling frequency, This represents the amplitude of the channel at the i-th sampling point; Calculate the amplitude ratio of each channel within one second: Where j represents the j-th channel, B represents the sampling bit depth, and M represents the maximum value of the data type; Calculate the weight quantization factor for each channel: in, This represents the weight quantization factor of the j-th channel. The mixing process based on mixing weights is specifically implemented as follows: multiply the value of each channel at the current sampling point by the corresponding channel's weight quantization factor and sum them to obtain the mixing result. : in, This represents the amplitude of the j-th channel at the i-th sampling point. This represents the weight quantization factor for the j-th channel.

Images