Delay synchronization and sound mixing processing method and system between acoustic equipment cascades

Abstract

The invention discloses a delay synchronization and sound mixing processing method and system between acoustic equipment cascading, and the system comprises a master device and a plurality of slave devices, and comprises the following steps: the master device establishes a communication network with the slave devices, the master device sends information to all the slave devices, and records the communication time delay of the master device from sending to receiving the reply of the slave devices; the master device sends a starting instruction to all the slave devices and receives slave device audio data sent by all the slave devices, and the master device synchronizes with all the slave devices at a preset time interval according to the communication time delay T (n); and the master device carries out sound mixing on all the synchronized slave device audio data sent by the slave device, and outputs the audio data after sound mixing. The method comprises the following steps: sending an instruction to slave equipment to confirm time delay, carrying out shift synchronization on channel data according to the time delay, carrying out time delay confirmation in real time in a data transmission process, and carrying out sound mixing according to a channel weight, so that the balance and quality of sound mixing are ensured, and meanwhile, the synchronization rate of multi-channel audio data is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to a method and system for delay synchronization and mixing processing between cascaded acoustic devices. Background Art

[0002] Mixing is the process of integrating audio from multiple sources into a stereo or mono track. These mixed sound signals may come from different instruments, vocals or orchestral music, and there are often multiple channels of audio that need to be mixed, so how to synchronize and balance multi-channel audio is particularly critical.

[0003] Wired cascade acoustic equipment is one of the application scenarios of mixing. In the existing POE cascade or other wired cascade acoustic equipment, there is a lack of data synchronization and mixing methods during recording and playback. The existing technology mostly uses ultrasonic signals as audio timestamps for synchronization. When the frequency supported by the speaker amplifier is low, this method cannot be used; or the hardware is optimized and cannot be conveniently applied to the software. At the same time, when the general mixing algorithm adds data, data overflow will occur, causing sound distortion and other problems, or it cannot be well adaptively adjusted according to the data of each channel. Therefore, there is a need for a method to improve the mixing quality of wired cascade acoustic equipment while ensuring the synchronization rate. Summary of the invention

[0004] To solve the above problems, the present invention provides a method and system for delay synchronization and mixing processing between cascaded acoustic devices. Multi-channel audio is synchronized by collecting communication delays and mixed according to channel weights, thereby solving the problem of uneven data synchronization and mixing quality in existing wired cascade mixing.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for delay synchronization and mixing processing between cascaded acoustic devices, comprising a master device and a plurality of slave devices, including:

[0007] The master device establishes a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending to receiving the slave device's reply. The communication delay between the master device and the nth slave device is recorded as T(n).

[0008] Furthermore, the master device and the slave device establish a communication network, which specifically includes: establishing a communication network through SOCKET.

[0009] Furthermore, the master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n), and also includes: obtaining the local delay L(n) of each slave device from receiving the start instruction to starting to send audio data.

[0010] Furthermore, the master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n), and its specific implementation method includes: the master device sends a synchronization instruction to all slave devices at preset time intervals, requiring all slave devices to report the number of frames of slave device audio data sent, updating the communication delay T(n), and performing frame calibration and alignment on all slave device audio data according to the number of audio data frames sent by each slave device, combined with the communication delay T(n) and the local delay L(n).

[0011] Furthermore, the specific implementation method of the frame calibration is: for each slave device, the master device adds the communication delay and local delay of the corresponding slave device, and then calibrates according to the number of frames of the slave device audio data sent as reported by the corresponding slave device; when the number of audio data frames received by the master device does not match the number of audio data frames sent by the slave device, the missing audio data frames are filled with zero values ​​and the excess audio data frames are discarded.

[0012] Furthermore, the specific implementation method of the alignment is: according to the communication delay T(n), the slave device audio data sent by each slave device is calibrated, and the master device delays or advances each slave device audio data until the slave device audio data frame order received by the master device at the same time is the same.

[0013] Furthermore, the master device mixes the slave device audio data sent by all the slave devices after synchronization, and the specific implementation method is: each slave device corresponds to a channel, the mixing weight of each channel is calculated, and the mixing process is performed according to the mixing weight.

[0014] Furthermore, the mixing weight is calculated in the following manner: for each channel, the average amplitude value A of all sampling points of a channel within one second is calculated:

[0015]

[0016] Where N is the sampling frequency, X i Represents the amplitude of the channel at the i-th sampling point;

[0017] Calculate the amplitude ratio of each channel within one second:

[0018]

[0019] Where, j represents the jth channel, B represents the sampling bit depth, and M represents the maximum value of the data type;

[0020] Calculate the weight quantization factor for each channel:

[0021]

[0022] Among them, Qj Represents the weight quantization factor of the jth channel.

[0023] Furthermore, the mixing process is performed according to the mixing weights, and its specific implementation method is: multiplying the value of each channel at the current sampling point by the weight quantization factor of the corresponding channel and adding them together, which is the mixing result Y i :

[0024]

[0025] Among them, X j,i represents the amplitude of the jth channel at the ith sampling point, Q j Represents the weight quantization factor of the jth channel.

[0026] Through the above technical scheme, the present invention has the following beneficial effects: the master device confirms the delay by sending instructions to the slave device, shifts and synchronizes the channel data according to the delay, confirms the delay in real time during data transmission, and mixes according to the channel weight to ensure the balance and quality of the mix, while ensuring the synchronization rate of multi-channel audio data. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The present invention is a schematic diagram of the overall flow of a method for delay synchronization and mixing processing between cascaded acoustic devices.

[0028] Figure 2 The present invention is a schematic diagram of the structure of a system for delay synchronization and mixing processing between cascaded acoustic devices in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1

[0032] See also Figure 1 , a delay synchronization and mixing processing method between cascaded acoustic devices, including a master device and a plurality of slave devices, including:

[0033] The master device establishes a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending to receiving the slave device's reply. The communication delay between the master device and the nth slave device is recorded as T(n).

[0034] The master device sends a start command to all slave devices, and the master device starts to receive the slave device audio data sent by each slave device. The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n) and monitors the data transmission status;

[0035] The master device mixes all the synchronized slave device audio data sent by the slave devices, and outputs the mixed audio data.

[0036] In an optional embodiment, the master device and the slave device establish a communication network, which specifically includes: establishing a communication network through SOCKET.

[0037] When multiple audio devices are cascaded to the main device via POE, sockets are used to establish a communication network, and socket communication can provide real-time, two-way data transmission, which is essential for synchronization and coordination between audio devices. In addition, sockets are usually based on the TCP / IP protocol, which provides reliable, connection-oriented communication services, ensuring the stability and integrity of data transmission.

[0038] In an optional embodiment, the master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n), and also includes: obtaining the local delay L(n) of each slave device from receiving the start instruction to starting to send audio data.

[0039] The local latency is the delay in the slave device processing instructions and returning audio recording results. By obtaining the local latency, you can understand the processing speed of each slave device, so that you can adjust the audio data returned by each slave device according to the processing speed to ensure that the audio between multiple slave devices can be synchronized to avoid audio misalignment problems.

[0040] In an optional embodiment, the master device synchronizes with all slave devices at preset time intervals based on the communication delay T(n). The specific implementation method includes: the master device sends synchronization instructions to all slave devices at preset time intervals, requiring all slave devices to report the number of frames of slave device audio data sent, updating the communication delay T(n), and performing frame calibration and alignment on all slave device audio data based on the number of audio data frames sent by each slave device in combination with the communication delay T(n) and the local delay L(n).

[0041] The master device synchronizes at preset time intervals, confirms the data transmission status of all slave devices, and updates the communication delay of each slave device, thereby updating the communication status of the cascade network in real time. The time of each frame of audio is fixed, so the number of frames is calibrated in combination with the delay to avoid synchronization errors caused by transmission delays and other reasons. When the master device receives the first frame of data from the slave device, the data T(n) time before the master device receives the audio data is discarded to synchronize the data.

[0042] In an optional embodiment, the specific implementation method of the frame calibration is: for each slave device, the master device adds the communication delay and local delay of the corresponding slave device, and then calibrates according to the number of frames of the slave device audio data sent as reported by the corresponding slave device. When the number of audio data frames received by the master device does not match the number of audio data frames sent by the slave device, the missing audio data frames are filled with zero values ​​and the excess audio data frames are discarded.

[0043] By combining local delay and communication delay, it is possible to confirm whether the currently received audio data frame is the same as the total data frame sent from the device based on the currently received data, and to supplement or discard the redundant or missing data frames.

[0044] In an optional embodiment, the alignment is specifically implemented as follows: according to the communication delay T(n), the slave device audio data sent by each slave device is calibrated, and the master device delays or advances each slave device audio data until the slave device audio data frame order received by the master device at the same time is the same.

[0045] By adjusting the front and back positions of the data, all audio data are guaranteed to be in the same position at the same time to avoid audio misalignment during mixing. By setting up a data buffer on the master device, the redundant data frames sent by the slave device are stored in the buffer, and the audio data is output in a first-in-first-out order, so that the audio data of the slave device with a faster transmission speed can be aligned with the slave device with a slower transmission speed. By delaying or advancing the data output by a corresponding amount of time, data synchronization between multiple nodes is achieved.

[0046] In an optional embodiment, the master device mixes the slave device audio data sent by all slave devices after synchronization, and the specific implementation method is: each slave device corresponds to a channel, the mixing weight of each channel is calculated, and the mixing processing is performed according to the mixing weight.

[0047] In a multi-channel audio system, each slave device usually corresponds to an audio channel. In order to implement mixing processing, we need to calculate a mixing weight for each channel. This weight determines the contribution of the channel to the final mix.

[0048] In an optional embodiment, the mixing weight is calculated as follows: for each channel, the average amplitude A of all sampling points of the channel within one second is calculated:

[0049]

[0050] Where N is the sampling frequency, X i Represents the amplitude of the channel at the i-th sampling point;

[0051] Calculate the amplitude ratio of each channel within one second:

[0052]

[0053] Where, j represents the jth channel, B represents the sampling bit depth, and M represents the maximum value of the data type;

[0054] Calculate the weight quantization factor for each channel:

[0055]

[0056] Among them, Q j Represents the weight quantization factor of the jth channel.

[0057] Audio amplitude represents the strength or volume of the audio signal. It reflects the energy of the sound wave. Audio data with greater intensity has a greater weight and a greater proportion in the mix.

[0058] In an optional embodiment, the mixing process is performed according to the mixing weights, and the specific implementation method is: multiplying the value of each channel at the current sampling point by the weight quantization factor of the corresponding channel and adding them together to obtain the mixing result Y i :

[0059]

[0060] Among them, X j,i represents the amplitude of the jth channel at the ith sampling point, Q j Represents the weight quantization factor of the jth channel.

[0061] Example 2

[0062] See also Figure 2 , a delay synchronization and mixing processing system between cascaded acoustic devices, including a master device and a plurality of slave devices, including:

[0063] The communication establishment module is used to establish a communication network between the master device and the slave device. The master device sends information to all slave devices, and records the communication delay from the master device sending to receiving the slave device reply. The communication delay between the master device and the nth slave device is recorded as T(n);

[0064] A data synchronization module is used to enable the master device to send a start command to all slave devices, and the master device starts to receive the slave device audio data sent by each slave device. The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n) and monitors the data transmission status;

[0065] The audio mixing module is used to enable the master device to mix all the slave device audio data sent by the slave devices after synchronization, and output the mixed audio data.

[0066] The embodiment disclosed in this specification is only an example of a unilateral feature of the present invention, and the protection scope of the present invention is not limited to this embodiment, and any other functionally equivalent embodiments fall within the protection scope of the present invention. For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. A method for delay synchronization and mixing processing between cascaded acoustic devices, comprising a master device and a plurality of slave devices, characterized in that: include: The master device establishes a communication network with the slave devices. The master device sends information to all slave devices and records the communication delay from the master device sending to receiving the slave device's reply. The communication delay between the master device and the nth slave device is recorded as T(n). The master device sends a start command to all slave devices, and the master device starts to receive the slave device audio data sent by each slave device. The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n) and monitors the data transmission status; The master device mixes all the synchronized slave device audio data sent by the slave devices, and outputs the mixed audio data.

2. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 1, characterized in that: The master device and the slave device establish a communication network, which specifically includes: establishing a communication network through SOCKET.

3. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 1, characterized in that: The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n), and also includes: obtaining the local delay L(n) of each slave device from receiving the start instruction to starting to send audio data.

4. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 3, characterized in that: The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n). The specific implementation method includes: the master device sends a synchronization instruction to all slave devices at preset time intervals, requiring all slave devices to report the number of frames of slave device audio data sent, updating the communication delay T(n), and performing frame calibration and alignment on all slave device audio data according to the number of audio data frames sent by each slave device, combined with the communication delay T(n) and the local delay L(n).

5. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 4, characterized in that: The specific implementation method of the frame calibration is: for each slave device, the master device adds the communication delay and local delay of the corresponding slave device, and then calibrates according to the number of frames of the slave device audio data that has been sent reported by the corresponding slave device. When the number of audio data frames received by the master device does not match the number of audio data frames sent by the slave device, the missing audio data frames are filled with zero values ​​and the excess audio data frames are discarded.

6. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 4, characterized in that: The specific implementation method of the alignment is: according to the communication delay T(n), the slave device audio data sent by each slave device is calibrated, and the master device delays or advances each slave device audio data until the slave device audio data frame order received by the master device at the same time is the same.

7. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 1, characterized in that: The master device mixes the slave device audio data sent by all the slave devices after synchronization. The specific implementation method is: each slave device corresponds to a channel, the mixing weight of each channel is calculated, and the mixing processing is performed according to the mixing weight.

8. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 7, characterized in that: The mixing weight is calculated as follows: for each channel, the average amplitude A of all sampling points of a channel within one second is calculated: Where N is the sampling frequency, X i Represents the amplitude of the channel at the i-th sampling point; Calculate the amplitude ratio of each channel within one second: Where, j represents the jth channel, B represents the sampling bit depth, and M represents the maximum value of the data type; Calculate the weight quantization factor for each channel: Among them, Q j Represents the weight quantization factor of the jth channel.

9. A method for delay synchronization and mixing processing between cascaded acoustic devices according to claim 8, characterized in that: The mixing process is performed according to the mixing weights. The specific implementation method is: multiply the value of each channel at the current sampling point by the weight quantization factor of the corresponding channel and add them together to obtain the mixing result Y. i : Among them, X j,i represents the amplitude of the jth channel at the i-th sampling point, Q j Represents the weight quantization factor of the jth channel.

10. A system for delay synchronization and mixing processing between cascaded acoustic devices, comprising a master device and a plurality of slave devices, characterized in that: include: The communication establishment module is used to establish a communication network between the master device and the slave device. The master device sends information to all slave devices, and records the communication delay from the master device sending to receiving the slave device reply. The communication delay between the master device and the nth slave device is recorded as T(n); A data synchronization module is used to enable the master device to send a start instruction to all slave devices, and the master device starts to receive the slave device audio data sent by each slave device. The master device synchronizes with all slave devices at preset time intervals according to the communication delay T(n) and monitors the data transmission status; The audio mixing module is used to enable the master device to mix all the slave device audio data sent by the slave devices after synchronization, and output the mixed audio data.

Images