Array microphone device cascade system and method

Abstract

The invention discloses an array microphone device cascading system and method. The system comprises a master device and at least one slave device. The first slave output end and the second slave output end of the slave device are connected to the slave device signal input end group of the master device; the first main output end of the main device is connected with the loudspeaker, and the second main output end is connected with the far end. Wherein the first master input end of the master device and the first slave input end of the slave device are used for inputting loudspeaker reference signals; and the master device and the slave device are configured to perform preset processing on the multiple paths of slave microphone signals at least according to the loudspeaker reference signal to generate a playing audio signal and an uplink audio signal. According to the array microphone equipment cascading system in the cascading mode, a larger coverage range is expanded, mutual isolation between the array microphone equipment is ensured, and sound amplification of a conference site and the audio quality of a far end are also ensured.

Description

Technical Field

[0001] The present application relates to the technical field of array microphones, and in particular to a cascading system and method for array microphone devices. Background Art

[0002] With the development of array microphone technology, array microphone equipment can be used in conference rooms to not only meet the local sound amplification function in the conference room, but also meet the needs of remote conferences. However, the coverage of a single array microphone device is limited, which limits the promotion and application of array microphone equipment in large spaces such as super-large conference rooms, tiered classrooms, and exhibition halls. Summary of the invention

[0003] The embodiments of the present application provide a system and method for cascading array microphone devices, which are used to solve at least one of the above-mentioned technical problems.

[0004] In a first aspect, an embodiment of the present application provides a cascade system of array microphone devices, including a master device and at least one slave device; the master device includes a first master input terminal, a slave device signal input terminal group, a first master output terminal and a second master output terminal; the slave device includes a first slave input terminal, a first slave output terminal and a second slave output terminal; wherein, The first slave output terminal and the second slave output terminal are connected to the slave device signal input terminal group, the first master output terminal is connected to a speaker, and the second master output terminal is connected to a remote end; Wherein, the first master input terminal and the first slave input terminal are used to input a loudspeaker reference signal; The slave device is configured to: perform preset processing on multiple slave microphone signals at least according to the speaker reference signal to obtain a first slave output signal and a second slave output signal, and send the signals to the master device, wherein the multiple slave microphone signals are collected by a microphone array inside the slave device; The master device is configured to: perform preset processing on multiple main microphone signals at least according to the speaker reference signal to obtain a first main output signal and a second main output signal, wherein the multiple main microphone signals are collected by a microphone array inside the master device; The master device is further configured to generate a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal.

[0005] In some embodiments, the master device further includes a second master input terminal, and the slave device further includes a second slave input terminal; the second master input terminal and the second slave input terminal are used to receive a downlink audio signal from the remote end; Generating a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal, including: Generate a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal; An uplink audio signal is generated according to the second master output signal and the second slave output signal.

[0006] In some embodiments, the master device further comprises a third master output, a third master input, a fourth master output, and a fourth master input; Generating a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal, comprising: Outputting the first main output signal from the third main output terminal to the third main input terminal, and generating a playback audio signal according to the first main output signal and the first slave output signal and the downlink audio signal obtained from the third main input terminal; Generating an uplink audio signal according to the second master output signal and the second slave output signal, comprising: The second main output signal is output from the fourth main output terminal to the fourth main input terminal, and an uplink audio signal is generated according to the second main output signal and the second slave output signal obtained from the fourth main input terminal.

[0007] In some embodiments, performing preset processing on the multi-channel main microphone signals at least according to the speaker reference signal to obtain the first main output signal and the second main output signal includes: Performing a first echo cancellation process on the multi-channel microphone signals at least according to the loudspeaker reference signal to obtain a first echo cancellation result; Performing sound source localization at least according to the multi-channel microphone signals; performing beamforming processing on the first echo cancellation result at least according to the sound source localization result; Performing a second echo cancellation process on the beamformed signal to obtain a first main output signal; An echo suppression process is performed on the beamformed signal to obtain a second main output signal.

[0008] In some embodiments, performing sound source localization at least according to the multi-channel microphone signals includes: Sound source localization is performed according to the multi-channel microphone signals and the downlink audio signal.

[0009] In some embodiments, the step of performing beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performing echo suppression processing on the beamformed signal includes: When the downlink audio signal indicates that there is far-end voice activity, performing a first echo suppression process on the beamformed signal; When the downlink audio signal indicates that there is no far-end voice activity, a first echo suppression process is performed on the beamformed signal; wherein the strength of the first echo suppression is greater than the strength of the second echo suppression.

[0010] In a second aspect, the present application further provides an array microphone device cascading method, which is applied to an array microphone cascading system, wherein the array microphone cascading system includes a master device and at least one slave device; the master device includes a first master input terminal, a slave device signal input terminal group, a first master output terminal and a second master output terminal; the slave device includes a first slave input terminal, a first slave output terminal and a second slave output terminal; the method includes: Connecting the first slave output terminal and the second slave output terminal to the slave device signal input terminal group, the first master output terminal is connected to the speaker, and the second master output terminal is connected to the remote end; wherein the first master input terminal and the first slave input terminal are used to input the speaker reference signal; The slave device is configured to: perform preset processing on multiple slave microphone signals at least according to the loudspeaker reference signal to obtain a first slave output signal and a second slave output signal, and send the signals to the master device, wherein the multiple slave microphone signals are collected by a microphone array inside the slave device; The master device is configured to: perform preset processing on multiple main microphone signals at least according to the speaker reference signal to obtain a first main output signal and a second main output signal, wherein the multiple main microphone signals are collected by a microphone array inside the master device; The master device is further configured to generate a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal.

[0011] In some embodiments, the master device further includes a second master input terminal, and the slave device further includes a second slave input terminal; the second master input terminal and the second slave input terminal are used to receive a downlink audio signal from the remote end; Generating a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal, including: Generate a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal; An uplink audio signal is generated according to the second master output signal and the second slave output signal.

[0012] In some embodiments, the master device further comprises a third master output, a third master input, a fourth master output, and a fourth master input; Generating a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal, comprising: Outputting the first main output signal from the third main output terminal to the third main input terminal, and generating a playback audio signal according to the first main output signal and the first slave output signal and the downlink audio signal obtained from the third main input terminal; Generating an uplink audio signal according to the second master output signal and the second slave output signal, comprising: The second main output signal is output from the fourth main output terminal to the fourth main input terminal, and an uplink audio signal is generated according to the second main output signal and the second slave output signal obtained from the fourth main input terminal.

[0013] In some embodiments, performing preset processing on the multi-channel main microphone signals at least according to the speaker reference signal to obtain the first main output signal and the second main output signal includes: Performing a first echo cancellation process on the multi-channel microphone signals at least according to the loudspeaker reference signal to obtain a first echo cancellation result; Performing sound source localization at least according to the multi-channel microphone signals; performing beamforming processing on the first echo cancellation result at least according to the sound source localization result; Performing a second echo cancellation process on the beamformed signal to obtain a first main output signal; An echo suppression process is performed on the beamformed signal to obtain a second main output signal.

[0014] In some embodiments, performing sound source localization at least according to the multi-channel microphone signals includes: Sound source localization is performed according to the multi-channel microphone signals and the downlink audio signal.

[0015] In some embodiments, the step of performing beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performing echo suppression processing on the beamformed signal includes: When the downlink audio signal indicates that there is far-end voice activity, performing a first echo suppression process on the beamformed signal; When the downlink audio signal indicates that there is no far-end voice activity, a first echo suppression process is performed on the beamformed signal; wherein the strength of the first echo suppression is greater than the strength of the second echo suppression.

[0016] The present application connects multiple array microphone devices through the cascade mode of master-slave devices. The master and slave devices independently perform preset processing (for example, echo cancellation and / or noise reduction processing) on ​​the collected multi-channel microphone signals based on the reference speaker signal, and obtain independent first master output signals, second master output signals, first slave output signals, and second slave output signals, respectively, and finally generate playback audio signals and uplink audio signals based on the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal through the master device. The cascade system of array microphone devices in this cascade mode expands a larger coverage range, and ensures the isolation between the array microphone devices, and ensures the sound reinforcement at the conference site and the audio quality at the far end. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A principle block diagram of an embodiment of the array microphone device cascade system of the present application; Figure 2 A principle block diagram of another embodiment of the array microphone device cascade system of the present application; Figure 3 A principle block diagram of another embodiment of the array microphone device cascade system of the present application; Figure 4 A principle block diagram of another embodiment of the array microphone device cascade system of the present application; Figure 5 A schematic diagram of the principle of an embodiment of an array microphone device of the present application; Figure 6 A schematic diagram of another embodiment of the array microphone device of the present application; Figure 7 A principle block diagram of another embodiment of the array microphone device of the present application; Figure 8 A principle block diagram of another embodiment of the array microphone device of the present application; Fig. 9 A principle block diagram of another embodiment of the array microphone device of the present application; Fig.10 A principle block diagram of another embodiment of the array microphone device of the present application; Fig.11 A flowchart of another embodiment of the signal processing method of the present application; Fig.12 This is a principle block diagram of another embodiment of the array microphone device cascade system of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0020] It should also be noted that, in this article, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements.

[0021] The present application proposes an array microphone device cascade system, which can be used for local meetings or remote meetings. Exemplarily, when used for remote meetings, a set of array microphone device cascade systems of the present application is set in the local meeting place, and a set of array microphone device cascade systems or a single array microphone device of the present application is set in the remote meeting place; the array microphone device cascade system of the local meeting place sends an uplink audio signal to the array microphone device cascade system or the single array microphone device of the remote meeting place through an uplink; the array microphone device cascade system or the single array microphone device of the remote meeting place sends a downlink audio signal to the array microphone device cascade system of the local meeting place through a downlink.

[0022] like Figure 1 As shown, an embodiment of the present application provides an array microphone device cascade system, including a master device 01 and at least one slave device 02; the master device 01 includes a first master input terminal Zin1, a slave device signal input terminal group Inn, a first master output terminal Zout1 and a second master output terminal Zout2; the slave device includes a first slave input terminal Cin1, a first slave output terminal Cout1 and a second slave output terminal Cout2; wherein the master device 01 and the slave device 02 are both array microphone devices; The first slave output terminal Cout1 and the second slave output terminal Cout2 are connected to the slave device signal input terminal group Inn, the first master output terminal Zout1 is connected to the speaker, and the second master output terminal Zout2 is connected to the remote end; wherein the first master input terminal Zin1 and the first slave input terminal Cin1 are used to input the speaker reference signal; The slave device 02 is configured to: perform preset processing on the multiple slave microphone signals at least according to the speaker reference signal to obtain the first slave output signal and the second slave output signal, and send them to the master device through the link between the first slave output terminal Cout1 and the second slave output terminal Cout2 of the slave device 02 and the slave device signal input terminal group Inn of the master device 01, and the multiple slave microphone signals are collected by the microphone array inside the slave device; exemplarily, the slave device signal input terminal group Inn may include two slave device signal input terminals, which are respectively connected to the first slave output terminal Cout1 and the second slave output terminal Cout2 of the slave device 02.

[0023] The master device 01 is configured to: perform preset processing on the multi-channel main microphone signals at least according to the speaker reference signal to obtain a first main output signal and a second main output signal, and the multi-channel main microphone signals are collected by the microphone array inside the master device; The master device 01 is further configured to generate a play audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal. Exemplarily, the play audio signal is used for local sound reinforcement, such as playing through a loudspeaker; the uplink audio signal is used to be sent to an array microphone device in a remote conference location.

[0024] The present application connects multiple array microphone devices through the cascade mode of master-slave devices. The master and slave devices independently perform preset processing (for example, echo cancellation and / or noise reduction processing) on ​​the collected multi-channel microphone signals based on the reference speaker signal, and obtain independent first master output signals, second master output signals, first slave output signals, and second slave output signals, respectively, and finally generate playback audio signals and uplink audio signals based on the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal through the master device. The cascade system of array microphone devices in this cascade mode expands a larger coverage range, and ensures the isolation between the array microphone devices, and ensures the sound reinforcement at the conference site and the audio quality at the far end.

[0025] like Figure 2 FIG. 1 is a schematic block diagram of another embodiment of the array microphone device cascade system of the present application. Figure 1The difference of the illustrated embodiment is that the master device 01 further includes a second master input terminal Zin2, and the slave device 02 further includes a second slave input terminal Cin2; the second master input terminal Zin2 and the second slave input terminal Cin2 are used to receive the downlink audio signal from the remote end; Generate a playback audio signal and an uplink audio signal at least according to a first master output signal, a second master output signal, a first slave output signal and a second slave output signal, including: generating a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal; generating an uplink audio signal according to the second master output signal and the second slave output signal.

[0026] Exemplarily, the first master output signal, the first slave output signal and the downlink audio signal are mixed to obtain the playback audio signal; and the second master output signal and the second slave output signal are mixed to obtain the uplink audio signal.

[0027] like Figure 3 The figure shows a principle block diagram of another embodiment of the array microphone device cascade system of the present application. In this embodiment, the master device 01 includes input terminals RX1 to RX6 and output terminals TX1 to TX4, wherein the input terminal RX1 corresponds to the first master input terminal Zin1 in the aforementioned embodiment, RX2 corresponds to the second master input terminal Zin2 in the aforementioned embodiment, the input terminals RX5 and RX6 correspond to the slave device signal input terminal group Inn in the aforementioned embodiment, the output terminal TX1 corresponds to the first master output terminal Zout1 in the aforementioned embodiment, and the output terminal TX2 corresponds to the second master output terminal Zout2 in the aforementioned embodiment; In this embodiment, the slave device 02 includes input terminals RX1' to RX2' and output terminals TX1' to TX2', wherein the input terminal RX1' corresponds to the first slave input terminal Cin1 in the aforementioned embodiment, the output terminal TX1' corresponds to the first slave output terminal Cout1 in the aforementioned embodiment, and the output terminal TX2' corresponds to the second slave output terminal Cout2 in the aforementioned embodiment.

[0028] Compared with the above embodiments, Figure 3 The master device 01 in the illustrated embodiment has additional input terminals RX3 and RX4 and output terminals TX3 and TX4. In some embodiments, the master device 01 further includes a third master output terminal, a third master input terminal, a fourth master output terminal and a fourth master input terminal; wherein the third output terminal and the fourth master output terminal correspond to the output terminals TX3 and TX4 respectively, and the third master input terminal and the fourth master input terminal correspond to the input terminals RX3 and RX4 respectively.

[0029] In some embodiments, generating a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal includes: Outputting the first main output signal from the third main output terminal to the third main input terminal, and generating a playback audio signal according to the first main output signal and the first slave output signal and the downlink audio signal obtained from the third main input terminal; Generating an uplink audio signal according to the second master output signal and the second slave output signal, comprising: The second main output signal is output from the fourth main output terminal to the fourth main input terminal, and an uplink audio signal is generated according to the second main output signal and the second slave output signal obtained from the fourth main input terminal.

[0030] The inventor found that the audio data (the first slave output signal and the second slave output signal) and the downlink audio signal sent by the slave device 02 are timestamped, while the audio data (the first master output signal and the second master output signal) of the master device 01 cannot obtain the network timestamp without being transmitted through the network, so data alignment and mixing processing cannot be performed. In order to solve this problem, in this embodiment, the sound reinforcement and call uplink audio (i.e., the first master output signal and the second master output signal) of the master device 01 are looped back, and the audio of the master and slave devices are synchronously collected according to the timestamps through Dante, so that alignment is achieved in this way. The sound reinforcement audio of the master device 01 is sent through TX3 and received by RX3, and the call uplink audio is sent by TX4 and received by RX4.

[0031] Further, Figure 3 The main device also includes an amplifier unit, a converter and a PC, and the main device realizes communication connection with the remote end through the PC. The amplifier unit is arranged between the first main output terminal of the main device 01 and the speaker, and pre-amplifies the audio signal input to the speaker. The converter is arranged between the second main output terminal of the main device 01 and the PC to realize signal conversion and realize Dante to USB signal conversion.

[0032] exist Figure 3 In the embodiment shown, the master device 01 obtains a local sound reinforcement signal 1 and an uplink audio signal 1 according to a downlink audio signal and a loudspeaker reference signal; the slave device 02 receives the downlink audio signal and the loudspeaker reference signal, and outputs a local sound reinforcement signal 2 and an uplink audio signal 2. Finally, the master device outputs a mixed signal 1 of the downlink audio signal, the local sound reinforcement signal 1 and the local sound reinforcement signal 2 through the first master output terminal TX1, and amplifies the sound through the power amplifier unit and the loudspeaker; the master device 01 outputs a mixed signal 2 of the uplink audio signal 1 and the uplink audio signal 2 through the second master output terminal TX2.

[0033] For example, regarding Figure 3 The data flow of the two devices in the embodiment shown is described as follows: 1) Define the master device in the cascade network through configuration 2) Downlink audio of the call is forwarded from the PC input to the master and slave devices through the converter. The ceiling microphone is completed through receiving channel 1 3) Sound reinforcement audio: the master and slave devices record the sound reinforcement audio after algorithm processing; the sound reinforcement audio of the slave device is sent to the master device through the network 4) Uplink audio of the call: the master and slave devices record the call audio after algorithm processing; the uplink audio of the call of the slave device is sent to the master device through the network 5) Reference audio, used for echo cancellation and feedback suppression, is the local audio played through the speaker. The local audio consists of several parts: one is the amplified audio after the master and slave devices record and process the algorithm, and the other is the call downlink audio. The two parts of audio are output to the speaker, master device and each slave device by the master device through strategy mixing, EQ processing, gain processing, etc. 6) The main device can output to the digital speaker through the Dante network, or it can output analog signals to the analog speaker through the DAC on the device 7) For the convenience of configuration, the following constraints are made: the master-slave device RX1 inputs the reference audio, and RX2 inputs the call downlink audio. The slave devices TX1 and TX2 output the sound reinforcement and call uplink audio respectively. The master devices RX5 and RX6 are connected to the sound reinforcement and call uplink audio of slave device 1 respectively, and RX7 and RX8 are connected to the sound reinforcement and call uplink audio of slave device 2 respectively, and so on. Up to 63 ceiling microphones can work together in the scenario without an audio processor.

[0034] like Figure 4 FIG. 1 is a block diagram showing another embodiment of the array microphone device cascade system of the present application. Figure 3 The embodiment shown differs in that Figure 3 The embodiment shown is a single slave device cascaded with a master device. Figure 4 The embodiment shown is that two slave devices are cascaded with a master device. In this embodiment, the slave device signal input terminal group Inn of the master device 01 further includes input terminals RX7 and RX8, which are used to connect the two output signals of the newly added slave device 03. The slave device input 03 receives the downlink audio signal and the loudspeaker reference signal, and outputs the local sound reinforcement signal 3 and the uplink audio signal 3. The other signal input, signal output and signal processing of the slave device 03 refer to the slave device 02, which will not be repeated here. In addition, the master device 01 finally outputs the mixed signal 1 of the downlink audio signal, the local sound reinforcement signal 1, the local sound reinforcement signal 2 and the local sound reinforcement signal 3 through the first main output terminal TX1, and realizes the sound reinforcement through the power amplifier unit and the loudspeaker; the master device 01 outputs the mixed signal 2 of the uplink audio signal 1, the uplink audio signal 2 and the uplink audio signal 3 through the second main output terminal TX2.

[0035] In some embodiments, the master device 01 performs preset processing on the multi-channel main microphone signals at least according to the speaker reference signal to obtain a first main output signal and a second main output signal, including: Performing a first echo cancellation process on the multi-channel microphone signals at least according to the loudspeaker reference signal to obtain a first echo cancellation result; Performing sound source localization at least according to the multi-channel microphone signals; performing beamforming processing on the first echo cancellation result at least according to the sound source localization result; Performing a second echo cancellation process on the beamformed signal to obtain a first main output signal; An echo suppression process is performed on the beamformed signal to obtain a second main output signal.

[0036] Similarly, the slave device 02 can perform signal processing according to the above steps to obtain a first slave output signal and a second slave output signal, which will not be described in detail here.

[0037] In some embodiments, performing sound source localization at least according to the multi-channel microphone signals includes: The sound source is localized according to the multi-channel microphone signal and the downlink audio signal. The sound source is localized according to the multi-channel microphone signal and the downlink audio signal, rather than directly according to the multi-channel microphone signal, thereby avoiding the influence of the far-end audio-downlink audio signal that may be contained in the multi-channel microphone signal on the accuracy of the sound source localization, thereby ensuring the accuracy of BF.

[0038] The following is further combined Figures 5 to 9 The relevant embodiments of the array microphone device in the present application exemplarily illustrate the internal structure of the master device and the slave device and the signal processing method.

[0039] like Figure 5 The schematic diagram of the principle of an embodiment of an array microphone device in the present application is shown, wherein the array microphone device includes multiple microphones and a processor, the processor includes a first input terminal In1, a first output terminal Out1 and a second output terminal Out2, wherein the first output terminal outputs a signal to a local speaker, and the second output terminal outputs a signal to a remote end, and the multiple microphones are used to collect multi-channel microphone signals. Exemplarily, the processor is a signal processor, and the array microphone device may also include signal acquisition and peripheral circuits that cooperate with the first input terminal, the first output terminal and the second output terminal, etc. The present application does not limit the specific circuit structure. The signal processing method for an array microphone device provided in the present application includes the following steps: S10, the processor collects a speaker reference signal through a communication path between the first output terminal and the first input terminal; S20, the processor performs a first preset processing on the multi-channel microphone signal at least according to the speaker reference signal to obtain a first output signal, and outputs the first output signal to the local speaker through the first output terminal; S30: The processor performs a second preset processing on the multi-channel microphone signals at least according to the speaker reference signal to obtain a second output signal, and outputs the second output signal to the remote end through the second output terminal.

[0040] This embodiment adopts a single input channel and dual output channel design, and only uses a reference input channel formed by a first input end to receive a loudspeaker reference signal, thereby realizing independent processing of a local sound reinforcement signal (first output signal) and a remote signal (second output signal), and can independently realize noise reduction and echo cancellation processing of the two signals. The system hardware design is simple and has good compatibility.

[0041] like Figure 6 FIG. 2 is a schematic diagram showing another embodiment of an array microphone device of the present application. Figure 5 The difference is that in this embodiment, the array microphone device further includes a second input port. Exemplarily, when the array microphone device includes a processor, the processor is provided with a second input port, and the processor can obtain the downlink audio signal from the remote end through the second input port.

[0042] Further, the processor performs a first preset processing on the multi-channel microphone signals at least according to the speaker reference signal to obtain a first output signal, including: the processor performs a first preset processing on the multi-channel microphone signals at least according to the speaker reference signal, the downlink audio signal, and obtains the first output signal; the processor performs a second preset processing on the multi-channel microphone signals at least according to the speaker reference signal to obtain a second output signal, including: the processor performs a second preset processing on the multi-channel microphone signals at least according to the speaker reference signal, the downlink audio signal, and obtains the second output signal.

[0043] In this embodiment, a dual-channel input and dual-channel output design is implemented. Among them, through this embodiment, a single microphone array system has two independent reference input channels (In1 and In2), which receive the speaker reference signal and the far-end downlink audio signal respectively, thereby effectively distinguishing the far-end voice and local sound reinforcement. In addition, this design also provides an independent signal processing path for mixed scenarios, without relying on an external DSP, and realizes efficient signal processing. Furthermore, through two independent output channels, the far-end call and local sound reinforcement signals are processed separately, ensuring that in complex scenarios, the sound quality of the far-end call and the local sound reinforcement effect do not interfere with each other.

[0044] like Figure 7FIG. 1 is a schematic block diagram of another embodiment of an array microphone device of the present application. Figure 7 The part in the dotted box corresponds to Figure 5 and Figure 6 In addition, Figure 7 In the illustrated embodiment, an audio signal (ie, a downlink audio signal) of a remote conference device is received through a downlink of the second input port In2 , and an audio signal is sent to the remote conference device through an uplink of the second output port Out2 . Figure 7 The illustrated embodiment also includes an AEC module, a BF module, a NN_AFC module, a DOA module and an AES module.

[0045] Among them, the AEC module (Acoustic Echo Cancellation) is used to eliminate echoes in multi-channel microphone signals. When the sound emitted by the local speaker is picked up by the local microphone and sent back to the remote end, an echo may be generated. The AEC module dynamically estimates the echo path through an adaptive filter and eliminates the echo emitted by the speaker from the local microphone signal. The AEC module ensures that the sound of the local speaker does not interfere with the remote call, providing a better call experience.

[0046] The DOA (Direction of Arrival) module is used to calculate the angle of arrival of the sound signal. For example, in a local sound reinforcement channel, the DOA module calculates the azimuth of the local sound source (such as the speaker) to provide input for the beamforming (BF) algorithm. The DOA module determines the direction of the sound source by analyzing the time difference and phase difference between different channels of the microphone array.

[0047] For example, by using the signal of the sampling channel 2 (corresponding to the downlink), the remote signal and the local sound source can be accurately distinguished, avoiding misjudgment of the loudspeaker playback signal as the local speaker signal, thereby more accurately tracking the real local speaker. Figure 7 As shown in FIG. 1 , the DOA module performs sound source localization based on the multi-channel microphone signals and the downlink audio signals, rather than directly performing sound source localization based on the multi-channel microphone signals, thereby avoiding the influence of the far-end audio-downlink audio signals that may be contained in the multi-channel microphone signals on the accuracy of sound source localization, thereby ensuring the accuracy of BF.

[0048] In some embodiments, the second input port In2 can accurately distinguish the real speaker sound source and the speaker echo signal in a mixed conference scenario. Because in a local sound reinforcement environment, the sound played by the speaker may be picked up by the microphone array, thereby generating a false sound source direction. If it is not processed, the DOA module may mistakenly locate the direction of the echo signal. In order to solve this problem, a reference signal (i.e., a downlink audio signal) is provided through the second input port In2 to indicate that the speaker plays the far-end voice, helping the microphone array to determine whether the sound currently received by the microphone array contains echo interference from the speaker. It has at least the following functions: •Prevent DOA from misjudging echo signals: By introducing the In2 signal, the microphone array can effectively distinguish the real sound source from the speaker echo, avoiding the DOA module's incorrect positioning due to echo, thereby improving the accuracy of sound source positioning.

[0049] • Improve positioning accuracy: When In2 indicates the presence of a far-end downlink signal, the microphone array can automatically eliminate interference from the speaker, ensuring that the DOA algorithm always focuses on the actual local sound source, effectively improving the accuracy of beamforming (BF) and the reliability of subsequent signal processing.

[0050] The BF (Beamforming) module is a beamforming algorithm that performs weighted combination of microphone array signals based on the azimuth estimated by DOA to form a "beam" to focus on the sound source in a specific direction. This process can enhance the signal of the target sound source while suppressing noise from other directions, thereby improving the signal quality of local sound reinforcement.

[0051] The NN_AFC (Neural Network Adaptive Feedback Cancellation) module is a feedback cancellation algorithm that combines deep learning technology. Compared with the traditional AFC algorithm, NN_AFC can better capture the complex characteristics of the feedback path through the neural network model and adapt to different environments and device configurations. The model can automatically learn the characteristics of feedback and make adaptive adjustments, thereby providing more efficient feedback cancellation effects in complex scenarios.

[0052] The AES (Acoustic Echo Suppression) module further processes the residual echo from the AEC. Although the AEC has removed most of the echo, there may still be some residual echo. AES further suppresses these residual echoes through advanced filtering and processing. AES ensures that there is almost no echo interference in the far-end call, providing a more natural call experience.

[0053] In some embodiments, the processor performs a first preset processing on the multi-channel microphone signal at least according to the speaker reference signal to obtain a first output signal, including: The processor performs a first echo cancellation process on the multi-channel microphone signal at least according to the loudspeaker reference signal to obtain a first echo cancellation result; wherein the first echo cancellation process can be implemented by using an AEC module; The processor performs sound source localization at least according to the multi-channel microphone signals; The processor performs beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performs second echo cancellation processing on the beamformed signal to obtain a first output signal. The second echo cancellation processing is (NN_AFC, neural network adaptive feedback cancellation) which is a feedback cancellation algorithm combined with deep learning technology. Compared with the traditional AFC algorithm, NN_AFC can better capture the complex characteristics of the feedback path through the neural network model and adapt to different environments and device configurations.

[0054] In some embodiments, the processor performs a second preset processing on the multi-channel microphone signal at least according to the speaker reference signal to obtain a second output signal, including: The processor performs a first echo cancellation process on the multi-channel microphone signal at least according to the loudspeaker reference signal to obtain a first echo cancellation result; wherein the first echo cancellation process can be implemented by using an AEC module; The processor performs sound source localization at least according to the multi-channel microphone signals; The processor performs beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performs echo suppression processing on the beamformed signal to obtain a second output signal.

[0055] In some embodiments, the processor performs beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performs echo suppression processing on the beamformed signal, including: When the downlink audio signal indicates that there is far-end voice activity, performing a first echo suppression process on the beamformed signal; When the downlink audio signal indicates that there is no far-end voice activity, a second echo suppression process is performed on the beamformed signal; wherein the strength of the first echo suppression is greater than the strength of the second echo suppression.

[0056] For example, when there is a voice signal in the far-end downlink (i.e., the downlink audio signal indicates that there is far-end voice activity), the suppression strength of the first echo suppression processing (AES, Acoustic Echo Suppression) is about 30-40dB to ensure that there is no echo leakage. In the sound reinforcement mode, if there is no far-end human voice speaking (i.e., the downlink audio signal indicates that there is no far-end voice activity), the suppression strength of the second echo suppression should be lowered to a maximum suppression strength of no more than 5dB, or completely turned off, so as to ensure the voice quality of the near-end human voice in the uplink channel.

[0057] In this embodiment, through the dual-recall signal input design, the AES module can distinguish the far-end voice played by the speaker from the local sound reinforcement signal in real time. When the far-end voice activity is detected in the recall channel 2 (corresponding to the downlink), the AES module will increase the echo suppression strength and reduce the residual echo; when the recall channel 2 is inactive but the recall channel 1 is active, the system recognizes it as a local sound reinforcement signal, and the AES module reduces the suppression strength to ensure the clarity of the local sound reinforcement voice.

[0058] like Figure 8 FIG. 1 is a schematic block diagram of another embodiment of an array microphone device of the present application. Figure 7 The difference between the embodiment in the embodiment is that the array microphone device of the present application is provided with a third output port (Out_DOA_info) for outputting the coordinate information of the sound source. Exemplarily, when the array microphone device includes a processor, the processor is configured with the third output port. Moreover, the processor performs sound source localization at least according to the multi-channel microphone signal to determine the coordinate information of the sound source and outputs it to the external device through the third output port.

[0059] By adding a DOA coordinate output channel in this embodiment, the coordinate information of the current speaker can be output in real time to assist external devices (such as cameras) to automatically and quickly track the speaker in the current meeting, thereby improving the accuracy and efficiency of meeting recording.

[0060] like Fig. 9The schematic diagram of another embodiment of the array microphone device of the present application is shown. In this embodiment, it also includes: before outputting the first output signal, the first output signal is subjected to local sound reinforcement preprocessing, and the local sound reinforcement preprocessing includes at least one of the following: noise suppression (NR, NN_NR), neural network reverberation suppression (NN_DER), automatic gain control (AGC) and frequency equalization processing (EQ); and / or, before outputting the second output signal, the second output signal is subjected to remote signal preprocessing, and the remote signal preprocessing includes at least one of the following: noise suppression (NR, NN_NR), neural network dynamic rebound suppression (NN_DER), automatic gain control (AGC) and frequency equalization processing (EQ).

[0061] This embodiment uses full-link audio signal processing to achieve all-round optimization from noise suppression to automatic gain control. For example, this application integrates a complete audio processing chain, including modules such as noise suppression (NR), dereverberation (NN_DER), equalizer (EQ), and automatic gain control (AGC). This full-link processing system ensures that the audio signal can be optimized at multiple levels throughout the entire process from input to output.

[0062] Effects: Significantly improved audio clarity: Through NR and dereverberation processing, the system can effectively improve audio clarity, especially in noisy or reverberant environments; Smooth volume adjustment: The AGC module automatically adjusts the gain to ensure that the audio output is not too high or too low, providing a more natural listening experience.

[0063] like Fig.10 FIG. 1 is a block diagram of another embodiment of an array microphone device of the present application. In this embodiment, independent signal processing modules are configured for the local sound reinforcement link and the uplink of the remote signal. Fig. 9 Compared with the embodiment, Fig.10 Will Fig. 9 The AEC in the system is split into the AFC for the local sound reinforcement link and the AEC for the uplink of the remote signal, the BF is split into two independent BFs, and the DOA is split into two independent DOAs.

[0064] It should be noted that Figure 7-Figure 10 The system block diagram of the array microphone device shown shows an array microphone unit with integrated complete mixed scene audio processing capabilities. The overall unit in the dotted box not only supports audio acquisition of multi-channel microphones, but also includes dual-channel reference audio acquisition, DSP signal processing, sound reinforcement channel output, and uplink call link signal output. The entire system block diagram has a high degree of integration and can independently handle complex audio scenarios of remote calls and local sound reinforcement, reducing dependence on external DSP processors.

[0065] The array microphone device system of the present application supports multiple signal transmission formats. External signal transmission can be transmitted using either analog audio signals or digital audio protocols (such as Dante, AES67). If analog audio signals are used, the array microphone unit also needs to add corresponding ADC and DAC modules to complete analog-to-digital conversion to ensure accurate signal processing.

[0066] like Fig.11 FIG. 4 is a flow chart of another embodiment of the signal processing method of the present application.

[0067] Step 1: Audio Collection Array microphone devices collect local audio signals through microphone arrays. These microphone arrays are usually composed of multiple microphones, and the layout is designed to capture sound source information from different directions. Each microphone will collect different audio signals, which will be transmitted to the subsequent signal processing module. With the help of the microphone array, the system can realize spatial audio perception for subsequent beamforming (BF) and azimuth angle estimation (DOA).

[0068] Step 1.2: Execute the AFC algorithm The Adaptive Feedback Cancellation (AFC) algorithm is used to detect and eliminate feedback in local sound reinforcement systems. This feedback is usually caused by the microphone picking up the sound from the speaker, causing the sound to circulate between the microphone and the speaker, forming a howling sound.

[0069] AFC dynamically adjusts the signal through an adaptive filter to reduce or eliminate the feedback signal so that the system can operate normally under high gain conditions.

[0070] Step 1.3: Calculate the local sound reinforcement channel DOA1 The DOA (Direction of Arrival) algorithm is used to calculate the angle of arrival of sound signals. In the local sound reinforcement channel, the DOA1 module calculates the azimuth of the local sound source (such as the speaker) to provide input for the beamforming (BF) algorithm.

[0071] DOA1 determines the direction of the sound source by analyzing the time and phase differences between different channels of the microphone array.

[0072] Step 1.4: Execute the beamforming (BF) algorithm The beamforming (BF) algorithm performs weighted combination of microphone array signals based on the azimuth angle estimated by DOA1 to form a "beam" to focus on the sound source in a specific direction.

[0073] This process enhances the signal from the target sound source while suppressing noise from other directions, thereby improving the signal quality of local sound reinforcement.

[0074] Step 1.5: Execute NN_AFC algorithm NN_AFC (Neural Network Adaptive Feedback Cancellation) is a feedback cancellation algorithm that combines deep learning technology. Compared with the traditional AFC algorithm, NN_AFC can better capture the complex characteristics of the feedback path through the neural network model and adapt to different environments and device configurations.

[0075] The model can automatically learn the characteristics of feedback and make adaptive adjustments to provide more efficient feedback elimination effects in complex scenarios.

[0076] Step 1.6: Execute the local sound reinforcement processing chain (NR, NN_NR, NN_DER, AGC, EQ) After beamforming, the local sound reinforcement signal needs to go through a series of processing links to ensure that the output audio signal is of good quality and suitable for use by local sound reinforcement equipment.

[0077] NR (Noise Reduction): Suppresses the noise of the signal collected by the microphone to reduce the interference of environmental noise.

[0078] NN_NR (Neural Network Noise Suppression): A noise suppression algorithm based on deep learning that can better separate speech signals from complex noise.

[0079] NN_DER (Neural Network Reverberation Suppression): Dynamically suppresses reverberation tails through a neural network model to ensure clear sound reinforcement signals.

[0080] AGC (Automatic Gain Control): Automatically adjusts the gain of the audio signal to ensure volume balance and prevent the signal from being too loud or too soft.

[0081] EQ (Equalization): Frequency equalization of audio signals, enhancing certain frequency bands or weakening other frequency bands to ensure the sound quality of the sound reinforcement signal.

[0082] Step 1.7: Local sound reinforcement output The local sound reinforcement signal that has passed through the entire processing chain is finally transmitted to the local sound reinforcement loudspeaker for playback.

[0083] The signal should have high-quality sound clarity, low noise and no howling, ensuring that users can hear clear and natural sound in the local environment.

[0084] Step 2.2: Execute the AEC algorithm The AEC (Acoustic Echo Cancellation) algorithm is used to eliminate echoes in far-end calls. When the sound from the local speaker is picked up by the local microphone and sent back to the far end, an echo may be generated. AEC dynamically estimates the echo path through an adaptive filter and eliminates the echo from the speaker from the local microphone signal.

[0085] AEC ensures that the sound from the local speaker does not interfere with the far-end call, providing a better call experience.

[0086] Step 2.3: Calculate the DOA2 of the remote call channel DOA2 (Far-end Azimuth Estimation) is used to calculate the azimuth of the far-end call source. By analyzing the angle of arrival of the far-end signal, the system can help the beamforming algorithm to enhance the sound in the direction of the far-end call.

[0087] DOA2 provides important input for beamforming of far-end calls, ensuring that the far-end voice is transmitted clearly.

[0088] Step 2.4: Execute the beamforming (BF) algorithm Based on the far-end azimuth estimated by DOA2, the beamforming (BF) algorithm performs direction-enhanced far-end call signals. By focusing on the far-end sound source, the system can suppress other interfering signals and ensure that the far-end voice signal is clearly audible.

[0089] This beamforming technology can significantly improve the signal-to-noise ratio (SNR) of far-end calls and ensure the quality of voice transmission.

[0090] Step 2.5: Execute AES algorithm and post-process residual echo The AES (Acoustic Echo Suppression) algorithm further processes the residual echo of the AEC. Although the AEC has removed most of the echo, there may still be some residual echo. AES further suppresses these residual echoes through advanced filtering and processing.

[0091] AES ensures that there is virtually no echo interference on the far end, providing a more natural calling experience.

[0092] Step 2.6: Execute remote call processing link (NR, NN_NR, NN_DER, AGC, EQ) After beamforming and AES processing, the far-end call signal still needs to pass through multiple processing modules to ensure the voice quality of the far-end output.

[0093] NR (Noise Reduction): Reduces background noise in the far-end call signal.

[0094] NN_NR (Neural Network Noise Suppression): Uses deep learning algorithms to separate speech signals from complex noise, further improving speech clarity.

[0095] NN_DER (Neural Network Dynamic Echo Suppression): Dynamically suppresses reverberation tails through a neural network model to ensure that the call signal sent to the far end is clear and has no reverberation tails.

[0096] AGC (Automatic Gain Control): Automatically adjusts the gain of the far-end call signal to ensure balanced voice volume.

[0097] EQ (Equalization): Performs frequency equalization on the remote call signal to ensure that the sound quality of the output signal is suitable for the remote device.

[0098] Step 2.7: Remote signal output The far-end signal after the processing link is finally sent to the far-end call device (such as a phone, video conferencing system). The signal has been processed by echo cancellation, beamforming, noise suppression, etc. to ensure that the voice signal is clear without echo and noise interference. The far-end user will receive high-quality call audio, bringing a better call experience.

[0099] like Fig.12 FIG. 1 is a block diagram of another embodiment of the cascade system of array microphone devices of the present application. The master device and the slave device in this embodiment refer to Fig.10 The array microphone device shown.

[0100] In a second aspect, the present application further provides an array microphone device cascading method, which is applied to an array microphone cascading system, wherein the array microphone cascading system includes a master device and at least one slave device; the master device includes a first master input terminal, a slave device signal input terminal group, a first master output terminal and a second master output terminal; the slave device includes a first slave input terminal, a first slave output terminal and a second slave output terminal; the method includes: Connecting the first slave output terminal and the second slave output terminal to the slave device signal input terminal group, the first master output terminal is connected to the speaker, and the second master output terminal is connected to the remote end; wherein the first master input terminal and the first slave input terminal are used to input the speaker reference signal; The slave device is configured to: perform preset processing on multiple slave microphone signals at least according to the loudspeaker reference signal to obtain a first slave output signal and a second slave output signal, and send the signals to the master device, wherein the multiple slave microphone signals are collected by a microphone array inside the slave device; The master device is configured to: perform preset processing on multiple main microphone signals at least according to the speaker reference signal to obtain a first main output signal and a second main output signal, wherein the multiple main microphone signals are collected by a microphone array inside the master device; The master device is further configured to generate a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal.

[0101] In some embodiments, the master device further includes a second master input terminal, and the slave device further includes a second slave input terminal; the second master input terminal and the second slave input terminal are used to receive a downlink audio signal from the remote end; Generating a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal, including: Generate a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal; An uplink audio signal is generated according to the second master output signal and the second slave output signal.

[0102] In some embodiments, the master device further comprises a third master output, a third master input, a fourth master output, and a fourth master input; Generating a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal, comprising: Outputting the first main output signal from the third main output terminal to the third main input terminal, and generating a playback audio signal according to the first main output signal and the first slave output signal and the downlink audio signal obtained from the third main input terminal; Generating an uplink audio signal according to the second master output signal and the second slave output signal, comprising: The second main output signal is output from the fourth main output terminal to the fourth main input terminal, and an uplink audio signal is generated according to the second main output signal and the second slave output signal obtained from the fourth main input terminal.

[0103] In some embodiments, performing preset processing on the multi-channel main microphone signals at least according to the speaker reference signal to obtain the first main output signal and the second main output signal includes: Performing a first echo cancellation process on the multi-channel microphone signals at least according to the loudspeaker reference signal to obtain a first echo cancellation result; Performing sound source localization at least according to the multi-channel microphone signals; performing beamforming processing on the first echo cancellation result at least according to the sound source localization result; Performing a second echo cancellation process on the beamformed signal to obtain a first main output signal; An echo suppression process is performed on the beamformed signal to obtain a second main output signal.

[0104] In some embodiments, performing sound source localization at least according to the multi-channel microphone signals includes: Sound source localization is performed according to the multi-channel microphone signals and the downlink audio signal.

[0105] In some embodiments, the step of performing beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performing echo suppression processing on the beamformed signal includes: When the downlink audio signal indicates that there is far-end voice activity, performing a first echo suppression process on the beamformed signal; When the downlink audio signal indicates that there is no far-end voice activity, a first echo suppression process is performed on the beamformed signal; wherein the strength of the first echo suppression is greater than the strength of the second echo suppression.

[0106] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0107] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cascade system of array microphone devices, characterized in that: It includes a master device and at least one slave device; the master device includes a first master input terminal, a slave device signal input terminal group, a first master output terminal and a second master output terminal; the slave device includes a first slave input terminal, a first slave output terminal and a second slave output terminal; wherein, The first slave output terminal and the second slave output terminal are connected to the slave device signal input terminal group, the first master output terminal is connected to a speaker, and the second master output terminal is connected to a remote end; Wherein, the first master input terminal and the first slave input terminal are used to input a loudspeaker reference signal; The slave device is configured to: perform preset processing on multiple slave microphone signals at least according to the loudspeaker reference signal to obtain a first slave output signal and a second slave output signal, and send the signals to the master device, wherein the multiple slave microphone signals are collected by a microphone array inside the slave device; The master device is configured to: perform preset processing on multiple main microphone signals at least according to the speaker reference signal to obtain a first main output signal and a second main output signal, wherein the multiple main microphone signals are collected by a microphone array inside the master device; The master device is further configured to generate a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal.

2. The system according to claim 1, characterized in that The master device further includes a second master input terminal, and the slave device further includes a second slave input terminal; the second master input terminal and the second slave input terminal are used to receive a downlink audio signal from the remote end; Generating a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal, including: Generate a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal; An uplink audio signal is generated according to the second master output signal and the second slave output signal.

3. The system according to claim 2, characterized in that The master device further includes a third master output terminal, a third master input terminal, a fourth master output terminal and a fourth master input terminal; Generating a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal, comprising: Outputting the first main output signal from the third main output terminal to the third main input terminal, and generating a playback audio signal according to the first main output signal and the first slave output signal and the downlink audio signal obtained from the third main input terminal; Generating an uplink audio signal according to the second master output signal and the second slave output signal, comprising: The second main output signal is output from the fourth main output terminal to the fourth main input terminal, and an uplink audio signal is generated according to the second main output signal and the second slave output signal obtained from the fourth main input terminal.

4. The system according to any one of claims 1 to 3, characterized in that: At least performing preset processing on the multi-channel main microphone signals according to the loudspeaker reference signal to obtain a first main output signal and a second main output signal, including: Performing a first echo cancellation process on the multi-channel microphone signals at least according to the loudspeaker reference signal to obtain a first echo cancellation result; Performing sound source localization at least according to the multi-channel microphone signals; performing beamforming processing on the first echo cancellation result at least according to the sound source localization result; Performing a second echo cancellation process on the beamformed signal to obtain a first main output signal; An echo suppression process is performed on the beamformed signal to obtain a second main output signal.

5. The system according to claim 4, characterized in that The performing of sound source localization at least according to the multi-channel microphone signals comprises: Sound source localization is performed according to the multi-channel microphone signals and the downlink audio signal.

6. The system according to claim 4, characterized in that The step of performing beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performing echo suppression processing on the beamformed signal includes: When the downlink audio signal indicates that there is far-end voice activity, performing a first echo suppression process on the beamformed signal; When the downlink audio signal indicates that there is no far-end voice activity, a second echo suppression process is performed on the beamformed signal; wherein the strength of the first echo suppression is greater than the strength of the second echo suppression.

7. A method for cascading array microphone devices, applied to an array microphone cascading system, characterized in that: The array microphone cascade system comprises a master device and at least one slave device; the master device comprises a first master input terminal, a slave device signal input terminal group, a first master output terminal and a second master output terminal; the slave device comprises a first slave input terminal, a first slave output terminal and a second slave output terminal; the method comprises: Connecting the first slave output terminal and the second slave output terminal to the slave device signal input terminal group, the first master output terminal is connected to the speaker, and the second master output terminal is connected to the remote end; wherein the first master input terminal and the first slave input terminal are used to input the speaker reference signal; The slave device is configured to: perform preset processing on multiple slave microphone signals at least according to the loudspeaker reference signal to obtain a first slave output signal and a second slave output signal, and send the signals to the master device, wherein the multiple slave microphone signals are collected by a microphone array inside the slave device; The master device is configured to: perform preset processing on multiple main microphone signals at least according to the speaker reference signal to obtain a first main output signal and a second main output signal, wherein the multiple main microphone signals are collected by a microphone array inside the master device; The master device is further configured to generate a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal.

8. The method according to claim 7, characterized in that The master device further includes a second master input terminal, and the slave device further includes a second slave input terminal; the second master input terminal and the second slave input terminal are used to receive a downlink audio signal from the remote end; Generating a playback audio signal and an uplink audio signal at least according to the first master output signal, the second master output signal, the first slave output signal, and the second slave output signal, including: Generate a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal; An uplink audio signal is generated according to the second master output signal and the second slave output signal.

9. The method according to claim 8, characterized in that The master device further includes a third master output terminal, a third master input terminal, a fourth master output terminal and a fourth master input terminal; Generating a playback audio signal according to the first master output signal, the first slave output signal and the downlink audio signal, comprising: Outputting the first main output signal from the third main output terminal to the third main input terminal, and generating a playback audio signal according to the first main output signal and the first slave output signal and the downlink audio signal obtained from the third main input terminal; Generating an uplink audio signal according to the second master output signal and the second slave output signal, comprising: The second main output signal is output from the fourth main output terminal to the fourth main input terminal, and an uplink audio signal is generated according to the second main output signal and the second slave output signal obtained from the fourth main input terminal.

10. The method according to any one of claims 7 to 9, characterized in that: At least performing preset processing on the multi-channel main microphone signals according to the loudspeaker reference signal to obtain a first main output signal and a second main output signal, including: Performing a first echo cancellation process on the multi-channel microphone signals at least according to the loudspeaker reference signal to obtain a first echo cancellation result; Performing sound source localization at least according to the multi-channel microphone signals; performing beamforming processing on the first echo cancellation result at least according to the sound source localization result; Performing a second echo cancellation process on the beamformed signal to obtain a first main output signal; An echo suppression process is performed on the beamformed signal to obtain a second main output signal.

11. The method according to claim 10, characterized in that The performing of sound source localization at least according to the multi-channel microphone signals comprises: Sound source localization is performed according to the multi-channel microphone signals and the downlink audio signal.

12. The method according to claim 10, characterized in that The step of performing beamforming processing on the first echo cancellation result at least according to the sound source localization result, and performing echo suppression processing on the beamformed signal includes: When the downlink audio signal indicates that there is far-end voice activity, performing a first echo suppression process on the beamformed signal; When the downlink audio signal indicates that there is no far-end voice activity, a first echo suppression process is performed on the beamformed signal; wherein the strength of the first echo suppression is greater than the strength of the second echo suppression.

Images