Multi-channel MEMS microphone random array and use method thereof

Abstract

The invention provides a multichannel MEMS microphone random array and a use method thereof. The multichannel MEMS microphone random array comprises an array board, a control board and a board-to-board connector. According to the array board, 64 MEMS microphones form a random array, every four microphones are connected with one Codec chip, every four Codec chips form a data line, and four data lines are obtained. The control panel generates a total position clock signal and transmits the total position clock signal to the array board through the connector, the total position clock signal is converted into four clocks on the array board, the four clocks are connected with corresponding data lines in a daisy chain mode, and effective connection and operation of the system are achieved. According to the scheme, the BCLK of the digital MEMS microphone is synchronized in a BCLK daisy chain routing and multiple Codec chip TDM cascading mode, and data transmission and control are realized in combination with the board-to-board connector and related modules, so that the PCB wiring difficulty and the system complexity and cost of a large microphone array are reduced, and the sampling synchronism of the microphone is ensured.

Description

Technical Field

[0001] The present application relates to the field of microphone arrays, and in particular to a multi-channel MEMS microphone random array and a method of using the same. Background Art

[0002] In the field of audio processing, microphone arrays play a vital role. Usually, microphones receive sound omnidirectionally, but in practical applications, it is often necessary to receive sound selectively in a certain direction, such as retaining only the speaker's voice or suppressing the sound of a fixed interference source. Multiple microphones are used to form an array, and this requirement can be achieved by relying on the time difference when the sound waves reach each microphone. The microphone array layout is divided into uniform array and random array. The uniform array is arranged in a regular geometric pattern and the spacing between adjacent microphones is equal, while the random array has no specific rules and standard spacing. It is widely used in audio processing and can provide rich spatial information contained in multi-channel audio input, supporting sound source localization, noise suppression, multi-speaker separation, and advanced audio algorithms such as machine learning and deep learning. Now, with the development of artificial intelligence, more research focuses on using array data to train models to automatically perform related tasks or develop new functions.

[0003] However, microphone arrays face many challenges in practical applications. From the perspective of microphone type, analog MEMS microphones need to be used in conjunction with ADCs. Common audio ADC analog input channels are mostly 2, 4, 6, or up to 8. Once the number of microphone array elements exceeds 8, the difficulty of PCB wiring will increase dramatically, and the synchronization of acquisition will also be difficult to guarantee. For digital MEMS microphones with PDM interfaces, each PDM bus can only mount up to two microphones. Too many elements also face the problems of difficult wiring and complex synchronization operations. Furthermore, when using microphone array data to train models, a large number of original audio data sets at different spatial positions must be recorded in an anechoic chamber in the early stage to ensure the accuracy of the sound source positioning training model. In addition, audio algorithm engineers must also obtain the original audio data collected by the array for offline simulation in order to optimize the algorithm effect.

[0004] Especially when designing large-size microphone arrays, the BCLK (bit clock line) design of digital MEMS microphones needs to be cautious. On the one hand, because long traces will cause signal attenuation, the signal at high frequencies will attenuate more significantly as the transmission distance increases, and the far-end clock signal is easily so weak that the receiving end cannot recognize it; at the same time, if there is no proper terminal matching, the reflection and impedance mismatch problems caused by long traces will cause standing waves to form on the signal line, seriously affecting the quality of the clock signal, especially for high-speed clocks; and long traces also increase the chance of crosstalk with other signal lines, interfering with clock stability. On the other hand, the microphone array has strict timing control. If the BCLK line is too long, it will increase the time difference between the clock reaching different microphones, destroy the consistency of the sampling points, and affect the phase relationship and spatial positioning ability of the audio data. In addition, the random array has no regular arrangement, which makes it much more difficult to synchronize the BCLK of all digital MEMS microphones. Once they are out of sync, it will cause phase deviation of the audio sampling data.

[0005] In summary, how to solve the problems of difficult wiring, poor acquisition synchronization and signal attenuation of multi-element microphones is an issue that needs to be urgently addressed in the prior art. Summary of the invention

[0006] The embodiment of the present application provides a multi-channel MEMS microphone random array and a method for using the same, which synchronizes the BCLK of a digital MEMS microphone by daisy-chain routing of a bit clock BCLK and TDM cascading of multiple Codec chips, and combines a board-to-board connector and related modules to realize data transmission and control, thereby reducing the difficulty of PCB wiring, system complexity and cost of a large microphone array, and ensuring the synchronization of microphone sampling.

[0007] In a first aspect, an embodiment of the present application provides a multi-channel MEMS microphone random array, comprising: An array board, a control board, and a board-to-board connector, wherein the array board is a random array composed of 64 MEMS microphones, and every four MEMS microphones are connected to a Codec chip, and every four Codec chips form a date data line to obtain a first date data line, a second date data line, a third date data line, and a fourth date data line; The control board is connected to the array board through a board-to-board connector, a total bit clock signal is generated in the control board and transmitted to the array board through the board-to-board connector, and the total bit clock signal is converted into a first bit clock, a second bit clock, a third bit clock and a fourth bit clock in the array board, the first bit clock is connected to the first date data line in a daisy chain manner, the second bit clock is connected to the second date data line in a daisy chain manner, the third bit clock is connected to the third date data line in a daisy chain manner, and the fourth bit clock is connected to the fourth date data line in a daisy chain manner.

[0008] Preferably, every four Codec chips form a date data line through TDM cascading.

[0009] Preferably, the control board obtains the total bit clock signal through the clock source of the SAI module, and the clock source of the SAI module is provided by a 48MHz RC Clock after being multiplied by the internal PLL of the Soc.

[0010] Preferably, the control board also obtains a total frame clock signal through a clock source of the SAI module, and transmits the total frame clock signal to each Codec chip in the array board.

[0011] Preferably, the total bit clock signal is converted into a first bit clock, a second bit clock, a third bit clock and a fourth bit clock in the array board through a clock buffer chip.

[0012] In a second aspect, an embodiment of the present application provides a method for using a multi-channel MEMS microphone random array, including: A gigabit network port is provided in the control board of the multi-channel MEMS microphone random array, and the PC-side host computer is directly connected to the multi-channel MEMS microphone random array through the gigabit network port. After the PC-side host computer sends a command to start recording, the array board in the multi-channel MEMS microphone random array starts recording; The first date data line, the second date data line, the third date data line and the fourth date data line in the array board respectively collect the original audio data and transmit it to the control board via the board-to-board connector, and then the control board transmits the original audio data to the PC host computer via the Gigabit network port.

[0013] Preferably, the array board transmits the collected raw audio data to the control board in DSP-A data format, and the control board transmits the raw audio data to the PC host computer through the Lwip protocol stack.

[0014] Preferably, the PC host computer saves the received original audio data as a wav file.

[0015] Preferably, the control board configures the registers of each Codec chip in the array board through an I2C module.

[0016] The main contributions and innovations of the present invention are as follows: The embodiment of the present application synchronizes the BCLK of a digital MEMS microphone by daisy-chain routing of the bit clock BCLK and TDM cascading of multiple Codec chips, thereby reducing the difficulty of PCB wiring of large microphone arrays, reducing the cost increase caused by complex wiring, making the system structure simpler, improving the feasibility and economy of hardware design, and ensuring the synchronization of sampling of each microphone. Even when the microphone positions are randomly arranged, the accuracy and consistency of audio data collection can be guaranteed, providing a reliable data basis for subsequent audio processing.

[0017] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a structural schematic diagram of a multi-channel MEMS microphone random array according to an embodiment of the present application; Figure 2 is a schematic diagram of a random array of 64 MEMS microphones according to an embodiment of the present application; Figure 3 This is a structural diagram of a multi-channel MEMS microphone random array connected to a PC host computer according to an embodiment of the present application; Figure 4 is a software workflow diagram of a control panel according to an embodiment of the present application; Figure 5 This is a diagram showing the actual installation effect of an array board and a control board implemented according to the present application. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0020] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0021] Embodiment 1:

[0022] The embodiment of the present application provides a multi-channel MEMS microphone random array, which synchronizes the BCLK of the digital MEMS microphone by daisy-chain routing of the bit clock BCLK and TDM cascade of multiple Codec chips, and realizes data transmission and control in combination with board-to-board connectors and related modules, thereby reducing the PCB wiring difficulty, system complexity and cost of large microphone arrays, and ensuring the synchronization of microphone sampling. Specifically, reference Figure 1 , the array comprising: An array board, a control board, and a board-to-board connector, wherein the array board is a random array composed of 64 MEMS microphones, and every four MEMS microphones are connected to a Codec chip, and every four Codec chips form a date data line to obtain a first date data line, a second date data line, a third date data line, and a fourth date data line; The control board is connected to the array board through a board-to-board connector, a total bit clock signal is generated in the control board and transmitted to the array board through the board-to-board connector, and the total bit clock signal is converted into a first bit clock, a second bit clock, a third bit clock and a fourth bit clock in the array board, the first bit clock is connected to the first date data line in a daisy chain manner, the second bit clock is connected to the second date data line in a daisy chain manner, the third bit clock is connected to the third date data line in a daisy chain manner, and the fourth bit clock is connected to the fourth date data line in a daisy chain manner.

[0023] In some specific embodiments, a schematic diagram of a random array of 64 MEMS microphones is shown in FIG. Figure 2 As shown, this solution arranges MEMS microphones in a random array, which can receive sound signals from multiple angles in complex scenes, and accurately locate the sound source by processing the data collected by each MEMS microphone, effectively capturing the sound direction of people in different directions and improving the accuracy of sound positioning.

[0024] In this solution, the codec chip is a chip that supports the TDM cascading function, so every four Codec chips in this solution form a date data line through TDM cascading.

[0025] Specifically, the use of 64 MEMS microphones in this solution will complicate the wiring of the PCB board. However, the TDM cascade method used in this solution can effectively reduce the number of data lines, simplify the wiring structure, and avoid confusion caused by too many lines. In addition, TDM cascade can effectively integrate data transmission, and can transmit microphone data processed by multiple Codec chips in chronological order on the same data line, ensuring the efficiency and orderliness of data transmission.

[0026] In some specific embodiments, in order to accurately synchronize the data processing and transmission timing of each chip, avoid data errors or delays caused by clock asynchrony, and ensure the accuracy and integrity of audio data acquisition and transmission, the first date data line, the second date data line, the third date data line, and the fourth date data line in the array board share the same bit clock and frame clock.

[0027] In some specific embodiments, the board-to-board connector in this solution is used to transmit the total bit clock and the total frame clock obtained by the control board to the array board, and can also transmit the data collected by the first date data line, the second date data line, the third date data line and the fourth date data line to the control board, thereby completing the connection between the control board and the array board.

[0028] In some embodiments, the control board obtains the total bit clock signal through the clock source of the SAI module, and the clock source of the SAI module is provided by a 48 MHz RC Clock after being multiplied by the internal PLL of the Soc.

[0029] Specifically, the SAI module is a module inside the Soc, and the PLL is a circuit module that can perform frequency synthesis and phase adjustment on the input clock signal. That is to say, the 48MHz input clock is multiplied by the PLL to obtain the clock signal of the specific frequency required by the SAI module to synchronize the audio data.

[0030] Furthermore, the control board also obtains the total frame clock signal through the clock source of the SAI module, and transmits the total frame clock signal to each Codec chip in the array board. The total frame clock signal works together with the total bit clock BCLK, and the total frame clock signal is used to define the frame structure of the audio data.

[0031] Specifically, the magnitude of the total bit clock signal BCLK is 12.288 MHz, and the magnitude of the total frame clock signal LRCLK is 48 kHz.

[0032] In some embodiments, the total bit clock signal is converted into a first bit clock, a second bit clock, a third bit clock and a fourth bit clock in the array board through a clock buffer chip.

[0033] Specifically, since the total bit clock signal needs to be transmitted to each Codec chip after passing through the control board, there may be a problem of clock signal attenuation. The clock buffer chip can buffer and amplify the total bit clock signal to enhance its driving capability, ensuring that the first bit clock, the second bit clock, the third bit clock and the fourth bit clock have sufficient strength and stability to drive the corresponding Codec chip, reducing the noise and interference caused by clock signal attenuation.

[0034] In some specific embodiments, the connection method is described by taking the first clock connected to the first date data line as an example. First, since four Codec chips are cascaded in the first date data line, the first Codec chip receives and processes the first clock and then forwards the first clock to the second Codec chip. Similarly, the second Codec chip receives and processes the first clock and then forwards the first clock to the third Codec chip. The third Codec chip receives and processes the first clock and then forwards the first clock to the fourth Codec chip, thereby completing the bit clock synchronization of all Codec chips on the first date data line. The connection method of the second clock connecting the second date data line, the third clock connecting the third date data line, and the fourth clock connecting the fourth date data line is the same as the connection method of the first clock connecting the first date data line, which will not be repeated here.

[0035] Specifically, sequentially transmitting the clock signal through the above connection scheme can effectively reduce the clock deviation caused by the difference in the clock transmission path, so that all Codec chips work at the same clock rhythm, ensuring the synchronization of audio sampling, thereby improving the accuracy of audio processing and avoiding problems such as audio data phase deviation caused by clock asynchrony. Moreover, since this scheme is a random array of MEMS microphones, the daisy chain clock connection method has good adaptability. No matter how the microphones are distributed on the array board, as long as the Codec chip and the microphone are connected reasonably, this clock connection method can ensure that the clock signal is transmitted to each Codec chip in an orderly manner, thereby providing a stable and synchronized clock for the corresponding microphone.

[0036] In some specific embodiments, the power supply between the array board and the control board is provided through a board-to-board connector. The power supply required by the array board is provided and managed by the control board. The control board is provided with a power interface and is powered by an external power supply. That is, the power supply of the entire array subsystem is provided by an external power supply through the power interface on the control board. The external power supply can be a power adapter or a battery pack, etc. The board-to-board connector consists of the following pin definitions: 1. bit clock BLCK, 2. frame clock LRCLK, 3. DATA1, 4. DATA2, 5. DATA3, 6. DATA4, 7. SDA, 8. SCL, 9. VCC, 10. GND.

[0037] Embodiment 2:

[0038] This application also proposes a method for using a multi-channel MEMS microphone random array, referring to Figure 3 , the method comprising: A gigabit network port is provided in the control board of the multi-channel MEMS microphone random array, and the PC-side host computer is directly connected to the multi-channel MEMS microphone random array through the gigabit network port. After the PC-side host computer sends a command to start recording, the array board in the multi-channel MEMS microphone random array starts recording; The first date data line, the second date data line, the third date data line and the fourth date data line in the array board respectively collect the original audio data and transmit it to the control board via the board-to-board connector, and then the control board transmits the original audio data to the PC host computer via the Gigabit network port.

[0039] Specifically, the PC host computer sends a command to start recording via the Gigabit network port using the UDP protocol.

[0040] In some embodiments, the array board transmits the collected raw audio data to the control board in DSP-A data format, and the control board transmits the raw audio data to the PC host computer through the Lwip protocol stack.

[0041] Specifically, the collected original audio data is transmitted to the control board in the DSP-A data format to facilitate the replacement of array boards with different arrangements, while providing a physical basis for data and control signal transmission.

[0042] Specifically, the Lwip protocol stack provides protocol support and encapsulation processing for data network transmission, ensuring that data can be transmitted reliably in the network.

[0043] Specifically, the PC host computer saves the received original audio data as a wav file for subsequent audio analysis, processing or playback operations.

[0044] In some embodiments, the control board configures the registers of each Codec chip in the array board through the I2C module.

[0045] Specifically, the I2C module consists of an I2C clock line SCL and an I2C data line SDA, which are connected to the I2C expansion chip of the array board through a board-to-board connector. The I2C expansion chip expands four I2C clock lines SCL1~4 and four I2C data lines SDA1~4. The SCL1 clock line and the SDA1 data line are configured with 4 Codec chips, and the I2C addresses of the Codec chips are 0x40, 0x41, 0x42, and 0x43 respectively.

[0046] Specifically, the I2C module, as a commonly used inter-chip communication interface, can accurately set parameters of the Codec chip so that it works in a suitable state and ensures the correct collection and processing of audio data.

[0047] In some specific embodiments, the software workflow diagram of the control panel is as follows: Figure 4 As shown, the FreeRTOS real-time operating system and the Lwip protocol stack are running in the control board. When the control board is powered on, the MCU initializes the peripheral pins such as the SAI module, the I2C module, and the ENET module, and initializes the internal modules including the SAI module, the DMA module, and the ENET module. Then, the DMA interrupt function is initialized, and the FreeRTOS kernel and the Lwip protocol stack are initialized. After the FreeRTOS operating system and the Lwip protocol stack are initialized, an audio acquisition data processing thread and an audio data network sending thread are created inside the control board. The content of the audio acquisition data processing thread is: after the DMA module acquires 480 fixed sampling points, a DMA interrupt is generated, the DMA module memory destination address is initialized, and the acquired data is placed in a ring buffer in the DMA interrupt processing function, a queue message is sent to notify the task, and the next interrupt cycle is started. The content of the audio data network sending thread is: when the queue message is received, a frame of audio acquisition data is taken out from the ring buffer. When the control board receives the command to start recording sent by the PC host computer through the UDP protocol, it encapsulates the serial number header of a frame of audio acquisition data, and sends it to the Lwip protocol stack by calling the Lwip protocol stack interface function. The Lwip protocol stack sends the encapsulated network data packet to the PC host computer through the UDP protocol. After the PC host computer receives the audio data, it writes a frame of audio acquisition data to the file. When the PC host computer sends a command to stop recording, the host computer software adds a wav header, saves the audio acquisition data as a wav file, and the control board stops sending audio data.

[0048] In some specific implementations, the actual effect diagram of the array board and the control board is as follows: Figure 5 As shown, in Figure 5 Among them, the soc core board 1 is the key part of the control board, which integrates many processing units and undertakes core functions such as data processing and clock management. It is the control center of the entire system operation; the gigabit network port 2 is used to realize high-speed data transmission between the control board 3 and the PC host computer, ensuring that audio data can interact quickly and stably between the two, meeting the data transmission requirements such as real-time recording; the control board 3 provides physical support and electrical connection for various electronic components to ensure that each component works stably on the board, and its wiring and layout design are closely related to the system function; the array board 4 is the installation carrier of 64 digital MEMS microphones and related Codec chips and other components; the board-to-board connector 5 is the key interface for data, clock signal and power transmission between the control board 3 and the array board 4. Its pin definition covers bit clock, frame clock, data line, I2C line, power and ground, etc. Through these connections, the control board 3 realizes effective control and data interaction of the array board 4 to ensure the coordinated work of the system.

[0049] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0050] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A multi-channel MEMS microphone random array, characterized in that: include: An array board, a control board, and a board-to-board connector, wherein the array board is a random array composed of 64 MEMS microphones, and every four MEMS microphones are connected to a Codec chip, and every four Codec chips form a date data line to obtain a first date data line, a second date data line, a third date data line, and a fourth date data line; The control board is connected to the array board through a board-to-board connector, a total bit clock signal is generated in the control board and transmitted to the array board through the board-to-board connector, and the total bit clock signal is converted into a first bit clock, a second bit clock, a third bit clock and a fourth bit clock in the array board, the first bit clock is connected to the first date data line in a daisy chain manner, the second bit clock is connected to the second date data line in a daisy chain manner, the third bit clock is connected to the third date data line in a daisy chain manner, and the fourth bit clock is connected to the fourth date data line in a daisy chain manner.

2. A multi-channel MEMS microphone random array according to claim 1, characterized in that: Every four Codec chips form a date data line through TDM cascading.

3. The multi-channel MEMS microphone random array according to claim 1, characterized in that: The control board obtains the total bit clock signal through the clock source of the SAI module, and the clock source of the SAI module is provided by the 48MHz RC Clock after being multiplied by the internal PLL of the Soc.

4. The multi-channel MEMS microphone random array according to claim 1, characterized in that: The control board also obtains a total frame clock signal through a clock source of the SAI module, and transmits the total frame clock signal to each Codec chip in the array board.

5. The multi-channel MEMS microphone random array according to claim 1, characterized in that: In the array board, the total bit clock signal is converted into a first bit clock, a second bit clock, a third bit clock and a fourth bit clock through a clock buffer chip.

6. A method for using a multi-channel MEMS microphone random array, characterized in that: include: A gigabit network port is provided in the control board of the multi-channel MEMS microphone random array, and the PC-side host computer is directly connected to the multi-channel MEMS microphone random array through the gigabit network port. After the PC-side host computer sends a command to start recording, the array board in the multi-channel MEMS microphone random array starts recording; The first date data line, the second date data line, the third date data line and the fourth date data line in the array board respectively collect the original audio data and transmit it to the control board via the board-to-board connector, and then the control board transmits the original audio data to the PC host computer via the Gigabit network port.

7. The method for using a multi-channel MEMS microphone random array according to claim 6, characterized in that: The array board transmits the collected raw audio data to the control board in DSP-A data format, and the control board transmits the raw audio data to the PC host computer through the Lwip protocol stack.

8. The method for using a multi-channel MEMS microphone random array according to claim 6, characterized in that: The PC host computer saves the received original audio data as a wav file.

9. The method for using a multi-channel MEMS microphone random array according to claim 6, characterized in that: The control board configures the registers of each Codec chip in the array board through the I2C module.

Images