Desktop robot microphone array sound source positioning system and method

Abstract

The invention relates to the technical field of robots, and particularly discloses a desktop robot microphone array sound source positioning system and method. Comprising a main control chip, acquisition microphones and a power supply module, an ESP32 main control board is connected with a circular array formed by the four acquisition microphones through double I2S bus pins, the array diameter of the four acquisition microphones is 50 mm, the included angle between every two adjacent acquisition microphones is 90 degrees, the acquisition microphones are grounded through GND lines, and the power supply module is connected with the ESP32 main control board. And the ESP32 main control board provides 3.3 V voltage for the acquisition microphone through a power supply line. A four-microphone circular array with the diameter of 50 mm and ESP32 double I2S synchronous acquisition are adopted, 360-degree full horizontal positioning is achieved, the hardware size is small, the miniaturization requirement of a desktop accompanying robot is met, a VAD voice activity detection module is introduced, algorithm operation is triggered only when human voice is detected, power consumption in a dormant state is small, and the endurance of the robot is prolonged.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to a desktop robot microphone array sound source localization system and method. Background Technology

[0002] Currently, the sound source localization function of desktop companion robots mostly adopts the following existing technical solutions: Single / dual microphone localization solution: Some desktop robots use a single microphone or linear dual microphone structure to realize sound source perception, but such solutions can only achieve rough direction judgment (such as distinguishing left and right), cannot cover the 360° full horizontal direction, and the positioning error usually exceeds 15°, which is difficult to meet the needs of precise interaction; High computing power chip driven localization solution: Some high-precision sound source localization systems rely on PCs, Raspberry Pis or dedicated DSP chips to run positioning algorithms such as SRP-PHAT. Although they can guarantee accuracy, such chips generally consume more than 50mA and are large in size, which is incompatible with the design requirements of desktop companion robots for "miniaturization and low power consumption".

[0003] Algorithm operation schemes without intelligent triggering: Most existing low-power sound source localization schemes based on MCUs (such as ESP32) do not integrate a voice activity detection (VAD) module. The algorithm runs continuously all time, resulting in high MCU power consumption and reducing robot battery life by more than 50%. Low computing power adaptation defects of SRP-PHAT algorithm: The traditional SRP-PHAT algorithm needs to process full-band audio data (0-20kHz) and traverse 360° direction with a step size of 1°. On low computing power MCUs such as ESP32, a single calculation takes more than 100ms, resulting in poor real-time performance and inability to support the robot's real-time interactive response. Limitations of visualization schemes: Existing sound source localization visualizations mostly rely on independent host computer software, which requires additional installation programs, is cumbersome to operate, and cannot be adapted to the lightweight interactive scenarios of desktop robots. Summary of the Invention

[0004] The purpose of this invention is to provide a sound source localization system and method for a desktop robot using a microphone array. It employs a 50mm diameter circular array of four microphones + ESP32 dual I2S synchronous acquisition to achieve 360° full horizontal positioning. The hardware is small in size, adapting to the miniaturization requirements of desktop companion robots. A VAD (Voice Activity Detection) module is introduced, triggering algorithm calculations only when human voices are detected, resulting in low power consumption in sleep mode and extended robot battery life. The SRP-PHAT algorithm is lightweighted and improved, using a 256-point hardware-accelerated FFT + 300-3400Hz bandpass PHAT weighting, adapting to low-computing-power scenarios for the ESP32. Real-time data push is achieved through the ESP32's built-in WebServer + WebSocket, and the Web client directly renders heatmaps, adapting to the lightweight interaction requirements of desktop robots, thus solving the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A desktop robot microphone array sound source localization system includes a main control chip, acquisition microphones, and a power supply module. The main control chip is an ESP32 main control board. The ESP32 main control board has a built-in WebServer module, a VAD voice activity detection module, an SRP-PHAT calculation module, an I2S driver module, and a WiFi module. The ESP32 main control board is connected to a circular array of four acquisition microphones through dual I2S bus pins. The array diameter of the four acquisition microphones is 50mm, and the angle between adjacent acquisition microphones is 90°. The acquisition microphones are grounded through a GND line. The ESP32 main control board provides 3.3V voltage to the acquisition microphones through a power supply line.

[0006] As a further aspect of the present invention: the threshold for short-time energy analysis of the audio stream by the VAD voice activity detection module is set to -40dBFS.

[0007] A method for using a desktop robot microphone array sound source localization system includes: S1: The ESP32 main control board synchronously acquires the time-domain audio signals of the four acquisition microphones through a dual I2S bus, and organizes the acquired serial data into four independent parallel audio streams; S2: The ESP32 main control board will send the acquired audio stream to the VAD voice activity detection module. Based on the basic computing power of the ESP32, this module performs short-time energy analysis on the audio stream with a threshold set to -40dBFS to determine whether the current signal is a human voice signal. If the detection result is "no", it will directly return to the multi-channel audio synchronous acquisition stage and maintain a low-power standby state. If the detection result is "yes", the core operation of the SRP-PHAT algorithm of the SRP-PHAT calculation module is triggered.

[0008] As a further aspect of the present invention: before step S1 is executed, hardware initialization configuration is performed, the dual I2S bus of the ESP32 main control board is configured to an audio acquisition mode with a sampling rate of 16kHz and a bit depth of 16bit, and then the physical parameters of the circular array of the four acquisition microphones are calibrated, the physical parameter being the array diameter, and the communication matching between the array and the ESP32 main control board is completed. Finally, the WiFi module of the ESP32 main control board is initialized, and the WebSocket service is started to support subsequent data push.

[0009] As a further aspect of the present invention, it also includes step S1: the SRP-PHAT algorithm execution steps in the SRP-PHAT calculation module are as follows: adding Hanning windows to the 4-channel time-domain audio, the hardware acceleration module of the ESP32 main control board performs 256-point FFT frequency domain conversion, converts the time-domain audio into a frequency domain signal, performs PHAT weighting processing on the frequency domain signal and retains only the phase information of the signal and eliminates amplitude interference, while only weighting is performed on the core frequency band of human voice from 300 to 3400 Hz.

[0010] As a further aspect of the present invention, it also includes step S3: after weighting, it enters a 360° directional cyclic operation, calculates the phase delay corresponding to each direction based on the physical layout of the 4-microphone circular array, with a step size of 1°, and applies the phase delay to the frequency domain signal to achieve "digital beamforming", that is, to simulate the "turning" of the microphone array in that direction. Then, it performs energy superposition on the frequency domain signals of the 4 channels and calculates the SRP turning response power value corresponding to that direction.

[0011] As a further aspect of the present invention, it also includes step S31: after traversing all directions of 360°, the direction with the largest SRP value is selected as the sound source direction to complete the localization operation. At the same time, the localization confidence is calculated, and the ratio of the maximum SRP value to the second largest SRP value is used as the confidence index. When the ratio is ≥3, it is determined to be a valid localization result; otherwise, it is marked as invalid localization.

[0012] As a further aspect of the present invention, step S4 is also included: after the positioning calculation is completed, the positioning results are integrated, and the sound source direction 0°~359° and the confidence index are encapsulated into structured data. Then, through the Web visualization data push stage, relying on the WiFi module of the ESP32 main control board, the data is pushed to the Web client in real time using the WebSocket protocol. The WebServer module, VAD module, SRP-PHAT calculation module, and I2S driver module built into the ESP32 main control board constitute the software layer, which interacts with the hardware layer through the data bus, and then pushes the data to the Web client of the interaction layer through WebSocket communication.

[0013] As a further aspect of the present invention, it also includes step S41: After the Web client receives the data, the client uses the heat map rendering module to display the energy distribution in the 360° direction in a color gradient manner to achieve an intuitive display of the positioning results. After the visualization is completed, the process will return to step S1 to continue the next round of signal acquisition and detection, forming a continuous closed-loop interaction.

[0014] Compared with the prior art, the beneficial effects of the present invention are: Employing a 50mm diameter 4-microphone circular array and dual I2S synchronous acquisition by ESP32, it achieves 360° full horizontal positioning. Its compact hardware size meets the miniaturization requirements of desktop companion robots. A VAD (Voice Activity Detection) module is introduced, triggering algorithm computation only when human voice is detected, resulting in low power consumption in sleep mode and extended robot battery life. The SRP-PHAT algorithm has been lightweighted and improved, employing a 256-point hardware-accelerated FFT and 300-3400Hz bandpass PHAT weighting, adapting to low-computing-power scenarios for ESP32. Real-time data push is achieved through the ESP32's built-in WebServer and WebSocket, with the web client directly rendering heatmaps, meeting the lightweight interaction needs of desktop robots. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram showing the connection between the microphone and the motherboard in a desktop robot microphone array sound source localization system. Figure 2 This is a system architecture block diagram of a desktop robot microphone array sound source localization system. Figure 3 This is a flowchart of the SRP-PHAT algorithm in a method for locating sound sources using a microphone array on a desktop robot. Figure 4 This is a web visualization interface diagram of a method for locating sound sources using a microphone array on a desktop robot. Figure 5 This is a flowchart of a method for locating sound sources using a microphone array on a desktop robot. Detailed Implementation

[0017] Please see Figures 1 to 5 In this embodiment: Based on the ESP32 main control board and a 4-microphone circular array, this method achieves low-power sound source localization and real-time visualization throughout the entire process through the coordinated operation of software modules. The complete implementation logic of this method is explained in detail below with reference to the attached diagram: The connection relationship between the ESP32 main control board and the 4-microphone circular array is shown in the attached figure. Figure 4The ESP32 main control board is connected to a circular array of four MEMS microphones via dual I2S bus pins (GPIO26 and GPIO27 correspond to microphones M1 and M2, and GPIO32 and GPIO33 correspond to microphones M3 and M4). The array diameter is 50mm, and the angle between adjacent microphones is 90°. The microphone array is provided with 3.3V through the red power supply line, and the black GND line is grounded to ensure the stability of the acquired signal. The hardware combination is small in size, which is suitable for the miniaturized installation requirements of desktop companion robots, and the overall standby power consumption is less than 5mA, which meets the low power consumption design goal.

[0018] The complete implementation process of this method is attached. Figure 5 As shown, continuous sound source localization and visualization are achieved in a closed-loop manner. The specific process is as follows: First, hardware initialization configuration is performed, which requires the completion of three core configurations: First, configure the dual I2S buses of the ESP32 main control board to an audio acquisition mode with a 16kHz sampling rate and a 16-bit depth. Second, calibrate the physical parameters (50mm diameter) of the 4-microphone circular array and complete the communication matching between the array and the ESP32 main control board; Third, initialize the WiFi module of the ESP32 main control board and start the WebSocket service to support subsequent data push.

[0019] After hardware initialization is complete, the multi-channel audio synchronous acquisition stage begins: relying on the above hardware connection, the ESP32 main control board synchronously acquires the time-domain audio signals of 4 microphones through dual I2S buses, and organizes the acquired serial data into 4 independent parallel audio streams. The acquisition frame rate in this stage is matched with the real-time performance of subsequent algorithm operations to ensure that the processing time of each frame of data does not exceed 50ms.

[0020] The acquired audio stream is sent to the VAD voice activity detection module. Based on the basic computing power of the ESP32 main control board, this module performs short-time energy analysis on the audio stream (threshold set to -40dBFS) to determine whether the current signal is a human voice signal. If the detection result is "no", it directly returns to the multi-channel audio synchronous acquisition stage and maintains a low-power standby state. If the detection result is "yes", it triggers the subsequent SRP-PHAT algorithm core operation to realize the low-power control logic of "human voice triggered operation".

[0021] The core localization operation of this method relies on the SRP-PHAT algorithm, and its specific sub-processes are shown in the appendix. Figure 2As shown: First, the four-channel time-domain audio input algorithm module is subjected to a Hanning window to reduce spectral leakage. Then, the hardware acceleration module of the ESP32 main control board performs a 256-point FFT frequency domain transformation to convert the time-domain audio into a frequency domain signal. Next, the frequency domain signal is subjected to PHAT weighting processing. This processing retains only the phase information of the signal and eliminates amplitude interference. At the same time, it only weights the core frequency band of human voice from 300 to 3400 Hz to reduce the amount of invalid calculation.

[0022] After weighting, a 360° cyclic calculation is performed: as shown in the attached diagram. Figure 4 As shown, based on the physical layout of a 4-microphone circular array, the phase delay (with a step size of 1°) corresponding to each direction is calculated, and this phase delay is applied to the frequency domain signal to achieve "digital beamforming", that is, the "turning" of the analog microphone array in that direction; then the frequency domain signals of the 4 channels are superimposed to calculate the SRP (turning response power) value corresponding to that direction.

[0023] After traversing all directions of 360°, the direction with the largest SRP value is selected as the sound source direction to complete the localization calculation; at the same time, the localization confidence is calculated, and the ratio of the maximum SRP value to the second largest SRP value is used as the confidence index. When the ratio is ≥3, it is judged as a valid localization result; otherwise, it is marked as invalid localization.

[0024] After the localization calculation is completed, the localization results are integrated: the sound source direction (0°~359°) and confidence index are encapsulated into structured data, and then, through the Web visualization data push process, relying on the WiFi module of the ESP32 main control board, the data is pushed to the Web client in real time using the WebSocket protocol; the software support for this process is attached. Figure 2 As shown: The ESP32 main control board's built-in WebServer module, VAD module, SRP-PHAT calculation module, and I2S driver module constitute the software layer, which interacts with the hardware layer through the data bus, and then pushes data to the Web client of the interaction layer through WebSocket communication.

[0025] After receiving the data, the web client executes the web client visualization step: as shown in the attached document. Figure 5 As shown, the client uses a heatmap rendering module to display the energy distribution in a 360° direction with a blue-red gradient, where low energy is blue and high energy is red. Simultaneously, a red pointer indicates the direction of the sound source (as shown in the attached diagram at 60°), providing a visual representation of the location results. After visualization, the process returns to the multi-channel audio synchronous acquisition stage to continue the next round of signal acquisition and detection, forming a continuous closed-loop interaction.

Claims

1. A desktop robot microphone array sound source localization system, comprising a main control chip, a acquisition microphone, and a power supply module, characterized in that: The main control chip is an ESP32 main control board. The ESP32 main control board has a built-in WebServer module, a VAD voice activity detection module, an SRP-PHAT calculation module, an I2S driver module, and a WiFi module. The ESP32 main control board is connected to a circular array of four acquisition microphones through dual I2S bus pins. The array diameter of the four acquisition microphones is 50mm, and the angle between adjacent acquisition microphones is 90°. The acquisition microphones are grounded through a GND line, and the ESP32 main control board provides 3.3V voltage to the acquisition microphones through a power supply line.

2. The desktop robot microphone array sound source localization system according to claim 1, characterized in that: The threshold for short-time energy analysis of the audio stream by the VAD speech activity detection module is set to -40 dBFS.

3. The method of using a desktop robot microphone array sound source localization system according to claims 1-2, characterized in that: include: S1: The ESP32 main control board synchronously acquires the time-domain audio signals of the four acquisition microphones through a dual I2S bus, and organizes the acquired serial data into four independent parallel audio streams; S2: The ESP32 main control board will send the acquired audio stream to the VAD voice activity detection module. Based on the basic computing power of the ESP32, this module performs short-time energy analysis on the audio stream with a threshold set to -40dBFS to determine whether the current signal is a human voice signal. If the detection result is "no", it will directly return to the multi-channel audio synchronous acquisition stage and maintain a low-power standby state. If the detection result is "yes", the core operation of the SRP-PHAT algorithm of the SRP-PHAT calculation module is triggered.

4. The method of using a desktop robot microphone array sound source localization system according to claim 3, characterized in that: Before step S1 is executed, hardware initialization configuration is performed. The dual I2S bus of the ESP32 main control board is configured to an audio acquisition mode with a sampling rate of 16kHz and a bit depth of 16bit. Then, the physical parameters of the circular array of the four acquisition microphones are calibrated. The physical parameters are the array diameter. Communication matching between the array and the ESP32 main control board is completed. Finally, the WiFi module of the ESP32 main control board is initialized, and the WebSocket service is started to support subsequent data push.

5. The method of using a desktop robot microphone array sound source localization system according to claim 3, characterized in that: It also includes step S1: The SRP-PHAT algorithm execution steps in the SRP-PHAT calculation module are as follows: Hanning windows are added to the 4-channel time-domain audio, the hardware acceleration module of the ESP32 main control board performs 256-point FFT frequency domain conversion, converts the time-domain audio into a frequency domain signal, performs PHAT weighting processing on the frequency domain signal and retains only the phase information of the signal and eliminates amplitude interference, and only weights the core frequency band of human voice from 300 to 3400 Hz.

6. The method of using a desktop robot microphone array sound source localization system according to claim 3, characterized in that: It also includes step S3: After weighting, it enters 360° directional cyclic calculation. Based on the physical layout of the 4-microphone circular array, it calculates the phase delay corresponding to each direction with a step size of 1°, and applies the phase delay to the frequency domain signal to realize "digital beamforming", that is, the analog microphone array "turns" in that direction. Then, it performs energy superposition on the frequency domain signals of the 4 channels and calculates the SRP turning response power value corresponding to that direction.

7. The method of using a desktop robot microphone array sound source localization system according to claim 6, characterized in that: It also includes step S31: After traversing all directions of 360°, the direction with the largest SRP value is selected as the sound source direction to complete the localization operation. At the same time, the localization confidence is calculated. The ratio of the maximum SRP value to the second largest SRP value is used as the confidence index. When the ratio is ≥3, it is determined to be a valid localization result; otherwise, it is marked as invalid localization.

8. The method of using a desktop robot microphone array sound source localization system according to claim 7, characterized in that: The process also includes step S4: After the positioning calculation is completed, the positioning results are integrated, and the sound source direction from 0° to 359° and the confidence index are encapsulated into structured data. Then, through the Web visualization data push stage, relying on the WiFi module of the ESP32 main control board, the data is pushed to the Web client in real time using the WebSocket protocol. The WebServer module, VAD module, SRP-PHAT calculation module, and I2S driver module built into the ESP32 main control board constitute the software layer, which interacts with the hardware layer through the data bus, and then pushes the data to the Web client of the interaction layer through WebSocket communication.

9. The method of using a desktop robot microphone array sound source localization system according to claim 8, characterized in that: It also includes step S41: After the Web client receives the data, the client uses the heat map rendering module to display the energy distribution in the 360° direction in a color gradient manner to achieve an intuitive display of the positioning results. After the visualization is completed, the process will return to step S1 to continue the next round of signal acquisition and detection, forming a continuous closed-loop interaction.

Images