Sound source localization method for humanoid robot

Abstract

The invention provides a sound source localization method for a humanoid robot, which relates to the technical field of data transmission, and comprises a hardware module and a software module, the hardware module comprises a microphone array, an acquisition unit and a signal processing board, the software module comprises a voice signal preprocessing unit, a microphone array time delay estimation unit and a microphone array position estimation unit; a microphone array in the sound source positioning module picks up voice signals in the space and converts the voice signals into electric signals; the acquisition module converts the electric signal into a digital signal and transmits the digital signal as an input signal to the signal processing board; the signal processing board processes and calculates the voice signal and transmits obtained sound source position information to the upper computer; and the upper computer displays the sound source position information in real time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of acoustic perception of bionic robots, and particularly relates to a sound source positioning technology for humanoid robots. BACKGROUND

[0002] In China, in the civil field, sound source positioning technology has been widely used in intelligent translation, voice input, smart home and other fields, and the combination of sound perception and bionic robots has just emerged. Recently, products such as music dancing robots have appeared on the market, which are limited to toys or commercial fields, and the background environment is relatively quiet during work, and the voice pickup distance is only within 5 meters. In the field of military unmanned technology, due to the complex use environment and high technical threshold, there is no formal equipment in China, and the research and application of sound source positioning technology combined with vision are relatively less, and there is still a large development space. Abroad, sound source positioning technology has been widely researched and applied. For example, in the field of unmanned driving, sound source positioning technology can be combined with vision to identify the position and direction of traffic signal lights, improving the safety of unmanned vehicles. In addition, in the field of rescue robots, sound source positioning technology can be used to locate the position of trapped personnel, improving the efficiency of rescue. Foreign research institutions and enterprises have made some important breakthroughs and application cases in sound source positioning technology. With the continuous development of artificial intelligence and robot technology, sound source positioning technology will be more widely applied. The future development trend mainly includes: improvement of accuracy, multi-modal fusion, real-time and robustness. Sound source positioning technology will be applied in more fields, such as intelligent transportation, intelligent manufacturing and medical assistance, bringing more convenience and benefits to people's life and work. With the continuous progress of technology and the expansion of application scenarios, sound source positioning technology will play an increasingly important role in the field of intelligent robots. SUMMARY

[0003] In view of the deficiencies of the prior art, the present application provides a sound source positioning method for a humanoid robot.

[0004] A sound source positioning method for a humanoid robot, comprising a hardware module and a software module, the hardware module comprising: a microphone for converting the picked-up voice signal into an electrical signal; a collection unit connected to the microphone electrical signal, for converting the electrical signal converted by the microphone into a digital signal; a signal processing board, the software module is integrated on the signal processing board, and the software module comprises a voice signal preprocessing unit and a microphone array position estimation unit, wherein: The voice signal preprocessing unit is electrically connected with the collecting unit, and is used for pre-filtering, frame windowing and endpoint detection on the digital signal input by the collecting unit. The microphone array position estimation unit is electrically connected with the voice signal preprocessing unit, estimates the position of the sound source based on the information preprocessed by the voice signal preprocessing unit, and obtains the position information of the sound source. The upper computer is electrically connected with the microphone array position estimation unit, and is used for displaying the obtained position information of the sound source in real time.

[0005] Preferably, the microphone array is composed of a plurality of microphones in an array form, each microphone is electrically connected with the collecting unit through a flexible FPC wire, and the microphone array is fixed on the center of the head of the humanoid robot and is isolated from the robot head shell through a damping silica gel pad.

[0006] Preferably, the microphone array comprises 7 microphones, and the 7 microphones are arranged in a hexagonal center-symmetric manner on the same horizontal plane, wherein one microphone is located at the center of the hexagon, and the remaining 6 microphones are respectively located at the vertices of the hexagon with a side length d, and the value range of d is 40mm≤d≤80mm.

[0007] Preferably, the software module further comprises a microphone array time delay estimation unit, which is electrically connected with the voice signal preprocessing unit and the microphone array position estimation unit respectively, and is used for feeding back the information superimposed by the microphone array time delay estimation information to the microphone array position estimation unit after the information preprocessed by the voice signal preprocessing unit.

[0008] Preferably, the hardware module comprises a microphone array, a collecting unit and a signal processing board, which are all fixed on the top of the head of the humanoid robot.

[0009] Preferably, the collecting unit is a multi-channel synchronous ADC chip, which converts the electric signal converted by the microphone array into a digital signal through A / D conversion, and transmits the digital signal as an input signal to the signal processing board.

[0010] The application further provides a sound source positioning method for a humanoid robot, which comprises the following steps: S1. The microphone picks up sound, and converts the picked-up voice signal into an electric signal; S2. The electric signal converted by the microphone array is transmitted to the collecting unit, and is converted into a digital signal by the collecting unit; S3. The collected data is transmitted to the signal processing board for preprocessing; S4. It is judged whether there is an effective language signal, if yes, the next step is executed, and if no, the step S3 is repeated; S5. Calculate the time delay estimation and position information from the language signal according to S4; S6. Display the positioning result.

[0011] Preferably, the voice signal preprocessing unit in the S3 step is executed according to the following steps: Pre-filtering: applying a sixth-order IIR band-pass filter of 300Hz-8kHz to each input digital signal to suppress out-of-band noise; Frame and windowing: dividing the signal into 32ms frames with a 16kHz sampling rate and 50% frame shift, and applying a Hamming window to each frame to reduce spectral leakage; Endpoint detection: calculating the short-time energy and zero-crossing rate of each frame, and if the energy of three consecutive frames is higher than the low threshold and lower than the high threshold, and the zero-crossing rate is in the preset interval, it is determined as the starting frame of the voice; if the three consecutive frames do not meet the condition, it is determined as the end frame, and only the voice segment between the start and end is retained for subsequent processing.

[0012] Preferably, the specific steps of S5 for microphone array time delay estimation are as follows: Time delay estimation: taking the center microphone as the reference, calculating the generalized cross-correlation-phase transform function for the remaining 6 microphone signals respectively, and taking the peak position to obtain 6 groups of time difference of arrival; Error elimination: median filtering is performed to eliminate abnormal values deviating from the median value by more than 50µs, and the retained time delay is obtained and transmitted.

[0013] Preferably, S5 further includes the following steps: Substitute the retained time delay into the spherical wavefront model, and solve the azimuth angle θ and the pitch angle φ of the sound source relative to the array center by least squares method; Coordinate conversion: convert θ and φ into three-dimensional space coordinates (x, y, z) according to the real-time attitude matrix of the robot head coordinate system and the global coordinate system; Output: send (x, y, z) to the host computer through UART with a period of 20ms for real-time display.

[0014] The present application provides a sound source positioning method for humanoid robots. It has the following advantages: (1) The present application adopts a 6(+1) microphone hexagonal ring array scheme, which can realize 360° positioning range compared with one-dimensional array structure, solving the problem of small positioning range of one-dimensional array, and the complexity is much lower than three-dimensional array, which is also the most commonly used microphone array topology structure.

[0015] (2) The interval of the microphone in the application not only considers the applicable environment of the humanoid robot, but also meets the sampling requirements of the sound source signal, so as to meet the positioning accuracy of the robot sound source positioning system. The sound source in this design is the sound emitted by human beings, and the signal frequency is generally 300Hz-3400Hz. According to the Nyquist sampling theorem, in order to keep the sampled speech signal complete and effective information, the minimum interval between adjacent microphones should be less than half of the wavelength of the sampling signal. According to the relationship between wavelength and frequency, the wavelength value range is 0.1m to 1.13m, and the distance range between the adjacent two microphones is 0.05m to 0.565m. According to the conditions and effects of the system device, the interval of the microphone array element is adjusted in this value range. Considering the structure of the humanoid robot, in order to facilitate installation and movement, the interval of the microphone array element is about D=5.2cm.

[0016] (3) The software scheme of the application selects the sound source positioning technology based on time delay estimation, mainly considering that the use environment of the bionic robot is the indoor complex acoustic environment. Compared with other algorithms, this method does not need to assume or know the statistical information of the sound source and noise in advance; there is no requirement for signal bandwidth, whether it is for wideband or narrowband signal; there is an analytical solution, the algorithm complexity is low, the calculation amount is small, and it can be conveniently applied to real-time system; and the positioning system accuracy can be flexibly improved by increasing the sampling frequency, increasing the array aperture or interpolation.

[0017] The above description is only a summary of the technical scheme of the application, in order to more clearly understand the technical means of the application, and can be implemented according to the content of the specification, the following will be described in detail with the preferred embodiment of the application. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical scheme in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description only some embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0019] Figure 1 is the workflow diagram of the application.

[0020] Figure 2 is the layout diagram of the application on the humanoid robot. DETAILED DESCRIPTION

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figure 1 This invention proposes a sound source localization method for humanoid robots, comprising a hardware module and a software module, wherein the hardware module includes: A microphone is used to convert picked-up speech signals into electrical signals; The acquisition unit is connected to the microphone's electrical signal and is used to convert the electrical signal converted by the microphone into a digital signal; A signal processing board, wherein the software module is integrated on the signal processing board, and the software module includes a speech signal preprocessing unit and a microphone array position estimation unit. The signal processing board is powered by a power supply. The speech signal preprocessing unit is electrically connected to the acquisition unit and is used to perform preprocessing such as pre-filtering, frame windowing and endpoint detection on the digital signal input by the acquisition unit. The microphone array position estimation unit is electrically connected to the speech signal preprocessing unit. It estimates the position of the sound source based on the information preprocessed by the speech signal preprocessing unit, thus obtaining the sound source's position information. The host computer is electrically connected to the microphone array position estimation unit and is used to display the obtained sound source position information in real time.

[0023] As an optional embodiment, the microphone array consists of several microphones arranged in an array. Each microphone is electrically connected to the acquisition unit via a flexible FPC cable. The microphone array is fixed at the center of the top of the humanoid robot's head and is isolated from the robot's head shell by a vibration-damping silicone pad.

[0024] As an optional embodiment, the microphone array includes 7 microphones, which are arranged in a hexagonal central symmetric manner on the same horizontal plane. One microphone is located at the center of the hexagon, and the other 6 microphones are located at the vertices of the hexagon with a side length of d, where the value of d is in the range of 40mm≤d≤80mm.

[0025] As an optional embodiment, the software module further includes a microphone array delay estimation unit, which is electrically connected to the speech signal preprocessing unit and the microphone array position estimation unit, respectively, and is used to feed back the microphone array delay estimation information after superimposing the information preprocessed by the speech signal preprocessing unit to the microphone array position estimation unit.

[0026] As an optional embodiment, the microphone array, the acquisition unit and the signal processing board included in the hardware module are fixed on the top of the head of the humanoid robot.

[0027] As an optional embodiment, the acquisition unit is a multi-channel synchronous ADC chip, which converts the electrical signals converted by the microphone array into digital signals through A / D conversion, and transmits the digital signals as input signals to the signal processing board.

[0028] The application further provides a sound source positioning method for a humanoid robot, comprising the following steps: S1. The microphone picks up sound, and converts the picked-up speech signal into an electrical signal; S2. The electrical signal converted by the microphone array is transmitted to the acquisition unit, and is converted into a digital signal by the acquisition unit; S3. The acquired data is transmitted to the signal processing board for preprocessing; S4. It is identified whether there is an effective language signal, if yes, the next step is performed, and if no, the step S3 is repeated; S5. The time delay estimation and position information are calculated according to the language signal of S4; S6. The positioning result is displayed.

[0029] As an optional embodiment, the speech signal preprocessing unit in the S3 step performs the following steps: Pre-filtering: a sixth-order IIR band-pass filter of 300Hz-8kHz is applied to each input digital signal to suppress out-of-band noise; Frame and windowing: the signal is cut into 32ms frames with a 16kHz sampling rate and a 50% frame shift, and a Hamming window is applied to each frame to reduce spectral leakage; End-point detection: the short-time energy and the zero-crossing rate of each frame are calculated, if the energy of three consecutive frames is higher than the low threshold and lower than the high threshold, and the zero-crossing rate is in the preset interval, it is determined that the frame is a speech starting frame; if the three consecutive frames do not satisfy the above conditions, it is determined that the frame is an ending frame, and only the speech segment between the starting frame and the ending frame is reserved for subsequent processing.

[0030] As an optional embodiment, the specific steps of the S5 for microphone array time delay estimation are as follows: Time delay estimation: taking the central microphone as a reference, the generalized cross-correlation-phase transform function is calculated for the remaining six microphone signals, and six groups of time difference of arrival are obtained by taking the peak value position; Error elimination: the median filter is applied to eliminate abnormal values deviating from the median value by more than 50µs, and the retained time delay is obtained and transmitted.

[0031] As an optional embodiment, the S5 further comprises the following steps: The reserved time delay is substituted into the spherical wave front model, and the azimuth angle θ and the pitch angle φ of the sound source relative to the array center are solved by using the least square method; Coordinate conversion: according to the real-time attitude matrix of the robot head coordinate system and the global coordinate system, θ and φ are converted into three-dimensional space coordinates (x, y, z); Output: (x, y, z) is sent to the host computer through UART at a period of 20 ms for real-time display.

[0032] The working principle of the sound source positioning technology for a humanoid robot is as follows: The sound source positioning system mainly calculates the sound source position based on the time difference of the voice information picked up by each microphone in the microphone array and the geometric relationship of the microphone array. Since there is a time delay between the transmission of the voice signal to each microphone, the time delay value is calculated by processing the voice signals collected by each microphone, then the distance difference of the sound source signal reaching each microphone is calculated by using the time delay value, and finally the azimuth angle of the sound source is located by a geometric method.

[0033] If a pair of microphones is regarded as two fixed points and the time delay difference is regarded as a constant, then the sound source trajectory determined by a pair of microphones and the corresponding time delay difference is on a hyperbola with the pair of microphones as the foci. According to the definition of hyperbola, the distance difference of each point on the hyperbola to the two foci is constant. Therefore, in an ideal case, if two microphones are regarded as foci, a hyperbola can be determined by the corresponding distance difference, and the sound source must appear at a certain point on the hyperbola.

[0034] Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A sound source localization device for a humanoid robot, characterized in that, It includes a hardware module and a software module, wherein the hardware module includes: A microphone is used to convert picked-up speech signals into electrical signals; The acquisition unit is connected to the microphone's electrical signal and is used to convert the electrical signal converted by the microphone into a digital signal; The signal processing board, wherein the software module is integrated on the signal processing board, and the software module includes a speech signal preprocessing unit and a microphone array position estimation unit, wherein: The speech signal preprocessing unit is electrically connected to the acquisition unit and is used to perform preprocessing on the digital signal input to the acquisition unit, such as pre-filtering, frame windowing, and endpoint detection. The microphone array position estimation unit is electrically connected to the speech signal preprocessing unit. It estimates the position of the sound source based on the information preprocessed by the speech signal preprocessing unit, thus obtaining the sound source's position information. The host computer is electrically connected to the microphone array position estimation unit and is used to display the obtained sound source position information in real time.

2. The sound source localization device for a humanoid robot according to claim 1, characterized in that, The microphone array consists of several microphones arranged in an array. Each microphone is electrically connected to the acquisition unit via a flexible FPC cable. The entire microphone array is fixed at the center of the humanoid robot's head and is isolated from the robot's head shell by a vibration-damping silicone pad.

3. A sound source localization device for a humanoid robot according to claim 2, characterized in that, The microphone array contains 7 microphones, which are arranged in a hexagonal central symmetric manner on the same horizontal plane. One microphone is located at the center of the hexagon, and the other 6 microphones are located at the vertices of the hexagon with a side length of d, where the value of d is 40mm≤d≤80mm.

4. A sound source localization device for a humanoid robot according to claim 2, characterized in that, The software module further includes a microphone array delay estimation unit, which is electrically connected to the speech signal preprocessing unit and the microphone array position estimation unit, respectively. The microphone array delay estimation unit is used to superimpose the microphone array delay estimation information on the information preprocessed by the speech signal preprocessing unit and then feed it back to the microphone array position estimation unit.

5. A sound source localization device for a humanoid robot according to claim 1, characterized in that, The hardware module, including the microphone array, acquisition unit, and signal processing board, is fixed at the top of the humanoid robot's head.

6. A sound source localization device for a humanoid robot according to claim 1, characterized in that, The acquisition unit is a multi-channel synchronous ADC chip, which converts the electrical signal generated by the microphone array into a digital signal through A / D conversion and transmits it as an input signal to the signal processing board.

7. A method for sound source localization in a humanoid robot, characterized in that, Includes the following steps: S1. The microphone picks up sound and converts the picked-up speech signal into an electrical signal; S2. The electrical signal converted by the microphone array is transmitted to the acquisition unit, and then converted into a digital signal by the acquisition unit; S3. Transfer the collected data to the signal processing board for preprocessing; S4 identifies whether there is a valid language signal. If there is, proceed to the next step; otherwise, repeat step S3. S5. Calculate the time delay estimate and location information based on the speech signal in S4; S6. Display the positioning results.

8. A sound source localization method for a humanoid robot according to claim 7, characterized in that, The speech signal preprocessing unit in step S3 performs the following steps: Pre-filtering: Apply a sixth-order IIR bandpass filter (300Hz–8kHz) to each input digital signal to suppress out-of-band noise; Framing and Windowing: The signal is divided into 32ms frames with a sampling rate of 16kHz and a frame shift of 50%, and a Hamming window is applied to each frame to reduce spectral leakage. Endpoint detection: Calculate the short-time energy and zero-crossing rate of each frame. If three consecutive frames simultaneously satisfy the conditions that the energy is higher than the low threshold and lower than the high threshold and the zero-crossing rate is within the preset range, it is determined to be a speech start frame. If three consecutive frames do not satisfy the conditions, it is determined to be an end frame, and only the speech segment between the start and end is retained for subsequent processing.

9. A sound source localization method for a humanoid robot according to claim 7, characterized in that, The specific steps for microphone array delay estimation using S5 are as follows: Time delay estimation: Taking the center microphone as a reference, the generalized cross-correlation-phase transformation function is calculated for the signals of the other 6 microphones respectively, and the peak position is taken to obtain 6 sets of arrival time differences; Error removal: Median filtering is performed to remove outliers that deviate from the median value by more than 50µs, thus obtaining the retained delay, which is then transmitted.

10. A sound source localization method for a humanoid robot according to claim 8, characterized in that, S5 further includes the following steps: Substitute the retained time delay into the spherical wavefront model, and use the least squares method to solve for the azimuth angle θ and elevation angle φ of the sound source relative to the array center; Coordinate transformation: Based on the real-time attitude matrix of the robot's head coordinate system and the global coordinate system, θ and φ are transformed into three-dimensional spatial coordinates (x, y, z). Output: (x, y, z) is sent to the host computer via UART at 20ms intervals for real-time display.

Images