A method and apparatus for robot wake-word sensitivity to adapt to scene noise

Abstract

The application discloses a method and device for robot wake-up word sensitivity self-adaption to scene noise, which mainly monitors and evaluates the noise level and type in the current environment in real time, and then dynamically adjusts the wake-up word recognition threshold according to the real-time noise evaluation result, so that the self-adaption adjustment mechanism is realized, the sensitivity can be automatically adjusted according to the change of the environmental noise without manual intervention, and when the energy of the environmental sound signal after noise reduction processing is determined to be higher than the wake-up word recognition threshold, it is determined whether to perform the wake-up word recognition, so that the robot can maintain a high wake-up rate in different noise environments, the robot can be self-adapted to various complex and changeable noise environments, and the intelligent level and user experience of the robot are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots, and particularly to a method and device for adapting the sensitivity of a robot's wake-up word to the noise in the scene. BACKGROUND

[0002] With the rapid development of intelligent robot technology, voice wake-up technology is the precursor of robot interaction with users. Voice wake-up technology refers to the technology of waking up a device through voice, and the wake-up sensitivity of a wake-up word is an important indicator of voice wake-up technology. When the matching degree of the voice signal emitted by the user and the wake-up word is high, that is, the score of the voice signal is greater than the wake-up sensitivity of the wake-up word, the device is then woken up. In actual applications, the environment in which the robot is located often has various noises, such as background sound, human voice interference, etc., which seriously affect the wake-up word recognition sensitivity of the robot, and thus the wake-up sensitivity of the wake-up word needs to be adjusted.

[0003] In the prior art, see the application publication number CN109243431A, a processing method, a control method, a recognition method, a device and an electronic device, which discloses adjusting the wake-up sensitivity of the device, including: setting the wake-up parameters of the wake-up engine of the device and / or setting the sound collection parameters of the sound sensor of the device according to the application scene information and the sensitivity setting information. It mainly improves the recognition rate through a pre-set fixed threshold, and is difficult to adapt to complex and variable noise environments, so that the wake-up rate of the robot in various noise environments is low, and the corresponding wake-up word recognition accuracy is also poor. SUMMARY

[0004] Therefore, it is necessary to provide a method for adapting the sensitivity of a robot's wake-up word to the noise in the scene, which dynamically adjusts the recognition threshold of the wake-up word, combines the characteristics of environmental noise, and realizes high-precision adaptive adjustment of the sensitivity of the wake-up word, thereby improving the wake-up rate of the robot in various noise environments.

[0005] To achieve the above-mentioned purpose, the present application provides a method for adapting the sensitivity of a robot's wake-up word to the noise in the scene, which comprises the following steps:

[0006] Real-time collection of environmental sound signals of the scene where the robot is located;

[0007] Extraction of effective features from the environmental sound signals to obtain a noise feature vector;

[0008] Real-time adjustment of the wake-up word recognition threshold according to the noise feature vector;

[0009] Noise reduction processing of the environmental sound signals;

[0010] When the energy of the noise-reduced ambient sound signal exceeds the wake word recognition threshold, the speech recognition model is used to recognize the wake word in the noise-reduced ambient sound signal and respond to the user.

[0011] Further, the step of extracting effective features from the environmental sound signal to obtain a noise feature vector includes the following steps:

[0012] Extract the frequency domain and time domain features of the environmental sound signal to obtain the corresponding features;

[0013] The extracted features are fused to form a noise feature vector.

[0014] Furthermore, extracting the frequency domain features of the ambient sound signal includes using spectral entropy and / or MFCC to extract features from the ambient sound signal.

[0015] Furthermore, extracting the temporal features of the environmental sound signal includes extracting features from the environmental sound signal using short-time energy.

[0016] Furthermore, the step of extracting effective features from the ambient sound signal to obtain a noise feature vector also includes a step of noise preprocessing of the ambient sound signal.

[0017] Furthermore, the step of noise preprocessing the environmental sound signal includes locating the noise source and separating noise sources from different directions.

[0018] Furthermore, noise source localization is achieved using a multi-microphone array based on signal arrival time difference, including the following steps:

[0019] Let x i (t) represents the ambient sound signal collected by the i-th microphone, d i Let be the distance between the i-th microphone and the noise source, and c be the speed of sound. Then, the arrival time difference Δt from the noise source to each microphone is... ij It can be represented as:

[0020]

[0021] Where △t ij d represents the time difference between the sound source and microphones i and j. i and d j These are the distances from the sound source to microphone i and microphone j, respectively, and c is the speed of sound in the medium.

[0022] Estimating the direction of the noise source using the GCC-PHAT algorithm:

[0023]

[0024] Among them, R ij (τ) is the normalized cross-correlation function of the sound signals received by microphones i and j, X i (f) and X j (f) are the Fourier transforms of the i-th and j-th microphone signals, respectively. X represents i The complex conjugate of (f), e j2πfτ It is the phase factor, τ=△t ij , which represents the time delay.

[0025] Furthermore, beamforming is used to separate noise sources from different directions:

[0026]

[0027] Where y(t) is the output ambient sound signal after beamforming, M represents the number of input signals, and x i (t) represents the ambient sound signal collected by the i-th microphone; w i It is the beamforming weight.

[0028] Furthermore, the step of adjusting the wake-word recognition threshold in real time based on the noise feature vector includes the following steps:

[0029] Introducing a sensitivity adjustment factor α, then

[0030] α=tf(F(t))

[0031] Where F(t) is the noise feature vector, and f(F(t)) is the mapping function from the noise feature vector to the sensitivity factor;

[0032] If linear regression is used for training, then the wake word recognition threshold T

[0033] T=T0α

[0034] Where T0 is the original wake word recognition threshold.

[0035] An apparatus for robot wake word sensitivity adaptive to scene noise, comprising the steps of implementing the method for robot wake word sensitivity adaptive to scene noise, including:

[0036] The environmental sound acquisition module is used to collect environmental sound signals in the robot's environment in real time.

[0037] The noise feature extraction module receives the ambient sound signal collected by the ambient sound acquisition module and extracts effective features from the ambient sound signal to obtain a noise feature vector.

[0038] An adaptive sensitivity adjustment module adjusts the wake-word recognition threshold in real time based on noise feature vectors.

[0039] The noise reduction module receives the ambient sound signal collected by the ambient sound acquisition module and performs noise reduction processing on the ambient sound signal;

[0040] The wake-up word recognition module receives the ambient sound signal after noise reduction by the noise reduction module. When the energy of the ambient sound signal exceeds the wake-up word recognition threshold, it performs wake-up word recognition on the noise-reduced ambient sound signal and responds to the user.

[0041] Unlike existing technologies, the above-mentioned technical solution mainly monitors and evaluates the noise level and type in the current environment in real time. Based on the real-time noise evaluation results, it dynamically adjusts the wake word recognition threshold to achieve an adaptive adjustment mechanism. This allows the robot to automatically adjust its sensitivity according to changes in environmental noise without human intervention. Furthermore, it determines whether to perform wake word recognition by judging whether the energy of the noise-reduced environmental sound signal exceeds the wake word recognition threshold. This enables the robot to maintain a high wake-up rate in different noise environments, allowing it to adapt to various complex and changing noise environments, thereby improving the robot's intelligence level and user experience. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the method for adapting robot wake word sensitivity to scene noise as described in the embodiment.

[0043] Figure 2 This is a schematic diagram illustrating the process of extracting effective features from environmental sound signals to obtain noise feature vectors as described in the embodiment.

[0044] Figure 3 This is a schematic diagram of the device module for adapting robot wake word sensitivity to scene noise as described in the embodiment. Detailed Implementation

[0045] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0046] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0047] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0048] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0049] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0050] Unless otherwise specified, the use of terms such as “comprising,” “including,” “having,” or other similar expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0051] Similar to the interpretation in the Patent Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceeding" are understood to exclude the stated number; expressions such as "above," "below," and "within" are understood to include the stated number. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also interpreted in this way, such as "multiple groups" and "multiple times," unless otherwise explicitly specified.

[0052] In the description of the embodiments of this application, the space-related expressions used, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," indicate the orientation or positional relationship based on the orientation or positional relationship shown in the specific embodiments or drawings. They are only for the purpose of describing the specific embodiments of this application or for the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0053] Unless otherwise expressly specified or limited, the terms "installation," "connection," "linking," "fixing," and "setting," as used in the description of the embodiments of this application, should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two components or the interaction between two components. For those skilled in the art to which this application pertains, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0054] See Figure 1 and Figure 2 As shown, this invention provides a method for robot wake word sensitivity to adapt to ambient noise, aiming to improve the accuracy and robustness of wake word detection in complex noisy environments. It mainly involves real-time monitoring and evaluation of the noise level and type in the current environment, and then dynamically adjusting the wake word recognition threshold based on the real-time noise evaluation results (e.g., appropriately increasing the wake word recognition threshold in cases of high background noise to reduce false wake-ups). This adaptive adjustment mechanism allows the robot to automatically adjust its sensitivity according to changes in ambient noise without manual intervention. Furthermore, it determines whether to perform wake word recognition by judging whether the energy of the noise-reduced ambient sound signal exceeds the wake word recognition threshold. This ensures the robot maintains a high wake-up rate in various noise environments, enabling it to adapt to diverse and complex noise conditions, thereby improving the robot's intelligence and user experience.

[0055] To further illustrate the method for robot wake word sensitivity adaptive scene noise according to the present invention, the present invention provides a specific implementation method, which includes the following steps:

[0056] S1. Real-time acquisition of ambient sound signals from the robot's location;

[0057] S2. Extract effective features from the environmental sound signal to obtain a noise feature vector;

[0058] S3. Adjust the wake-word recognition threshold in real time based on the noise feature vector;

[0059] S4. Perform noise reduction processing on the ambient sound signal;

[0060] S5. When the energy of the noise-reduced ambient sound signal exceeds the wake-word recognition threshold, the speech recognition model is used to recognize the wake-word of the noise-reduced ambient sound signal, respond to the user, and trigger the speech recognition process.

[0061] In step S1 above, collecting the ambient sound signal of the robot's location can be achieved by selecting a suitable microphone to collect the ambient sound of the scene and forming an ambient sound signal; the microphone can be a condenser microphone, a dynamic microphone, etc. The microphone can be a single microphone or a microphone array formed by multiple microphones (such as a 6-microphone ring array formed by 6 microphones).

[0062] In step S2 above, the step of extracting effective features from the above environmental sound signal to obtain a noise feature vector includes extracting frequency domain features and time domain features of the environmental sound signal, obtaining corresponding features, and fusing the extracted features to form a noise feature vector.

[0063] Frequency domain features of ambient sound signals can be extracted using one or more of the following methods: STFT (Short Time Fourier Transform), spectral entropy, and MFCC (Mel Frequency Cepstral Coefficients). Time domain features of ambient sound signals can be extracted using one or more of the following methods: frequency features, short-time energy, and transient features.

[0064] In some embodiments, the frequency domain features of the ambient sound signal are extracted using both spectral entropy and MFCC (Mel frequency cepstral coefficients); wherein,

[0065] Spectral entropy is used to extract features from environmental sound signals:

[0066]

[0067] Where P(k) is the normalized energy of the k-th frequency band.

[0068] MFCC (Mel-frequency cepstral coefficients) is used to extract features from ambient sound signals for more accurate wake-word detection.

[0069]

[0070] Where S(n) is the spectral value and N is the number of Mel filters.

[0071] The temporal features of environmental sound signals are extracted using short-time energy, enhancing the ability to identify different types of noise.

[0072]

[0073] This formula represents the total energy of a signal x(t) over a discrete time interval (from n = 0 to n = N - 1). E(t) represents the energy of the signal x(t) at time t (in discrete signal processing, this usually represents the total energy of the signal over a period of time); x(n) represents the signal value at discrete time n, which is a time-varying sequence, usually obtained by sampling the continuous-time signal x(t); n is the discrete-time index, ranging from 0 to N - 1, representing each time point of the sampled signal; N represents the number of sampling points of the signal, or the number of sampling points within a time window. |x(n)| 2 Let |x(n)| represent the instantaneous power of the signal x(n) at time point n. For a complex signal x(n), |x(n)| is the amplitude (modulus) of the signal. 2 That is the signal power.

[0074] The extracted features are then fused to form a noise feature vector F(t):

[0075] F(t) = [E(t), H(t), Cm]

[0076] Where E(t) is the short-time energy, H(t) is the spectral entropy, and Cm is the MFCC.

[0077] Before the step of extracting effective features from the ambient sound signal to obtain a noise feature vector, a noise preprocessing step is included. This noise preprocessing step may include, but is not limited to, noise source identification, noise blocking, separation of noise sources from different directions, and improving the signal-to-noise ratio, thereby providing a more accurate, stable, and reliable signal for subsequent feature extraction. In some embodiments, the noise preprocessing step includes noise source localization and separation of noise sources from different directions; noise source localization can be achieved using multiple microphones arranged in an array, forming a multi-microphone array, which can be a linear array or a circular array; then, the noise source is located by using the arrival time difference of the ambient sound signals collected by the multi-microphone array, accurately locating the noise source and providing better noise suppression. Specifically, the steps include:

[0078] Let x i (t) represents the ambient sound signal collected by the i-th microphone, d i Let be the distance between the i-th microphone and the noise source, and c be the speed of sound. Then, the arrival time difference Δt from the noise source to each microphone is... ij It can be represented as:

[0079]

[0080] Where △t ijd represents the time difference between the sound source and microphones i and j. i and d j These are the distances from the sound source to microphone i and microphone j, respectively, and c is the speed of sound in the medium.

[0081] Estimating the direction of the noise source using the GCC-PHAT algorithm:

[0082]

[0083] Among them, R ij (τ) is the normalized cross-correlation function of the sound signals received by microphones i and j, X i (f) and X j (f) are the Fourier transforms of the i-th and j-th microphone signals, respectively. X represents i The complex conjugate of (f), e j2πfτ It is the phase factor, τ=△t ij , which represents the time delay.

[0084] Cross-correlation function R ij (τ) is used to estimate the time difference τ between two microphones, and its maximum value corresponds to the time offset where the two signals are most similar. This time offset is the TDOA we need, i.e., Δt. ij .

[0085] Specific steps:

[0086] Signal reception: Two microphones i and j receive the sound signal and convert it to a frequency domain representation X. i (f) and X j (f).

[0087] Cross-correlation calculation: By calculating the cross-correlation function R ij (τ), find the τ corresponding to the maximum value.

[0088] Time difference determination: The τ corresponding to the maximum value is Δt. ij This is consistent with the formula △t in ij They are the same physical quantity.

[0089] By using multiple such time differences, the spatial location of the sound source can be deduced.

[0090] In some embodiments, the step of noise preprocessing the ambient sound signal includes employing beamforming to enhance the speech signal from the target direction, suppress noise from other directions, and improve the signal-to-noise ratio. Spatial filtering is achieved using the spatial filtering capability of a multi-microphone array to separate noise sources from different directions, reducing mutual interference between different noise sources during feature extraction, thereby improving the accuracy of feature extraction.

[0091]

[0092] Where y(t) is the ambient sound signal output after beamforming, and x i (t) represents the ambient sound signal collected by the i-th microphone, M represents the number of input signals, and w i It is the beamforming weight.

[0093] In step S3 above, the step of adjusting the wake-word recognition threshold in real time based on the noise feature vector includes the following steps:

[0094] Introducing a sensitivity adjustment factor α, then

[0095] α=f(F(t))

[0096] Where F(t) is the noise feature vector, and f(F(t)) is the mapping function from the noise feature vector to the sensitivity factor;

[0097] If linear regression is used for training, then the wake word recognition threshold T

[0098] T=T0α

[0099] Where T0 is the original wake word recognition threshold.

[0100] In step S4 above, the step of denoising the ambient sound signal can increase the energy of the signal by noise reduction and / or gain enhancement, which can improve the quality of the input speech, enabling the speech recognition algorithm to more accurately recognize and understand the speech content, thereby improving recognition accuracy and enhancing the recognition effect of wake words.

[0101] In step S5 above, the speech recognition model is trained using a large amount of labeled data to recognize wake words and other speech content. When the energy of the noise-reduced ambient sound signal exceeds the wake word recognition threshold, the speech recognition model is used to identify the wake word in the noise-reduced ambient sound signal to respond to the user. This means determining whether the energy of the noise-reduced ambient sound signal exceeds the wake word recognition threshold. If so, the trained speech recognition model is used to identify the wake word in the noise-reduced ambient sound signal to determine whether it contains a wake word. If the wake word is detected, corresponding response operations are performed to respond to the user, such as playing sound or triggering the speech recognition process.

[0102] See Figure 3 As shown, the present invention also provides a device for robot wake word sensitivity adaptation to scene noise, which, when executed, implements the above-mentioned method steps for robot wake word sensitivity adaptation to scene noise, including...

[0103] The environmental sound acquisition module is used to collect environmental sound signals in the robot's environment in real time.

[0104] The noise feature extraction module receives the ambient sound signal collected by the ambient sound acquisition module and extracts effective features from the ambient sound signal to obtain a noise feature vector.

[0105] An adaptive sensitivity adjustment module adjusts the wake-word recognition threshold in real time based on noise feature vectors.

[0106] The noise reduction module receives the ambient sound signal collected by the ambient sound acquisition module and performs noise reduction processing on the ambient sound signal;

[0107] The wake-up word recognition module receives the ambient sound signal after noise reduction by the noise reduction module. When the energy of the ambient sound signal exceeds the wake-up word recognition threshold, it performs wake-up word recognition on the noise-reduced ambient sound signal and responds to the user.

[0108] It should be noted that although the above embodiments have been described herein, this does not limit the scope of patent protection of the present invention. Therefore, any changes and modifications made to the embodiments described herein based on the innovative concept of the present invention, or equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included within the scope of patent protection of the present invention.

Claims

1. A method for robot wake word sensitivity to adapt to scene noise, characterized in that, Includes the following steps: Real-time acquisition of ambient sound signals from the robot's environment; Effective features are extracted from the ambient sound signal to obtain a noise feature vector; The wake word recognition threshold is adjusted in real time based on the noise feature vector. The ambient sound signal is subjected to noise reduction processing; When the energy of the noise-reduced ambient sound signal exceeds the wake word recognition threshold, the speech recognition model is used to recognize the wake word in the noise-reduced ambient sound signal and respond to the user. The step of extracting effective features from the ambient sound signal to obtain a noise feature vector includes the following steps: Feature extraction of ambient sound signals is performed using spectral entropy: ； in, For the first Normalized energy of each frequency band; MFCC is used to extract features from environmental sound signals: ； in, For spectral values, The number of Mel filters; Feature extraction of environmental sound signals using short-time energy: ； in, In discrete time The signal value at that location, The value range is from 0 to -1, Indicates signal In discrete time Instantaneous power; The extracted features are then fused to form a noise feature vector. : [ , , ]； in, For short-term energy, For spectral entropy, For MFCC; The step of adjusting the wake-word recognition threshold in real time based on the noise feature vector includes the following steps: Introducing a sensitivity adjustment factor ,but ； in For noise feature vectors, This is the mapping function from the noise feature vector to the sensitivity factor; Using linear regression for training, the wake-word recognition threshold is... : = ； in The threshold for identifying the original wake word.

2. The method for robot wake word sensitivity adaptive scene noise according to claim 1, characterized in that, The step of extracting effective features from the ambient sound signal to obtain a noise feature vector includes the following steps: Extract the frequency domain and time domain features of the environmental sound signal to obtain the corresponding features; The extracted features are fused to form a noise feature vector.

3. The method for robot wake word sensitivity adaptive scene noise according to claim 2, characterized in that, Extracting frequency domain features of ambient sound signals includes using spectral entropy and / or MFCC to extract features from ambient sound signals.

4. The method for robot wake word sensitivity adaptive scene noise according to claim 2, characterized in that, Extracting the temporal features of environmental sound signals includes extracting features from short-time energy data.

5. The method for robot wake word sensitivity adaptive scene noise according to claim 1, characterized in that, The step of extracting effective features from the ambient sound signal to obtain a noise feature vector also includes a step of noise preprocessing of the ambient sound signal.

6. The method for robot wake word sensitivity adaptive scene noise according to claim 5, characterized in that, The steps of noise preprocessing for environmental sound signals include locating noise sources and separating noise sources from different directions.

7. The method for robot wake word sensitivity adaptive scene noise according to claim 6, characterized in that, A multi-microphone array is used to locate noise sources based on the time difference of signal arrival. Includes the following steps: set up For the first Ambient sound signals collected by one microphone Let be the speed of sound propagation; then the time difference between the noise source and each microphone. Represented as: ； in This indicates that sound travels from the sound source to the microphone. and microphone The time difference between them and These are the sound source and the microphone. and microphone The distance; Estimating the direction of the noise source using the GCC-PHAT algorithm: df; in, It's a microphone. and microphone The normalized cross-correlation function of the received audio signal. and The first The and the first Fourier transform of each microphone signal express The complex conjugate, It is the phase factor, τ= , which represents the time delay.

8. The method for robot wake word sensitivity adaptive scene noise according to claim 6, characterized in that, Beamforming is used to separate noise sources from different directions: ； in, It is the ambient sound signal output after beamforming. Indicates the number of input signals. It is the first Ambient sound signals collected by one microphone; It is the beamforming weight.

9. An apparatus for adapting robot wake word sensitivity to scene noise, wherein the method for adapting robot wake word sensitivity to scene noise as described in any one of claims 1 to 8 is implemented, characterized in that, include The environmental sound acquisition module is used to collect environmental sound signals in the robot's environment in real time. The noise feature extraction module receives the ambient sound signal collected by the ambient sound acquisition module and extracts effective features from the ambient sound signal to obtain a noise feature vector. An adaptive sensitivity adjustment module adjusts the wake-word recognition threshold in real time based on noise feature vectors. The noise reduction module receives the ambient sound signal collected by the ambient sound acquisition module and performs noise reduction processing on the ambient sound signal; The wake-up word recognition module receives the ambient sound signal after noise reduction by the noise reduction module. When the energy of the ambient sound signal exceeds the wake-up word recognition threshold, it performs wake-up word recognition on the noise-reduced ambient sound signal and responds to the user.

Images