Standby awakening method for voice module of smart home equipment

Through the VAD unit monitoring sound energy and combining the coordinated work of DSP and NPU, low-power standby and accurate wake-up of smart home devices is achieved, solving the problems of high power consumption and false wake-up rates in the existing technology, and the dual wake-up word mechanism is used to reduce The false wake-up rate improves the wake-up accuracy and efficiency of the device.

CN119007718APending Publication Date: 2024-11-22SHANGHAI SHENSILICON SEMICON CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410657897.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-05-24
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The standby wake-up method of voice modules of existing smart home devices has problems with high power consumption and false wake-up rate, especially in noise scenarios, which leads to frequent wake-up of the device, high power consumption and high false wake-up rate.

Method used

The VAD unit monitors the surrounding sound energy in real time. If the threshold is exceeded, the DSP unit will enter the low-frequency working mode, extract audio characteristics and calculate the model posterior information. It will enter the full-speed working mode only after confirming the wake-up word, and combine the two wake-up words. Reduce the false wake-up rate.

Benefits of technology

It realizes low power consumption standby in quiet scenarios, reduces the power consumption of the voice module, and effectively reduces the false wake-up rate through the dual wake-up word mechanism, improving the accuracy and efficiency of device wake-up.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007718A_ABST
    Figure CN119007718A_ABST
Patent Text Reader

Abstract

The invention discloses a standby wake-up method for a voice module of smart home equipment. The method comprises the following steps: monitoring the energy of surrounding sound in real time, and if the sound energy exceeds a threshold value, enabling a DSP (Digital Signal Processor) unit to perform a low-frequency working mode; the audio features of the surrounding sound are extracted and sent to the NPU calculation unit for calculation, and then model posterior information is obtained. And the DSP unit executes the first wake-up word network search, and after the first wake-up word is searched, the DSP unit enters a full-speed working mode. And the DSP unit executes second wake-up word network search, and if the second wake-up word is searched in a preset time, the corresponding smart home device is started. The VAD unit is used for monitoring surrounding sound energy in real time, and the voice module can be continuously in a low-power-consumption standby state. And by adopting the hardware feature extraction module and the npu calculation unit, the voice module can detect whether the voice module is awakened or not according to the posterior information of the model only by working at a very low dominant frequency, so that the power consumption is further reduced. By using the combination of the two wake-up words, the false wake-up rate can be effectively reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of smart home, and in particular relates to a standby wake-up method for a voice module of a smart home device. Background Art

[0002] With the rapid development of smart homes, voice plays an increasingly important role in smart homes. Now many smart home devices support voice standby wake-up. There are two main solutions for standby wake-up for voice modules on the market. One is that the voice module always runs at full speed and wakes up the smart home device after detecting the wake-up word. The other is that the voice module is in standby state first. When the sound energy exceeds the hardware VAD detection threshold, the voice module runs at full speed and wakes up the smart home device after detecting the wake-up word.

[0003] When the voice module is running at full speed to detect the wake-up word, the standby power consumption of the smart home will increase. If the hardware VAD is used to detect the sound energy first, and then the voice module is run at full speed after exceeding the threshold, the power consumption is reduced compared to the first method. However, in a noisy scene, the voice module will also be running at full speed for a long time, and the power consumption is still high.

[0004] In addition, since voice wake-up may cause false wake-up, when the smart home device is in standby mode, the above two methods have a high probability of falsely waking up the smart home device. Usually, the tolerance for false wake-up is relatively low when the smart home is in standby mode. Summary of the invention

[0005] The technical purpose of the present invention is to provide a method for waking up a voice module of a smart home device in standby mode, so as to solve the problems of easy false wake-up and high power consumption in the prior art.

[0006] To solve the above problems, the technical solution of the present invention is:

[0007] A method for waking up a voice module of a smart home device from standby mode comprises the following steps:

[0008] Monitor the energy of surrounding sounds in real time to determine whether the sound energy exceeds the threshold. If so, the DSP unit will be set to low-frequency working mode.

[0009] Extract the audio features of the surrounding sounds and send them to the NPU computing unit for calculation, thereby obtaining the model posterior information, which is then sent to the DSP unit;

[0010] The DSP unit performs a network search for the first wake-up word. If the first wake-up word is found, the DSP unit is caused to enter a full-speed working mode. Otherwise, the DSP unit is caused to return to a standby mode.

[0011] After entering the full-speed working mode, the DSP unit performs a network search for the second wake-up word. If the second wake-up word is searched within the preset time, the corresponding smart home device is started. Otherwise, the DSP unit returns to the standby mode.

[0012] The VAD unit monitors the energy of the surrounding sounds in real time and makes judgments. At this time, the DSP unit is in standby mode.

[0013] Among them, extracting the audio features of the surrounding sounds specifically includes: after receiving the audio information of the surrounding sounds, the audio information is framed, windowed and processed by fast Fourier transform in sequence to obtain the frequency spectrum distribution of the audio information, and then extracted to obtain the Mel-frequency cepstrum coefficients.

[0014] Among them, it is sent to the NPU computing unit for calculation, and then the model posterior information is obtained as follows: the extracted Mel-frequency cepstral coefficients are input into the NPU computing unit, the NPU computing unit stores the language model, the input Mel-frequency cepstral coefficients are calculated through the language model, and then the model posterior information is output, and the model posterior information includes the probability of each syllable.

[0015] Among them, the DSP unit executes the first wake-up word network search specifically as follows: receiving the model posterior information output by the NPU computing unit, the NPU unit storing the first wake-up word acoustic model, performing a prefix tree search on the model posterior information through the first wake-up word acoustic model, and calculating the probability of the first wake-up word.

[0016] Among them, the DSP unit executes the second wake-up word network search specifically as follows: echo elimination and beamforming are performed on the model posterior information in turn, the second wake-up word acoustic model is stored in the NPU unit, and the model posterior information after signal processing is calculated and searched through the second wake-up word acoustic model to calculate the probability of the second wake-up word.

[0017] Among them, the second wake-up word acoustic model is larger than the first wake-up word acoustic model, and the strategy of the NPU unit in searching for the second wake-up word is more complex than the strategy in searching for the first wake-up word.

[0018] An electronic device includes a processor and a memory, the memory stores computer instructions that can be executed on the processor, and the processor is used to call the computer instructions in the memory to execute any of the above-mentioned smart home device voice module standby wake-up methods.

[0019] A storage medium is used to store computer instructions, wherein when the computer instructions are executed, the device where the storage medium is located is controlled to execute any of the above-mentioned methods for waking up the voice module of a smart home device from standby mode.

[0020] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art:

[0021] The present invention uses the VAD unit to monitor the surrounding sound energy in real time. In a quiet scene, the voice module can be continuously in a low-power standby state with low power consumption. Furthermore, after the VAD unit monitoring is passed, a model posterior information is calculated using the hardware feature extraction module and the npu computing unit. At this time, the voice module only needs to work at a very low main frequency, and it can detect whether to wake up according to the model posterior information, further reducing power consumption.

[0022] After the first wake-up word wakes up, the voice module enters full speed state and continues to detect the second wake-up word. After the second wake-up word wakes up, the smart home device wakes up. By using two wake-up words in combination, the false wake-up rate can be effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0024] Figure 1 The present invention discloses a method for waking up a voice module of a smart home device from standby mode. DETAILED DESCRIPTION

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0026] In order to simplify the drawings, only the parts related to the present invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0027] The following is a further detailed description of a smart home device voice module standby wake-up method proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims.

[0028] Example

[0029] Conventional voice-activated smart devices use a full-speed wake-up mode, which cannot achieve a low-power solution. When in use, there are problems of high power consumption and many false wake-ups. In some existing technologies, hardware VAD is set to achieve secondary wake-up, thereby reducing the power consumption of the entire device when in standby mode. However, human voices and noise in the air can trigger VAD, resulting in frequent wake-ups of the device when the noise in the surrounding environment is strong, and the power consumption is still very high, which does not achieve true low power consumption.

[0030] Based on this, see Figure 1 This embodiment provides a method for waking up a voice module of a smart home device from standby mode, comprising the following steps:

[0031] First, the VAD unit (Voice Activity Detection) is used to monitor the energy of the surrounding sound in real time and determine whether the sound energy exceeds the threshold. If the energy exceeds the threshold, the DSP unit (Digital Signal Processing) enters the low-frequency working mode and proceeds to the next step. If the energy does not exceed the threshold, the VAD unit always maintains the monitoring state, and the DSP unit is in the standby dormant state.

[0032] After the DSP enters the low-frequency working mode, the hardware feature extraction module extracts the audio features of the surrounding sounds, and then sends them to the NPU computing unit (Neura l Processing Unit) for calculation, thereby obtaining the model posterior information. Specifically, after the hardware feature extraction module receives the audio information of the surrounding sounds, it performs frame division, windowing, and fast Fourier transform processing on the audio information in turn to obtain the frequency spectrum distribution of the audio information, and then extracts it to obtain the Mel Frequency Cepstral Coefficient (MFCC).

[0033] Among them, the purpose of framing the audio signal is to divide several speech sampling points into one frame, and within this frame, the characteristics of the speech signal can be considered to be stable. Preferably, the frame length of the signal framing is 16ms. After the framing is completed, the result of the speech framing will not be directly used for fast Fourier transform, which will cause spectrum leakage. Therefore, we introduce the window function here, specifically adding the Hanning window function. The Hanning window can be regarded as a special case of the raised cosine window. The Hanning window can be regarded as the sum of the spectra of three rectangular time windows, or the sum of three sinc(t) type functions, and the two terms in the brackets are moved to the left and right by π / T relative to the first spectrum window, so that the side lobes cancel each other and eliminate high-frequency interference and leakage energy. After the windowing is completed, a fast Fourier transform (fft) is performed to convert the signal from the time domain to the frequency domain, that is, to obtain the spectrum distribution of the audio information. Finally, the result of the fft is mel-filtered to extract the mel-frequency cepstrum coefficients, that is, the characteristics of the audio information.

[0034] Next, the extracted Mel frequency cepstral coefficients need to be input into the NPU computing unit, which stores a language model (a neural network, the specific network structure can be adjusted, and the commonly used multi-layer feedforward neural network dfsmn structure) and is loaded by the software when the hardware is initialized. The language model calculates the input Mel frequency cepstral coefficients and then outputs the model posterior information, which includes the probability of each syllable. After the calculation is completed, the NPU computing unit sends the model posterior information to the DSP unit.

[0035] The model posterior information is sent to the DSP unit. At this time, the DSP unit is still in low-frequency working mode. The acoustic model of the first wake-up word is stored in the NPU unit. The NPU calculates the model posterior, and the DSP performs a prefix tree search on the posterior to calculate the probability of the first wake-up word. Among them, the prefix tree is also called a dictionary tree, which is a data structure. Its characteristic is that the search time is only related to the height of the tree, and the time complexity is o(h). It can efficiently match characters with the same prefix. In the decoding process of voice wake-up, the prefix tree is used to quickly search for matching vocabulary sequences. Since the algorithm complexity is only o(n), the DSP unit can be completed at a very low main frequency. If the first wake-up word is successfully searched, the DSP unit is put into full-speed working mode. Otherwise, if the search fails, the DSP unit returns to standby sleep mode.

[0036] Next, when the DSP unit enters the full-speed working mode, it performs a network search for the second wake-up word, and needs to perform echo cancellation and beamforming processing on the model posterior information in sequence. Among them, beamforming enhances the audio according to the direction, which can eliminate the interference of noise in the non-target direction, improve the quality of the wake-up audio, and thus improve the wake-up rate. The NPU unit stores the acoustic model of the second wake-up word, and the model posterior information after signal processing is calculated and searched through the acoustic model of the second wake-up word to calculate the probability of the second wake-up word. If the second wake-up word is searched within a preset short time (2s is used as an example in this embodiment), the corresponding smart home device is started. On the contrary, if the second wake-up word is not searched within the preset time, it is determined that the wake-up voice command has not been received, and then the DSP unit is returned to standby mode to maintain low power consumption standby. Among them, echo cancellation is to eliminate the sound emitted by the device of this embodiment itself without affecting the sound transmitted from the outside.

[0037] The difference between the first wake-up word acoustic model and the second wake-up word acoustic model is that the second wake-up word acoustic model will be larger, and the strategy for searching the second wake-up word will be more complicated than that for searching the first wake-up word.

[0038] Furthermore, the present embodiment also provides an electronic device, which includes a processor and a memory, wherein the memory stores computer instructions that can be executed on the processor, and the processor is used to call the computer instructions in the memory to execute any of the above-mentioned smart home device voice module standby wake-up methods.

[0039] In addition, this embodiment also provides a storage medium, which is used to store computer instructions, wherein when the computer instructions are executed, the device where the storage medium is located is controlled to execute any of the above-mentioned smart home device voice module standby wake-up methods.

[0040] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the protection scope of the present invention.

Claims

1. A method for waking up a voice module of a smart home device from standby mode, characterized in that: The steps include: Monitor the energy of surrounding sounds in real time to determine whether the sound energy exceeds the threshold. If so, the DSP unit will be set to low-frequency working mode. Extracting audio features of surrounding sounds, sending them to the NPU computing unit for calculation, and then obtaining model posterior information, which is then sent to the DSP unit; The DSP unit performs a first wake-up word network search. If the first wake-up word is found, the DSP unit is caused to enter a full-speed working mode. Otherwise, the DSP unit is caused to return to a standby mode. After entering the full-speed working mode, the DSP unit performs a second wake-up word network search. If the second wake-up word is searched within a preset time, the corresponding smart home device is started. Otherwise, the DSP unit returns to the standby mode.

2. The method for waking up the voice module of a smart home device from standby mode according to claim 1, characterized in that: The VAD unit monitors the energy of the surrounding sounds in real time and makes a judgment. At this time, the DSP unit is in standby mode.

3. The method for waking up the voice module of a smart home device from standby mode according to claim 1, characterized in that: The method of extracting the audio features of the surrounding sounds specifically includes: after receiving the audio information of the surrounding sounds, sequentially performing frame division, windowing and fast Fourier transform processing on the audio information to obtain the frequency spectrum distribution of the audio information, and then extracting the Mel-frequency cepstrum coefficients.

4. The method for waking up the voice module of a smart home device from standby mode according to claim 3, characterized in that: The method of sending the information to the NPU computing unit for calculation to obtain the model posterior information is as follows: the extracted Mel-frequency cepstral coefficients are input into the NPU computing unit, the NPU computing unit stores a language model, the input Mel-frequency cepstral coefficients are calculated by the language model, and then the model posterior information is output, wherein the model posterior information includes the probability of each syllable.

5. The method for waking up the voice module of a smart home device from standby mode according to claim 1, characterized in that: The DSP unit performs the first wake-up word network search specifically as follows: receiving the model posterior information output by the NPU computing unit, wherein the NPU unit stores a first wake-up word acoustic model, performing a prefix tree search on the model posterior information through the first wake-up word acoustic model, and calculating the probability of the first wake-up word.

6. The method for waking up the voice module of a smart home device from standby mode according to claim 5, characterized in that: The DSP unit executes the second wake-up word network search specifically as follows: echo elimination and beamforming are performed on the model posterior information in sequence, the NPU unit stores a second wake-up word acoustic model, and the model posterior information after signal processing is calculated and searched through the second wake-up word acoustic model to calculate the probability of the second wake-up word.

7. The method for waking up the voice module of a smart home device from standby mode according to claim 6, characterized in that: The second wake-up word acoustic model is larger than the first wake-up word acoustic model, and the strategy of the NPU unit in searching for the second wake-up word is more complex than the strategy in searching for the first wake-up word.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory stores computer instructions that can be executed on the processor, and the processor is used to call the computer instructions in the memory to execute the smart home device voice module standby wake-up method as described in any one of claims 1-8.

9. A storage medium, characterized in that: The storage medium is used to store computer instructions, wherein when the computer instructions are executed, the device where the storage medium is located is controlled to execute the smart home device voice module standby wake-up method according to any one of claims 1 to 7.