Voice wake-up methods, systems, storage media and electronic devices

Abstract

This application provides a voice wake-up method, apparatus, storage medium, and electronic device. The method includes: acquiring a voice signal to be recognized; extracting a target feature vector from the voice signal to be recognized; inputting the target feature vector into a speech recognition neural network to output a speech classification result corresponding to the voice signal to be recognized, wherein the speech recognition neural network consists of a first convolutional layer, a first batch of normalization layers, a second convolutional layer, a third convolutional layer, a second batch of normalization layers, a concatenation layer, and a fully connected layer connected sequentially, wherein the second and third convolutional layers are binary quantized convolutional layers; and determining whether to wake up a target device based on the speech classification result. The technical solution provided by this application can reduce the computational load and memory usage in the voice wake-up process, thereby improving the applicability of voice wake-up based on neural networks.

Description

Technical Field

[0001] This application relates to the field of voice technology, and more specifically, to a voice wake-up method, system, storage medium, and electronic device. Background Technology

[0002] Choosing a suitable framework has become a crucial step in KWS (Knowledge-Based Wake-up) systems. With the increasing success of deep learning in classification tasks, neural network-based KWS systems demonstrate superior performance compared to GMM-HMM-based systems in terms of wake-up rate, generalization ability, and running speed. For different recognition tasks and application environments, the algorithm design directly impacts not only the overall system's recognition results but also the resource costs required for hardware implementation. For example, computational load indirectly determines hardware power consumption. Therefore, to facilitate hardware design and implementation, a lightweight KWS system architecture that is easy to implement on mobile devices with low hardware resources and low power consumption should be chosen. Based on this, designing a lightweight neural network to achieve voice wake-up functionality is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] The embodiments of this application provide a voice wake-up method, system, storage medium, and electronic device. Based on the technical solution provided by this application, the computational and operational load of the speech recognition neural network can be reduced, thereby improving the applicability of voice wake-up based on the neural network, and thus enabling the voice wake-up method of this application to be applied to devices with low memory and low computing power.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to a first aspect of the embodiments of this application, a voice wake-up method is provided. The method includes: acquiring a voice signal to be recognized; extracting a target feature vector from the voice signal to be recognized; inputting the target feature vector into a speech recognition neural network to output a speech classification result corresponding to the voice signal to be recognized through the speech recognition neural network, wherein the speech recognition neural network consists of a first convolutional layer, a first batch normalization layer, a second convolutional layer, a third convolutional layer, a second batch normalization layer, a concatenation layer, and a fully connected layer connected in sequence, wherein the second convolutional layer and the third convolutional layer are binary quantized convolutional layers; and determining whether to wake up a target device based on the speech classification result.

[0006] In some embodiments of this application, based on the aforementioned scheme, the step of extracting the target feature vector of the speech signal to be identified includes: inputting the speech signal to be identified into a preset feature extraction module; and performing spectral analysis on the speech signal to be identified through the preset feature extraction module to obtain Mel-frequency cepstral coefficients as the target feature vector.

[0007] In some embodiments of this application, based on the foregoing scheme, the method further includes: performing binary quantization on the feature vector input to the second convolutional layer to obtain a first feature vector; and performing convolution operation on the first feature vector in the second convolutional layer using XOR operation and counting operation to obtain a second feature vector.

[0008] In some embodiments of this application, based on the foregoing scheme, the method further includes: performing binary quantization on the second feature vector to obtain a third feature vector; and performing convolution operation on the third feature vector in the third convolutional layer using XOR operation and counting operation.

[0009] In some embodiments of this application, based on the aforementioned scheme, the splicing layer is used to add and splice the second feature vector and the feature vector output from the second batch normalization layer to increase the recognition accuracy of the speech recognition neural network.

[0010] In some embodiments of this application, based on the foregoing scheme, determining whether to wake up the target device according to the speech classification result includes: determining the matching score between the speech signal to be recognized and each set semantic tag from the speech classification result; determining the maximum value among the matching scores; determining the set semantic tag corresponding to the maximum value as the target semantic tag; and if the target semantic tag is a wake word, waking up the target device based on the target semantic tag.

[0011] In some embodiments of this application, based on the foregoing scheme, waking up the target device based on the target semantic tag includes: obtaining the first time of the last time the target device was woken up based on the target semantic tag; if the time length between the first time and the current time is greater than a preset threshold, then waking up the target device based on the target semantic tag.

[0012] According to a second aspect of the embodiments of this application, a voice wake-up system is provided. The system includes: an acquisition module for acquiring a voice signal to be recognized; a feature extraction module for extracting a target feature vector from the voice signal to be recognized; a voice recognition module for inputting the target feature vector into a voice recognition neural network to output a voice classification result corresponding to the voice signal to be recognized through the voice recognition neural network, wherein the voice recognition neural network consists of a first convolutional layer, a first batch normalization layer, a second convolutional layer, a third convolutional layer, a second batch normalization layer, a concatenation layer, and a fully connected layer connected in series, wherein the second convolutional layer and the third convolutional layer are binary quantized convolutional layers; and a wake-up module for determining whether to wake up a target device based on the voice classification result.

[0013] According to a third aspect of the embodiments of this application, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores at least one piece of program code, the at least one piece of program code being loaded and executed by a processor to perform the operations performed by the method described in any of the first aspects above.

[0014] According to a fourth aspect of the present application, an electronic device is provided, including one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, the at least one piece of program code being loaded and executed by the one or more processors to perform the operation as described in any of the first aspects above.

[0015] The technical solution of this application first acquires the speech signal to be recognized; then extracts the target feature vector of the speech signal to be recognized; then inputs the target feature vector into a speech recognition neural network, so that the speech recognition neural network outputs a speech classification result corresponding to the speech signal to be recognized; finally, it determines whether to wake up the target device based on the speech classification result. In this application, the structure of the speech recognition neural network used to recognize the acquired speech signal to be recognized is composed of a first convolutional layer, a first batch of normalization layers, a second convolutional layer, a third convolutional layer, a second batch of normalization layers, a concatenation layer, and a fully connected layer connected in sequence. This speech recognition neural network uses a deep separable convolutional neural network (DSCNN), which reduces the memory and operation load by about 6 / 7 compared to the traditional convolutional neural network (CNN), significantly reducing the memory and operation load of the device used to implement the voice wake-up function.

[0016] In addition, in the technical solution of this application, the second convolutional layer and the third convolutional layer are subjected to binary quantization processing, which further reduces the amount of device memory and operation required to implement the voice wake-up function, thereby greatly improving the applicability of the voice wake-up function based on neural networks, so that the voice wake-up method provided by this application can be applied to devices with low memory and low computing power.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0019] Figure 1 A flowchart illustrating a voice wake-up method according to an embodiment of this application is shown;

[0020] Figure 2 A schematic diagram of the structure of a speech recognition neural network according to an embodiment of this application is shown;

[0021] Figure 3 A schematic diagram illustrating a scenario of XOR operation and counting operation according to an embodiment of this application is shown;

[0022] Figure 4 A detailed flowchart illustrating a process for determining whether to wake up a target device based on the voice classification result, according to an embodiment of this application, is shown.

[0023] Figure 5 A comparative schematic diagram of a convolutional neural network according to an embodiment of this application and a speech recognition neural network according to this application is shown;

[0024] Figure 6 A block diagram of a voice wake-up system according to an embodiment of this application is shown;

[0025] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0026] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0027] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0028] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0029] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0030] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such uses of these terms can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described.

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0033] The following detailed description of some embodiments of this application will be provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0034] See Figure 1 The diagram shows a flowchart of a voice wake-up method according to an embodiment of the present application, specifically including S110 to S140.

[0035] S110, acquire the speech signal to be recognized.

[0036] It should be noted that the technical solution of this application can be applied to devices with voice wake-up function, including but not limited to refrigerators, televisions, speakers, mobile phones and other low-memory devices.

[0037] In some implementations, a user can send a voice signal to the device, which can then receive the voice signal to be recognized.

[0038] In some implementations, the device can actively collect voice signals from the surrounding environment as the voice signal to be recognized.

[0039] In some implementations, if the content of the voice signal to be recognized obtained by the device is empty, it is regarded as a blank voice signal, that is, a silent voice signal.

[0040] In some embodiments, the voice signal to be identified may be voice information including a wake word, voice information containing only a wake word, voice information without a wake word, or silence information without any information. Specifically, this application does not limit the specifics.

[0041] It should be noted that the wake word refers to a keyword that can wake up the device and control the device to perform corresponding actions, such as start, exit, close, run, hibernate, pause, hello, Xiao Ai, etc.

[0042] See also Figure 1 S120, extract the target feature vector of the speech signal to be recognized.

[0043] It should be noted that, in this embodiment, the purpose of feature extraction on the speech signal to be recognized is to transform the speech signal from discrete points in the time domain to the frequency domain, perform fixed-point processing, and finally output a target feature vector that coincides with the information of the speech signal to be recognized. This embodiment does not limit the specific implementation method for extracting the target feature vector of the speech signal to be recognized, but at least the following three implementation methods are exemplary.

[0044] The first implementation method is to use linear prediction coefficients (LPC) to extract features from the speech signal to be recognized, and obtain the target feature vector.

[0045] The second implementation method is to input the speech signal to be recognized into a set feature extraction model, extract its features, and obtain the linear predictive cepstral coefficient (LPCC) as the target feature vector.

[0046] The third implementation method is as follows: The speech signal to be recognized is input into a preset feature extraction module; the preset feature extraction module performs spectral analysis on the speech signal to be recognized to obtain Mel-frequency cepstral coefficients, which are used as the target feature vector.

[0047] The calculation process for Mel-frequency cepstral coefficients (MFCCs) in the preset feature extraction module is as follows: pre-emphasis of the speech signal, windowing of the speech signal in frames, Fourier transform, passing the transformed frequency domain signal through a Mel filter bank, performing logarithmic operations to change multiplication to addition, and performing Discrete Cosine Transform (DCT) to boost the low-frequency components of the signal. Finally, the speech signal to be recognized is extracted as an MFCC feature vector, which serves as the target feature vector.

[0048] See also Figure 1 S130, the target feature vector is input into the speech recognition neural network to output a speech classification result corresponding to the speech signal to be recognized. The speech recognition neural network consists of a first convolutional layer, a first batch of normalization layers, a second convolutional layer, a third convolutional layer, a second batch of normalization layers, a concatenation layer, and a fully connected layer connected in sequence. The second convolutional layer and the third convolutional layer are binary quantized convolutional layers.

[0049] In some implementations, the speech recognition neural network can be obtained through the following steps, including S1 to S3.

[0050] S1, Obtain sample speech data.

[0051] S2, extract the sample feature vector of the sample speech data.

[0052] S3, the sample feature vector is input into a pre-constructed initial neural network to train the initial neural network using the sample feature vector, thereby obtaining the speech recognition neural network. It can be understood that the network structure of the initial neural network is consistent with the structure of the speech recognition neural network.

[0053] To enable those skilled in the art to better understand this embodiment, the background technology of the speech recognition neural network design structure used in this application will be described below.

[0054] Currently, the neural networks commonly used in devices with voice wake-up functionality are typically Convolutional Neural Networks (CNNs). Due to the sharing of weights, their storage requirements are significantly reduced compared to Deep Neural Networks (DNNs). However, the number of floating-point operations in CNNs becomes extremely large with the increase in the number of layers, and the complex multiplication and accumulation operations lead to a substantial increase in computational cost. Nevertheless, CNNs have better recognition accuracy than DNNs. Therefore, this application proposes to use a Deeply Separable Convolutional Neural Network (DSCNN) in the speech recognition neural network.

[0055] For the specific structure of the speech recognition neural network used in this application, please refer to [link / reference]. Figure 2 As shown.

[0056] In some implementations, a target feature vector of 1*40*98 (where 1 represents the number of channels, 40 represents the length, and 98 represents the width) is input to the first convolutional layer. The parameters of the first convolutional layer are set to 32*3*3 (where 32 represents the number of channels, 3 represents the length, and 3 represents the width), with a stride of 1. It should be noted that this first convolutional layer is a standard convolutional layer and does not require binary quantization.

[0057] In some implementations, the second convolutional layer is a depthwise convolution (DC). The feature vector obtained after processing by the first convolutional layer and the first batch of normalization layers has parameters of 32*4*51 (32 represents the number of input channels, 4 represents the length, and 51 represents the width). The parameters set for the second convolutional layer are 32*3*3 (32 represents the number of channels, 3 represents the length, and 3 represents the width), with a stride of 1.

[0058] In some implementations, the third convolutional layer is a pointwise convolution (PC), the number of channels of the feature vector input to the third convolutional layer is 32, the parameters of the third convolutional layer are set to 1*1 (representing length and width respectively), and the stride is 1. The number of channels in the third convolutional layer can be designed according to the actual situation, such as 32, 64, etc., and this application does not make specific limitations here.

[0059] In some implementations, the second convolutional layer and the third convolutional layer are specifically implemented as binary-quantized convolutional layers as follows:

[0060] Binary Neural Networks (BNNs) are used to quantize the weights and biases in the second and third convolutional layers into two values, +1 and -1, specifically using the following formula:

[0061]

[0062] Where X1 represents the weights or biases of the binary quantized output, and Xr represents the weights or biases in the second or third convolutional layer.

[0063] In some implementations, the feature vector input to the second convolutional layer is binarized to obtain a first feature vector; in the second convolutional layer, the first feature vector is convolved using XOR and counting operations to obtain a second feature vector.

[0064] Specifically, the binary quantization of the feature vector input to the second convolutional layer can be performed using the formula described above.

[0065] Specifically, in the second convolutional layer, the XNOR and Bitcount operations performed on the input first feature vector can be found in the following example. Figure 3 It will be executed as shown.

[0066] In some implementations, the second feature vector is binarized to obtain a third feature vector; in the third convolutional layer, the third feature vector is convolved using XOR and counting operations.

[0067] Specifically, the second feature vector can be binary quantized using the formula described above.

[0068] Specifically, the XOR operation and counting operation performed on the input third feature vector in the third convolutional layer can be found in [reference needed]. Figure 3It will be executed as shown.

[0069] It should be noted that because the recognition rate of the speech recognition model after binarization is lower than that of the full-precision model, this method adopts a shortcut quantization approach. Specifically, the concatenation layer is designed to add and concatenate the second feature vector and the feature vector output from the second batch normalization layer to increase the recognition accuracy of the speech recognition neural network.

[0070] A shortcut is used to connect the outputs of the second convolutional layer and the second batch normalization layer, that is, the real-valued outputs of the second convolutional layer and the second batch normalization layer are directly added together. This increases the numerical representation range of the network. By using shortcut quantization to transmit real-valued features, the amount of information in the network can be greatly preserved, thus improving the network's expressive power. Finally, the concatenated result is input into the fully connected layer.

[0071] In some implementations, the speech classification result output by the fully connected layer includes matching scores between the speech signal to be recognized and various defined semantic labels. For example, it can be represented as a 12-dimensional vector.

[0072] To enable those skilled in the art to better understand this embodiment, the following description will be provided in conjunction with Table 1.

[0073] It should be noted that the semantic tags represent pre-defined voice category tags. For example, they can be designed as shown in Table 1, with 10 keywords (each keyword is a wake-up word, meaning that it can wake up the target device and perform the corresponding semantic action), and other words besides the keywords are defined as non-keywords. Blank voice (i.e., voice signal with empty content) is designed as silence.

[0074] For example, keywords could be "start, run, exit, hello, hibernate, pause, Xiao Ai, etc."

[0075] It is understood that after the acquired speech to be recognized is recognized by the speech recognition neural network designed in this application, it can output 12 semantic labels and corresponding matching scores. It is understood that the higher the matching score, the lower the probability that the speech signal to be recognized contains or is equal to a certain semantic label.

[0076] In some implementations, the matching scores can be normalized so that the results are displayed as values ​​in the range of 0-1.

[0077]

[0078] Table 1

[0079] See also Figure 1 S140, determine whether to wake up the target device based on the voice classification result.

[0080] In some implementations, the specific implementation of determining whether to wake up the target device based on the voice classification result can be as follows: Figure 4 Perform the steps shown.

[0081] For details, see Figure 4 The diagram illustrates a detailed process for determining whether to wake up a target device based on the voice classification result according to an embodiment of the present application, including S141 to S144.

[0082] S141, determine the matching score between the speech signal to be recognized and each set semantic label from the speech classification results.

[0083] S142, determine the maximum value among the various matching scores.

[0084] S143, the set semantic label corresponding to the maximum value is determined as the target semantic label.

[0085] S144, if the target semantic tag is a wake word, then wake up the target device based on the target semantic tag.

[0086] To enable those skilled in the art to better understand this embodiment, examples will be provided below in conjunction with Table 1 above.

[0087] See Table 1, where keywords 1-10, non-keywords, and silence are all set semantic tags. Of course, semantic tags can also be designed according to actual conditions, and this application does not impose any restrictions here.

[0088] In Table 1, the semantic tag matching score for keyword 4 is the highest, meaning that the probability that the input speech signal to be recognized is equal to or contains keyword 4 is the highest. Therefore, the semantic tag for keyword 4 is taken as the target semantic tag. Assuming keyword 4 means "exit," the target device can be woken up and controlled to perform the exit action.

[0089] In some embodiments, the specific implementation of waking up the target device based on the target semantic tag can also be performed according to the following steps, including S1441 to S1442.

[0090] S1441, Obtain the first time of the last time the target device was woken up based on the target semantic tag.

[0091] S1442, if the time length between the first time and the current time is greater than a preset threshold, then wake up the target device based on the target semantic tag.

[0092] In some embodiments, the preset threshold can be set to 1 minute, 3 minutes, 5 minutes, etc., but this application does not limit the specific value.

[0093] It should be noted that the main purpose of this embodiment is to avoid executing the same wake-up command on the target device multiple times in a short period of time. For example, suppose the smart speaker just completed an exit action 1 second ago, and now needs to be woken up to perform an exit action again. This would undoubtedly cause the smart speaker to perform an invalid action. Therefore, this embodiment can improve the operating efficiency of the target device.

[0094] To enable those skilled in the art to more intuitively understand the beneficial effects of the technical solution of this application, namely, the greatest benefit being the reduction of memory and operational workload in the process of implementing voice wake-up function, the following will combine... Figure 5 The following tables 2 and 3 illustrate this.

[0095]

[0096] Table 2

[0097] As shown in Table 2, although DNNs do not achieve optimal accuracy and often consume a large amount of memory, they have fewer operations, i.e., fewer inferences, making them very suitable for systems with limited computing power. On the other hand, CNNs achieve higher accuracy than DNNs, but at the cost of a large number of operations and memory requirements. LSTM and CRNN achieve a balance between memory and operations while maintaining good accuracy. The speech recognition neural network used in this application, based on DSCNN, achieves accuracy on the Google dataset at a level comparable to other models, while significantly improving storage and computational efficiency. These advantages demonstrate the significant hardware implementation advantages of DSCNN and exhibit good scalability.

[0098]

[0099] Table 3

[0100] See Figure 5 The figure compares the differences in memory usage and number of operations between CNN and DSCNN when the input and output results are the same. The statistical results of the differences are shown in Table 3. As can be seen from Table 3, DSCNN can achieve 1 / 7 of the memory usage and number of operations of CNN neural network. Moreover, the degree of compression of parameters and floating-point operations will increase exponentially with the increase of convolution kernel dimension and the increase of convolution kernel channel depth.

[0101] In some embodiments of this application, the technical solution first acquires a speech signal to be recognized; then extracts the target feature vector of the speech signal to be recognized; then inputs the target feature vector into a speech recognition neural network to output a speech classification result corresponding to the speech signal to be recognized; finally, it determines whether to wake up the target device based on the speech classification result. In this application, the speech recognition neural network used to recognize the acquired speech signal to be recognized consists of a first convolutional layer, a first batch of normalization layers, a second convolutional layer, a third convolutional layer, a second batch of normalization layers, a concatenation layer, and a fully connected layer connected sequentially. This speech recognition neural network uses a Deep Separable Convolutional Neural Network (DSCNN), which reduces memory and operational workload by about 6 / 7 compared to traditional convolutional neural networks (CNN), significantly reducing the memory and operational workload required to implement the voice wake-up function.

[0102] In addition, in the technical solution of this application, the second convolutional layer and the third convolutional layer are subjected to binary quantization processing, which further reduces the amount of device memory and operation required to implement the voice wake-up function, thereby greatly improving the applicability of the voice wake-up function based on neural networks, so that the voice wake-up method provided by this application can be applied to devices with low memory and low computing power.

[0103] Specifically, binary quantization offers the following advantages:

[0104] (1) Using low-bit-width data instead of high-bit-width data reduces deployment difficulty and hardware overhead.

[0105] (2 If an extreme binary quantization algorithm is used, the storage space can be reduced by 32 times in theory, and single-bit operations can be replaced by XOR logic operations, thus saving a lot of computation. Smaller memory consumption and lower computation means lower power consumption.

[0106] (3) Quantization can improve the running speed of the network. If the network is deployed on a large-scale GPU server, it will not have a significant impact. However, if it is deployed on an embedded system, the low computational cost of the quantized network will show a great advantage in inference speed.

[0107] (4) It greatly preserves the amount of information in the network and improves the network's expressive ability.

[0108] The following describes an embodiment of the apparatus described in this application, which can be used to execute the voice wake-up method in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the voice wake-up method described above.

[0109] Figure 6 A block diagram of a voice wake-up system according to an embodiment of this application is shown.

[0110] Reference Figure 6 As shown, a voice wake-up system 600 according to an embodiment of this application includes: an acquisition module 601, a feature extraction module 602, a speech recognition module 603, and a wake-up module 604.

[0111] The system includes an acquisition module 601 for acquiring a speech signal to be recognized; a feature extraction module 602 for extracting a target feature vector from the speech signal to be recognized; a speech recognition module 603 for inputting the target feature vector into a speech recognition neural network to output a speech classification result corresponding to the speech signal to be recognized. The speech recognition neural network consists of a first convolutional layer, a first batch of normalization layers, a second convolutional layer, a third convolutional layer, a second batch of normalization layers, a concatenation layer, and a fully connected layer, all connected in series. The second and third convolutional layers are binary quantized convolutional layers. The concatenation layer adds and concatenates the second feature vector and the feature vector output from the second batch of normalization layers to increase the recognition accuracy of the speech recognition neural network. A wake-up module 604 is used to determine whether to wake up the target device based on the speech classification result.

[0112] In some embodiments of this application, based on the foregoing scheme, the feature extraction module 602 is further configured to: input the speech signal to be identified to a preset feature extraction module; and perform spectral analysis on the speech signal to be identified through the preset feature extraction module to obtain Mel-frequency cepstral coefficients as target feature vectors.

[0113] In some embodiments of this application, based on the foregoing scheme, the speech recognition module 603 is further configured to: perform binary quantization processing on the feature vector input to the second convolutional layer to obtain a first feature vector; and in the second convolutional layer, perform convolution operation on the first feature vector using XOR operation and counting operation to obtain a second feature vector.

[0114] In some embodiments of this application, based on the foregoing scheme, the speech recognition module 603 is further configured to: perform binary quantization processing on the second feature vector to obtain a third feature vector; and perform convolution operation on the third feature vector in the third convolutional layer using XOR operation and counting operation.

[0115] In some embodiments of this application, based on the foregoing scheme, the wake-up module 604 is further configured to: determine the matching score between the speech signal to be recognized and each set semantic tag from the speech classification result; determine the maximum value among the matching scores; determine the set semantic tag corresponding to the maximum value as the target semantic tag; and if the target semantic tag is a wake-up word, wake up the target device based on the target semantic tag.

[0116] In some embodiments of this application, based on the foregoing scheme, the wake-up module 604 is further configured to: obtain the first time of the last wake-up of the target device based on the target semantic tag; if the time length between the first time and the current time is greater than a preset threshold, then wake up the target device based on the target semantic tag.

[0117] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0118] It should be noted that, Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0119] like Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 702 or programs loaded from storage portion 708 into Random Access Memory (RAM) 703, such as performing the methods described in the above embodiments. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0120] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0121] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs various functions defined in the system of this application.

[0122] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0124] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0125] In another aspect, this application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the voice wake-up method described in the above embodiments.

[0126] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the voice wake-up method described in the above embodiments.

[0127] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0128] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0129] Other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A voice wake-up method, characterized in that, The method includes: Acquire the speech signal to be recognized; Extract the target feature vector of the speech signal to be recognized; The target feature vector is input into a speech recognition neural network to output a speech classification result corresponding to the speech signal to be recognized. The speech recognition neural network consists of a first convolutional layer, a first batch of normalization layers, a second convolutional layer, a third convolutional layer, a second batch of normalization layers, a concatenation layer, and a fully connected layer connected in sequence. The second convolutional layer and the third convolutional layer are binary quantized convolutional layers. Determine whether to wake up the target device based on the voice classification results; The method further includes: The feature vector input to the second convolutional layer is binarized to obtain the first feature vector; In the second convolutional layer, the first feature vector is convolved using XOR and counting operations to obtain the second feature vector. The splicing layer is used to add and splice the second feature vector and the feature vector output from the second batch normalization layer to increase the recognition accuracy of the speech recognition neural network.

2. The method according to claim 1, characterized in that, The extraction of the target feature vector of the speech signal to be recognized includes: The speech signal to be recognized is input into a preset feature extraction module; The preset feature extraction module performs spectral analysis on the speech signal to be identified to obtain Mel-frequency cepstral coefficients, which are used as the target feature vector.

3. The method according to claim 1, characterized in that, The method further includes: The second feature vector is subjected to binary quantization to obtain the third feature vector; In the third convolutional layer, the third feature vector is convolved using XOR and counting operations.

4. The method according to claim 1, characterized in that, The step of determining whether to wake up the target device based on the voice classification result includes: The matching score between the speech signal to be recognized and each set semantic label is determined from the speech classification results; Determine the maximum value among the various matching scores; The set semantic label corresponding to the maximum value is determined as the target semantic label; If the target semantic tag is a wake word, then the target device is woken up based on the target semantic tag.

5. The method according to claim 4, characterized in that, The step of waking up the target device based on the target semantic tag includes: Obtain the first moment when the target device was last woken up based on the target semantic tag; If the time elapsed between the first time and the current time is greater than a preset threshold, the target device is woken up based on the target semantic tag.

6. A voice wake-up system, characterized in that, The system includes: The acquisition module is used to acquire the speech signal to be recognized; The feature extraction module is used to extract the target feature vector of the speech signal to be recognized; The speech recognition module is used to input the target feature vector into the speech recognition neural network, so as to output the speech classification result corresponding to the speech signal to be recognized through the speech recognition neural network. The speech recognition neural network consists of a first convolutional layer, a first batch of normalization layers, a second convolutional layer, a third convolutional layer, a second batch of normalization layers, a concatenation layer, and a fully connected layer connected in series. The second convolutional layer and the third convolutional layer are binary quantized convolutional layers. A wake-up module is used to determine whether to wake up the target device based on the voice classification results. The speech recognition module is also used for: The feature vector input to the second convolutional layer is binarized to obtain the first feature vector; In the second convolutional layer, the first feature vector is convolved using XOR and counting operations to obtain the second feature vector. The splicing layer is used to add and splice the second feature vector and the feature vector output from the second batch normalization layer to increase the recognition accuracy of the speech recognition neural network.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to perform the operations performed by the method as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, The method includes one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to implement the method as claimed in any one of claims 1 to 5.

Images