A voice wake-up method and device based on lightweight deep residual convolutional network

Abstract

The present application relates to the fields of deep learning and voice wake-up technology. More specifically, the present application relates to a voice wake-up method and device based on a lightweight deep residual convolutional network. The method comprises obtaining voice sample data; inputting the voice sample data as training samples into a voice wake-up model based on a lightweight deep residual convolutional network for training; and using the trained voice wake-up model based on a lightweight deep residual convolutional network to wake up the target voice; wherein the voice wake-up model based on a lightweight deep residual convolutional network is provided with a residual causal convolution module. The method and device proposed in the present application are both efficient and accurate, thereby significantly improving the efficiency of voice wake-up.

Description

Technical Field

[0001] The present application relates to the field of deep learning and voice wake-up technology. More specifically, the present application relates to a voice wake-up method and device based on a lightweight deep residual convolutional network. Background Art

[0002] With the rise of the concept of human-computer interaction, more and more companies are focusing on the research and development of human-computer interaction products, with voice interaction products being a current R&D hotspot. Typically, before human-computer interaction, the device is dormant. Only when a keyword is detected does the device wake up and await user instructions, thus reducing device energy consumption. Therefore, the efficiency and accuracy of the device waking up after hearing the keyword will affect the user experience.

[0003] Currently, time series models such as RNN, GRU, and LSTM have been proven to perform well in modeling speech features. However, due to their inability to perform parallel operations, their training and prediction efficiency is low. High-precision models require more hidden layer neurons for learning, so they cannot meet the requirements of high precision and real-time performance at the same time. Summary of the Invention

[0004] Based on the above technical problems, the present invention aims to wake up the target speech based on a speech wake-up model of a lightweight deep residual convolutional network.

[0005] A first aspect of the present invention provides a voice wake-up method based on a lightweight deep residual convolutional network, the method comprising:

[0006] Obtain voice sample data;

[0007] Inputting the speech sample data as training samples into a voice wake-up model based on a lightweight deep residual convolutional network for training;

[0008] Use the trained voice wake-up model based on lightweight deep residual convolutional network to wake up the target voice;

[0009] Among them, the voice wake-up model based on the lightweight deep residual convolutional network is provided with a residual causal convolution module.

[0010] In some embodiments of the present invention, the voice wake-up model based on the lightweight deep residual convolutional network further includes a first feedforward network and a feature extraction module. The trained voice wake-up model based on the lightweight deep residual convolutional network is used to wake up the target voice, including:

[0011] Inputting the target speech into the feature extraction module and converting it into original speech features;

[0012] Inputting the original speech feature into the first feedforward network and performing nonlinear transformation to obtain the first speech feature;

[0013] The first speech feature is input into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech.

[0014] In some embodiments of the present invention, the residual causal convolution module includes five layers of residual causal convolution blocks, a nonlinear transformation block, an average pooling layer and a perception layer; the five layers of residual causal convolution blocks are arranged in sequence, and the convolution step size of each layer of residual causal convolution blocks increases by a multiple of 2; the nonlinear transformation block includes two layers of second feedforward networks, and each layer of the second feedforward network is also integrated with a batch normalization processing block.

[0015] In some embodiments of the present invention, inputting the first speech feature into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech includes:

[0016] In a first-layer residual causal step, the first speech feature is convolved with a first-layer residual causal convolution block to obtain a second speech feature, and the result of the summation of the second speech feature and the first speech feature is input into a second-layer residual causal convolution block;

[0017] In a second-layer residual causal step, a result of adding the second speech feature to the first speech feature is convolved in a second-layer residual causal convolution block to obtain a third speech feature, and a result of adding the third speech feature to the first speech feature and the second speech feature is input into a third-layer residual causal convolution block;

[0018] According to the first layer residual causal step and the second layer residual causal step, a fourth speech feature, a fifth speech feature and a sixth speech feature are obtained in sequence;

[0019] Connecting the second voice feature, the third voice feature, the fourth voice feature, the fifth voice feature, and the sixth voice feature to obtain a seventh voice feature;

[0020] The seventh speech feature is processed sequentially by a nonlinear transformation block, an average pooling layer, and a perception layer to obtain a keyword probability corresponding to the target speech.

[0021] In some embodiments of the present invention, each layer of the five-layer residual causal convolution block includes a causal dilation convolution block and a gate activation function, and the gate activation function formula is:

[0022] z=tanh(W f *x)⊙σ(W g *x)

[0023] Where σ(·) is the sigmoid function, W f represents the convolution kernel used for filtering, W g Represents the convolution kernel used for gate calculation, ⊙ represents the dot product, x represents the expansion result of the causal expansion convolution block, and z represents the gate activation function.

[0024] In some embodiments of the present invention, inputting the target speech into the feature extraction module and converting it into original speech features includes:

[0025] Pre-emphasize, frame, window, and perform fast Fourier transform on the target speech in sequence;

[0026] The result obtained after fast Fourier transform is used to extract speech features based on Mel-frequency cepstral coefficients to obtain the original speech features.

[0027] In some embodiments of the present invention, before inputting the voice sample data as a training sample into a voice wake-up model based on a lightweight deep residual convolutional network for training, it also includes: performing a voice enhancement transformation operation on the voice sample data, wherein the voice enhancement transformation operation at least includes intonation transformation, volume transformation, fade-in and fade-out transformation, speaking speed transformation, head and tail silence cutting transformation, bandpass filtering transformation and bandstop filtering transformation.

[0028] A second aspect of the present invention provides a voice wake-up device based on a lightweight deep residual convolutional network, the device comprising:

[0029] An acquisition module, used to obtain voice sample data;

[0030] A training module, configured to input the speech sample data as a training sample into a voice wake-up model based on a lightweight deep residual convolutional network for training;

[0031] The wake-up module is used to wake up the target speech using a trained speech wake-up model based on a lightweight deep residual convolutional network;

[0032] Among them, the voice wake-up model based on the lightweight deep residual convolutional network is provided with a residual causal convolution module.

[0033] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0034] Obtain voice sample data;

[0035] Inputting the speech sample data as training samples into a voice wake-up model based on a lightweight deep residual convolutional network for training;

[0036] Use the trained voice wake-up model based on lightweight deep residual convolutional network to wake up the target voice;

[0037] Among them, the voice wake-up model based on the lightweight deep residual convolutional network is provided with a residual causal convolution module.

[0038] A fourth aspect of the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0039] Obtain voice sample data;

[0040] Inputting the speech sample data as training samples into a voice wake-up model based on a lightweight deep residual convolutional network for training;

[0041] Use the trained voice wake-up model based on lightweight deep residual convolutional network to wake up the target voice;

[0042] Among them, the voice wake-up model based on the lightweight deep residual convolutional network is provided with a residual causal convolution module.

[0043] The technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0044] This application first obtains voice sample data, then inputs the voice sample data as training samples into a voice wake-up model based on a lightweight deep residual convolutional network for training. Finally, the trained voice wake-up model based on a lightweight deep residual convolutional network is used to wake up the target voice, achieving both high efficiency and accuracy, thereby significantly improving the efficiency of voice wake-up. In particular, the residual causal convolution module is set in the voice wake-up model based on the lightweight deep residual convolutional network, which has strong modeling capabilities without adding excessive complexity. It also meets the requirements of high precision and real-time performance, and can be used for voice wake-up applications in a variety of scenarios.

[0045] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0047] Figure 1 A schematic diagram of the steps of a voice wake-up method based on a lightweight deep residual convolutional network in an exemplary embodiment of the present application is shown;

[0048] Figure 2 A schematic diagram of the structure of a voice wake-up model based on a lightweight deep residual convolutional network in an exemplary embodiment of the present application is shown;

[0049] Figure 3 A schematic diagram showing the working process of a feature extraction module in an exemplary embodiment of the present application is shown;

[0050] Figure 4 A schematic diagram of the overall working process of a voice wake-up method based on a lightweight deep residual convolutional network in an exemplary embodiment of the present application is shown;

[0051] Figure 5 A schematic diagram of the causal expansion process of a five-layer residual causal convolution block in an exemplary embodiment of the present application is shown;

[0052] Figure 6 The following figure shows the structure and working process of each layer of residual causal convolution block in an exemplary embodiment of the present application;

[0053] Figure 7 A schematic diagram of the structure of a voice wake-up device based on a lightweight deep residual convolutional network in an exemplary embodiment of the present application is shown;

[0054] Figure 8 A schematic structural diagram of a computer device provided by an exemplary embodiment of the present application is shown;

[0055] Figure 9 A schematic diagram of a storage medium provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0056] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present application. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present application. It is obvious to those skilled in the art that the present application can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present application, some technical features known in the art are not described.

[0057] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of the features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0058] Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described herein. The accompanying drawings are not drawn to scale, and certain details may be magnified and omitted for the purpose of clarity. The shapes of the various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are merely exemplary and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0059] The following is attached with the instruction manual Figure 1 -Attached Figure 9 Several examples are given to describe exemplary implementations of the present application. It should be noted that the following application scenarios are only provided to facilitate understanding of the spirit and principles of the present application, and the implementations of the present application are not limited in this respect. On the contrary, the implementations of the present application can be applied to any applicable scenario.

[0060] Currently, voice interaction products are a hot research and development topic. Traditional methods use large vocabulary speech recognition (LVCSR) technology, using a GMM-HMM learning approach. However, GMM suffers from insufficient modeling capabilities, high misrecognition rates, and inefficient search for solutions in large corpora. Modern voice wake-up technology employs an end-to-end model based on deep learning. There are two wake-up approaches. One uses speech recognition to model keyword detection as a binary classification problem of whether the text is a keyword. The device wakes up when the text is identified as a keyword. This approach places high demands on the speech model and struggles to meet real-time requirements. However, modeling keyword detection as a binary classification problem of whether the text is a keyword is difficult. The model compares the output probability with a set threshold to determine whether a keyword is present. This approach significantly reduces the complexity of the required model.

[0061] In the two aforementioned methods, time-series models such as RNN, GRU, and LSTM have proven to be excellent at modeling speech features. However, due to their inability to perform parallel operations, training and prediction efficiency are low, and high-precision models require more hidden layers for learning. Therefore, they cannot simultaneously meet the requirements of high accuracy and real-time performance. Deep residual convolutional networks, proposed by WaveNet, have achieved promising results in the field of voice cloning. However, WaveNet's multi-layered and complex structure makes it difficult to apply to voice wake-up devices.

[0062] Therefore, in some exemplary embodiments of the present application, a voice wake-up method based on a lightweight deep residual convolutional network is provided, such as Figure 1 As shown, the method includes: S1, obtaining speech sample data; S2, inputting the speech sample data as training samples into a speech wake-up model based on a lightweight deep residual convolutional network for training; S3, using the trained speech wake-up model based on a lightweight deep residual convolutional network to wake up the target speech; wherein the speech wake-up model based on a lightweight deep residual convolutional network is provided with a residual causal convolution module. These steps are described in detail below.

[0063] The first step is to obtain voice sample data.

[0064] The voice sample data here include keyword voice sample data and non-keyword voice sample data. For keyword voice sample data, this application recorded a total of 1,500 clean keyword voice data samples and 1,500 noisy keyword voice data samples for 50 experimenters, with an average of 60 voice samples recorded for each experimenter. The voice recording device is a professional microphone with a sampling frequency of 16 kHz and 1 sampling channel. During the recording process, the experimenter ensured the diversity of the voice as much as possible by changing the tone, speaking speed, sound volume, etc. The duration of the voice data sample is between 1-7 seconds, the average duration of all keyword voice samples is about 3 seconds, and the total duration is about 2.5 hours.

[0065] For non-keyword speech data, this application collects multiple public speech data sets, public noise data sets, and background noise recorded in a noisy environment. The public speech data sets include: aidatatang_200zh, aishel3, aishel4, primewords_md_2018_set1, free_st_mandrain, magic_mandrain, thchs30. Among them, the aishel4 data set is an 8-channel long speech segment recorded in a classroom environment, so it was preprocessed. The specific operations include: first splitting the 8-channel speech data into multiple single-channel speech data, and then randomly slicing the long speech data according to the 1-7s interval, and finally obtaining non-keyword speech data that can be used for training. The public noise data set and the background noise recorded in a noisy environment also underwent the above preprocessing process. The total length of all non-keyword speech data is about 750h.

[0066] In a preferred implementation, the speech sample data also needs to be subjected to a speech enhancement transformation operation so that the speech sample can be strengthened and expanded. Here, the speech enhancement transformation operation at least includes intonation transformation, volume transformation, fade-in and fade-out transformation, speech speed transformation, head and tail silence cutting transformation, bandpass filtering transformation, and bandstop filtering transformation.

[0067] In the second step, the speech sample data is input as a training sample into a speech wake-up model based on a lightweight deep residual convolutional network for training.

[0068] like Figure 2 As shown, the voice wake-up model based on a lightweight deep residual convolutional network includes a feature extraction module, a first feedforward network, and a residual causal convolution module. During speech sample training, the speech sample data is first input into the feature extraction module and converted into raw speech features. The raw speech features are then input into the first feedforward network, where they undergo a nonlinear transformation to obtain first speech features. The first speech features are then input into the residual causal convolution module for processing to obtain the keyword probability corresponding to the speech sample. The feedforward network is a simple neural network in which neurons are arranged in layers. Each neuron is connected only to neurons in the previous layer, receiving the output of the previous layer and outputting it to the next layer. There is no feedback between layers. It is one of the most widely used and rapidly developing artificial neural networks. The "first" in the first feedforward network is only to distinguish it from the feedforward network within the residual causal convolution module, which is referred to as the second feedforward network. Both the first and second feedforward networks incorporate a batch normalization processing block.

[0069] like Figure 3As shown, in the process of realizing speech feature extraction, the feature extraction module sequentially performs pre-emphasis, framing, windowing, and fast Fourier transform on the target speech; the result obtained after fast Fourier transform is based on Mel ( Figure 3 Mel frequency cepstral coefficients (MFCC) are used to extract speech features to obtain the original speech features. Mel frequency cepstral coefficients (MFCC) are in line with the principle of human hearing and can effectively extract speech acoustic features. Figure 3 The DFT in the figure represents discrete Fourier transform. Speech features are extracted based on Mel-frequency cepstral coefficients. In the process of obtaining the original speech features, the frequency range of the extracted audio is 20-8000 Hz, the number of sampling points used in the fast Fourier transform is 512, the window length is 32 ms, the window shift is 20 ms, the number of Mel filter groups is 26, the number of Mel cepstral coefficients is 26, and the first 13 Mel cepstral coefficients are used as input features for training.

[0070] refer to Figure 4 and Figure 5 The residual causal convolution module includes five layers of residual causal convolution blocks, nonlinear transformation blocks, average pooling layers and perception layers, where each layer of residual causal convolution blocks is as follows: Figure 6 As shown in the figure, the five layers of residual causal convolution blocks are set in sequence, and the convolution step size of each layer of residual causal convolution blocks increases by a multiple of 2. The nonlinear transformation block includes two layers of second feedforward networks, and each layer of second feedforward networks is also integrated with a batch normalization processing block. Considering that the gate activation function memorizes effective information, each layer of residual causal convolution blocks in the five layers of residual causal convolution blocks is set with a causal expansion convolution block and a gate activation function. The gate activation function formula is:

[0071] z=tanh(W f *x)⊙σ(W g *x)

[0072] Where σ(·) is the sigmoid function, W f represents the convolution kernel used for filtering, W g Represents the convolution kernel used for gate calculation, ⊙ represents the dot product, x represents the expansion result of the causal expansion convolution block, and z represents the gate activation function.

[0073] Figure 5 The causal expansion process of the five-layer causal convolution block is shown. When the step size is 1, the causal expansion convolution is a normal causal convolution. The convolution step size of each layer of residual causal convolution block increases by a multiple of 2. The step size set of the five-layer residual causal convolution module is {1, 2, 4, 8, 16}. Since speech data is sequential data, using causal convolution can ensure that the model learns speech features in chronological order. For example, let X = {x1, x2, ..., x n} is the input speech feature, n is the time length, and the predicted value p(x t+1 |x1,x2,…,x t ) depends only on information before time t and does not depend on {x t+1 ,x t+2 ,…,x n Causal extended convolution increases the step size of the convolution kernel based on ordinary convolution and ignores the features of the intermediate span. For example, when calculating the output with a step size of 2 and a time of 3, The input features at time 2 are ignored, and f is a convolution operation.

[0074] Figure 4 The overall working process of the voice wake-up method based on the lightweight deep residual convolutional network is shown. The target voice is input into the voice wake-up model based on the lightweight deep residual convolutional network. The target voice will first enter the feature extraction module for feature extraction to obtain the original voice features. The original voice features will then enter the feedforward network, that is, the original voice features are input into the feedforward network for nonlinear transformation to obtain the first voice features. The first voice features are then input into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target voice. It should be noted that Figure 4 The feedforward network in

[15] has the same structure as the first and second feedforward networks described above. The first and second feedforward networks are only distinguished in form. All of the feedforward networks, the first feedforward network, and the second feedforward network are integrated with a batch normalization block. The process of inputting the raw speech features into the feedforward network and performing a nonlinear transformation to obtain the first speech features includes batch normalization processing on the raw speech features.

[0075] The first speech feature is input into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech, including: a first-layer residual causal step, performing a convolution operation on the first speech feature through the first-layer residual causal convolution block to obtain a second speech feature, and inputting the result of the sum operation of the second speech feature and the first speech feature into the second-layer residual causal convolution block; a second-layer residual causal step, performing a convolution operation on the result of the sum operation of the second speech feature and the first speech feature through the second-layer residual causal convolution block to obtain a third speech feature, and inputting the result of the sum operation of the third speech feature with the first and second speech features into the third-layer residual causal convolution block; according to the first-layer residual causal step and the second-layer residual causal step, the fourth speech feature, the fifth speech feature and the sixth speech feature are obtained in sequence. It can be understood here that the result of the sum operation of the fourth speech feature with the first speech feature, the second speech feature and the third speech feature is input into the fourth-layer residual causal convolution block, and so on; the second speech feature, the third speech feature, the fourth speech feature, the fifth speech feature and the sixth speech feature are connected (i.e. Figure 4 The seventh speech feature is processed by a nonlinear transformation block, an average pooling layer and a perception layer in sequence to obtain the keyword probability corresponding to the speech sample.

[0076] If the output of the first layer of residual causal convolution is z 1 , then the output after skip connection is:

[0077] Z=sum([z 1 ,z 2 ,z 3 ,z 4 ,z 5 ])

[0078] A feedforward network is used before and after the deep residual convolutional network for nonlinear learning. Batch normalization is also used to accelerate model learning and avoid overfitting. The output features contain three dimensions: batch dimension, time dimension, and feature dimension. Average pooling is used in the time dimension to obtain the average time feature, and the final keyword probability is obtained through the perception layer. The average pooling formula is as follows:

[0079]

[0080] Among them, X is the feature after causal convolution fine-grained learning, and the perception layer formula is:

[0081] p i =σ(W i X i +b)

[0082] Among them, i is the sample index and σ is the sigmoid function.

[0083] This model uses cross entropy loss and Adam learner for optimization. The formula for cross entropy loss is as follows:

[0084]

[0085] Among them, y represents the true label value, Represents the predicted label value.

[0086] Of course, it is understandable that during the training process, the keyword labels corresponding to the voice samples must be input so that the model can be trained normally. In addition, preferably, the number of causal expansion convolution kernels is 128, and the number of hidden layer nodes of the feedforward network is 128.

[0087] In the third step, the target voice is awakened using the trained voice awakening model based on a lightweight deep residual convolutional network.

[0088] The voice wake-up model based on a lightweight deep residual convolutional network must be trained before it can be used in actual application scenarios. Inputting the target speech into the model follows a similar process as inputting speech sample data. The trained voice wake-up model based on a lightweight deep residual convolutional network is used to wake up the target speech. This includes: inputting the target speech into the feature extraction module to convert it into raw speech features; inputting the raw speech features into the first feedforward network for nonlinear transformation to obtain the first target speech features; and inputting the first target speech features into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech.

[0089] The first target speech feature is input into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech, including: a first-layer residual causal step, performing a convolution operation on the first target speech feature through the first-layer residual causal convolution block to obtain a second target speech feature, and inputting the result of the sum operation of the second target speech feature and the first target speech feature into the second-layer residual causal convolution block; a second-layer residual causal step, performing a convolution operation on the result of the sum operation of the second target speech feature and the first target speech feature through the second-layer residual causal convolution block to obtain a third target speech feature, and inputting the result of the sum operation of the third target speech feature with the first target speech feature and the second speech feature into the third-layer residual causal convolution block; according to the first-layer residual causal step and the second-layer residual causal step, the fourth target speech feature, the fifth target speech feature and the sixth target speech feature are obtained in sequence; the second target speech feature, the third target speech feature, the fourth target speech feature, the fifth target speech feature and the sixth target speech feature are connected to obtain the seventh target speech feature; the seventh target speech feature is processed in sequence by the nonlinear transformation block, the average pooling layer and the perception layer to obtain the keyword probability corresponding to the target speech.

[0090] In order to verify the accuracy and real-time performance of the speech wake-up model based on the lightweight deep residual convolutional network of this application, this application tested 1,000 additional keyword voices collected. The experimental results show that the wake-up accuracy of the model of this application is 95%. The model of this application is used on ARM microcontrollers such as Raspberry Pi. It takes about 0.1s to recognize 7s of speech. The model prediction time required using only a single-layer unidirectional GRU with 128 hidden nodes is 0.4s. The prediction efficiency of the model of this application is much higher than that of the time series model. At the same time, this application runs the model in a real environment, and its false wake-up rate is only about 5 / 24h. It can be seen that the method proposed in this application is both efficient and accurate, thereby greatly improving the prediction efficiency.

[0091] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

[0092] In some exemplary embodiments of the present application, a voice wake-up device based on a lightweight deep residual convolutional network is also provided, such as Figure 7 As shown, the device includes:

[0093] Acquisition module 601, used to acquire voice sample data;

[0094] A training module 602 is configured to input the speech sample data as a training sample into a voice wake-up model based on a lightweight deep residual convolutional network for training;

[0095] A wake-up module 603 is used to wake up the target speech using a trained speech wake-up model based on a lightweight deep residual convolutional network;

[0096] Among them, the voice wake-up model based on the lightweight deep residual convolutional network is provided with a residual causal convolution module.

[0097] It should also be emphasized that the system provided in the embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology and machine learning / deep learning.

[0098] Please refer to the following Figure 8 , which shows a schematic diagram of a computer device provided by some embodiments of the present application. Figure 8 As shown, the computer device 2 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203, and the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and when the processor 200 runs the computer program, it executes the voice wake-up method based on the lightweight deep residual convolutional network provided in any of the aforementioned embodiments of the present application.

[0099] The memory 201 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The system network element communicates with at least one other network element via at least one communication interface 203 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0100] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The voice wake-up method based on a lightweight deep residual convolutional network disclosed in any of the aforementioned embodiments of the present application may be applied to the processor 200 or implemented by the processor 200.

[0101] The processor 200 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 200 or by software instructions. The above processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 201 , and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0102] The present application also provides a computer-readable storage medium corresponding to the voice wake-up method based on the lightweight deep residual convolutional network provided in the above embodiment. Please refer to Figure 9 , Figure 9The computer-readable storage medium shown is an optical disc 30 on which a computer program (i.e., a program product) is stored. When the computer program is executed by a processor, it executes the voice wake-up method based on a lightweight deep residual convolutional network provided in any of the aforementioned embodiments.

[0103] In addition, examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0104] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the voice wake-up method based on a lightweight deep residual convolutional network provided in any of the aforementioned embodiments, including: obtaining voice sample data; inputting the voice sample data as a training sample into a voice wake-up model based on a lightweight deep residual convolutional network for training; using the trained voice wake-up model based on a lightweight deep residual convolutional network to wake up the target voice; wherein the voice wake-up model based on a lightweight deep residual convolutional network is provided with a residual causal convolution module.

[0105] It should be noted that the algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other device. Various general-purpose devices may also be used in conjunction with the teachings herein. Based on the above description, the structure required to construct such devices is apparent. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages is intended to disclose the best implementation methods of this application. In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this specification.

[0106] Similarly, it should be understood that in order to streamline the present application and aid understanding of one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the present application.

[0107] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subassemblies. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0108] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the creation device of the virtual machine according to the embodiment of the present application. The application can also be implemented as a device or device program for executing part or all of the methods described herein. The program implementing the application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0109] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A voice wake-up method based on a lightweight deep residual convolutional network, characterized in that: The method comprises: Obtain voice sample data; Inputting the speech sample data as training samples into a voice wake-up model based on a lightweight deep residual convolutional network for training; Use the trained voice wake-up model based on lightweight deep residual convolutional network to wake up the target voice; The voice wake-up model based on the lightweight deep residual convolutional network is provided with a residual causal convolution module; The voice wake-up model based on the lightweight deep residual convolutional network also includes a first feedforward network and a feature extraction module. The trained voice wake-up model based on the lightweight deep residual convolutional network is used to wake up the target voice, including: Inputting the target speech into the feature extraction module and converting it into original speech features; Inputting the original speech feature into the first feedforward network, wherein the first feedforward network is integrated with a batch normalization processing block, and performing nonlinear transformation to obtain the first speech feature; The first speech feature is input into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech.

2. The voice wake-up method based on lightweight deep residual convolutional network according to claim 1 is characterized in that The residual causal convolution module includes five layers of residual causal convolution blocks, a nonlinear transformation block, an average pooling layer and a perception layer; the five layers of residual causal convolution blocks are arranged in sequence, and the convolution step size of each layer of residual causal convolution blocks increases by a multiple of 2; the nonlinear transformation block includes two layers of second feedforward networks, and each layer of the second feedforward network is also integrated with a batch normalization processing block.

3. The voice wake-up method based on lightweight deep residual convolutional network according to claim 2 is characterized in that Inputting the first speech feature into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech includes: In a first-layer residual causal step, the first speech feature is convolved with a first-layer residual causal convolution block to obtain a second speech feature, and the result of the summation of the second speech feature and the first speech feature is input into a second-layer residual causal convolution block; In a second-layer residual causal step, a result of adding the second speech feature to the first speech feature is convolved in a second-layer residual causal convolution block to obtain a third speech feature, and a result of adding the third speech feature to the first speech feature and the second speech feature is input into a third-layer residual causal convolution block; According to the first layer residual causal step and the second layer residual causal step, a fourth speech feature, a fifth speech feature and a sixth speech feature are obtained in sequence; Connecting the second voice feature, the third voice feature, the fourth voice feature, the fifth voice feature, and the sixth voice feature to obtain a seventh voice feature; The seventh speech feature is processed sequentially by a nonlinear transformation block, an average pooling layer, and a perception layer to obtain a keyword probability corresponding to the target speech.

4. The voice wake-up method based on lightweight deep residual convolutional network according to claim 2 is characterized in that Each layer of the five-layer residual causal convolution block includes a causal dilation convolution block and a gate activation function. The gate activation function formula is: z=tanh(W f *x)☉σ(W g *x) Where σ(·) is the sigmoid function, W f represents the convolution kernel used for filtering, W g Represents the convolution kernel used for gate calculation, ⊙ represents the dot product, x represents the expansion result of the causal expansion convolution block, and z represents the gate activation function.

5. The voice wake-up method based on lightweight deep residual convolutional network according to claim 1, characterized in that The step of inputting the target speech into the feature extraction module and converting the target speech into original speech features comprises: Pre-emphasize, frame, window, and perform fast Fourier transform on the target speech in sequence; The result obtained after fast Fourier transform is used to extract speech features based on Mel-frequency cepstral coefficients to obtain the original speech features.

6. The voice wake-up method based on lightweight deep residual convolutional network according to claim 1, characterized in that Before inputting the voice sample data as a training sample into a voice wake-up model based on a lightweight deep residual convolutional network for training, the method further includes: performing a voice enhancement transformation operation on the voice sample data, wherein the voice enhancement transformation operation at least includes intonation transformation, volume transformation, fade-in and fade-out transformation, speaking speed transformation, head and tail silence cutting transformation, bandpass filtering transformation and bandstop filtering transformation.

7. A voice wake-up device based on a lightweight deep residual convolutional network, characterized in that: The device comprises: An acquisition module, used to obtain voice sample data; A training module, configured to input the speech sample data as a training sample into a voice wake-up model based on a lightweight deep residual convolutional network for training; The wake-up module is used to wake up the target speech using a trained speech wake-up model based on a lightweight deep residual convolutional network; The voice wake-up model based on the lightweight deep residual convolutional network is provided with a residual causal convolution module; The voice wake-up model based on the lightweight deep residual convolutional network also includes a first feedforward network and a feature extraction module. The trained voice wake-up model based on the lightweight deep residual convolutional network is used to wake up the target voice, including: Inputting the target speech into the feature extraction module and converting it into original speech features; Inputting the original speech feature into the first feedforward network, wherein the first feedforward network is integrated with a batch normalization processing block, and performing nonlinear transformation to obtain the first speech feature; The first speech feature is input into the residual causal convolution module for processing to obtain the keyword probability corresponding to the target speech.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images