Command word recognition method, device, mobile terminal and readable storage medium

Abstract

The present application discloses a command word recognition method, device, mobile terminal and readable storage medium, the method comprising: converting a received command word into a standard matrix; using each feature extraction layer of a command word offline recognition model to perform channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix to obtain a feature tensor of the command word; using the output layer of the command word offline recognition model to determine the category of the command word based on the feature tensor of the command word. In the present application, on the one hand, the use of channel-by-channel convolution operation and point-by-point convolution operation to replace the traditional convolution operation can greatly reduce the memory space occupied by the command word offline recognition model; on the other hand, the command word offline recognition model has low hardware requirements and does not require improved hardware configuration, which can effectively reduce hardware costs; on the other hand, when recognizing speech with command words, there is no need to continuously send voice data to the cloud computing server, thereby ensuring the privacy of the user.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition, and in particular to a command word recognition method, device, mobile terminal and readable storage medium. Background Art

[0002] At present, voice recognition, as a commonly used human-computer interaction technology, has been widely used in various electronic products. Users can control electronic products by issuing voice with command words. This voice control method has been loved by consumers for its natural and convenient interaction method, and has gradually become the mainstream interaction control method in the era of smart products.

[0003] However, in order to achieve a sufficiently high accuracy, existing command word recognition algorithms (for example, GMM-HMM and command word recognition algorithms based on deep learning) need to occupy a large amount of memory in the mobile terminal, resulting in a reduction in the available memory of the mobile terminal and a reduction in the working efficiency of the mobile terminal. Especially for some mobile terminals with smaller memory, the memory of the mobile terminal is very limited. After setting up the existing command word recognition algorithm, it is difficult for the mobile terminal to ensure that it can continue to maintain normal operation.

[0004] Although the memory usage problem can be solved by improving hardware configuration and cloud computing services, improving hardware configuration will increase equipment costs, and cloud computing services require a network to be used and voice data to be uploaded, which has poor security. Summary of the invention

[0005] In view of the above problems, the present application proposes a command word recognition method, device, mobile terminal and readable storage medium. The method only occupies a small amount of memory in the mobile terminal and realizes effective recognition of command words in an offline state.

[0006] The present application embodiment provides a command word recognition method, the method comprising:

[0007] Convert the received command words into a standard matrix;

[0008] Using each feature extraction layer of the command word offline recognition model to perform channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix to obtain a feature tensor of the command word;

[0009] The category of the command word is determined based on the feature tensor of the command word using the output layer of the command word offline recognition model.

[0010] The command word recognition method described in the embodiment of the present application, the command word offline recognition model includes N feature extraction layers, each feature extraction layer includes a channel-by-channel convolution operation layer, a point-by-point convolution operation layer, a normalization layer, an activation layer, and a maximum pooling layer, and the use of each feature extraction layer of the command word offline recognition model to perform channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix to obtain the feature tensor of the command word includes:

[0011] Using a first channel-by-channel convolution operation layer, a first point-by-point convolution operation layer, a first normalization layer, a first activation layer, and a first maximum pooling layer of a first feature extraction layer to perform feature extraction on the standard matrix to obtain a first feature tensor;

[0012] Using the nth channel-by-channel convolution operation layer, the nth point-by-point convolution operation layer, the nth normalization layer, the nth activation layer, and the nth maximum pooling layer of the nth feature extraction layer to extract features from the n-1th feature tensor to obtain the nth feature tensor, 2≤n≤N;

[0013] The Nth feature tensor is used as the feature tensor of the command word.

[0014] The command word recognition method described in the embodiment of the present application, wherein the n-1th feature tensor is extracted by using the n-th channel-by-channel convolution operation layer, the n-th point-by-point convolution operation layer, the n-th normalization layer, the n-th activation layer, and the n-th maximum pooling layer of the n-th feature extraction layer to obtain the n-th feature tensor, comprises:

[0015] Using the nth channel-by-channel convolution operation layer to perform a channel-by-channel convolution operation on the n-1th feature tensor to obtain an nth channel-by-channel convolution tensor;

[0016] Using the nth point-by-point convolution operation layer to perform a point-by-point convolution operation on the nth channel-by-channel convolution tensor to obtain an nth point-by-point convolution tensor;

[0017] Using the nth normalization layer to perform a normalization operation on the nth point-by-point convolution tensor to obtain an nth normalized feature tensor;

[0018] Using the nth activation layer, performing an activation operation on the nth normalized feature tensor to obtain an nth activated feature tensor;

[0019] The nth maximum pooling layer is used to perform an activation operation on the nth activated feature tensor to obtain the nth feature tensor.

[0020] In the command word recognition method described in the embodiment of the present application, the formula for each channel-by-channel convolution operation is as follows:

[0021]

[0022] xd is the feature tensor of the input channel-by-channel convolution layer, wd is the convolution kernel of the channel-by-channel convolution operation, DW ijk (x d ) is x d The output of the channel-by-channel convolution operation with wd, w H is the height of the convolution kernel wd, w W is the width of the convolution kernel wd, stride h is the convolution kernel wd at x d The step in the height direction, stride w is the convolution kernel wd at x d The step in the width direction, i is the index in the height direction of the feature tensor, j is the index in the width direction of the feature tensor, i' is the index in the height direction of the convolution kernel, j' is the index in the height direction of the convolution kernel, and K represents the kth channel;

[0023] The formula for each point-by-point convolution operation is as follows:

[0024]

[0025] x p is the feature tensor of the input point-by-point convolution layer, wp is the convolution kernel of the point-by-point convolution operation, x C is the number of channels of the input feature, b k is the convolution bias of the kth channel, PW ijk (x p ) is x p The output result of the point-by-point convolution operation with wd, and k' is the summation index.

[0026] In the command word recognition method described in the embodiment of the present application, the value range of N is 4 to 8.

[0027] The command word recognition method described in the embodiment of the present application, wherein the step of converting the received command word into a standard matrix includes:

[0028] Performing pre-emphasis processing on the audio of the received command word to obtain a standard time-domain audio signal;

[0029] Converting the time-domain audio signal into a frequency-domain signal using Fourier transform;

[0030] Performing discrete cosine transformation on the frequency domain signal to obtain a standard matrix.

[0031] In the command word recognition method described in the embodiment of the present application, the output layer includes a dimensionality reduction layer, a linear layer, and a probability output layer, and the output layer using the command word offline recognition model determines the category of the command word based on the feature tensor, including:

[0032] Converting the feature tensor of the command word into a feature vector using the dimension reduction layer;

[0033] Utilizing the linear layer to calculate the probability that the feature vector is each category;

[0034] The probability output layer is used to normalize the probabilities of various categories and output the categories of the command words and the probabilities corresponding to the categories.

[0035] The present application also provides a command word recognition device, the device comprising:

[0036] A conversion module, used for converting the received command words into a standard matrix;

[0037] A calculation module, used for performing channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix by using each feature extraction layer of the command word offline recognition model to obtain a feature tensor of the command word;

[0038] A determination module is used to determine the category of the command word based on the feature tensor using the output layer of the command word offline recognition model.

[0039] The embodiment of the present application also provides a mobile terminal, including a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the command word recognition method described in the embodiment of the present application is executed.

[0040] The embodiment of the present application also provides a readable storage medium storing a computer program, which executes the command word recognition method described in the embodiment of the present application when running on a processor.

[0041] The present application converts the received command words into a standard matrix; uses each feature extraction layer of the command word offline recognition model to perform channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix to obtain the feature tensor of the command word; uses the output layer of the command word offline recognition model to determine the category of the command word based on the feature tensor of the command word. On the one hand, each feature extraction layer of the command word offline recognition model uses channel-by-channel convolution operation and point-by-point convolution operation to replace the traditional convolution operation, which can greatly reduce the memory space occupied by the command word offline recognition model; on the other hand, the command word offline recognition model has low hardware requirements and does not require improved hardware configuration, which can effectively reduce hardware costs; on the other hand, when recognizing speech with command words, there is no need to continuously send voice data to the cloud computing server, which ensures the privacy of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0043] Figure 1 A schematic diagram of the structure of a command word recognition model proposed in an embodiment of the present application is shown;

[0044] Figure 2 A schematic diagram showing a flow chart of a command word recognition method proposed in an embodiment of the present application is shown;

[0045] Figure 3 A schematic diagram of a process for obtaining a standard matrix in a command word recognition method proposed in an embodiment of the present application is shown;

[0046] Figure 4 A schematic diagram showing a flow chart of obtaining a feature tensor of a command word in a command word recognition method proposed in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the structure of a command word recognition device proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0050] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0051] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0052] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.

[0053] The command word recognition method disclosed in the embodiment of the present application can convert the received command word (for example, connect, hang up or call back) into a standard matrix, and then input the standard matrix into a preset command word offline recognition model. The standard matrix is ​​processed by the command word offline recognition model to determine the category to which the command word belongs, that is, to determine whether the command word is to connect, hang up or call back.

[0054] The following will describe in detail the conversion process from command words to standard matrices, the command word offline recognition model and the command word recognition method with specific embodiments.

[0055] Example 1

[0056] See also Figure 1 In an embodiment of the present application, a command word recognition model is proposed. The command word offline recognition model includes an output layer and multiple feature extraction layers. The multiple feature extraction layers are connected in sequence, and the output layer is connected to the last feature extraction layer.

[0057] Furthermore, each feature extraction layer includes a channel-by-channel convolution operation layer, a point-by-point convolution operation layer, a normalization layer, an activation layer, and a maximum pooling layer. The output layer includes a dimensionality reduction layer, a linear layer, and a probability output layer.

[0058] The channel-by-channel convolution operation layer of each feature extraction layer first performs a channel-by-channel convolution operation on the input feature tensor, and then performs a point-by-point convolution operation on the result of the channel-by-channel convolution operation. Compared with the traditional feature extraction process, the amount of computation in this feature extraction process is significantly reduced, so that the entire command word offline recognition model only occupies less memory space.

[0059] For example, traditional feature extraction uses the following formula to perform convolution operation:

[0060]

[0061] Among them, x is the input three-dimensional feature tensor, wt is the convolution kernel of the traditional convolution operation, y is the output result of the convolution of x and wt, and x Cis the number of channels of the input feature, w H is the height of the convolution kernel, w W is the width of the convolution kernel, stride h is the stride of the convolution kernel wt in the height direction of x, stride w is the step of the convolution kernel wt in the width direction of x, i is the index in the height direction of the feature tensor, j is the index in the width direction of the feature tensor, i' is the index in the height direction of the convolution kernel, j' is the index in the height direction of the convolution kernel, b k is the convolution bias of the kth channel.

[0062] In the case where each weight of the convolution kernel wt occupies the storage space of int8, all convolution kernels of the traditional convolution operation need to occupy w H × W × C Bit memory, where w C =x C ×F, where F represents the number of convolution kernels. In order to further reduce the space occupied by the feature extraction process, the traditional convolution can be approximated by two steps: channel-by-channel convolution (DW) and point-by-point convolution (PW), namely:

[0063] y ijk =PW ijk (DW ijk (x)

[0064] in, wd is the convolution kernel for channel-by-channel convolution operation, and wp is the convolution kernel for point-by-point convolution operation.

[0065] In channel-by-channel and point-by-point convolution operations, the convolution kernel occupies w H × W +w C Bit memory, thereby effectively reducing the memory occupied by the entire command word offline recognition model.

[0066] It should be noted that the total number of feature extraction layers of the command word offline recognition model can be adjusted as needed. If the total number of feature extraction layers is too small, the recognition accuracy of the command word offline recognition model for the command word may be reduced. However, if the total number of feature extraction layers is too large, the command word offline recognition model will occupy more memory space, resulting in excessive memory space of the mobile terminal. The total number of feature extraction layers can range from 4 to 8, including 4 and 8. When identifying the connect, hang up or call back command words, the total number of feature extraction layers of 5 is the best effect.

[0067] Preferably, the command word offline recognition model includes five feature extraction layers, namely a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, and a fifth feature extraction layer.

[0068] The first feature extraction layer includes a first channel-by-channel convolution operation layer, a first point-by-point convolution operation layer, a first normalization layer, a first activation layer and a first maximum pooling layer, the second feature extraction layer includes a second channel-by-channel convolution operation layer, a second point-by-point convolution operation layer, a second normalization layer, a second activation layer and a second maximum pooling layer, the third feature extraction layer includes a third channel-by-channel convolution operation layer, a third point-by-point convolution operation layer, a third normalization layer, a third activation layer and a third maximum pooling layer, the fourth feature extraction layer includes a fourth channel-by-channel convolution operation layer, a fourth point-by-point convolution operation layer, a fourth normalization layer, a fourth activation layer and a fourth maximum pooling layer, and the fifth feature extraction layer includes a fifth channel-by-channel convolution operation layer, a fifth point-by-point convolution operation layer, a fifth normalization layer, a fifth activation layer and a fifth maximum pooling layer.

[0069] Exemplarily, when the command word offline recognition model is used to recognize the connect, hang up or call back command words, the size of the convolution kernel of each channel-by-channel convolution operation layer can be 3×3, and the size of the convolution kernel of each point-by-point convolution operation can be 1×1. In the first feature extraction layer, the number of convolution kernels in the first point-by-point convolution operation layer is 16, the stride is 2×2, the size of the first maximum pooling layer is 2×1, and the output tensor dimension of the first feature extraction layer is 24×7×16; in the second feature extraction layer, the number of convolution kernels in the second point-by-point convolution operation layer is 32, the stride is 1×1, the size of the second maximum pooling layer is 2×1, and the output tensor dimension of the second feature extraction layer is 12×7×32; in the third feature extraction layer, the number of convolution kernels in the third point-by-point convolution operation layer is 64, the stride is 1×1, the size of the third maximum pooling layer is 2×1, and the output tensor dimension of the third feature extraction layer is 6×7×64; in the fourth feature extraction layer, the number of convolution kernels in the third point-by-point convolution operation layer is 64, the stride is 1×1, the size of the third maximum pooling layer is 2×1, and the output tensor dimension of the third feature extraction layer is 6×7×64; in the fourth feature extraction layer, the number of convolution kernels in the third point-by-point convolution operation layer is 64, the stride is 1×1, the size of the third maximum pooling layer is 2×1, and the output tensor dimension of the third feature extraction layer is 6×7×64. In the extraction layer, the number of convolution kernels in the fourth point-by-point convolution operation layer is 64, the stride is 1×1, the size of the fourth maximum pooling layer is 2×1, and the output tensor dimension of the first feature extraction layer is 3×3×64; in the fifth feature extraction layer, the number of convolution kernels in the fifth point-by-point convolution operation layer is 128, the stride is 1×1, the size of the fifth maximum pooling layer is 2×1, and the output tensor dimension of the fifth feature extraction layer is 1×1×128; in the dimensionality reduction layer, the three-dimensional tensor 1×1×128 is transformed into a 128×1 vector; in the linear layer, the probability of the command words answering, hanging up, calling back and others is calculated based on the 128×1 vector, and finally the output layer is used to output the probability of the command words answering, hanging up, calling back and others.

[0070] In this embodiment, compared with the existing traditional convolutional layer, the convolution kernel of each feature extraction layer of the command word offline recognition model takes up less memory space than that of the traditional convolutional layer. Furthermore, when multiple feature extraction layers are used, multiple traditional convolutional layers are replaced, which greatly reduces the memory space occupied by the command word offline recognition model.

[0071] Example 2

[0072] See also Figure 2 Based on the command word recognition model in Example 1, this embodiment further proposes a command word recognition method, which includes the following steps S100 to S300:

[0073] Step S100, converting the received command word into a standard matrix.

[0074] A microphone or other sound receiving device may be used to collect speech with command words, and the command words may be converted into a standard matrix.

[0075] The standard matrix can be a matrix determined based on the Mel-frequency cepstral coefficients (MFCC). Among them, the Mel frequency is proposed based on the auditory characteristics of the human ear, and the Mel frequency has a nonlinear correspondence with the Hz frequency. The Mel-frequency cepstral coefficients (MFCC) are the Hz spectrum features calculated using this nonlinear correspondence. The Mel-frequency cepstral coefficients are mainly used for voice data feature extraction and reducing the operation dimension. For example: for a frame with 512 dimensions (sampling points) of data, the most important 40 dimensions (generally speaking) of data can be extracted after MFCC, thereby achieving the purpose of dimensionality reduction.

[0076] Exemplarily, in order to ensure that the command word offline recognition model can accurately and quickly recognize the command words, the size of the standard matrix can be 98×13×1.

[0077] Step S200, using each feature extraction layer of the command word offline recognition model to perform channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix to obtain the feature tensor of the command word.

[0078] The feature extraction layers of the command word offline recognition model are connected in sequence, and each feature extraction layer can perform channel-by-channel convolution operations and point-by-point convolution operations on the received data.

[0079] It can be understood that compared with the existing traditional convolutional layer, the convolution kernel of each feature extraction layer of the command word offline recognition model occupies less memory space than the traditional convolutional layer. Furthermore, when multiple feature extraction layers are used, multiple traditional convolutional layers are replaced, which greatly reduces the memory space occupied by the command word offline recognition model.

[0080] Step S300, using the output layer of the command word offline recognition model to determine the category of the command word based on the feature tensor of the command word.

[0081] The output layer of the command word offline recognition model includes a dimensionality reduction layer, a linear layer and a probability output layer.

[0082] The dimension reduction layer may be used to convert the feature tensor of the command word into a feature vector, for example, transforming a three-dimensional tensor 1×1×128 into a 128×1 vector.

[0083] The linear layer is then used to calculate the probability that the feature vector is of each category, that is, the linear layer is used to calculate the proportion of the command words being answer, hang up, call back, and other instructions.

[0084] Finally, the probability output layer is used to normalize the probabilities of each category and output the category of the command word and the probability corresponding to the category. The probability output layer may be a Softmax function.

[0085] For example, see Figure 3 , step S100 includes the following steps S110 to S130:

[0086] Step S110 , performing pre-emphasis processing on the audio of the received command word to obtain a standard time-domain audio signal.

[0087] Exemplarily, a 16000kHz voice signal with a command word collected by a microphone is pre-emphasized through a pre-emphasis filter to obtain a standard time domain audio signal, wherein the pre-emphasis filter can be expressed by the following formula:

[0088] y[n]=x[n]–a*x[n-1], wherein x[n] is the speech signal obtained by the nth sampling, x[n-1] is the speech signal obtained by the n-1th sampling, and y[n] is the standard time domain audio signal corresponding to the speech signal obtained by the nth sampling. In this embodiment, the value of a can be any value between 0.9 and 1, including 0.9 and 1. Preferably, a=0.97. When a is 0.97, the command word offline recognition model has the highest recognition accuracy for the command word.

[0089] It should be noted that the pre-emphasis filter can compensate for the high-frequency part of the speech signal that is suppressed by the human vocal system, while highlighting the high-frequency resonance peak, thereby improving the accuracy of command word recognition.

[0090] Step S120: Convert the time domain audio signal into a frequency domain signal by using Fourier transform.

[0091] The standard time domain audio signal can be processed by dividing the frame into frames, and the length of each frame is generally 20 to 30 ms, and the frame overlap is 60% of the frame length. Preferably, when the frame length is 25 ms, the recognition effect of the command word offline recognition model is the best.

[0092] Furthermore, the time domain audio signal of each frame is converted into a frequency domain signal using Fourier transform.

[0093] Step S130: performing discrete cosine transformation on the frequency domain signal to obtain a standard matrix.

[0094] All frequency bands from 0 Hz to 8 kHz of the Fourier transform output pass through the Mel filter bank, where the Mel filter bank can be expressed by the following formula:

[0095] Mel(f)=2595*log10(1+f / 700), where f is the frequency band of 0 Hz to 8 kHz output by Fourier transform.

[0096] Furthermore, the output of the Mel filter bank is modulo squared and converted to dB scale through log10. The result of the conversion to dB scale is subjected to discrete cosine transform (DCT), and the first 13 coefficients of DCT are taken, retaining only the low-frequency part, and then the first coefficient is replaced with the energy value of the current frame. This process can effectively reduce the impact of different speaking volumes on the recognition effect and ensure the accuracy of the command word offline recognition model in recognizing command words.

[0097] For example, see Figure 4 The above step S200 includes the following steps S210 to S220:

[0098] Step S210, using the nth channel-by-channel convolution operation layer, the nth point-by-point convolution operation layer, the nth normalization layer, the nth activation layer and the nth maximum pooling layer of the nth feature extraction layer to extract features from the n-1th feature tensor to obtain the nth feature tensor, 1≤n≤N, the zeroth feature tensor is the standard matrix.

[0099] Exemplarily, when n=1, the first channel-by-channel convolution operation layer can be used to perform a channel-by-channel convolution operation on the standard matrix to obtain a first channel-by-channel convolution tensor, the first point-by-point convolution operation layer can be used to perform a point-by-point convolution operation on the first channel-by-channel convolution tensor to obtain a first point-by-point convolution tensor, the first normalization layer can be used to normalize the first point-by-point convolution tensor to obtain a first normalized feature tensor, the first activation layer can be used to activate the first normalized feature tensor to obtain a first activated feature tensor, and the first maximum pooling layer can be used to activate the first activated feature tensor to obtain the first feature tensor.

[0100] Furthermore, when 2≤n≤N, the n-1th feature tensor is extracted using the nth channel-by-channel convolution operation layer, the nth point-by-point convolution operation layer, the nth normalization layer, the nth activation layer, and the nth maximum pooling layer of the nth feature extraction layer to obtain the nth feature tensor.

[0101] Exemplarily, the nth channel-by-channel convolution operation layer can be used to perform a channel-by-channel convolution operation on the n-1th feature tensor to obtain the nth channel-by-channel convolution tensor, the nth point-by-point convolution operation layer can be used to perform a point-by-point convolution operation on the nth channel-by-channel convolution tensor to obtain the nth point-by-point convolution tensor, the nth normalization layer can be used to perform a normalization operation on the nth point-by-point convolution tensor to obtain the nth normalized feature tensor, the nth activation layer can be used to perform an activation operation on the nth normalized feature tensor to obtain the nth activated feature tensor, and the nth maximum pooling layer can be used to perform an activation operation on the nth activated feature tensor to obtain the nth feature tensor.

[0102] It can be understood that step S210 is executed N times, and the standard matrix, the first feature tensor to the N-1th feature tensor are respectively subjected to channel-by-channel convolution operation layer, point-by-point convolution operation layer, normalization operation, activation operation and maximum pooling operation to obtain the first feature tensor to the Nth feature tensor.

[0103] It should be noted that the formula for each channel-by-channel convolution operation is as follows:

[0104]

[0105] x d is the feature tensor of the input channel-by-channel convolution layer, wd is the convolution kernel of the channel-by-channel convolution operation, DW ijk (x d ) is x d The output of the channel-by-channel convolution operation with wd, w H is the height of the convolution kernel wd, w Wis the width of the convolution kernel wd, stride h is the convolution kernel wd at x d The step in the height direction, stride w is the convolution kernel wd at x d The step in the width direction, i is the index in the height direction of the feature tensor, j is the index in the width direction of the feature tensor, i' is the index in the height direction of the convolution kernel, j' is the index in the height direction of the convolution kernel, and K represents the kth channel.

[0106] It should be noted that the formula for each point-by-point convolution operation is as follows:

[0107]

[0108] x p is the feature tensor of the input point-by-point convolution layer, wp is the convolution kernel of the point-by-point convolution operation, x C is the number of channels of the input feature, b k is the convolution bias of the kth channel, PW ijk (x p ) is x p The output result of the point-by-point convolution operation with wd, and k' is the summation index.

[0109] Step S220: Using the Nth feature tensor as the feature tensor of the command word.

[0110] In this embodiment, each feature extraction layer of the command word offline recognition model uses channel-by-channel convolution operation and point-by-point convolution operation to replace the traditional convolution operation. Since the memory space occupied by the convolution kernel of the channel-by-channel convolution operation and the point-by-point convolution operation is smaller than the memory space occupied by the convolution kernel of the traditional convolution, when multiple feature extraction layers are used, multiple traditional convolution operations are replaced by channel-by-channel convolution operation and point-by-point convolution operation, which greatly reduces the memory space occupied by the command word offline recognition model. In addition, the command word offline recognition model has low hardware requirements and does not require improved hardware configuration, which can effectively reduce hardware costs. When recognizing voice with command words, there is no need to continuously send voice data to the cloud computing server, which effectively guarantees the privacy of users.

[0111] Example 3

[0112] See also Figure 5 In an embodiment of the present application, a command word recognition device 10 is proposed, including: a conversion module 11, a calculation module 12 and a determination module 13.

[0113] The conversion module 11 is used to convert the received command word into a standard matrix; the operation module 12 is used to use the various feature extraction layers of the command word offline recognition model to perform channel-by-channel convolution operations and point-by-point convolution operations on the standard matrix to obtain the feature tensor of the command word; the determination module 13 is used to use the output layer of the command word offline recognition model to determine the category of the command word based on the feature tensor.

[0114] The command word recognition device 10 disclosed in this embodiment is used to execute the command word recognition method described in the above embodiment through the coordinated use of a conversion module 11, an operation module 12 and a determination module 13. The implementation scheme and beneficial effects involved in the above embodiment are also applicable to this embodiment and will not be repeated here.

[0115] Example 4

[0116] An embodiment of the present application further provides a mobile terminal, including a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the command word recognition method described in the above embodiment is executed.

[0117] The mobile terminal can be a mobile phone, a pad, a Bluetooth headset, etc.

[0118] It should be noted that the memory of the Bluetooth call headset is very limited, and the command word offline recognition model disclosed in the present application can be saved on the Bluetooth call headset. On the one hand, since the command word offline recognition model disclosed in the present application only occupies a small amount of memory space and will not affect other functions of the Bluetooth call headset, the Bluetooth call headset can accurately recognize voice commands such as answering, hanging up and calling back; on the other hand, the command word offline recognition model disclosed in the present application has low hardware requirements and does not need to improve the hardware configuration of the Bluetooth call headset, which can effectively reduce the cost of the Bluetooth call headset; on the other hand, when recognizing voice with command words, there is no need to continuously send voice data to the cloud computing server, thereby ensuring the privacy of Bluetooth call headset users.

[0119] Example 5

[0120] An embodiment of the present application further provides a readable storage medium storing a computer program, which executes the command word recognition method described in the above embodiment when running on a processor.

[0121] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0123] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0124] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A command word recognition method, characterized in that: The method comprises: Convert the received command words into a standard matrix; Using each feature extraction layer of the command word offline recognition model to perform channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix to obtain a feature tensor of the command word; Determining the category of the command word based on the feature tensor of the command word using the output layer of the command word offline recognition model; The output layer includes a dimensionality reduction layer, a linear layer and a probability output layer, and the output layer using the command word offline recognition model determines the category of the command word based on the feature tensor, including: Converting the feature tensor of the command word into a feature vector using the dimension reduction layer; Utilizing the linear layer to calculate the probability that the feature vector is each category; The probability output layer is used to normalize the probabilities of various categories and output the categories of the command words and the probabilities corresponding to the categories.

2. The command word recognition method according to claim 1, characterized in that: The command word offline recognition model includes N feature extraction layers, each of which includes a channel-by-channel convolution operation layer, a point-by-point convolution operation layer, a normalization layer, an activation layer, and a maximum pooling layer. The method of using each feature extraction layer of the command word offline recognition model to perform channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix to obtain the feature tensor of the command word includes: Using the nth channel-by-channel convolution operation layer, the nth point-by-point convolution operation layer, the nth normalization layer, the nth activation layer and the nth maximum pooling layer of the nth feature extraction layer to extract features from the n-1th feature tensor to obtain the nth feature tensor, 1≤n≤N, and the zeroth feature tensor is the standard matrix; The Nth feature tensor is used as the feature tensor of the command word.

3. The command word recognition method according to claim 2, characterized in that: The method uses the nth channel-by-channel convolution operation layer, the nth point-by-point convolution operation layer, the nth normalization layer, the nth activation layer and the nth maximum pooling layer of the nth feature extraction layer to extract features from the n-1th feature tensor to obtain the nth feature tensor, including: Using the nth channel-by-channel convolution operation layer to perform a channel-by-channel convolution operation on the n-1th feature tensor to obtain an nth channel-by-channel convolution tensor; Using the nth point-by-point convolution operation layer to perform a point-by-point convolution operation on the nth channel-by-channel convolution tensor to obtain an nth point-by-point convolution tensor; Using the nth normalization layer to perform a normalization operation on the nth point-by-point convolution tensor to obtain an nth normalized feature tensor; Using the nth activation layer, performing an activation operation on the nth normalized feature tensor to obtain an nth activated feature tensor; The nth maximum pooling layer is used to perform an activation operation on the nth activated feature tensor to obtain the nth feature tensor.

4. The command word recognition method according to claim 2, characterized in that: The formula for each channel-by-channel convolution operation is as follows: x d is the feature tensor of the input channel-by-channel convolution layer, wd is the convolution kernel of the channel-by-channel convolution operation, DW ijk (x d ) is x d The output of the channel-by-channel convolution operation with wd, w H is the height of the convolution kernel wd, w W is the width of the convolution kernel wd, stride h is the convolution kernel wd at x d The step in the height direction, stride w is the convolution kernel wd at x d The step in the width direction, i is the index in the height direction of the feature tensor, j is the index in the width direction of the feature tensor, i' is the index in the height direction of the convolution kernel, j' is the index in the height direction of the convolution kernel, and K represents the kth channel; The formula for each point-by-point convolution operation is as follows: x p is the feature tensor of the input point-by-point convolution layer, wp is the convolution kernel of the point-by-point convolution operation, x C is the number of channels of the input feature, b k is the convolution bias of the kth channel, PW ijk (x p ) is x p The output result of the point-by-point convolution operation with wd, and k' is the summation index.

5. The command word recognition method according to claim 2, characterized in that: The value range of N is 4 to 8.

6. The command word recognition method according to claim 1, characterized in that: The step of converting the received command word into a standard matrix includes: Performing pre-emphasis processing on the audio of the received command word to obtain a standard time-domain audio signal; Converting the time-domain audio signal into a frequency-domain signal using Fourier transform; Performing discrete cosine transformation on the frequency domain signal to obtain a standard matrix.

7. A command word recognition device, characterized in that: The device comprises: A conversion module, used for converting the received command words into a standard matrix; A calculation module, used for performing channel-by-channel convolution operation and point-by-point convolution operation on the standard matrix by using each feature extraction layer of the command word offline recognition model to obtain a feature tensor of the command word; A determination module, configured to determine the category of the command word based on the feature tensor using an output layer of the command word offline recognition model; Wherein, the output layer includes a dimensionality reduction layer, a linear layer and a probability output layer, and the determination module is further used for: The feature tensor of the command word is converted into a feature vector by using the dimensionality reduction layer; the probability of the feature vector being each category is calculated by using the linear layer; the probability output layer is used to normalize the probability of each category and output the category of the command word and the probability corresponding to the category.

8. A mobile terminal, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the command word recognition method according to any one of claims 1 to 6 is executed.

9. A readable storage medium, characterized in that: It stores a computer program, which, when running on a processor, executes the command word recognition method according to any one of claims 1 to 6.

Images