Speech recognition method, device, equipment and storage medium for target device

Abstract

The present invention provides a speech recognition method, apparatus, device, and storage medium for a target device. The method comprises: acquiring an audio signal, obtaining acoustic features of the audio signal, inputting the acoustic features into a first acoustic model, and obtaining a phoneme-level feature and a phoneme-level probability vector associated with the phoneme-level feature; if the current operating mode of the target device is a wake-up mode, inputting the phoneme-level probability vector into a preset decoder to recognize a wake-up word, and converting the operating mode of the target device to a command word mode when the wake-up word is recognized; if the current operating mode of the target device is a command word mode, inputting the phoneme-level feature into a second acoustic model to trigger the target device to execute an action corresponding to the command word when a command word is recognized. The present invention can solve the current crosstalk problem caused by unequal length comparison when command words of different lengths pass through a decoding network, effectively improving the accuracy of command word recognition.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition method, apparatus, device and storage medium for a target device. Background Art

[0002] Existing voice wake-up and command word methods generally consist of two parts: an acoustic model and a decoding network. After extracting the acoustic features of the speech data, they are input into a pre-built acoustic model to obtain posterior probabilities. The posterior probabilities are then input into the decoding network to obtain the acoustic scores of each wake-up word, command word path, and absorption path in the network. Finally, the acoustic scores of each path are compared, and the path with the highest score is the recognition result. However, this approach results in unequal length comparisons of command words of different lengths when passing through the decoding network, resulting in severe and difficult-to-solve crosstalk between different command words. Summary of the Invention

[0003] The present invention provides a speech recognition method, apparatus, device and storage medium for a target device, which are used to solve the problem in the prior art that command words of different lengths have unequal length comparisons when passing through a decoding network, resulting in serious and difficult-to-solve crosstalk between different command words.

[0004] In a first aspect, the present invention provides a speech recognition method for a target device, the method comprising:

[0005] Acquiring an audio signal and obtaining acoustic features of the audio signal;

[0006] Inputting the acoustic feature into a trained first acoustic model to obtain a phoneme-level feature and a phoneme-level probability vector associated with the phoneme-level feature;

[0007] If the current operating mode of the target device is a wake-up mode, inputting the phoneme-level probability vector into a preset decoder to identify a wake-up word, and converting the operating mode of the target device to a command word mode if the wake-up word is identified;

[0008] If the current working mode of the target device is the command word mode, the phoneme-level features are input into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and the command word is identified based on the whole-word-level probability vector to trigger the target device to perform the action corresponding to the command word when the command word is identified.

[0009] In one embodiment of the present invention, inputting the phoneme-level features corresponding to each frame signal into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and identifying the command word based on the whole-word-level probability vector includes:

[0010] Based on a convolutional neural network, the second acoustic model obtains a syllable-level feature for the speech signal and a syllable-level probability vector associated with the syllable-level feature according to the phoneme-level feature;

[0011] Based on a global average pooling method, the second acoustic model obtains a whole-word-level feature for the speech signal and a whole-word-level probability vector associated with the whole-word-level feature according to the syllable-level feature;

[0012] The command word corresponding to the element with the largest probability value in the whole word level probability vector is used as the command word finally recognized.

[0013] In one embodiment of the present invention, acquiring the audio signal and obtaining the acoustic features of the audio signal includes:

[0014] Performing noise reduction and audio energy-based sliding window screening processing on the audio signal to obtain an audio signal of a speech segment as the speech signal and filtering out an audio signal of a non-speech segment;

[0015] Acoustic features of the speech signal are extracted based on energy spectrum features to obtain acoustic features of each frame of the signal.

[0016] In one embodiment of the present invention, inputting the acoustic feature into the trained first acoustic model to obtain the phoneme-level feature and the phoneme-level probability vector associated with the phoneme-level feature includes:

[0017] For each frame signal acoustic feature, the first acoustic model converts the acoustic feature into a phoneme-level feature of a triphone pattern based on a preset triphone rule and obtains a phoneme-level probability vector associated with the phoneme-level feature;

[0018] The phoneme-level probability vector indicates the probability value of each type of three-phoneme unit in the phoneme-level feature.

[0019] In one embodiment of the present invention, inputting the phoneme-level probability vector into a preset decoder to recognize the wake-up word includes:

[0020] Set the initial position of the comparison to 0, and perform the following operations on the phoneme-level probability vector corresponding to each frame signal in sequence:

[0021] Based on the phoneme-level probability vector corresponding to the frame signal, starting from the initial comparison position, the probability values corresponding to the phoneme units in the wake-up word path in the decoder are sequentially compared with the maximum probability values of all phoneme units in the absorption path in the decoder;

[0022] If the probability value of the currently compared phoneme unit in the wake-up word path is greater than the maximum probability value, then the probability values corresponding to subsequent phoneme units in the wake-up word path are sequentially compared with the maximum probability value;

[0023] If it is found that the probability value of the currently compared phoneme unit in the wake-up word path is not greater than the maximum probability value, the comparison of the frame signal is terminated and the initial position of the comparison is updated to the serial number of the currently compared phoneme unit in the probability vector, and the above comparison operation is continued on the phoneme-level probability vector corresponding to the next frame signal until the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, or the comparison of all frame signals is completed;

[0024] The wake-up word path is composed of all the phoneme units that constitute the wake-up word in series, and the absorption path is composed of all the phoneme units except all the phoneme units that constitute the wake-up word, and each phoneme unit except all the phoneme units that constitute the wake-up word constitutes an absorption path.

[0025] In one embodiment of the present invention, inputting the phoneme-level probability vector into a preset decoder to recognize the wake-up word further includes:

[0026] If the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, it means that the wake-up word is recognized from the voice signal.

[0027] In one embodiment of the present invention, the method includes:

[0028] The first acoustic model is trained based on the following method:

[0029] Using the first preset voice data to train the preset acoustic model to distinguish the voice data other than the wake-up word;

[0030] and using the second preset voice data to perform intensive training on the preset acoustic model to distinguish relevant phoneme units of the wake-up word;

[0031] Clustering the output phoneme units to obtain classifications of N phoneme units, and using the classifications of the N phoneme units as N output nodes of the preset acoustic model;

[0032] The first preset voice data is general corpus data that does not contain a wake-up word, and the second preset voice data is corpus data that contains a wake-up word.

[0033] In one embodiment of the present invention, the method further includes:

[0034] The second acoustic model is trained based on the following method:

[0035] Inputting the phoneme-level features output by the first acoustic model into the convolution layer and the pooling layer of the second acoustic model to obtain syllable-level features;

[0036] Inputting the syllable-level feature into the fully connected layer of the second acoustic model to obtain a corresponding syllable-level probability vector;

[0037] Inputting the syllable-level features into the global average pooling layer of the second acoustic model to obtain corresponding whole-word-level features;

[0038] Inputting the whole word level feature into the fully connected layer of the second acoustic model to obtain a corresponding whole word level probability vector;

[0039] The syllable-level probability vector and the preset syllable-level label are calculated according to preset rules to obtain a syllable-level loss function, and the whole-word-level probability vector and the preset whole-word-level label vector are calculated according to preset rules to obtain a whole-word-level loss function;

[0040] Calculating the phoneme-level probability vector obtained by the first acoustic model and the preset phoneme-level label vector according to preset rules to obtain a phoneme-level loss function;

[0041] With the minimization of the syllable-level loss function, the whole-word-level loss function, and the phoneme-level loss function as the optimization goal, the first acoustic model and the second acoustic model are jointly trained until the convergence condition is met, thereby obtaining the trained first acoustic model and the second acoustic model.

[0042] In a second aspect, the present invention provides a speech recognition apparatus for a target device, the apparatus comprising:

[0043] A preprocessing module, configured to acquire an audio signal and obtain acoustic features of the audio signal;

[0044] An identification module, wherein the identification module is used to:

[0045] Inputting the acoustic feature into a trained first acoustic model to obtain a phoneme-level feature and a phoneme-level probability vector associated with the phoneme-level feature;

[0046] If the current operating mode of the target device is a wake-up mode, inputting the phoneme-level probability vector into a preset decoder to identify a wake-up word, and converting the operating mode of the target device to a command word mode if the wake-up word is identified;

[0047] If the current working mode of the target device is the command word mode, the phoneme-level features are input into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and the command word is identified based on the whole-word-level probability vector to trigger the target device to perform the action corresponding to the command word when the command word is identified.

[0048] In one embodiment of the present invention, the identification module is further configured to:

[0049] Based on a convolutional neural network, the second acoustic model obtains a syllable-level feature for the speech signal and a syllable-level probability vector associated with the syllable-level feature according to the phoneme-level feature;

[0050] Based on a global average pooling method, the second acoustic model obtains a whole-word-level feature for the speech signal and a whole-word-level probability vector associated with the whole-word-level feature according to the syllable-level feature;

[0051] The command word corresponding to the element with the largest probability value in the whole word level probability vector is used as the command word finally recognized.

[0052] In one embodiment of the present invention, the pre-processing module is further configured to:

[0053] Performing noise reduction and audio energy-based sliding window screening processing on the audio signal to obtain an audio signal of a speech segment as the speech signal and filtering out an audio signal of a non-speech segment;

[0054] Acoustic features of the speech signal are extracted based on energy spectrum features to obtain acoustic features of each frame of the signal.

[0055] In one embodiment of the present invention, the identification module is further configured to:

[0056] For each frame signal acoustic feature, the first acoustic model converts the acoustic feature into a phoneme-level feature of a triphone pattern based on a preset triphone rule and obtains a phoneme-level probability vector associated with the phoneme-level feature;

[0057] The phoneme-level probability vector indicates the probability value of each type of three-phoneme unit in the phoneme-level feature.

[0058] In one embodiment of the present invention, the identification module is further configured to:

[0059] Set the initial position of the comparison to 0, and perform the following operations on the phoneme-level probability vector corresponding to each frame signal in sequence:

[0060] Based on the phoneme-level probability vector corresponding to the frame signal, starting from the initial comparison position, the probability values corresponding to the phoneme units in the wake-up word path in the decoder are sequentially compared with the maximum probability values of all phoneme units in the absorption path in the decoder;

[0061] If the probability value of the currently compared phoneme unit in the wake-up word path is greater than the maximum probability value, then the probability values corresponding to subsequent phoneme units in the wake-up word path are sequentially compared with the maximum probability value;

[0062] If it is found that the probability value of the currently compared phoneme unit in the wake-up word path is not greater than the maximum probability value, the comparison of the frame signal is terminated and the initial position of the comparison is updated to the serial number of the currently compared phoneme unit in the probability vector, and the above comparison operation is continued on the phoneme-level probability vector corresponding to the next frame signal until the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, or the comparison of all frame signals is completed;

[0063] The wake-up word path is composed of all the phoneme units that constitute the wake-up word in series, and the absorption path is composed of all the phoneme units except all the phoneme units that constitute the wake-up word, and each phoneme unit except all the phoneme units that constitute the wake-up word constitutes an absorption path.

[0064] In one embodiment of the present invention, the apparatus further includes a first training module, and the first training module trains the first acoustic model based on the following method:

[0065] Using the first preset voice data to train the preset acoustic model to distinguish the voice data other than the wake-up word;

[0066] and using the second preset voice data to perform intensive training on the preset acoustic model to distinguish relevant phoneme units of the wake-up word;

[0067] Clustering the output phoneme units to obtain classifications of N phoneme units, and using the classifications of the N phoneme units as N output nodes of the preset acoustic model;

[0068] The first preset voice data is general corpus data that does not contain a wake-up word, and the second preset voice data is corpus data that contains a wake-up word.

[0069] In one embodiment of the present invention, the apparatus further includes a second training module, and the second training module trains the second acoustic model based on the following method:

[0070] Inputting the phoneme-level features output by the first acoustic model into the convolution layer and the pooling layer of the second acoustic model to obtain syllable-level features;

[0071] Inputting the syllable-level feature into the fully connected layer of the second acoustic model to obtain a corresponding syllable-level probability vector;

[0072] Inputting the syllable-level features into the global average pooling layer of the second acoustic model to obtain corresponding whole-word-level features;

[0073] Inputting the whole word level feature into the fully connected layer of the second acoustic model to obtain a corresponding whole word level probability vector;

[0074] The syllable-level probability vector and the preset syllable-level label are calculated according to preset rules to obtain a syllable-level loss function, and the whole-word-level probability vector and the preset whole-word-level label vector are calculated according to preset rules to obtain a whole-word-level loss function;

[0075] Calculating the phoneme-level probability vector obtained by the first acoustic model and the preset phoneme-level label vector according to preset rules to obtain a phoneme-level loss function;

[0076] With the minimization of the syllable-level loss function, the whole-word-level loss function, and the phoneme-level loss function as the optimization goal, the first acoustic model and the second acoustic model are jointly trained until the convergence condition is met, thereby obtaining the trained first acoustic model and the second acoustic model.

[0077] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the speech recognition method for a target device as described above are implemented.

[0078] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for speech recognition of a target device.

[0079] The speech recognition method, apparatus, device and storage medium for a target device provided by the present invention construct a first acoustic model and a second acoustic model, so that the first acoustic model and the second acoustic model have more powerful feature learning capabilities, can better learn the differences between different command words, and reduce the crosstalk rate of command word recognition.

[0080] Moreover, in the command word mode, the present invention does not need to go through the decoding network, but directly identifies the command word category to which the speech segment belongs through the second acoustic model, which solves the current crosstalk problem caused by unequal length comparison when command words of different lengths pass through the decoding network, and effectively improves the accuracy of command word recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying any creative work.

[0082] Figure 1 is a flow chart of a speech recognition method for a target device provided by an embodiment of the present invention;

[0083] Figure 2 is a schematic diagram of the process of training the first acoustic model according to an embodiment of the present invention;

[0084] Figure 3 is a schematic diagram of a decoding network provided by an embodiment of the present invention;

[0085] Figure 4 is a schematic diagram of a decoding process provided by an embodiment of the present invention;

[0086] FIG5( a ) is a schematic diagram of a process for training a second acoustic model according to an embodiment of the present invention;

[0087] FIG5( b ) is a schematic diagram of training a first acoustic model and a second acoustic model according to an embodiment of the present invention;

[0088] Figure 6 is a flow chart of a speech recognition method for a target device provided by an embodiment of the present invention;

[0089] Figure 7 is a structural diagram of a speech recognition device for a target device provided by an embodiment of the present invention;

[0090] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0091] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0092] The terms "first," "second," and the like in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments described herein can be practiced in an order other than that shown or described herein.

[0093] As AI technologies mature, more and more smart devices are entering users' lives, making human-machine interactions increasingly commonplace. Voice is considered one of the most natural ways for people to communicate. Spoken language possesses unique properties. As a naturally learned part of human development, it offers a low barrier to entry in any interactive scenario, making it a promising future trend in human-machine interaction. Many smart voice-activated devices (such as air conditioners) now offer both voice wake-up and command recognition. This allows them to not only understand their names but also execute commands spoken by the user (e.g., "Set the temperature to 26°C"). This eliminates the need for users to press various buttons, significantly improving the user experience.

[0094] The following combination Figures 1-8 The present invention describes a speech recognition method, apparatus, device and storage medium for a target device.

[0095] See also Figure 1 , Figure 1 1 is a flow chart of a speech recognition method for a target device provided by an embodiment of the present invention. The present invention provides a speech recognition method for a target device, the method comprising:

[0096] Step 101: Acquire an audio signal and obtain acoustic features of the audio signal.

[0097] Exemplarily, the audio signal is filtered to obtain a speech signal, and the speech signal is framed and subjected to feature extraction processing to obtain acoustic features of each frame signal.

[0098] Step 102: Input the acoustic features into a trained first acoustic model to obtain phoneme-level features and phoneme-level probability vectors associated with the phoneme-level features.

[0099] Step 103: If the current working mode of the target device is the wake-up mode, the phoneme-level probability vector is input into a preset decoder to identify the wake-up word, and when the wake-up word is identified, the working mode of the target device is converted to the command word mode.

[0100] Step 104: If the current working mode of the target device is the command word mode, the phoneme-level features are input into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and the command word is identified based on the whole-word-level probability vector to trigger the target device to perform the action corresponding to the command word when the command word is identified.

[0101] The speech recognition method for a target device described in the present invention has two modes: voice wake-up and command word recognition. When the target device is in standby mode, it is in wake-up mode. Upon receiving voice data containing a wake-up word, the target device wakes up successfully and switches to command word mode to recognize the command word in the voice data and execute the corresponding instruction.

[0102] Furthermore, since the first acoustic model and the second acoustic model of the present invention adopt a step-by-step multi-scale modeling method, that is, the first acoustic model adopts a phoneme-level modeling method, and the second acoustic model adopts a syllable-level and whole-word-level modeling method, they can better learn the differences between different command words and effectively improve the accuracy of command word recognition.

[0103] The above steps 101 to 104 are described in detail below.

[0104] In the above step 101, an audio signal is acquired to obtain the acoustic features of the audio signal. Specifically, the following steps are performed:

[0105] Step 1011 , performing noise reduction and audio energy-based sliding window screening processing on the audio signal to obtain the audio signal of the speech segment as the speech signal and filtering out the audio signal of the non-speech segment.

[0106] For example, the audio signal can be obtained by picking up sound through a microphone, and the sound pickup device can be a tablet computer, a smart speaker, a television, or an air conditioner.

[0107] For example, the audio after noise reduction picked up by a microphone is passed through a sliding window to calculate the audio energy within the window to distinguish between speech segments and non-speech segments. Only speech segments can be sent to the next level to extract the acoustic features of the speech signal.

[0108] Exemplarily, the VAD model uses an energy and model-based detection method to determine whether a speech segment exists in the acquired audio signal and extract the speech segment from the audio signal. For example, the VAD model calculates the likelihood ratio of speech and noise for each frame of the acquired audio signal and determines whether the audio signal is a speech segment based on the likelihood ratio. If the audio signal is a non-speech segment, such as silence or ambient noise, it is filtered out. If the audio signal is a speech segment, the speech segment audio signal is used as the speech signal.

[0109] Step 1012: extract acoustic features of the speech signal based on the energy spectrum features to obtain acoustic features of each frame signal.

[0110] Exemplarily, the acoustic features used for wake-up word recognition may be spectral features of a speech signal, such as Mel Frequency Cepstrum Coefficient (MFCC) features or Perceptual Linear Predictive (PLP) features.

[0111] To extract the signal, the speech signal is first framed. Pre-emphasized, the framed signal is then transformed into the frequency domain using an FFT (Fast Fourier Transform), and then squared and logarithmized to obtain the energy spectrum. If the input audio signal is multi-microphone speech passing through a microphone array, beamforming can be used to reduce the noise of the multi-microphone speech and combine it into an enhanced single-microphone speech. The above feature extraction process is then repeated to obtain the spectral characteristics of each frame of the speech signal.

[0112] In the above step 102, the acoustic features are input into the trained first acoustic model to obtain phoneme-level features and phoneme-level probability vectors associated with the phoneme-level features. Specifically, the process includes:

[0113] Step 1021: For the acoustic features of each frame signal, the first acoustic model converts the acoustic features into phoneme-level features of a triphone pattern based on a preset triphone rule and obtains a phoneme-level probability vector associated with the phoneme-level features.

[0114] The phoneme-level probability vector indicates the probability value of each type of three-phoneme unit in the phoneme-level feature.

[0115] For example, the first acoustic model is usually characterized by a GMM-HMM model, that is, a Hidden Markov Model (HMM) is used to characterize the state transition between speech units, and a Gaussian Mixture Model (GMM) is used to characterize the state output probability of a speech unit. Initially, the target device is in wake-up mode. Since the first acoustic model is mostly used in low-power scenarios, the embodiment of the present invention may also adopt a shuffle-net structure with fewer parameters. Shuffle-net is a computationally efficient CNN (convolutional neural network model). The design goal of ShuffleNet is also to use limited computing resources to achieve the best model accuracy. This requires a good balance between speed and accuracy, which can greatly reduce the computational complexity of the model while maintaining accuracy.

[0116] Exemplarily, the first acoustic model performs phoneme-level classification, and the modeling unit is a phoneme. Specifically, in the embodiment of the present invention, triphone units are used for modeling. The triphone units represent context-dependent phoneme units of each phoneme unit.

[0117] Exemplarily, the input of the first acoustic model is the acoustic features of each frame signal. When training the first acoustic model, the training data used are the first preset voice data and the second preset voice data.

[0118] For example, Figure 2 As shown, Figure 2 : is a flow chart of training the first acoustic model according to an embodiment of the present invention. The embodiment of the present invention trains the first acoustic model based on the following method:

[0119] Step 201: Use first preset voice data to train the preset acoustic model to distinguish the voice data other than the wake-up word.

[0120] In step 202, the preset acoustic model is reinforced with training using the second preset voice data to distinguish the relevant phoneme units of the wake-up word.

[0121] Step 203 , clustering the output phoneme units to obtain classifications of N phoneme units, and using the classifications of the N phoneme units as N output nodes of the preset acoustic model.

[0122] The first preset voice data is general corpus data that does not contain a wake-up word, and the second preset voice data is corpus data that contains a wake-up word.

[0123] For example, assuming the wake-up word is "iFLYTEK", and its phoneme units are "k", "e", "d", "a", "x", "un", "f", and "ei", then the three-phoneme units can be represented as "ked", "eda", "dax", "ax-un", "x-un-f", and "un-f-ei". The three-phoneme model can be used to obtain context-related information. Since the scale of all Chinese three-phoneme units is very large, the embodiment of the present invention will perform clustering processing when used, that is, the number of clusters N obtained is 3004, indicating that the output nodes N of the acoustic model are 3004, and each node represents a type of three-phoneme unit.

[0124] Therefore, after the acoustic features of N frames of speech signals pass through the first acoustic model, the probability value of each frame signal being divided into each type of three-phoneme unit will be obtained, forming an N*3004-dimensional probability vector. This probability vector will be input into the subsequent decoding network to obtain the recognition result of whether the speech signal contains the wake-up word.

[0125] In the above step 103, if the current working mode of the target device is the wake-up mode, the phoneme-level probability vector is input into a preset decoder to identify the wake-up word, and if the wake-up word is identified, the working mode of the target device is converted to the command word mode. Specifically, it includes:

[0126] Set the initial position of the comparison to 0, and perform the following operations on the phoneme-level probability vector corresponding to each frame signal in sequence:

[0127] Step 1031, based on the phoneme-level probability vector corresponding to the frame signal, starting from the initial comparison position, the probability values corresponding to the phoneme units in the wake-up word path in the decoder are compared in sequence with the maximum probability values of all phoneme units in the absorption path in the decoder.

[0128] Step 1032: If the probability value of the currently compared phoneme unit in the wake-up word path is greater than the maximum probability value, then the probability values corresponding to subsequent phoneme units in the wake-up word path are sequentially compared with the maximum probability value.

[0129] Step 1033: If it is found that the probability value of the currently compared phoneme unit in the wake-up word path is not greater than the maximum probability value, the comparison of the frame signal is terminated and the initial position of the comparison is updated to the serial number of the currently compared phoneme unit in the probability vector, and the above comparison operation is continued on the phoneme-level probability vector corresponding to the next frame signal until the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, or the comparison of all frame signals is terminated.

[0130] The wake-up word path is composed of all the phoneme units that constitute the wake-up word in series, and the absorption path is composed of all the phoneme units except all the phoneme units that constitute the wake-up word, and each phoneme unit except all the phoneme units that constitute the wake-up word constitutes an absorption path.

[0131] For example, the wake-up word path is composed of all the three-phone units of the wake-up word in the 3004 types of three-phone units connected in series. The absorption path is composed of the other three-phone units except the wake-up word three-phone units, and each type of three-phone unit is an absorption path.

[0132] Step 1034: If the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, it means that the wake-up word is recognized from the voice signal.

[0133] The decoding process of the wake-up word in steps 1031 to 1034 is as follows: Figure 3 、 Figure 4 As shown, Figure 3 is a schematic diagram of a decoding network provided by an embodiment of the present invention, Figure 4 It is a schematic diagram of the decoding process provided by an embodiment of the present invention.

[0134] like Figure 3 As shown, the decoding network (i.e., decoder) adopts a keyword (wake-up word path) + filler (absorption path) network. Among them, the upper dotted box is the wake-up word path, and the lower dotted box is the absorption path. The probability of each phoneme unit corresponding to each frame of speech signal is input into the decoding network for decoding, with a total of N*3004 (N represents a total of N frames of speech signal) dimensional probability vector. The decoding network uses a dynamic programming algorithm to calculate the acoustic score of the acoustic features of each phoneme unit on each path, and takes the path with the highest acoustic score as the optimal path. If the optimal path is the wake-up word path, the recognition result is the wake-up word on the path; if the recognition result is the absorption path, the recognition result is a non-wake-up word.

[0135] For example, Figure 4 As shown in the figure, assuming that the wake-up word is "iFLYTEK", its three-phoneme units are "ked", "eda", "dax", "ax-un", "x-un-f", and "un-f-ei", and the corresponding probability values are K0 to K5.

[0136] After inputting the N * 3004 - dimensional probability vector obtained in the above step 102 into the decoding network, starting from the first frame of the N frames of voice signals obtained, compare the probability value K0 of the first triphone unit in the wake - up word path with the maximum probability value F among all absorption paths. If F >= K0, it is considered that the wake - up word has not started to appear, and continue to compare K0 and F of the next frame. If F < K0, then compare the probability value K1 of the next triphone unit in the wake - up word path with F, and so on, until the probability value K5 of the last triphone unit in the wake - up word path is greater than F, then it is considered that there is a wake - up word in the voice signal and the wake - up is successful.

[0137] In the above step 104, if the current working mode of the target device is the command word mode, then input the phoneme - level feature into the second acoustic model to obtain a whole - word - level probability vector based on the phoneme - level feature, and identify the command word based on the whole - word - level probability vector, so as to trigger the target device to execute the action corresponding to the command word when the command word is recognized. Specifically, it includes:

[0138] Step 1041, based on a convolutional neural network, the second acoustic model obtains a syllable - level feature for the voice signal and a syllable - level probability vector associated with the syllable - level feature according to the phoneme - level feature.

[0139] Step 1042, based on the global average pooling method, the second acoustic model obtains a whole - word - level feature for the voice signal and a whole - word - level probability vector associated with the whole - word - level feature according to the syllable - level feature.

[0140] Step 1043, take the command word corresponding to the element with the largest probability value in the whole - word - level probability vector as the finally recognized command word.

[0141] Therefore, after passing through the decoding network of the above steps 1031 - 1034, if a wake - up word is recognized, enter the command word mode, and output the command word recognition result through the second acoustic model. For the second acoustic model, the input of the second acoustic model is the phoneme - level hidden - layer feature cached by the first acoustic model, and the training objective is to perform syllable - level classification and whole - word - level classification, which is combined with the phoneme - level classification of the first acoustic model to form a hierarchical multi - scale modeling model, so that the hierarchical multi - scale modeling model composed of the first acoustic model and the second acoustic model has a more powerful feature learning ability, can better learn the differences between different command words, and reduce the crosstalk rate of command word recognition.

[0142] Exemplarily, the syllable-level classification of the second acoustic model adopts syllable modeling, which is at the character level. For example, if the command word is "turn on the air conditioner", the syllable modeling units are represented as "da", "kai", "kong", and "tiao". In the embodiments of the present invention, the total number of syllable modeling units for all Chinese characters is 1326, but the present invention is not limited to 1326. The whole-word-level classification of the second acoustic model adopts whole-word modeling. A command word, regardless of the number of characters, is one modeling unit, and the number of modeling units is the number of command words.

[0143] Exemplarily, FIG. 5(a) is a schematic flowchart of training the second acoustic model in the embodiments of the present invention, and FIG. 5(b) is a schematic diagram of training the first acoustic model and the second acoustic model in the embodiments of the present invention. As shown in FIGS. 5(a) and 5(b). The schematic flowchart of training the second acoustic model in the embodiments of the present invention. The embodiments of the present invention train the second acoustic model based on the following method:

[0144] Step 501, input the phoneme-level features output by the first acoustic model into the convolutional layer and pooling layer of the second acoustic model to obtain syllable-level features.

[0145] For example, after inputting the phoneme-level features (with a size of N*64, where N represents the number of frames of the speech signal) output by the first acoustic model into the second acoustic model, it first passes through a convolutional neural network with a preset number of layers (for example, three layers) (such as Figure 6 the CNN+Pooling shown, where the convolutional kernel size of the convolutional layer CNN is 3 and the stride of the pooling layer Pooling is 2), to obtain syllable-level features with a size of (N / 8)*64.

[0146] Step 502, input the syllable-level features into the fully connected layer of the second acoustic model to obtain the corresponding syllable-level probability vector.

[0147] For example, the syllable-level features with a size of (N / 8)*64 output from step 501, on the one hand, pass through a fully connected layer (Fully connect) with a size of 64*1326 (assuming the total number of syllable modeling units is 1326) to obtain a syllable-level probability vector with a size of (N / 8)*1326.

[0148] Step 503, input the syllable-level features into the global average pooling layer of the second acoustic model to obtain the corresponding whole-word-level features.

[0149] For example, the syllable-level features with a size of (N / 8)*64 output from step 501, on the other hand, pass through the global average pooling layer (AvgPooling) to obtain whole-word-level features with a size of 1*64.

[0150] Step 504: Input the whole-word-level features into the fully connected layer of the second acoustic model to obtain the corresponding whole-word-level probability vector.

[0151] For example, the whole word level features obtained in step 503 are passed through a fully connected layer (Fullyconnect) of size 64*W to obtain a whole word level probability vector of size 1*W, where W represents the number of command words.

[0152] In step 505, the syllable-level probability vector and the preset syllable-level label are calculated according to preset rules to obtain a syllable-level loss function, and the whole-word-level probability vector and the preset whole-word-level label vector are calculated according to preset rules to obtain a whole-word-level loss function.

[0153] Step 506 : The phoneme-level probability vector obtained by the first acoustic model and the preset phoneme-level label vector are calculated according to preset rules to obtain a phoneme-level loss function.

[0154] Step 507: With minimizing the syllable-level loss function, the whole-word-level loss function, and the phoneme-level loss function as the optimization goal, the first acoustic model and the second acoustic model are jointly trained until the convergence condition is met, thereby obtaining the trained first acoustic model and the second acoustic model.

[0155] Therefore, during training, the optimization goal is to minimize the syllable-level loss function (loss) of the second acoustic model + the whole-word-level loss function (loss) of the second acoustic model + the phoneme-level loss function (loss) of the first acoustic model. During testing, there is no need to go through the decoding network. The 1*W whole-word-level probability vector (W is the number of command words) obtained by the whole-word modeling module of the command word multi-level modeling model is directly used. The command word corresponding to the one-dimensional probability value with the largest probability value in the whole-word-level probability vector is the command word recognition result. The present invention can avoid the crosstalk problem caused by unequal length comparison when command words of different lengths pass through the decoding network in the prior art, effectively improving the accuracy of command word recognition.

[0156] The speech recognition method for a target device according to the present invention is described below by taking an example.

[0157] Figure 6 FIG. 1 is a flow chart of a speech recognition method for a target device provided by an embodiment of the present invention. Figure 6 shown.

[0158] Step 601: Acquire an audio signal and pre-process the audio signal.

[0159] For example, an audio signal is acquired through a microphone of a sound pickup device such as a tablet computer, smart speaker, TV, or air conditioner. The audio signal is then pre-processed by noise reduction, sliding window, etc. to obtain an audio signal of a speech segment, which is a speech signal.

[0160] Step 602: Frame the pre-processed speech signal and perform feature extraction to obtain acoustic features of each frame signal.

[0161] Step 603: Input the acoustic features of each frame signal into the trained first acoustic model to obtain the phoneme-level features corresponding to each frame signal and the phoneme-level probability vector associated with the phoneme-level features.

[0162] Step 604: If the current working mode of the target device is the awake mode, the phoneme-level features corresponding to each frame signal and the phoneme-level probability vector associated with the phoneme-level features are input into a preset decoding network.

[0163] In step 605 , the decoding network decodes the phoneme-level features corresponding to each frame signal and the phoneme-level probability vector associated with the phoneme-level features to identify the wake-up word.

[0164] In step 606, the decoding network uses a dynamic programming algorithm to calculate the acoustic score of the acoustic features of each phoneme unit on each path, and the path with the highest acoustic score is taken as the optimal path.

[0165] Step 607: Determine whether the voice signal contains the wake-up word. If yes, proceed to step 608; otherwise, return to step 604.

[0166] Step 608: convert the operating mode of the target device to a command word mode when the wake-up word is recognized.

[0167] Step 609: If the current working mode of the target device is the command word mode, the phoneme-level features corresponding to each frame signal output by the first acoustic model are input into the second acoustic model.

[0168] In step 610 , the second acoustic model obtains a whole-word-level probability vector based on the phoneme-level features, and recognizes the command word based on the whole-word-level probability vector.

[0169] Step 611: The second acoustic model outputs a command word recognition result.

[0170] When the second acoustic model recognizes a command word, the target device is triggered to execute an action corresponding to the command word.

[0171] The speech recognition apparatus for a target device provided by the present invention is described below. The speech recognition apparatus for a target device described below and the speech recognition method for a target device described above can be referred to in correspondence with each other.

[0172] Figure 7 FIG. 1 is a structural diagram of a speech recognition device for a target device provided by an embodiment of the present invention. Figure 7 The embodiment of the present invention provides a speech recognition apparatus for a target device, wherein the apparatus 700 includes a pre-processing module 710 and a recognition module 720 .

[0173] The pre-processing module 710 is configured to acquire an audio signal and obtain acoustic features of the audio signal.

[0174] The identification module 720 is configured to:

[0175] Inputting the acoustic feature into a trained first acoustic model to obtain a phoneme-level feature and a phoneme-level probability vector associated with the phoneme-level feature;

[0176] If the current operating mode of the target device is a wake-up mode, inputting the phoneme-level probability vector into a preset decoder to identify a wake-up word, and converting the operating mode of the target device to a command word mode if the wake-up word is identified;

[0177] If the current working mode of the target device is the command word mode, the phoneme-level features are input into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and the command word is identified based on the whole-word-level probability vector to trigger the target device to perform the action corresponding to the command word when the command word is identified.

[0178] Exemplarily, the identification module 720 is further configured to:

[0179] Based on a convolutional neural network, the second acoustic model obtains a syllable-level feature for the speech signal and a syllable-level probability vector associated with the syllable-level feature according to the phoneme-level feature;

[0180] Based on a global average pooling method, the second acoustic model obtains a whole-word-level feature for the speech signal and a whole-word-level probability vector associated with the whole-word-level feature according to the syllable-level feature;

[0181] The command word corresponding to the element with the largest probability value in the whole word level probability vector is used as the command word finally recognized.

[0182] Exemplarily, the pre-processing module 710 is further configured to:

[0183] Performing noise reduction and audio energy-based sliding window screening processing on the audio signal to obtain an audio signal of a speech segment as the speech signal and filtering out an audio signal of a non-speech segment;

[0184] Acoustic features of the speech signal are extracted based on energy spectrum features to obtain acoustic features of each frame of the signal.

[0185] Exemplarily, the identification module 720 is further configured to:

[0186] For each frame signal acoustic feature, the first acoustic model converts the acoustic feature into a phoneme-level feature of a triphone pattern based on a preset triphone rule and obtains a phoneme-level probability vector associated with the phoneme-level feature;

[0187] The phoneme-level probability vector indicates the probability value of each type of three-phoneme unit in the phoneme-level feature.

[0188] Exemplarily, the identification module 720 is further configured to:

[0189] Set the initial position of the comparison to 0, and perform the following operations on the phoneme-level probability vector corresponding to each frame signal in sequence:

[0190] Based on the phoneme-level probability vector corresponding to the frame signal, starting from the initial comparison position, the probability values corresponding to the phoneme units in the wake-up word path in the decoder are sequentially compared with the maximum probability values of all phoneme units in the absorption path in the decoder;

[0191] If the probability value of the currently compared phoneme unit in the wake-up word path is greater than the maximum probability value, then the probability values corresponding to subsequent phoneme units in the wake-up word path are sequentially compared with the maximum probability value;

[0192] If it is found that the probability value of the currently compared phoneme unit in the wake-up word path is not greater than the maximum probability value, the comparison of the frame signal is terminated and the initial position of the comparison is updated to the serial number of the currently compared phoneme unit in the probability vector, and the above comparison operation is continued on the phoneme-level probability vector corresponding to the next frame signal until the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, or the comparison of all frame signals is completed;

[0193] The wake-up word path is composed of all the phoneme units that constitute the wake-up word in series, and the absorption path is composed of all the phoneme units except all the phoneme units that constitute the wake-up word, and each phoneme unit except all the phoneme units that constitute the wake-up word constitutes an absorption path.

[0194] Exemplarily, the apparatus 700 further includes a first training module 730, which trains the first acoustic model based on the following method:

[0195] Using the first preset voice data to train the preset acoustic model to distinguish the voice data other than the wake-up word;

[0196] and using the second preset voice data to perform intensive training on the preset acoustic model to distinguish relevant phoneme units of the wake-up word;

[0197] Clustering the output phoneme units to obtain classifications of N phoneme units, and using the classifications of the N phoneme units as N output nodes of the preset acoustic model;

[0198] The first preset voice data is general corpus data that does not contain a wake-up word, and the second preset voice data is corpus data that contains a wake-up word.

[0199] Exemplarily, the apparatus 700 further includes a second training module 740, and the second training module 740 trains the second acoustic model based on the following method:

[0200] Inputting the phoneme-level features output by the first acoustic model into the convolution layer and the pooling layer of the second acoustic model to obtain syllable-level features;

[0201] Inputting the syllable-level feature into the fully connected layer of the second acoustic model to obtain a corresponding syllable-level probability vector;

[0202] Inputting the syllable-level features into the global average pooling layer of the second acoustic model to obtain corresponding whole-word-level features;

[0203] Inputting the whole word level feature into the fully connected layer of the second acoustic model to obtain a corresponding whole word level probability vector;

[0204] The syllable-level probability vector and the preset syllable-level label are calculated according to preset rules to obtain a syllable-level loss function, and the whole-word-level probability vector and the preset whole-word-level label vector are calculated according to preset rules to obtain a whole-word-level loss function;

[0205] Calculating the phoneme-level probability vector obtained by the first acoustic model and the preset phoneme-level label vector according to preset rules to obtain a phoneme-level loss function;

[0206] With the minimization of the syllable-level loss function, the whole-word-level loss function, and the phoneme-level loss function as the optimization goal, the first acoustic model and the second acoustic model are jointly trained until the convergence condition is met, thereby obtaining the trained first acoustic model and the second acoustic model.

[0207] Figure 8 An example of a physical structure diagram of an electronic device is shown below. Figure 8 As shown, the electronic device may include: a processor (Processor) 810, a communication interface (Communications Interface) 820, a memory (Memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the speech recognition method for the target device, the method including:

[0208] Acquiring an audio signal, filtering the audio signal to obtain a speech signal, and performing frame division and feature extraction processing on the speech signal to obtain acoustic features of each frame signal;

[0209] Inputting the acoustic features of each frame signal into the trained first acoustic model to obtain a phoneme-level feature corresponding to each frame signal and a phoneme-level probability vector associated with the phoneme-level feature;

[0210] If the current operating mode of the target device is the wake-up mode, inputting the phoneme-level features corresponding to each frame signal and the phoneme-level probability vector associated with the phoneme-level features into a preset decoder to identify the wake-up word, and switching the operating mode of the target device to the command word mode if the wake-up word is identified;

[0211] If the current working mode of the target device is the command word mode, the phoneme-level features corresponding to each frame signal are input into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and the command word is identified based on the whole-word-level probability vector to trigger the target device to perform the action corresponding to the command word when the command word is identified.

[0212] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0213] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the speech recognition method for the target device provided by the above methods.

[0214] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned speech recognition method for a target device.

[0215] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0216] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0217] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A speech recognition method for a target device, characterized in that: The method comprises: Acquiring an audio signal and obtaining acoustic features of the audio signal; Inputting the acoustic feature into a trained first acoustic model to obtain a phoneme-level feature and a phoneme-level probability vector associated with the phoneme-level feature; If the current operating mode of the target device is a wake-up mode, inputting the phoneme-level probability vector into a preset decoder to identify a wake-up word, and converting the operating mode of the target device to a command word mode if the wake-up word is identified; If the current working mode of the target device is the command word mode, the phoneme-level features are input into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and the command word is identified based on the whole-word-level probability vector, so as to trigger the target device to perform the action corresponding to the command word when the command word is identified; wherein, the second acoustic model adopts a convolutional neural network structure, converts the phoneme-level features into syllable-level features through convolution layers and pooling layers, and then obtains the whole-word-level features through global average pooling.

2. The speech recognition method for a target device according to claim 1, wherein: Inputting the phoneme-level features into a second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and recognizing a command word based on the whole-word-level probability vector includes: Based on a convolutional neural network, the second acoustic model obtains a syllable-level feature for the audio signal and a syllable-level probability vector associated with the syllable-level feature according to the phoneme-level feature; Based on a global average pooling method, the second acoustic model obtains a whole-word-level feature for the audio signal and a whole-word-level probability vector associated with the whole-word-level feature according to the syllable-level feature; The command word corresponding to the element with the largest probability value in the whole word level probability vector is used as the command word finally recognized.

3. The speech recognition method for a target device according to claim 1, wherein: The acquiring of the audio signal and obtaining the acoustic features of the audio signal includes: Performing noise reduction and audio energy-based sliding window screening processing on the audio signal to obtain an audio signal of a speech segment as the audio signal and filtering out an audio signal of a non-speech segment; Acoustic features of the audio signal are extracted based on the energy spectrum features to obtain acoustic features of each frame signal.

4. The speech recognition method for a target device according to claim 3, wherein: Inputting the acoustic feature into the trained first acoustic model to obtain a phoneme-level feature and a phoneme-level probability vector associated with the phoneme-level feature includes: For each frame signal acoustic feature, the first acoustic model converts the acoustic feature into a phoneme-level feature of a triphone pattern based on a preset triphone rule and obtains a phoneme-level probability vector associated with the phoneme-level feature; The phoneme-level probability vector indicates the probability value of each type of three-phoneme unit in the phoneme-level feature.

5. The speech recognition method for a target device according to claim 4, characterized in that: Inputting the phoneme-level probability vector into a preset decoder to identify the wake-up word includes: Set the initial position of the comparison to 0, and perform the following operations on the phoneme-level probability vector corresponding to each frame signal in sequence: Based on the phoneme-level probability vector corresponding to the frame signal, starting from the initial comparison position, the probability values corresponding to the phoneme units in the wake-up word path in the decoder are sequentially compared with the maximum probability values of all phoneme units in the absorption path in the decoder; If the probability value of the currently compared phoneme unit in the wake-up word path is greater than the maximum probability value, then the probability values corresponding to subsequent phoneme units in the wake-up word path are sequentially compared with the maximum probability value; If it is found that the probability value of the currently compared phoneme unit in the wake-up word path is not greater than the maximum probability value, the comparison of the frame signal is terminated and the initial position of the comparison is updated to the serial number of the currently compared phoneme unit in the probability vector, and the above comparison operation is continued on the phoneme-level probability vector corresponding to the next frame signal until the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, or the comparison of all frame signals is completed; The wake-up word path is composed of all the phoneme units that constitute the wake-up word in series, and the absorption path is composed of all the phoneme units except all the phoneme units that constitute the wake-up word, and each phoneme unit except all the phoneme units that constitute the wake-up word constitutes an absorption path.

6. The speech recognition method for a target device according to claim 5, characterized in that: Inputting the phoneme-level probability vector into a preset decoder to identify the wake-up word further includes: If the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, it means that the wake-up word is recognized from the audio signal.

7. The speech recognition method for a target device according to claim 1, wherein: The method comprises: The first acoustic model is trained based on the following method: Training the first acoustic model using first preset voice data to distinguish voice data other than the wake-up word; and using second preset voice data to perform intensive training on the first acoustic model to distinguish relevant phoneme units of the wake-up word; Clustering the output phoneme units to obtain classifications of N phoneme units, and using the classifications of the N phoneme units as N output nodes of the first acoustic model; The first preset voice data is general corpus data that does not contain a wake-up word, and the second preset voice data is corpus data that contains a wake-up word.

8. The speech recognition method for a target device according to claim 1, wherein: The method further comprises: The second acoustic model is trained based on the following method: Inputting the phoneme-level features output by the first acoustic model into the convolution layer and the pooling layer of the second acoustic model to obtain syllable-level features; Inputting the syllable-level feature into the fully connected layer of the second acoustic model to obtain a corresponding syllable-level probability vector; Inputting the syllable-level features into the global average pooling layer of the second acoustic model to obtain corresponding whole-word-level features; Inputting the whole word level feature into the fully connected layer of the second acoustic model to obtain a corresponding whole word level probability vector; The syllable-level probability vector and the preset syllable-level label are calculated according to preset rules to obtain a syllable-level loss function, and the whole-word-level probability vector and the preset whole-word-level label vector are calculated according to preset rules to obtain a whole-word-level loss function; Calculating the phoneme-level probability vector obtained by the first acoustic model and the preset phoneme-level label vector according to preset rules to obtain a phoneme-level loss function; With the minimization of the syllable-level loss function, the whole-word-level loss function, and the phoneme-level loss function as the optimization goal, the first acoustic model and the second acoustic model are jointly trained until the convergence condition is met, thereby obtaining the trained first acoustic model and the second acoustic model.

9. A speech recognition device for a target device, characterized in that: The device comprises: A preprocessing module, configured to acquire an audio signal and obtain acoustic features of the audio signal; An identification module, wherein the identification module is used to: Inputting the acoustic feature into a trained first acoustic model to obtain a phoneme-level feature and a phoneme-level probability vector associated with the phoneme-level feature; If the current operating mode of the target device is a wake-up mode, inputting the phoneme-level probability vector into a preset decoder to identify a wake-up word, and converting the operating mode of the target device to a command word mode if the wake-up word is identified; If the current working mode of the target device is the command word mode, the phoneme-level features are input into the second acoustic model to obtain a whole-word-level probability vector based on the phoneme-level features, and the command word is identified based on the whole-word-level probability vector, so as to trigger the target device to perform the action corresponding to the command word when the command word is identified; wherein, the second acoustic model adopts a convolutional neural network structure, converts the phoneme-level features into syllable-level features through convolution layers and pooling layers, and then obtains the whole-word-level features through global average pooling.

10. The speech recognition device for a target device according to claim 9, characterized in that: The identification module is further used to: Based on a convolutional neural network, the second acoustic model obtains a syllable-level feature for the audio signal and a syllable-level probability vector associated with the syllable-level feature according to the phoneme-level feature; Based on a global average pooling method, the second acoustic model obtains a whole-word-level feature for the audio signal and a whole-word-level probability vector associated with the whole-word-level feature according to the syllable-level feature; The command word corresponding to the element with the largest probability value in the whole word level probability vector is used as the command word finally recognized.

11. The speech recognition device for a target device according to claim 10, wherein: The preprocessing module is further used for: performing noise reduction and audio energy-based sliding window screening processing on the audio signal to obtain an audio signal of a speech segment as the audio signal and filtering out an audio signal of a non-speech segment; Acoustic features of the audio signal are extracted based on the energy spectrum features to obtain acoustic features of each frame signal.

12. The speech recognition device for a target device according to claim 11, characterized in that: The identification module is further used to: For each frame signal acoustic feature, the first acoustic model converts the acoustic feature into a phoneme-level feature of a triphone pattern based on a preset triphone rule and obtains a phoneme-level probability vector associated with the phoneme-level feature; The phoneme-level probability vector indicates the probability value of each type of three-phoneme unit in the phoneme-level feature.

13. The speech recognition device for a target device according to claim 12, characterized in that: The identification module is further used to: Set the initial position of the comparison to 0, and perform the following operations on the phoneme-level probability vector corresponding to each frame signal in sequence: Based on the phoneme-level probability vector corresponding to the frame signal, starting from the initial comparison position, the probability values corresponding to the phoneme units in the wake-up word path in the decoder are sequentially compared with the maximum probability values of all phoneme units in the absorption path in the decoder; If the probability value of the currently compared phoneme unit in the wake-up word path is greater than the maximum probability value, then the probability values corresponding to subsequent phoneme units in the wake-up word path are sequentially compared with the maximum probability value; If it is found that the probability value of the currently compared phoneme unit in the wake-up word path is not greater than the maximum probability value, the comparison of the frame signal is terminated and the initial position of the comparison is updated to the serial number of the currently compared phoneme unit in the probability vector, and the above comparison operation is continued on the phoneme-level probability vector corresponding to the next frame signal until the probability values corresponding to all phoneme units in the wake-up word path are greater than the maximum probability value, or the comparison of all frame signals is completed; The wake-up word path is composed of all the phoneme units that constitute the wake-up word in series, and the absorption path is composed of all the phoneme units except all the phoneme units that constitute the wake-up word, and each phoneme unit except all the phoneme units that constitute the wake-up word constitutes an absorption path.

14. The speech recognition device for a target device according to claim 9, wherein: The apparatus further includes a first training module, wherein the first training module trains the first acoustic model based on the following method: Training the first acoustic model using first preset voice data to distinguish voice data other than the wake-up word; and using second preset voice data to perform intensive training on the first acoustic model to distinguish relevant phoneme units of the wake-up word; Clustering the output phoneme units to obtain classifications of N phoneme units, and using the classifications of the N phoneme units as N output nodes of the first acoustic model; The first preset voice data is general corpus data that does not contain a wake-up word, and the second preset voice data is corpus data that contains a wake-up word.

15. The speech recognition device for a target device according to claim 9, characterized in that: The apparatus further includes a second training module, wherein the second training module trains the second acoustic model based on the following method: Inputting the phoneme-level features output by the first acoustic model into the convolution layer and the pooling layer of the second acoustic model to obtain syllable-level features; Inputting the syllable-level feature into the fully connected layer of the second acoustic model to obtain a corresponding syllable-level probability vector; Inputting the syllable-level features into the global average pooling layer of the second acoustic model to obtain corresponding whole-word-level features; Inputting the whole word level feature into the fully connected layer of the second acoustic model to obtain a corresponding whole word level probability vector; The syllable-level probability vector and the preset syllable-level label are calculated according to preset rules to obtain a syllable-level loss function, and the whole-word-level probability vector and the preset whole-word-level label vector are calculated according to preset rules to obtain a whole-word-level loss function; Calculating the phoneme-level probability vector obtained by the first acoustic model and the preset phoneme-level label vector according to preset rules to obtain a phoneme-level loss function; With the minimization of the syllable-level loss function, the whole-word-level loss function, and the phoneme-level loss function as the optimization goal, the first acoustic model and the second acoustic model are jointly trained until the convergence condition is met, thereby obtaining the trained first acoustic model and the second acoustic model.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the speech recognition method for a target device according to any one of claims 1 to 8 are implemented.

17. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the speech recognition method for a target device according to any one of claims 1 to 8 are implemented.

Images