Speech recognition method and apparatus, electronic device, and storage medium

Abstract

The application provides a speech recognition method and device, electronic equipment and storage medium, the method comprising: obtaining a to-be-recognized speech; based on a noise coding model, performing noise coding on acoustic features of the to-be-recognized speech to obtain noise features of the to-be-recognized speech, the noise features being used to represent features of noise carried by the to-be-recognized speech; based on a speech recognition model, applying the acoustic features and the noise features of the to-be-recognized speech to perform speech recognition to obtain a speech recognition result of the to-be-recognized speech; and the speech recognition model is trained based on acoustic features and noise features of a second sample speech and a speech recognition label of the second sample speech. The method, device, electronic equipment and storage medium provided by the application combine noise features and acoustic features to perform speech recognition, thereby improving the noise resistance of speech recognition and subsequent speech processing, such as speech wake-up.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech processing, and in particular to a speech recognition method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Voice wake-up refers to a user waking up an electronic device by speaking a wake-up word, so that the electronic device enters a state of waiting for a voice instruction or directly executes a predetermined voice instruction.

[0003] In order to improve the noise resistance performance of the voice wake-up technology, the current solution is to add noise to clean wake-up speech to simulate the speaking voice in a noisy environment, and to use the noise-added wake-up speech as a positive example sample for training of a voice wake-up model.

[0004] However, since the noise in the positive example sample is artificially added, it is different from the wake-up speech actually collected in the environment, and the common features brought by the programmed noise addition mode are also easy to be learned by the voice wake-up model, resulting in that the noise resistance effect of voice wake-up is limited. SUMMARY

[0005] The present application provides a speech recognition method, device, electronic equipment and storage medium to solve the defect that the noise resistance effect of voice wake-up in the prior art is limited.

[0006] The present application provides a speech recognition method, comprising:

[0007] obtaining a to-be-recognized speech;

[0008] based on a noise coding model, coding the acoustic features of the to-be-recognized speech to obtain noise features of the to-be-recognized speech, the noise features being used to represent the features of the noise carried by the to-be-recognized speech;

[0009] based on a speech recognition model, applying the acoustic features and noise features of the to-be-recognized speech to perform speech recognition to obtain a speech recognition result of the to-be-recognized speech; the speech recognition model being trained based on the acoustic features and noise features of a second sample speech and a speech recognition label of the second sample speech.

[0010] According to the speech recognition method provided by the present application, the speech recognition model comprises a noise reduction feature extraction layer and a classification layer;

[0011] The based on a speech recognition model, applying the acoustic features and noise features of the to-be-recognized speech to perform speech recognition to obtain a speech recognition result of the to-be-recognized speech, comprises:

[0012] based on the noise reduction feature extraction layer, extracting features that are different from the noise features from the acoustic features to obtain noise reduction features.

[0013] Based on the classification layer, the noise reduction feature is applied for feature classification to obtain the speech recognition result.

[0014] According to the speech recognition method provided by the application, the noise reduction feature extraction layer comprises a feature conversion layer, a feature merging layer and a high layer extraction layer.

[0015] Based on the noise reduction feature extraction layer, the feature that is different from the noise feature is extracted from the acoustic feature to obtain a noise reduction feature, comprising:

[0016] Based on the feature conversion layer, the acoustic feature and / or the noise feature is converted to obtain acoustic conversion features and noise conversion features in the same feature space.

[0017] Based on the feature merging layer, the acoustic conversion features and the noise conversion features in the same feature space are merged to obtain merged features.

[0018] Based on the high layer extraction layer, the high layer feature of the merged features is extracted to obtain the noise reduction feature.

[0019] According to the speech recognition method provided by the application, the feature conversion layer comprises an acoustic feature conversion layer and a noise feature conversion layer.

[0020] Based on the feature conversion layer, the acoustic feature and / or the noise feature is converted to obtain acoustic conversion features and noise conversion features in the same feature space, comprising:

[0021] Based on the acoustic feature conversion layer, the acoustic feature is converted to obtain the acoustic conversion feature.

[0022] Based on the noise feature conversion layer, the noise feature is converted to obtain the noise conversion feature.

[0023] According to the speech recognition method provided by the application, the noise encoding model and the feature classification model constitute a speech classification model, the speech classification model is trained based on the acoustic feature of the first sample speech and the noise label representing whether the first sample speech is a noisy speech, and the feature classification model is used to determine whether the first sample speech is a noisy speech based on the noise feature of the first sample speech output by the noise encoding model.

[0024] According to the speech recognition method provided by the application, the noise encoding model is determined based on the following steps:

[0025] code the acoustic features of the first sample speech based on the initial noise coding model in the initial classification model to obtain initial noise features;

[0026] determine a speech classification result of the first sample speech based on the initial noise coding model in the initial classification model and the initial noise features;

[0027] perform parameter iteration on the initial classification model based on the noise label of the first sample speech and the speech classification result to obtain a speech classification model;

[0028] determine the noise coding model based on the initial noise coding model after parameter iteration in the speech classification model.

[0029] According to the speech recognition method provided by the application, the training step of the speech recognition model comprises:

[0030] determine an initial recognition model;

[0031] code the acoustic features of the second sample speech based on the noise coding model to obtain noise features of the second sample speech;

[0032] perform speech recognition based on the initial recognition model and the acoustic features and noise features of the second sample speech to obtain a speech recognition result of the second sample speech;

[0033] perform parameter iteration on the initial recognition model based on the speech recognition result and the speech recognition label of the second sample speech to obtain the speech recognition model.

[0034] According to the speech recognition method provided by the application, the parameter iteration on the initial recognition model based on the speech recognition result and the speech recognition label of the second sample speech to obtain the speech recognition model comprises:

[0035] perform parameter iteration on the noise coding model and the initial recognition model based on the speech recognition result and the speech recognition label of the second sample speech to obtain a noise coding model after parameter iteration and the speech recognition model;

[0036] use the noise coding model after parameter iteration as the noise coding model.

[0037] According to the speech recognition method provided by the application, the speech recognition result is a phoneme classification result;

[0038] the speech recognition based on the speech recognition model, the acoustic features and noise features of the speech to be recognized to obtain the speech recognition result of the speech to be recognized, further comprises:

[0039] perform voice wake-up based on the phoneme classification result.

[0040] The application also provides a speech recognition device, comprising:

[0041] a speech obtaining unit, configured to obtain to-be-recognized speech;

[0042] a noise encoding unit, configured to perform noise encoding on the acoustic feature of the to-be-recognized speech based on a noise encoding model, to obtain a noise feature of the to-be-recognized speech, the noise feature being used to represent the feature of noise carried by the to-be-recognized speech;

[0043] a speech recognition unit, configured to perform speech recognition based on a speech recognition model, by using the acoustic feature and the noise feature of the to-be-recognized speech, to obtain a speech recognition result of the to-be-recognized speech, the speech recognition model being trained based on the acoustic feature and the noise feature of a second sample speech and a speech recognition label of the second sample speech.

[0044] The application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the speech recognition method according to any one of the above when executing the program.

[0045] The application also provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the speech recognition method according to any one of the above.

[0046] The application also provides a computer program product, comprising a computer program, and the computer program is executable on a processor to implement the speech recognition method according to any one of the above.

[0047] The speech recognition method, device, electronic device, and storage medium provided by the application encode the noise feature used to determine whether the to-be-recognized speech is noisy speech by using the noise encoding model in the speech classification model, and perform speech recognition by combining the noise feature and the acoustic feature of the to-be-recognized speech, so as to improve the noise resistance of speech recognition, and further improve the reliability of the speech recognition result and the noise resistance of subsequent speech processing such as voice wake-up. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0049] Figure 1is one of flowcharts of the speech recognition method provided by the present application;

[0050] Figure 2 is a flowchart of the speech recognition based on the speech recognition model provided by the present application;

[0051] Figure 3 is a flowchart of the noise reduction feature extraction based on the noise reduction feature extraction layer provided by the present application;

[0052] Figure 4 is a structural diagram of the speech recognition model provided by the present application;

[0053] Figure 5 is a flowchart of the speech recognition model training method provided by the present application;

[0054] Figure 6 is another flowchart of the speech recognition method provided by the present application;

[0055] Figure 7 is a structural diagram of the speech recognition device provided by the present application;

[0056] Figure 8 is a structural diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0058] In order to improve the anti-noise performance of the speech wake-up technology, the current scheme is to add noise to the clean wake-up speech to simulate the speaking voice in the noise environment, and the wake-up speech after adding noise is used as the positive example sample for speech wake-up model training. In addition, various audio collected from various environments can also be directly used as negative example samples for speech wake-up model training. After obtaining the positive example samples and negative example samples, the speech wake-up model can be trained based on them, so as to realize the speech wake-up with anti-noise performance.

[0059] However, since the noise in the positive example sample is artificially added, there is a difference from the real wake-up speech collected from the environment, and the common features brought by the programmed noise adding method are also easy to be learned by the speech wake-up model, which limits the anti-noise effect of the speech wake-up.

[0060] To further improve the noise resistance of voice wake-up, this invention provides a voice recognition method that enhances noise resistance during the voice recognition process. The recognition results obtained can be used for voice wake-up or for implementing other human-computer interaction functions; this invention does not specifically limit this application.

[0061] Figure 1 This is one of the flowcharts illustrating the speech recognition method provided by the present invention, such as... Figure 1 As shown, the method includes:

[0062] Step 110: Obtain the speech to be recognized.

[0063] The speech to be recognized may include the speech that needs to be recognized. The speech to be recognized can be obtained by a sound pickup device, which may be a smartphone, tablet, or smart appliance such as a speaker, television, or air conditioner. After the sound pickup device obtains the speech to be recognized through a microphone array, it may also amplify and / or reduce noise, etc. The embodiments of the present invention do not specifically limit this.

[0064] Step 120: Based on the noise coding model, noise coding is performed on the acoustic features of the speech to be identified to obtain the noise features of the speech to be identified. The noise features are used to characterize the noise carried by the speech to be identified.

[0065] After obtaining the speech to be recognized, the acoustic features of the speech can be determined. These acoustic features can be the acoustic features of each frame of the speech to be recognized extracted by Fast Fourier Transform (FFT) after the speech to be recognized is segmented and windowed. Examples include Mel Frequency Cepstrum Coefficient (MFCC) features or Perceptual Linear Predictive (PLP) features.

[0066] Based on this, the acoustic features of the speech to be recognized can be input into a pre-trained noise coding model. The noise coding model then encodes the input acoustic features to produce noise features, thus obtaining the noise features of the speech to be recognized output by the noise coding model. Here, the noise features of the speech to be recognized are used to characterize the specific features of the noise carried in the speech to be recognized, and these noise features can be used to determine whether the speech to be recognized is noisy.

[0067] It can be understood that the acoustic feature of the to-be-recognized speech is a feature for each frame of speech in the to-be-recognized speech, and correspondingly, the noise feature obtained by noise coding the acoustic feature is also a feature for each frame of speech in the to-be-recognized speech, that is, the noise feature of the to-be-recognized speech can be used to determine whether each frame of speech in the to-be-recognized speech is noisy speech.

[0068] In step 130, based on a speech recognition model, the acoustic feature and the noise feature of the to-be-recognized speech are applied to perform speech recognition, to obtain a speech recognition result of the to-be-recognized speech; the speech recognition model is trained based on the acoustic feature and the noise feature of the second sample speech and a speech recognition label of the second sample speech.

[0069] After obtaining the noise feature of the to-be-recognized speech based on the noise coding model, the noise feature and the acoustic feature of the to-be-recognized speech can be combined to perform speech recognition. It can be understood that both the noise feature and the acoustic feature are features for each frame of speech in the to-be-recognized speech, and therefore, the speech recognition result obtained therefrom is also for each frame of speech in the to-be-recognized speech, for example, can recognize the phonemes corresponding to each frame of speech in the to-be-recognized speech, or can recognize the acoustic states corresponding to each frame of speech in the to-be-recognized speech, which is not limited in the embodiments of the present application. Here, the speech recognition result can be applied to subsequent speech processing links, for example, can be applied to speech translation, human-computer interaction, speech wake-up, etc.

[0070] Compared with the related art in which speech recognition is only performed by using the acoustic feature of the to-be-recognized speech, in the embodiments of the present application, not only the acoustic feature but also the noise feature is used. The combination of the acoustic feature and the noise feature in the speech recognition model enables the speech recognition model to extract the features in the acoustic feature that are different from the noise feature, that is, to extract the features in the acoustic feature that are irrelevant to noise, and to perform speech recognition based on the features. Since the features in the acoustic feature that are related to noise are filtered in the process of speech recognition by taking the noise feature as a reference, the noise resistance of speech recognition is enhanced, and the result of speech recognition is more reliable.

[0071] Here, the speech recognition combined with the acoustic feature and the noise feature of the to-be-recognized speech is implemented based on a pre-trained speech recognition model, and before step 130 is performed, the speech recognition model needs to be obtained through the following steps:

[0072] The second sample voice can be collected in advance, the acoustic feature and the noise feature of the second sample voice are extracted, and a voice recognition label is set for the second sample voice. It should be noted that the second sample voice can be the same as or different from the first sample voice used for training the noise coding model. The noise feature of the second sample voice can be extracted by using the noise coding model. The voice recognition label of the second sample voice is set for each frame of voice in the second sample voice, and is used to reflect the phoneme or acoustic state corresponding to each frame of voice in the second sample voice. The embodiments of the present application do not make a specific limitation in this regard.

[0073] After the acoustic feature and the noise feature of the second sample voice and the voice recognition label of the second sample voice are obtained, the initial recognition model can be trained based on the acoustic feature, the noise feature and the voice recognition label of the second sample voice, and the trained initial recognition model can be used as the voice recognition model.

[0074] The method provided by the embodiments of the present application encodes the noise feature used for judging whether the to-be-recognized voice is a noisy voice by using the noise coding model in the voice classification model, and performs voice recognition in combination with the noise feature and the acoustic feature of the to-be-recognized voice, so as to improve the noise resistance of voice recognition, and further improve the reliability of the voice recognition result and the noise resistance of subsequent voice processing such as voice wake-up.

[0075] According to any one of the above embodiments, the voice recognition model comprises a noise reduction feature extraction layer and a classification layer.

[0076] Correspondingly, Figure 2 is a flowchart of a process of voice recognition based on a voice recognition model provided by the present application, as shown in Figure 2 The step 130 comprises:

[0077] In step 131, the noise reduction feature extraction layer is used to extract features that are different from the acoustic feature and the noise feature, and obtain noise reduction features.

[0078] In step 132, the classification layer is used to perform feature classification by using the noise reduction features, and obtain the voice recognition result.

[0079] The voice recognition model can comprise two parts, i.e., a noise reduction feature extraction layer and a classification layer. The input of the voice recognition model, i.e., the input of the noise reduction feature extraction layer, comprises the acoustic feature and the noise feature of the to-be-recognized voice. The output of the noise reduction feature extraction layer is features that are different from the acoustic feature and the noise feature, i.e., noise reduction features. The output of the noise reduction feature extraction layer is the input of the classification layer. The output of the classification layer is the output of the voice recognition model, i.e., the voice recognition result.

[0080] Further, in the process of voice recognition, the acoustic feature and the noise feature of the voice to be recognized are first input to the noise reduction feature extraction layer, the noise reduction feature extraction layer can perform differential feature extraction on the input acoustic feature and noise feature, so as to obtain and output a feature reflecting the difference between the acoustic feature and the noise feature. It can be understood that the acoustic feature itself is mixed with the feature reflected by the human voice and the noise in the voice to be recognized, and the noise feature more reflects the feature of the noise carried by the voice to be recognized. The difference between the acoustic feature and the noise feature, that is, the feature reflected by the human voice in the voice to be recognized, is the extraction of the differential feature. It can be understood that the feature obtained by filtering the noise-related feature reflected in the acoustic feature is the noise reduction feature.

[0081] Here, the noise reduction feature extraction layer can be a multi-layer convolutional neural network (CNN) in a cascade structure, can also be a deep neural network (DNN), and can also be a combination structure of CNN and DNN, etc., and the embodiments of the present application do not make specific limitations.

[0082] Subsequently, the noise reduction feature output by the noise reduction feature extraction layer can be input to the classification layer, the classification layer can perform recognition on the input noise reduction feature, so as to obtain and output the voice recognition result. The classification layer here can be a hidden Markov model (HMM), can also be a multi-layer DNN or other model structure capable of realizing the classification function based on the noise reduction feature, and the embodiments of the present application do not make specific limitations.

[0083] The method provided by the embodiments of the present application realizes noise reduction in the process of voice recognition by extracting the feature different from the noise feature, improves the noise resistance of voice recognition, and further improves the reliability of the voice recognition result and the noise resistance of subsequent voice processing such as voice wake-up.

[0084] Based on any of the above embodiments, the noise reduction feature extraction layer includes a feature conversion layer, a feature merging layer and a high layer extraction layer.

[0085] Correspondingly, Figure 3 is a flowchart of extracting noise reduction features based on the noise reduction feature extraction layer provided by the present application, as Figure 3 shown, step 131 includes:

[0086] Step 1311, based on the feature conversion layer, performing feature conversion on the acoustic feature and / or the noise feature to obtain acoustic conversion features and noise conversion features in the same feature space;

[0087] Step 1312, based on the feature merging layer, the acoustic conversion feature and the noise conversion feature under the same feature space are merged to obtain a merged feature;

[0088] Step 1313, based on the high layer extraction layer, the merged feature is high layer feature extracted to obtain the noise reduction feature.

[0089] The noise reduction feature extraction layer includes a feature conversion layer, a feature merging layer and a high layer extraction layer. The input of the noise reduction feature extraction layer, i.e. the input of the feature conversion layer, includes acoustic features and noise features of the voice to be recognized. The feature conversion layer can perform feature conversion on the input acoustic features and noise features, thereby outputting acoustic conversion features and noise conversion features under the same feature space. It can be understood that the feature conversion layer exists to convert the acoustic features and noise features to the same feature space through feature conversion, so as to facilitate subsequent comparison of the acoustic features and noise features to extract distinctive features.

[0090] Here, the converted feature space can be the feature space in which the acoustic features are located or the feature space in which the noise features are located, or a feature space other than the above two feature spaces. It can be understood that, in order to obtain acoustic conversion features and noise conversion features under the same feature space, the acoustic features can be converted only to the feature space in which the noise features are located, in which case the converted acoustic features are denoted as acoustic conversion features and the noise features themselves are denoted as noise conversion features; or the noise features can be converted only to the feature space in which the acoustic features are located, in which case the converted noise features are denoted as noise conversion features and the acoustic features themselves are denoted as acoustic conversion features; or the acoustic features and the noise features can be converted to the same specific feature space, in which case the converted acoustic features are denoted as acoustic conversion features and the converted noise features are denoted as noise conversion features, which are not limited by the embodiments of the present application.

[0091] The output of the feature conversion layer, i.e. the input of the feature merging layer, can perform feature merging on the acoustic conversion features and the noise conversion features under the same feature space output by the feature conversion layer, thereby obtaining and outputting a merged feature that can reflect both the acoustic features and the noise features. Here, the feature merging operation can be feature splicing, or the acoustic conversion features and the noise conversion features can be weighted according to the pre-trained weights and then spliced, which are not limited by the embodiments of the present application.

[0092] The output of the feature merging layer, i.e. the input of the high layer extraction layer, can perform high layer feature extraction on the merged feature output by the feature merging layer, thereby obtaining and outputting a noise reduction feature that can reflect the difference between the acoustic features and the noise features.

[0093] Based on any of the above embodiments, the feature conversion layer comprises an acoustic feature conversion layer and a noise feature conversion layer.

[0094] Step 1311 comprises:

[0095] Based on the acoustic feature conversion layer, the acoustic feature is converted to obtain the acoustic converted feature.

[0096] Based on the noise feature conversion layer, the noise feature is converted to obtain the noise converted feature.

[0097] The feature conversion layer can include two parts, namely an acoustic feature conversion layer and a noise feature conversion layer, for respectively converting the acoustic feature and the noise feature input into the feature conversion layer, so that the acoustic converted feature and the noise converted feature after feature conversion are in the same feature space.

[0098] The acoustic feature conversion layer and the noise feature conversion layer are arranged to make the feature conversion of the acoustic feature and the noise feature more flexible and easier to fit the characteristics of the acoustic feature and the noise feature, which helps to improve the noise reduction performance of speech recognition.

[0099] Based on any of the above embodiments, Figure 4 is a structural diagram of a speech recognition model provided by the present application, as Figure 4 shown, the speech recognition model comprises a DNN 1, a DNN2, a feature merging layer, a multi-layer CNN and a multi-layer DNN. Among them, DNN1 and DNN2 are both DNN networks, and 1 and 2 here are only used to distinguish the two DNN networks.

[0100] It can be understood that, Figure 4 DNN1 and DNN2 in correspond to the acoustic feature conversion layer and the noise feature conversion layer, DNN1 converts the acoustic feature in the form of feature coding, and DNN2 converts the noise feature in the form of feature coding, and the acoustic converted feature and the noise converted feature obtained after conversion by DNN1 and DNN2 are in the same feature space.

[0101] Subsequently, the feature merging layer merges the acoustic converted feature and the noise converted feature to obtain the merged feature.

[0102] Figure 4The multi-layer CNN in the noise encoding model corresponds to a high-level extraction layer, and the multi-layer CNN in the cascade structure can perform feature extraction on the merged features layer by layer, so as to obtain noise reduction features with higher dimensions and wider fields of view, which can summarize the difference between acoustic features and noise features. For example, the multi-layer CNN can be a cascade structure of three-layer CNNs, wherein the first-layer CNN is used to extract features of the merged features, the second-layer CNN is used to further extract features with wider fields of view on the basis of the features extracted by the first-layer CNN, and the third-layer CNN is used to extract noise reduction features with higher dimensions and wider fields of view on the basis of the features extracted by the second-layer CNN.

[0103] Figure 4 The multi-layer DNN in the noise encoding model corresponds to a classification layer, and the multi-layer DNN in the cascade structure can classify the noise reduction features extracted by the multi-layer CNN, so as to output the speech recognition result. The multi-layer DNN here can be a 4-layer DNN, can be a 5-layer DNN, or can be a CNN with another number of layers, and the embodiments of the present application do not make specific limitations thereto.

[0104] Based on any of the above embodiments, the noise encoding model and the feature classification model constitute a speech classification model, the speech classification model is trained based on acoustic features of a first sample speech and noise labels representing whether the first sample speech is noisy speech, and the feature classification model is used to determine whether the first sample speech is noisy speech based on noise features of the first sample speech output by the noise encoding model.

[0105] In order to be able to realize noise encoding for the to-be-recognized speech, the noise encoding model needs to be obtained through the following steps before step 120 is performed:

[0106] The first sample speech can be collected in advance, and noise labels are set for the first sample speech to determine whether the first sample speech is noisy speech. It needs to be noted that the noise labels here are set for each frame of speech in the first sample speech respectively, and are used to reflect whether each frame of speech in the first sample speech is noisy speech or clean speech. The noisy speech here refers to speech carrying noise, and the clean speech refers to speech without noise.

[0107] In addition, an initial classification model can be constructed, which can functionally include two parts of encoding and classification. The encoding part is used to perform noise encoding on the input acoustic features, and the classification part is used to classify the noise features output by the encoding part. Subsequently, the initial classification model can be trained based on the first sample speech and the noise labels, and the initial classification model after training can be used as the speech classification model.

[0108] Here, the speech classification model is a trained model that can determine whether the speech is noisy speech or clean speech. Like the initial model, the speech classification model can also functionally include two parts of encoding and classification. The part of the speech classification model that encodes is the noise encoding model, and the part that classifies is the feature classification model.

[0109] Thus, it can be understood that the noise encoding model can be cascaded with the feature classification model to implement the function of determining whether the speech is noisy speech or clean speech as the speech classification model. Taking the first sample speech input in the training process of the speech classification model as an example, the noise encoding model in the speech classification model undertakes the task of noise encoding and outputs the noise feature of the first sample speech, and the feature classification model undertakes the task of determining whether the first sample speech is noisy speech based on the noise feature of the first sample speech.

[0110] That is, the noise encoding model is part of the speech classification model, and in the training process of the speech classification model, the noise encoding model learns to encode the input acoustic feature to extract a noise feature that can be used to determine whether the speech is noisy speech.

[0111] Based on any of the above embodiments, the noise encoding model is determined based on the following steps:

[0112] Based on the initial noise encoding model in the initial classification model, the acoustic feature of the first sample speech is noise encoded to obtain an initial noise feature;

[0113] Based on the initial feature classification model in the initial classification model, the initial noise feature is applied to determine the speech classification result of the first sample speech;

[0114] Based on the noise label of the first sample speech and the speech classification result, the initial classification model is iteratively parameterized to obtain a speech classification model;

[0115] Based on the initial noise encoding model in the speech classification model after parameter iteration, the noise encoding model is determined.

[0116] The initial classification model is the initial model required to train the speech classification model. The initial classification model includes two parts, namely an initial noise encoding model and an initial feature classification model. The initial noise encoding model can be understood as the encoding part of the initial classification model, which is used to encode the input acoustic feature, and the initial feature classification model can be understood as the classification part of the initial classification model, which is used to classify the noise feature output by the initial noise encoding model. Here, the initial noise encoding model can be a CNN, DNN, etc., and the initial feature classification model can be a softmax layer.

[0117] After obtaining the initial classification model including two parts of the initial noise encoding model and the initial feature classification model, the first sample voice pre-collected and the noise label used for representing whether the first sample voice is a noisy voice are applied to train the initial classification model:

[0118] Firstly, the acoustic feature of the first sample voice is input into the initial noise encoding model of the initial classification model, the acoustic feature of the first sample voice is noise encoded by the initial noise encoding model, and the initial noise feature of the first sample voice is obtained and output. It can be understood that the initial noise encoding model is an initial model before the noise encoding model is trained, and in order to distinguish from the noise feature output by the noise encoding model, the noise feature obtained by the initial noise encoding model is called the initial noise feature.

[0119] Secondly, the initial noise feature of the first sample voice is input into the initial feature classification model of the initial classification model, whether the first sample voice is a noisy voice is judged by the initial feature classification model based on the initial noise feature of the first sample voice, and the voice classification result of the first sample voice is obtained and output.

[0120] After obtaining the voice classification result of the first sample voice based on the initial classification model, the voice classification result and the pre-labeled noise label are compared, the loss function value is calculated according to the difference between the two, and the initial classification model is regarded as a whole based on the loss function value for parameter iteration. The initial classification model after completing the parameter iteration is called a voice classification model.

[0121] The voice classification model after the parameter iteration has the same structure as the initial classification model, so the voice classification model sample can be divided into two parts, the initial noise encoding model after the parameter iteration and the initial feature classification model after the parameter iteration. For the initial noise encoding model after the parameter iteration, this part can be directly used as the noise encoding model.

[0122] Based on any one of the above embodiments, Figure 5 is a flowchart of the voice recognition model training method provided by the application, as Figure 5 shown, the training steps of the voice recognition model include:

[0123] Step 510, determining an initial recognition model;

[0124] Step 520, based on the noise encoding model, noise encoding is performed on the acoustic feature of the second sample voice to obtain the noise feature of the second sample voice;

[0125] At step 530, based on the initial recognition model, acoustic features and noise features of the second sample voice are applied to perform voice recognition, to obtain a voice recognition result of the second sample voice.

[0126] At step 540, based on the voice recognition result of the second sample voice and a voice recognition label, parameter iteration is performed on the initial recognition model, to obtain the voice recognition model.

[0127] The initial recognition model is an initial model required for training a voice recognition model. Model parameters of the initial recognition model can be pre-set or initialized, and the embodiments of the present application do not make specific limitations thereon.

[0128] Training of the initial recognition model requires acoustic features, noise features of the second sample voice, and a voice recognition label of the second sample voice. The noise features of the second sample voice can be obtained by performing noise coding on the acoustic features according to a pre-trained noise coding model.

[0129] After obtaining the acoustic features and the noise features of the second sample voice, the acoustic features and the noise features of the second sample voice can be input into the initial recognition model, and the initial recognition model performs voice recognition based on the acoustic features and the noise features of the second sample voice, to obtain and output a voice recognition result of the second sample voice.

[0130] After obtaining the voice recognition result of the second sample voice, the voice recognition result can be compared with a pre-labeled voice recognition label, a loss function value is calculated according to the difference between the two, and parameter iteration is performed on the initial recognition model based on the loss function value. The initial recognition model after the parameter iteration is denoted as a voice recognition model.

[0131] It can be understood that in step 540, the parameters of the noise coding model can be fixed, and the parameter iteration can be performed on the initial recognition model alone; or the parameters of the noise coding model can not be fixed, and the noise coding model and the initial recognition model can be regarded as a whole to perform parameter iteration, and the embodiments of the present application do not make specific limitations thereon.

[0132] Based on any of the above embodiments, step 540 includes:

[0133] Based on the voice recognition result of the second sample voice and a voice recognition label, parameter iteration is performed on the noise coding model and the initial recognition model, to obtain a noise coding model after the parameter iteration, and the voice recognition model.

[0134] The noise coding model after the parameter iteration is taken as the noise coding model.

[0135] In the parameter iteration process based on the speech recognition result and the speech recognition label, the noise coding model and the initial recognition model can be regarded as a whole, that is, the output of the noise coding model is connected with the input of the initial recognition model, so as to realize the parameter iteration of the noise coding model and the initial recognition model. In this process, the parameter iteration of the noise coding model can be understood as the parameter fine-tuning of the noise coding model, and after the parameter iteration, the noise coding model can cover the original noise coding model and participate in the noise coding in the subsequent speech recognition process, and the initial recognition model after the parameter iteration, that is, the speech recognition model used in the subsequent speech recognition.

[0136] The method provided by the embodiment of the application can further improve the reliability of the noise coding model and the speech recognition model in the subsequent speech recognition, and ensure the accuracy of the speech recognition.

[0137] According to any of the above embodiments, the speech recognition result is a phoneme classification result.

[0138] After step 130, the method further includes:

[0139] Based on the phoneme classification result, the speech is woken up.

[0140] The noise coding model and the speech recognition model are both for each frame of speech in the speech to be recognized, wherein the speech recognition model can recognize the phoneme corresponding to each frame of speech in the speech to be recognized according to the acoustic features and the noise features of each frame of speech in the speech to be recognized, so as to obtain the speech recognition result, that is, the phoneme classification result of each frame of speech in the speech to be recognized, which can be the phoneme corresponding to each frame of speech or the probability distribution of various phonemes corresponding to each frame of speech, and the embodiment of the application does not make a specific limitation.

[0141] After obtaining the phoneme classification result of the speech to be recognized, the phoneme classification result can be decoded to obtain the transcription text of the speech to be recognized, and whether the wake-up operation is performed can be determined according to whether the wake-up word is contained in the transcription text, or the phoneme classification result can be directly compared with the phoneme sequence for performing the wake-up operation, and if the comparison is consistent, the wake-up operation is performed.

[0142] The method provided by the embodiment of the application applies the speech recognition to the speech wake-up, and improves the noise resistance of the speech wake-up.

[0143] According to any of the above embodiments, Figure 6 is a second flowchart of the speech recognition method provided by the application, as Figure 6As shown, first, the to-be-recognized speech is determined, which can be in WAV format or in FLAC (Free Lossless Audio Codec), APE, MP3, etc.

[0144] Then, the acoustic feature of the to-be-recognized speech can be input into the noise coding model to obtain the noise feature output by the noise coding model. Here, the acoustic feature can be an MFCC feature.

[0145] Next, the acoustic feature and the noise feature of the to-be-recognized speech can be input into the speech recognition model to obtain the phoneme classification result output by the speech recognition model. Thereafter, the phoneme classification result can be decoded, and in a case where the decoding result meets the wake-up condition, a wake-up operation is performed.

[0146] Based on any of the above embodiments, Figure 7 is a structural schematic diagram of a speech recognition device provided by the present application, as Figure 7 shown, the device comprises:

[0147] The speech acquisition unit 710 is configured to acquire to-be-recognized speech.

[0148] The noise coding unit 720 is configured to perform noise coding on the acoustic feature of the to-be-recognized speech based on a noise coding model to obtain a noise feature of the to-be-recognized speech, wherein the noise feature is used to represent the feature of the noise carried by the to-be-recognized speech.

[0149] The speech recognition unit 730 is configured to perform speech recognition based on a speech recognition model by applying the acoustic feature and the noise feature of the to-be-recognized speech to obtain a speech recognition result of the to-be-recognized speech, wherein the speech recognition model is trained based on the acoustic feature and the noise feature of a second sample speech and a speech recognition label of the second sample speech.

[0150] The device provided by the embodiment of the present application encodes the noise feature for judging whether the to-be-recognized speech is noisy speech by the noise coding model in the speech classification model, and performs speech recognition in combination with the noise feature and the acoustic feature of the to-be-recognized speech to improve the anti-noise capability of speech recognition, and further improve the reliability of the speech recognition result and the anti-noise capability of subsequent speech processing such as speech wake-up.

[0151] Based on any of the above embodiments, the speech recognition model comprises a noise reduction feature extraction layer and a classification layer.

[0152] The speech recognition unit 730 comprises:

[0153] a noise reduction subunit configured to extract features distinguishing the acoustic features from the noise features based on the noise reduction feature extraction layer to obtain noise reduction features;

[0154] a classification subunit configured to perform feature classification on the noise reduction features based on the classification layer to obtain the speech recognition result.

[0155] According to any one of the above embodiments, the noise reduction feature extraction layer comprises a feature conversion layer, a feature merging layer and a high-level extraction layer.

[0156] The noise reduction subunit is configured to:

[0157] perform feature conversion on the acoustic features and / or the noise features based on the feature conversion layer to obtain acoustic converted features and noise converted features in a same feature space;

[0158] perform feature merging on the acoustic converted features and the noise converted features in the same feature space based on the feature merging layer to obtain merged features;

[0159] perform high-level feature extraction on the merged features based on the high-level extraction layer to obtain the noise reduction features.

[0160] According to any one of the above embodiments, the feature conversion layer comprises an acoustic feature conversion layer and a noise feature conversion layer.

[0161] The noise reduction subunit is specifically configured to:

[0162] perform feature conversion on the acoustic features based on the acoustic feature conversion layer to obtain the acoustic converted features;

[0163] perform feature conversion on the noise features based on the noise feature conversion layer to obtain the noise converted features.

[0164] According to any one of the above embodiments, the noise encoding model and the feature classification model constitute a speech classification model, the speech classification model is trained based on acoustic features of a first sample speech and a noise label representing whether the first sample speech is a noisy speech, and the feature classification model is configured to determine whether the first sample speech is a noisy speech based on noise features of the first sample speech output by the noise encoding model.

[0165] According to any one of the above embodiments, the apparatus further comprises a noise encoding model determination unit configured to:

[0166] perform noise encoding on the acoustic features of the first sample speech based on an initial noise encoding model in an initial classification model to obtain initial noise features;

[0167] Based on an initial feature classification model in the initial classification model, the initial noise feature is applied to determine a speech classification result of the first sample speech;

[0168] Based on the noise label of the first sample speech and the speech classification result, the initial classification model is iterated in parameters to obtain a speech classification model;

[0169] Based on the initial noise coding model iterated in parameters in the speech classification model, the noise coding model is determined.

[0170] Based on any of the above embodiments, the apparatus further comprises a speech recognition model determination unit, configured to:

[0171] Determine an initial recognition model;

[0172] Based on the noise coding model, the acoustic feature of the second sample speech is noise coded to obtain a noise feature of the second sample speech;

[0173] Based on the initial recognition model, the acoustic feature and the noise feature of the second sample speech are applied to perform speech recognition to obtain a speech recognition result of the second sample speech;

[0174] Based on the speech recognition result and the speech recognition label of the second sample speech, the initial recognition model is iterated in parameters to obtain the speech recognition model.

[0175] Based on any of the above embodiments, the speech recognition model determination unit is specifically configured to:

[0176] Based on the speech recognition result and the speech recognition label of the second sample speech, the noise coding model and the initial recognition model are iterated in parameters to obtain a noise coding model iterated in parameters, and the speech recognition model;

[0177] The noise coding model iterated in parameters is taken as the noise coding model.

[0178] Based on any of the above embodiments, the speech recognition result is a phoneme classification result;

[0179] The apparatus further comprises a wake-up unit, configured to:

[0180] Based on the phoneme classification result, speech wake-up is performed.

[0181] Figure 8 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 8As shown, the electronic device can include a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 complete mutual communication through the communications bus 840. The processor 810 can invoke a logic instruction in the memory 830 to execute a speech recognition method, which includes: obtaining to-be-recognized speech; performing noise coding on acoustic features of the to-be-recognized speech based on a noise coding model to obtain noise features of the to-be-recognized speech, the noise features being used to represent features of noise carried by the to-be-recognized speech; and performing speech recognition by applying the acoustic features and the noise features of the to-be-recognized speech based on a speech recognition model to obtain a speech recognition result of the to-be-recognized speech; the speech recognition model being trained based on acoustic features and noise features of second sample speech and a speech recognition label of the second sample speech.

[0182] In addition, the logic instruction in the memory 830 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0183] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the speech recognition method provided by the above-mentioned methods, which includes: obtaining to-be-recognized speech; performing noise coding on acoustic features of the to-be-recognized speech based on a noise coding model to obtain noise features of the to-be-recognized speech, the noise features being used to represent features of noise carried by the to-be-recognized speech; and performing speech recognition by applying the acoustic features and the noise features of the to-be-recognized speech based on a speech recognition model to obtain a speech recognition result of the to-be-recognized speech; the speech recognition model being trained based on acoustic features and noise features of second sample speech and a speech recognition label of the second sample speech.

[0184] In another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the speech recognition method provided by the above method, and the method comprises: obtaining a to-be-recognized speech; performing noise coding on an acoustic feature of the to-be-recognized speech based on a noise coding model to obtain a noise feature of the to-be-recognized speech, the noise feature being used to represent a feature of noise carried by the to-be-recognized speech; and performing speech recognition on the acoustic feature and the noise feature of the to-be-recognized speech based on a speech recognition model to obtain a speech recognition result of the to-be-recognized speech; the speech recognition model is trained based on an acoustic feature and a noise feature of a second sample speech and a speech recognition label of the second sample speech.

[0185] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0186] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some part of the embodiment.

[0187] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A voice recognition method, characterized by, include: Acquire the speech to be recognized; Based on a noise coding model, the acoustic features of the speech to be identified are coded with noise to obtain the noise features of the speech to be identified. The noise features are used to characterize the noise carried by the speech to be identified. Based on the speech recognition model, the acoustic features and noise features of the speech to be recognized are applied to perform speech recognition, and the speech recognition result of the speech to be recognized is obtained; the speech recognition model is trained based on the acoustic features and noise features of the second sample speech, as well as the speech recognition label of the second sample speech; The speech recognition model includes a noise reduction feature extraction layer and a classification layer; The process of using a speech recognition model to perform speech recognition by applying the acoustic and noise features of the speech to be recognized, and obtaining the speech recognition result of the speech to be recognized, includes: Based on the noise reduction feature extraction layer, features that distinguish the acoustic features from the noise features are extracted to obtain noise reduction features; Based on the classification layer, the noise reduction features are applied to perform feature classification to obtain the speech recognition result. The noise reduction feature extraction layer includes a feature transformation layer, a feature merging layer, and a high-level extraction layer; The noise reduction feature extraction layer extracts features that distinguish the acoustic features from the noise features to obtain noise reduction features, including: Based on the feature conversion layer, the acoustic features and / or the noise features are converted to obtain acoustic converted features and noise converted features in the same feature space. The same feature space includes the feature space where the acoustic features are located, or the feature space where the noise features are located, or a feature space other than the feature space where the acoustic features are located and the feature space where the noise features are located. Based on the feature merging layer, acoustic conversion features and noise conversion features under the same feature space are merged to obtain merged features; Based on the high-level extraction layer, high-level feature extraction is performed on the merged features to obtain the noise reduction features.

2. The voice recognition method of claim 1, wherein, The feature conversion layer includes an acoustic feature conversion layer and a noise feature conversion layer; The step of performing feature transformation on the acoustic features and / or the noise features based on the feature transformation layer to obtain acoustic transformed features and noise transformed features in the same feature space includes: Based on the acoustic feature conversion layer, the acoustic features are transformed to obtain the acoustic converted features; Based on the noise feature conversion layer, the noise features are converted to obtain the noise converted features.

3. The voice recognition method of claim 1, wherein, The noise coding model and the feature classification model constitute a speech classification model. The speech classification model is trained based on the acoustic features of the first sample speech and noise labels that characterize whether the first sample speech is noisy speech. The feature classification model is used to determine whether the first sample speech is noisy speech based on the noise features of the first sample speech output by the noise coding model.

4. The voice recognition method of claim 3, wherein, The noise coding model is determined based on the following steps: Based on the initial noise coding model in the initial classification model, the acoustic features of the first sample speech are coded with noise to obtain the initial noise features; Based on an initial noise feature in an initial classification model, the initial noise feature is applied to determine a speech classification result of the first sample speech; Based on the noise label of the first sample speech and the speech classification result, the initial classification model is iterated to obtain a speech classification model; Based on the initial noise coding model after parameter iteration in the speech classification model, the noise coding model is determined.

5. The voice recognition method of claim 1, wherein, The training step of the speech recognition model comprises: An initial recognition model is determined; Based on the noise coding model, the acoustic feature of the second sample speech is noise coded to obtain a noise feature of the second sample speech; Based on the initial recognition model, the acoustic feature and the noise feature of the second sample speech are applied to perform speech recognition to obtain a speech recognition result of the second sample speech; Based on the speech recognition result and the speech recognition label of the second sample speech, the initial recognition model is iterated to obtain the speech recognition model.

6. The voice recognition method of claim 5, wherein, The iteration of the initial recognition model based on the speech recognition result and the speech recognition label of the second sample speech to obtain the speech recognition model comprises: Based on the speech recognition result and the speech recognition label of the second sample speech, the noise coding model and the initial recognition model are iterated to obtain a noise coding model after parameter iteration and the speech recognition model; The noise coding model after parameter iteration is used as the noise coding model.

7. The voice recognition method according to any one of claims 1 to 6, characterized by, The speech recognition result is a phoneme classification result; After the speech recognition result of the to-be-recognized speech is obtained by applying the acoustic feature and the noise feature of the to-be-recognized speech to the speech recognition model, the method further comprises: Based on the phoneme classification result, speech wake-up is performed.

8. A speech recognition apparatus characterized by comprising: The method comprises: A speech acquisition unit is configured to acquire a to-be-recognized speech; A noise coding unit is configured to code the acoustic feature of the to-be-recognized speech based on a noise coding model to obtain a noise feature of the to-be-recognized speech, wherein the noise feature is used to represent the feature of noise carried by the to-be-recognized speech; A speech recognition unit is configured to apply the acoustic feature and the noise feature of the to-be-recognized speech to the speech recognition model to obtain a speech recognition result of the to-be-recognized speech; The speech recognition model is trained based on the acoustic feature and the noise feature of the second sample speech and the speech recognition label of the second sample speech; The speech recognition model comprises a noise reduction feature extraction layer and a classification layer; The speech recognition unit is specifically configured to: Based on the noise reduction feature extraction layer, a feature that is different from the noise feature is extracted from the acoustic feature to obtain a noise reduction feature; Based on the classification layer, the noise reduction feature is applied to feature classification to obtain the speech recognition result; The noise reduction feature extraction layer comprises a feature conversion layer, a feature merging layer and a high-level extraction layer; The speech recognition unit is further specifically configured to: The acoustic feature and / or the noise feature are converted based on the feature conversion layer, to obtain acoustic converted features and noise converted features in a same feature space, the same feature space including a feature space in which the acoustic feature is located, or a feature space in which the noise feature is located, or a feature space other than the feature space in which the acoustic feature is located and the feature space in which the noise feature is located; The acoustic converted features and the noise converted features in the same feature space are merged based on the feature merging layer, to obtain merged features; The merged features are high-level feature extracted based on the high-level feature extraction layer, to obtain the noise reduction features.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the speech recognition method according to any one of claims 1 to 7 when executing the program.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the speech recognition method according to any one of claims 1 to 7 when executed by the processor.

11. A computer program product comprising a computer program, characterized in that, The computer program implements the speech recognition method according to any one of claims 1 to 7 when executed by the processor.

Images