Multi-scene speech recognition method and device, computer device and storage medium

Abstract

The application discloses a multi-scene speech recognition method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining to-be-recognized speech data; adopting a pre-trained multi-scene acoustic model to process the to-be-recognized speech data, and obtaining an acoustic recognition result; performing speech recognition on the to-be-recognized speech data, and obtaining to-be-recognized text data; adopting a pre-trained multi-scene language model to process the to-be-recognized text data, and obtaining a language recognition result; and decoding the acoustic recognition result and the language recognition result, and obtaining a speech recognition result. The method integrates speech recognition models of all scenes, selects a model according to different scenes and different dialogues, selects an optimal model to recognize speech, combines multiple models into a single model, and reduces deployment cost and model maintenance cost in the later period.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of speech recognition, and in particular to a multi-scene speech recognition method and device, computer equipment and a storage medium. BACKGROUND

[0002] Speech recognition is the basis for realizing artificial intelligence, and many robots, Internet of Things and mobile devices use speech as an interactive interface. The industry recording and annotation data required for speech recognition is relatively scarce, resulting in low recognition accuracy when the speech recognition system is applied to different fields. Even within the same field, there will be some differences in language expression between different products. The existing solutions for multi-scene speech recognition are as follows: one is to deploy models by scene, but the number of scenes is large, and the demand for manpower for deploying models will also increase, and the maintenance cost will also increase. The second is to train a large model by aggregating data from all scenes. Although different scenes have similar data, there are still many different parts, so the model cannot be generalized. When adjusting the accuracy of a scene, it cannot guarantee that the accuracy of other scenes will not decrease, so it cannot meet the business needs to the greatest extent. That is, the existing multi-scene speech recognition accuracy is low and cannot meet the business needs. SUMMARY

[0003] The embodiments of the present application provide a multi-scene speech recognition method, device, computer equipment and storage medium to solve the problem of low speech recognition accuracy in multiple scenes.

[0004] A multi-scene speech recognition method comprises:

[0005] obtaining speech data to be recognized;

[0006] processing the speech data to be recognized using a pre-trained multi-scene acoustic model to obtain an acoustic recognition result;

[0007] performing speech recognition on the speech data to be recognized to obtain text data to be recognized;

[0008] processing the text data to be recognized using a pre-trained multi-scene language model to obtain a language recognition result;

[0009] decoding the acoustic recognition result and the language recognition result to obtain a speech recognition result.

[0010] A multi-scene speech recognition device comprises:

[0011] a speech data to be recognized obtaining module for obtaining speech data to be recognized;

[0012] An acoustic recognition result acquisition module is configured to acquire an acoustic recognition result by processing the to-be-recognized voice data using a pre-trained multi-scene acoustic model.

[0013] A to-be-recognized text data acquisition module is configured to acquire to-be-recognized text data by performing voice recognition on the to-be-recognized voice data.

[0014] A language recognition result acquisition module is configured to acquire a language recognition result by processing the to-be-recognized text data using a pre-trained multi-scene language model.

[0015] A voice recognition result acquisition module is configured to acquire a voice recognition result by decoding the acoustic recognition result and the language recognition result.

[0016] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the multi-scene voice recognition method.

[0017] The multi-scene voice recognition method, device, computer device, and storage medium improve the accuracy of multi-scene voice recognition by acquiring to-be-recognized voice data, inputting the to-be-recognized voice data into a pre-trained multi-scene acoustic model to obtain a voice recognition result, acquiring to-be-recognized text data by performing voice recognition on the to-be-recognized voice data, inputting the to-be-recognized text data into a multi-scene language model to obtain a language recognition result, decoding the acoustic recognition result and the language recognition result to obtain a voice recognition result, and incorporating acoustic and text information to ensure the accuracy of language recognition and adding scene information to improve the accuracy of voice recognition in different scenes. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is an application environment diagram of a multi-scene voice recognition method in an embodiment of the present application.

[0020] Figure 2 is a flowchart of a multi-scene voice recognition method in an embodiment of the present application.

[0021] Figure 3 is another flowchart of a multi-scene voice recognition method in an embodiment of the present application.

[0022] Figure 4is another flow chart of the multi-scene speech recognition method in an embodiment of the present application;

[0023] Figure 5 is another flow chart of the multi-scene speech recognition method in an embodiment of the present application;

[0024] Figure 6 is another flow chart of the multi-scene speech recognition method in an embodiment of the present application;

[0025] Figure 7 is another flow chart of the multi-scene speech recognition method in an embodiment of the present application;

[0026] Figure 8 is another flow chart of the multi-scene speech recognition method in an embodiment of the present application;

[0027] Figure 9 is a schematic diagram of the multi-scene speech recognition device in an embodiment of the present application;

[0028] Figure 10 is a schematic diagram of the computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0030] The multi-scene speech recognition method provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 . Specifically, the multi-scene speech recognition method is applied in a multi-scene speech recognition system, which includes a client and a server as shown in Figure 1 . The client and the server communicate through a network to realize multi-scene speech recognition. The client, also known as the user end, is a program that provides local services for clients corresponding to the server. The client can be installed on, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0031] In an embodiment, as shown in Figure 2 , a speech recognition method is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0032] S201: Obtain voice data to be recognized;

[0033] S202: Process the voice data to be recognized by using a pre-trained multi-scene acoustic model to obtain an acoustic recognition result;

[0034] S203: Perform voice recognition on the voice data to be recognized to obtain text data to be recognized;

[0035] S204: Process the text data to be recognized by using a pre-trained multi-scene language model to obtain a language recognition result;

[0036] S205: Decode the acoustic recognition result and the language recognition result to obtain a voice recognition result.

[0037] The voice data to be recognized is voice data prepared for a recognition task.

[0038] As an example, in step S201, the server can receive voice data to be recognized uploaded by a user through a client or obtained from other channels, to prepare for a subsequent recognition task.

[0039] The multi-scene acoustic model is an acoustic model trained using acoustic features extracted from multi-scene training voice data. The acoustic features are extracted from multi-scene training voice data, and the acoustic features are obtained by a pre-training model and clustering to obtain a scene category. The acoustic model is obtained by fusing the intermediate layer of the basic acoustic model and then training the full connection layer. The acoustic recognition result is a recognition result obtained by processing the voice data to be recognized by the multi-scene acoustic model.

[0040] As an example, in step S202, the server inputs the voice data to be recognized into the pre-trained multi-scene acoustic model to obtain an acoustic recognition result output by the multi-scene acoustic model, to improve the accuracy of multi-scene voice recognition. In this example, the multi-scene acoustic model is an acoustic model trained based on multi-scene training voice data, so that when recognizing multi-scene voice data to be recognized, the accuracy of the acoustic recognition result output can be ensured.

[0041] The text data to be recognized is text data obtained by performing voice recognition on the voice data to be recognized.

[0042] As an example, in step S203, the server performs voice recognition on the voice data to be recognized. Specifically, a voice recognition tool can be used to recognize the voice data to be recognized to obtain text data to be recognized, to prepare for multi-scene text recognition.

[0043] The multi-scene language model is a language model obtained by training on training text data from multiple scenarios. This multi-scene language model includes a base language model, scenario-specific language models, and rescoring. The language recognition result is obtained by processing the text data to be recognized using the pre-trained multi-scene language model.

[0044] As an example, in step S204, the server inputs the text data to be recognized into a pre-trained multi-scene language model and outputs the language recognition result. Specifically, the text data is input into the multi-scene language model, where the basic language models encompassing multiple scenes segment the input text data by length, generate text features using the pre-trained model, classify the features to obtain category information, and then re-score using the corresponding trained scene language models. In this example, because the multi-scene language model is a language model trained on training text data from multiple scenes, the accuracy of the output language recognition result can be guaranteed when recognizing text data corresponding to multiple scenes.

[0045] The speech recognition result is obtained by decoding the acoustic recognition result and the language recognition result using a decoder.

[0046] As an example, in step S205, the server uses a decoder to decode the acoustic recognition result and the speech recognition result to obtain the speech recognition result. By using the acoustic recognition result and the speech recognition result for decoding, acoustic and text information are integrated, ensuring the accuracy of the speech recognition result; multi-scene recognition will not affect its accuracy.

[0047] In the multi-scene speech recognition method provided in this embodiment, the speech data to be recognized is first acquired, and then input into a pre-trained multi-scene acoustic model to obtain acoustic recognition results, ensuring the accuracy of the acoustic recognition results and making them unrestricted by the scene in which the speech data to be recognized is located. Next, the speech data to be recognized is subjected to speech recognition to obtain text data to be recognized, and then the text data to be recognized is input into a multi-scene language model to obtain language recognition results, ensuring the accuracy of the language recognition results and making them unrestricted by the scene in which the text data to be recognized is located. Finally, the acoustic recognition results and language recognition results are decoded to obtain the speech recognition result, incorporating acoustic and text information to ensure the accuracy of the speech recognition result. Since both the multi-scene acoustic model and the multi-scene language model incorporate scene information, the accuracy of the speech recognition result in different scenes is guaranteed.

[0048] In one embodiment, such as Figure 3 As shown, in step S202, a pre-trained multi-scene acoustic model is used to process the speech data to be recognized and obtain the acoustic recognition result, including:

[0049] S301: Feature extraction is performed on the to-be-identified voice data to obtain to-be-identified acoustic features;

[0050] S302: Scene coding processing is performed on the to-be-identified acoustic features to obtain an acoustic scene coding vector.

[0051] S303: The to-be-identified acoustic features are processed by using a target scene acoustic model corresponding to the acoustic scene coding vector to obtain an acoustic recognition result.

[0052] The to-be-identified acoustic features are acoustic features obtained by performing feature extraction on the to-be-identified voice data.

[0053] As an example, in step S301, the server performs feature extraction on the to-be-identified voice data to obtain to-be-identified acoustic features. Specifically, the server first extracts acoustic features MFCC of the training voice data by using an MFCC extraction method, converts the obtained acoustic features MFCC into a frequency spectrum FBANK (Filter Bank) by performing IDCT (inverse discrete cosine transform) on the acoustic features MFCC, and obtains to-be-identified acoustic features. In this example, after obtaining the frequency spectrum FBANK (Filter Bank) as the to-be-identified acoustic features, the server can also perform spectral perturbation on the obtained frequency spectrum FBANK by using a SpecAugment technology, and determines the frequency spectrum after the spectral perturbation as the to-be-identified acoustic features. The SpecAugment is an enhancement method based on the logarithmic mel spectrum of input audio rather than the original audio itself. This method is simple and has low computational cost, because it directly acts on the logarithmic mel spectrum and does not require any additional data. In this example, the spectral perturbation is added to make the model more robust to noise.

[0054] The acoustic scene coding vector is a vector formed by performing scene coding processing on the to-be-identified acoustic features. The acoustic scene coding vector incorporates scene information, and the data can be fine-tuned according to the scene information.

[0055] As an example, in step S302, the server performs coding processing on the to-be-identified acoustic features to obtain an acoustic scene coding vector. The acoustic scene coding vector can be understood as a coding vector including acoustic scene information, and the acoustic scene coding vector can be used to verify the feasibility of multi-scene voice recognition.

[0056] The acoustic recognition result is a result obtained by identifying the to-be-identified acoustic features by using a target scene acoustic model corresponding to the acoustic scene coding vector. The target scene acoustic model is an acoustic model corresponding to the acoustic scene coding vector.

[0057] As an example, in step S303, the server processes the to-be-identified acoustic feature by using the target scene acoustic model corresponding to the acoustic scene coding vector, to obtain an acoustic recognition result. The acoustic recognition result incorporates the information of the acoustic scene, thereby improving the multi-scene speech recognition degree.

[0058] In the multi-scene speech recognition method provided in this embodiment, the to-be-identified acoustic feature is extracted from the to-be-identified speech data, and the robustness of the multi-scene acoustic model is improved by performing spectrum perturbation during the extraction of the to-be-identified acoustic feature. Then, the to-be-identified acoustic feature is coded to obtain an acoustic scene coding vector, thereby improving the feasibility of multi-scene speech recognition by incorporating the scene information. Finally, the to-be-identified acoustic feature is processed by using the target scene acoustic model corresponding to the acoustic scene coding vector to obtain an acoustic recognition result, thereby improving the multi-scene speech recognition degree, i.e., the to-be-identified acoustic feature of the feature scene is processed by using the acoustic model of the specific scene of the target scene acoustic model corresponding to the acoustic scene coding vector, which can guarantee the accuracy of the recognized acoustic recognition result.

[0059] In an embodiment, as shown in FIG. 2, Figure 4 In step S204, the to-be-identified text data is processed by using the pre-trained multi-scene language model to obtain a language recognition result, including:

[0060] S401: identifying the to-be-identified text data to obtain a to-be-identified text vector;

[0061] S402: determining a language scene vector corresponding to the to-be-identified text vector according to the to-be-identified text vector;

[0062] S403: identifying the to-be-identified text vector by using the target scene language model corresponding to the language scene vector to obtain a language recognition result.

[0063] The to-be-identified text vector is obtained by inputting the to-be-identified text data into a pre-trained model.

[0064] As an example, in step S401, the server inputs the to-be-identified text data into a pre-trained statistical language model, performs length segmentation on the to-be-identified text data, and then inputs the segmented to-be-identified text data into a text pre-training model to convert the to-be-identified text data into a to-be-identified text vector.

[0065] The language scene vector is obtained by processing the to-be-identified text vector to determine the vector corresponding to the scene to which the to-be-identified text vector belongs. The language scene vector is obtained by extracting features by using a pre-training model and calculating the similarity.

[0066] As an example, in step S402, the server calculates the similarity between the to-be-identified text vector and the vectors in the scenario-related language model, and obtains a language scenario vector that is closest to the to-be-identified text vector. The efficiency of multi-scenario speech recognition is improved.

[0067] The language recognition result is a result obtained by the target scenario language model corresponding to the language scenario vector from recognizing the to-be-identified text vector. The target scenario acoustic model is a language model corresponding to the language scenario vector.

[0068] As an example, in step S403, the server obtains the target scenario language model corresponding to the language scenario vector, inputs the to-be-identified text vector into the target language model for recognition, and finally obtains the language recognition result.

[0069] In the multi-scenario speech recognition method provided in this embodiment, the pre-training model is used to obtain the to-be-identified text vector, thereby providing a basis for multi-scenario speech recognition. The similarity between the to-be-identified text vector and the existing text vector in the model is calculated to obtain the language scenario vector, thereby improving the efficiency of multi-scenario speech recognition. Finally, the language scenario vector is used to obtain the corresponding target scenario speech model, and the language recognition result is obtained, thereby guaranteeing the feasibility of multi-scenario language recognition.

[0070] In an embodiment, as shown in Figure 5 Before obtaining the to-be-identified speech data, that is, the multi-scenario speech recognition method further includes:

[0071] S501: Obtain training speech data corresponding to multiple scenarios.

[0072] S502: Perform feature extraction on the training speech data to obtain training acoustic features.

[0073] S503: Perform scenario coding processing on the training acoustic features to obtain acoustic scenario coding vectors.

[0074] S504: Input the training acoustic features and the acoustic scenario coding vectors into a neural network model for training to obtain a multi-scenario acoustic model, the multi-scenario acoustic model including at least one acoustic scenario coding vector and a target scenario acoustic model corresponding to the acoustic scenario coding vector.

[0075] The training speech data is speech data used for model training. The training speech data can be training speech data corresponding to multiple scenarios, which can be training speech data obtained from different users, different modules, and different time periods.

[0076] As an example, in step S501, the server obtains speech data corresponding to multiple scenarios, which is used as training speech data of the multi-scenario acoustic model.

[0077] The training acoustic feature is an acoustic feature extracted from the training speech data feature. The training acoustic feature can be an acoustic feature formed by the acoustic feature MFCC.

[0078] As an example, in step S502, the server extracts features from the acquired multi-scene training speech data to obtain training acoustic features. The acoustic features can be a spectrum obtained by transforming the acoustic MFCC, or a spectrum formed by perturbing the transformed spectrum. In this example, the server first extracts the acoustic feature MFCC of the training speech data using the MFCC extraction method, converts the acoustic feature MFCC into a spectrum FBANK (Filter Bank) by IDCT (Inverse Discrete Cosine Transform), and then uses the SpecAugment technology to perform spectrum perturbation on the obtained spectrum FBANK. In this example, the spectrum perturbation is added to make the model more robust to noise.

[0079] The acoustic scene encoding vector is a vector formed by encoding the training acoustic feature with a scene. The acoustic scene encoding vector incorporates scene information and can be used to fine-tune the data.

[0080] As an example, in step S503, the server encodes the training acoustic feature to obtain an acoustic scene encoding vector and acoustic scene information, thereby achieving the purpose of training for a specific scene. The server can fine-tune the training scene, and only the scene-related parameters will change when training a scene, while other parameters remain unchanged, thereby improving the feasibility of multi-scene acoustic model training.

[0081] The multi-scene acoustic model is obtained by inputting the training acoustic feature and the acoustic scene encoding vector into a neural network for training. The multi-scene acoustic model includes at least one acoustic scene encoding vector and a target scene acoustic model corresponding to the acoustic scene encoding vector. In this example, a speech recognition neural network is constructed using CNN, TDNN, LSTM, and a fully connected layer, and the target scene acoustic model is the acoustic model corresponding to the acoustic scene encoding vector.

[0082] As an example, in step S504, the server inputs the training acoustic feature and the acoustic scene encoding vector into a speech recognition neural network composed of CNN, DNN, LSTM, and a fully connected layer to obtain a multi-scene acoustic model. The multi-scene acoustic model includes at least one acoustic scene encoding vector and a target scene acoustic model corresponding to the acoustic scene encoding vector, thereby providing a basis for multi-scene speech recognition.

[0083] The multi-scene speech recognition method provided in the embodiment first acquires training speech data corresponding to multiple scenes, thereby providing a basis for subsequent training of a multi-scene acoustic model; then feature extraction is performed on the acquired training speech data to acquire training acoustic features, and spectrum perturbation can be performed in the training acoustic feature acquisition process to improve the robustness of the model to noise; then scene coding processing is performed on the training acoustic features to obtain acoustic scene coding vectors, so that the acoustic features incorporate scene information and the feasibility of training of the multi-scene acoustic model is improved; and the training acoustic features and the acoustic scene coding vectors are input to a neural network for training to obtain a multi-scene acoustic model, which includes at least one acoustic scene coding vector and a target scene acoustic model corresponding to the acoustic scene coding vector, thereby providing a basis for multi-scene speech recognition.

[0084] In an embodiment, as shown in Figure 6 In step S503, scene coding processing is performed on the training acoustic features to obtain acoustic scene coding vectors, including:

[0085] S601: performing clustering processing on the training acoustic features to determine a scene category corresponding to the training acoustic features;

[0086] S602: encoding the scene category corresponding to the training acoustic features to obtain an acoustic scene coding vector.

[0087] The scene category is obtained by clustering processing, and in this example, the clustering processing adopts a k-means algorithm. The k-means algorithm is a clustering algorithm based on division, which divides n data objects into k clusters with a parameter k, so that the similarity within the cluster is high and the similarity between the clusters is low.

[0088] As an example, in step S601, the server adopts the k-means algorithm to perform clustering processing on the training acoustic features to form multiple clusters. Training acoustic features with high similarity will be in one cluster, and there will be one same scene category. In the subsequent process, only the scene category corresponding to each cluster needs to be processed, thereby improving the efficiency of multi-scene speech recognition.

[0089] In this example, the encoding method adopts One-Hot encoding, which is also called one-bit effective encoding. The method uses an N-bit state register to encode multiple states. Each state has its own register bit, and only one bit is effective at any time, thereby solving the problem of difficulty of attribute data processing by the classifier and also playing a role in feature expansion to a certain extent.

[0090] As an example, in step S602, the server encodes the scene categories corresponding to the training acoustic features by using one-hot encoding to obtain acoustic scene encoding vectors, thereby providing feasibility for subsequent multi-scene speech recognition.

[0091] In the multi-scene speech recognition method provided in this embodiment, the k-means algorithm is used to cluster the training acoustic features to determine the scene categories corresponding to the training acoustic features, thereby improving the multi-scene speech recognition efficiency. Finally, the one-hot encoding method is used to encode the scene categories corresponding to the training acoustic features to obtain acoustic scene encoding vectors, thereby guaranteeing the feasibility of multi-scene speech recognition.

[0092] In an embodiment, as shown in Figure 7 In step S504, the training acoustic features and the acoustic scene encoding vectors are input into the neural network model for training to obtain a multi-scene acoustic model. The multi-scene acoustic model includes at least one acoustic scene encoding vector and a target scene acoustic model corresponding to the acoustic scene encoding vector, and includes the following:

[0093] S701: input the training acoustic features into a first neural network model for training to obtain first acoustic features;

[0094] S702: input the first acoustic features into a second neural network model for training to obtain second acoustic features;

[0095] S703: fuse the second acoustic features and the acoustic scene encoding vectors to obtain third acoustic features;

[0096] S704: input the third acoustic features into a third neural network model for training to obtain fourth acoustic features;

[0097] S705: fuse the fourth acoustic features and the acoustic scene encoding vectors to obtain fifth acoustic features;

[0098] S706: input the fifth acoustic features into a full connection layer for processing to obtain a target scene acoustic model corresponding to each acoustic scene encoding vector, and obtain a multi-scene acoustic model according to the target scene acoustic model corresponding to at least one acoustic scene encoding vector.

[0099] The first acoustic features are obtained by inputting the training acoustic features into the first neural network model for training. In this example, the first neural network model is a CNN (Convolutional Neural Network) which is a convolutional neural network. The CNN is used to extract features of things in a certain model.

[0100] As an example, in step S701, the server inputs the training acoustic feature into the CNN, a first neural network model, for training to obtain a first acoustic feature. The CNN can better extract information from the training acoustic feature and learn some local information.

[0101] The second acoustic feature is obtained by inputting the training acoustic feature into a second neural network model for training, and the second acoustic feature is an intermediate layer feature of the second neural network model. In this example, the second neural network model is a TDNN (Time Delay Neural Networks). The TDNN is a time delay neural network, which is operated by multiple interconnected layers composed of clusters, like other neural networks.

[0102] As an example, in step S702, the server inputs the first acoustic feature into the TDNN, a second neural network model, for training to obtain a second acoustic feature. The TDNN aims to improve the field of view of the multi-scene acoustic model.

[0103] The third acoustic feature is obtained by fusing the second acoustic feature and the acoustic scene encoding vector. The fusion processing is to use a full connection layer to change the vector dimensions to be consistent and to perform addition processing.

[0104] As an example, in step S703, the server uses a full connection layer to convert the dimension of the scene encoding vector, changes the dimension of the acoustic scene encoding vector to be consistent with the dimension of the second acoustic feature, and then adds the second acoustic feature and the acoustic scene encoding vector to obtain the third acoustic feature.

[0105] The fourth acoustic feature is obtained by inputting the training acoustic feature into a third neural network model for training, and the fourth acoustic feature is an intermediate layer feature of the third neural network model. In this example, the third neural network model is an LSTM (Long short-term memory). The LSTM is a special RNN (Recurrent Neural Network) that is mainly used to solve the problems of gradient disappearance and gradient explosion in long sequence training.

[0106] As an example, in step S704, the server inputs the third acoustic feature into the LSTM, a third neural network model, for training to obtain a fourth acoustic feature. This can make the model learn the context information and improve the feasibility of multi-scene acoustic model training.

[0107] The fifth acoustic feature is obtained by fusing the fourth acoustic feature and the acoustic scene encoding vector. The fusion processing is to use a full connection layer to change the vector dimensions to be consistent and to perform addition processing.

[0108] As an example, in step S705, the server converts the dimension of the scene encoding vector by using a full connection layer, makes the dimension of the acoustic scene encoding vector consistent with the dimension of the fourth acoustic feature, and then adds the fourth acoustic feature and the scene encoding vector to obtain a fifth acoustic feature.

[0109] As an example, in step S706, the server inputs the fifth acoustic feature into a full connection layer for processing to obtain a target scene acoustic model corresponding to the acoustic scene encoding vector, thereby obtaining a multi-scene acoustic model and providing a basis for multi-scene speech recognition.

[0110] In the multi-scene speech recognition method provided in this embodiment, the training acoustic feature is input into a first neural network model to obtain a first acoustic feature, so that the multi-scene acoustic model can better extract information from the acoustic feature and learn some local information; the first acoustic feature is input into a second neural network model to obtain a second acoustic feature, so that the multi-scene acoustic model has a larger field of view; the second acoustic feature and the acoustic scene encoding vector are fused to obtain a third acoustic feature, the third acoustic feature is input into a third neural network model to obtain a fourth acoustic feature, so that the multi-scene acoustic model learns context information; the fourth acoustic feature and the acoustic scene encoding vector are fused to obtain a fifth acoustic feature, so that the multi-scene acoustic model obtains scene information; and the fifth acoustic feature is input into a full connection layer to obtain a target scene acoustic model corresponding to each acoustic scene encoding vector, thereby obtaining a multi-scene acoustic model and providing a basis for multi-scene speech recognition.

[0111] In an embodiment, as shown in FIG. 8, before obtaining the to-be-recognized speech data, the multi-scene speech recognition method further includes: Figure 8

[0112] S801: obtaining training text data corresponding to multiple scenes;

[0113] S802: training the training text data to obtain a training text vector;

[0114] S803: clustering the training text vector to obtain a scene category corresponding to the training text vector;

[0115] S804: encoding the scene category corresponding to the training text vector to obtain a language scene vector;

[0116] S805: inputting the training text data corresponding to the language scene vector into a language model for training to obtain a multi-scene language model, the multi-scene language model including at least one language scene vector and a target scene language model corresponding to each language scene vector.

[0117] ​The training text data is obtained according to multiple scenes, and the training text data corresponding to the multiple scenes can be training voice data obtained for different users, different modules and different time periods.

[0118] As an example, in step S801, the server obtains training text data corresponding to multiple scenes.

[0119] The training text vector is obtained by training the training text data, the text training is performed by a statistical language model, and the text vector is generated by a text pre-training model. In this example, the statistical language model adopts an N-gram model, and the text pre-training model adopts a Bert model. The N-gram is an algorithm based on a statistical language model. The basic idea is to perform a sliding window operation on the content in the text according to the byte, forming a byte segment sequence with a length of N. The re-scoring technology uses the ngram re-scoring mechanism in the open source speech recognition framework kaldi. BERT (Bidirectional Encoder Representation from Transformers) is a pre-trained language representation model. It emphasizes that it is no longer pre-trained by using a traditional one-way language model or by shallowly splicing two one-way language models.

[0120] As an example, in step S802, the server trains a general N-gram model for the training text data. The purpose is to segment the input training data to a certain length, and then input the segmented text to the Bret model to generate a training text vector, providing a basis for the multi-scene language training model.

[0121] The scene category corresponding to the training text vector is obtained by clustering the training text vector. In this example, the clustering method adopted is the k-means algorithm.

[0122] As an example, in step S803, the server adopts the k-means algorithm to cluster the training text vector, forming multiple clusters. Training text vectors with high similarity will be in one cluster, and there will be a same scene category. In the subsequent process, only the scene category corresponding to each cluster needs to be processed, thereby improving the multi-scene speech recognition efficiency.

[0123] The language scene vector is obtained by encoding the scene category corresponding to the training text vector by an encoder.

[0124] As an example, in step S804, the server encodes the scene category corresponding to the training text vector by an encoder to obtain a language scene vector, thereby improving the feasibility of training the multi-scene language model.

[0125] The multi-scene language model is obtained by inputting the training text vector corresponding to the language scene vector into the language model.

[0126] As an example, in step S805, the service inputs the training text data corresponding to the language scene vector into the language model to train and obtain a multi-scene language model. In this example, the training process uses a re-scoring mechanism. First, an N-Gram is used to generate a recognition result. Then, a Bert is used to generate text features for the sentence. Then, the sentence is classified to obtain category information. Finally, a multi-scene speech model is used for re-scoring. In the multi-scene speech recognition method provided in this embodiment, first, a pre-trained BERT model is used to train the text data corresponding to the multi-scene to obtain a training text vector, which provides a basis for training a multi-scene language model. Second, a k-means clustering algorithm is used to cluster the training text vector to obtain a scene code corresponding to the training text, which improves the multi-scene speech recognition efficiency. Finally, the scene category corresponding to the training text vector is encoded to obtain a language scene vector. Then, the training text data corresponding to the language scene vector is input into the language model to train and obtain a multi-scene language model, which provides a basis for multi-scene speech recognition.

[0127] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0128] In an embodiment, a multi-scene speech recognition device is provided, which corresponds to the multi-scene speech recognition method described in the above embodiment. As shown in Figure 9 The multi-scene speech recognition method device includes a to-be-recognized speech data acquisition module 901, an acoustic recognition result acquisition module 902, a to-be-recognized text data acquisition module 903, a language recognition result acquisition module 904, and a speech recognition result acquisition module 905. The functions of each module are described in detail as follows.

[0129] The to-be-recognized speech data acquisition module 901 is configured to acquire to-be-recognized speech data.

[0130] The acoustic recognition result acquisition module 902 is configured to process the to-be-recognized speech data by using a pre-trained multi-scene acoustic model to obtain an acoustic recognition result.

[0131] The to-be-recognized text data acquisition module 903 is configured to perform speech recognition on the to-be-recognized speech data to obtain to-be-recognized text data.

[0132] The language recognition result obtaining module 904 is configured to process the to-be-recognized text data by using a pre-trained multi-scene language model, and obtain a language recognition result.

[0133] The speech recognition result obtaining module 905 is configured to decode the acoustic recognition result and the language recognition result, and obtain a speech recognition result.

[0134] In an embodiment, the acoustic recognition result obtaining module 902 comprises:

[0135] The to-be-processed speech waveform obtaining unit is configured to perform feature extraction on the to-be-recognized speech data, and obtain to-be-recognized acoustic features.

[0136] The acoustic scene obtaining unit is configured to perform scene coding processing on the to-be-recognized acoustic features, and obtain an acoustic scene coding vector.

[0137] The acoustic recognition result obtaining unit is configured to perform processing on the to-be-recognized acoustic features by using a target scene acoustic model corresponding to the acoustic scene coding vector, and obtain an acoustic recognition result.

[0138] In an embodiment, the language recognition result obtaining module 904 comprises:

[0139] The to-be-recognized text vector obtaining unit is configured to perform recognition on the to-be-recognized text data, and obtain a to-be-recognized text vector.

[0140] The language scene vector determining unit is configured to determine a language scene vector corresponding to the to-be-recognized text vector according to the to-be-recognized text vector.

[0141] The language recognition result obtaining unit is configured to perform recognition on the to-be-recognized text vector by using a target scene language model corresponding to the language scene vector, and obtain a language recognition result.

[0142] In an embodiment, the multi-scene speech recognition apparatus further comprises:

[0143] The training speech data obtaining unit is configured to obtain training speech data corresponding to multiple scenes.

[0144] The training acoustic feature obtaining unit is configured to perform feature extraction on the training speech data, and obtain training acoustic features.

[0145] The training acoustic scene obtaining unit is configured to perform scene coding processing on the training acoustic features, and obtain an acoustic scene coding vector.

[0146] The multi-scene acoustic model obtaining unit is configured to input the training acoustic features and the acoustic scene coding vector into a neural network model for training, and obtain a multi-scene acoustic model, wherein the multi-scene acoustic model comprises at least one acoustic scene coding vector and a target scene acoustic model corresponding to the acoustic scene coding vector.

[0147] In an embodiment, the training acoustic scene obtaining unit comprises:

[0148] a scene category determining subunit configured to perform clustering processing on the training acoustic feature to determine a scene category corresponding to the training acoustic feature;

[0149] a training acoustic scene obtaining subunit configured to encode the scene category corresponding to the training acoustic feature to obtain an acoustic scene encoding vector.

[0150] In an embodiment, the multi-scene acoustic model obtaining unit comprises:

[0151] a first acoustic feature obtaining subunit configured to input the training acoustic feature into a first neural network model to perform training to obtain a first acoustic feature;

[0152] a second acoustic feature obtaining subunit configured to input the first acoustic feature into a second neural network model to perform training to obtain a second acoustic feature;

[0153] a third acoustic feature obtaining subunit configured to perform fusion processing on the second acoustic feature and the acoustic scene encoding vector to obtain a third acoustic feature;

[0154] a fourth acoustic feature obtaining subunit configured to input the third acoustic feature into a third neural network model to perform training to obtain a fourth acoustic feature;

[0155] a fifth acoustic feature obtaining subunit configured to perform fusion processing on the fourth acoustic feature and the acoustic scene encoding vector to obtain a fifth acoustic feature;

[0156] a multi-scene acoustic model obtaining subunit configured to input the fifth acoustic feature into a full connection layer to perform processing to obtain a target scene acoustic model corresponding to each acoustic scene encoding vector, and obtain a multi-scene acoustic model according to the target scene acoustic model corresponding to at least one acoustic scene encoding vector.

[0157] In an embodiment, the multi-scene speech recognition device further comprises:

[0158] a training text data obtaining unit configured to obtain training text data corresponding to multiple scenes;

[0159] a training text vector obtaining unit configured to perform training on the training text data to obtain a training text vector;

[0160] a scene category obtaining unit configured to perform clustering on the training text vector to obtain a scene category corresponding to the training text vector;

[0161] a language scene vector obtaining unit configured to encode the scene category corresponding to the training text vector to obtain a language scene vector.

[0162] The multi-scene language model obtaining unit is configured to input the training text data corresponding to the language scene vector into the language model to train the language model, and obtain a multi-scene language model, wherein the multi-scene language model comprises at least one language scene vector and a target scene language model corresponding to each language scene vector

[0163] The specific limitation of the multi-scene speech recognition device can refer to the limitation of the multi-scene speech recognition method in the foregoing, and will not be repeated here. Each module in the multi-scene speech recognition device described above can be realized by software, hardware, and a combination thereof, in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0164] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 10 The computer device comprises a processor, a memory, a network interface, and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store data used or generated in the process of performing multi-scene speech recognition. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a multi-scene speech recognition method.

[0165] In an embodiment, a computer device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor implements the multi-scene speech recognition method in the above-mentioned embodiments when executing the computer program, for example, S201-S205 shown in Figure 2 , or Figures 3 to 8 For the sake of brevity, the functions of the modules / units in this embodiment will not be repeated here. Alternatively, the processor implements the functions of each module / unit in the multi-scene speech recognition device in this embodiment when executing the computer program, for example, the functions of the to-be-recognized speech data obtaining module 901, the acoustic recognition result obtaining module 902, the to-be-recognized text data obtaining module 903, the language recognition result obtaining module 904, and the speech recognition result obtaining module 905 shown in Figure 9 , for the sake of brevity, the functions of the modules / units in this embodiment will not be repeated here.

[0166] In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the multi-scene speech recognition method in the above embodiment, for example Figure 2 as shown in S201-S205, or Figures 3 to 8 as shown in the above embodiment, and details are not repeated here. Alternatively, the computer program is executed by the processor to implement the functions of each module / unit in the above embodiment of the multi-scene speech recognition device, for example Figure 9 as shown in the above embodiment, and details are not repeated here. Alternatively, the computer program is executed by the processor to implement the functions of each module / unit in the above embodiment of the multi-scene speech recognition device, for example

[0167] A person of ordinary skill in the art can understand that all or part of the processes in the above embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above embodiments of each method. In the embodiments provided in the present application, any reference to a memory, storage, database, or other medium can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional units and modules is exemplified. In actual applications, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above described functions.

[0169] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A multi-scenario speech recognition method, characterized by, The method comprises the following steps: obtaining training voice data corresponding to multiple scenes; extracting features from the training voice data to obtain training acoustic features; performing scene coding processing on the training acoustic features to obtain acoustic scene coding vectors; inputting the training acoustic features into a first neural network model for training to obtain first acoustic features; inputting the first acoustic features into a second neural network model for training to obtain second acoustic features; performing fusion processing on the second acoustic features and the acoustic scene coding vectors to obtain third acoustic features; inputting the third acoustic features into a third neural network model for training to obtain fourth acoustic features; performing fusion processing on the fourth acoustic features and the acoustic scene coding vectors to obtain fifth acoustic features; performing processing on the fifth acoustic features in a full connection layer to obtain a target scene acoustic model corresponding to each acoustic scene coding vector; obtaining a multi-scene acoustic model according to at least one target scene acoustic model corresponding to the acoustic scene coding vector, wherein the multi-scene acoustic model comprises at least one acoustic scene coding vector and the target scene acoustic model corresponding to the acoustic scene coding vector; obtaining to-be-recognized voice data; processing the to-be-recognized voice data by using a pre-trained multi-scene acoustic model to obtain an acoustic recognition result; performing voice recognition on the to-be-recognized voice data to obtain to-be-recognized text data; processing the to-be-recognized text data by using a pre-trained multi-scene language model to obtain a language recognition result; decoding the acoustic recognition result and the language recognition result to obtain a voice recognition result.

2. The multi-scene speech recognition method of claim 1, wherein, The processing of the to-be-recognized voice data by using the pre-trained multi-scene acoustic model to obtain the acoustic recognition result comprises the following steps: extracting features from the to-be-recognized voice data to obtain to-be-recognized acoustic features; performing scene coding processing on the to-be-recognized acoustic features to obtain acoustic scene coding vectors; processing the to-be-recognized acoustic features by using a target scene acoustic model corresponding to the acoustic scene coding vector to obtain an acoustic recognition result.

3. The multi-scene speech recognition method of claim 1, wherein, The processing of the to-be-recognized text data by using the pre-trained multi-scene language model to obtain the language recognition result comprises the following steps: recognizing the to-be-recognized text data to obtain a to-be-recognized text vector; determining a language scene vector corresponding to the to-be-recognized text vector according to the to-be-recognized text vector; recognizing the to-be-recognized text vector by using a target scene language model corresponding to the language scene vector to obtain a language recognition result.

4. The multi-scene speech recognition method of claim 1, wherein, The scene coding processing on the training acoustic features to obtain acoustic scene coding vectors comprises the following steps: performing clustering processing on the training acoustic features to determine a scene category corresponding to the training acoustic features; performing coding on the scene category corresponding to the training acoustic features to obtain acoustic scene coding vectors.

5. The multi-scene voice recognition method according to claim 1, before the step of obtaining to-be-recognized voice data, the method further comprises the following steps: obtaining training text data corresponding to multiple scenes; training the training text data to obtain training text vectors; The training text vectors are clustered to obtain a scenario category corresponding to the training text vectors; The scenario category corresponding to the training text vectors is encoded to obtain a language scenario vector; The training text data corresponding to the language scenario vector is input into a language model to obtain a multi-scenario language model, the multi-scenario language model comprising at least one language scenario vector and a target scenario language model corresponding to each language scenario vector.

6. A multi-scenario speech recognition apparatus, characterized by comprising: Comprise: The voice data to be recognized is obtained by the voice data to be recognized acquisition module; The acoustic recognition result is obtained by processing the voice data to be recognized by using the pre-trained multi-scenario acoustic model. The text data to be recognized is obtained by performing speech recognition on the voice data to be recognized. The language recognition result is obtained by processing the text data to be recognized by using the pre-trained multi-scenario language model. The speech recognition result is obtained by decoding the acoustic recognition result and the language recognition result. The multi-scenario voice recognition device further comprises: obtaining training voice data corresponding to a multi-scenario; extracting features from the training voice data to obtain training acoustic features; performing scenario encoding processing on the training acoustic features to obtain acoustic scenario encoding vectors; inputting the training acoustic features into a first neural network model to obtain first acoustic features; inputting the first acoustic features into a second neural network model to obtain second acoustic features; performing fusion processing on the second acoustic features and the acoustic scenario encoding vectors to obtain third acoustic features; inputting the third acoustic features into a third neural network model to obtain fourth acoustic features; performing fusion processing on the fourth acoustic features and the acoustic scenario encoding vectors to obtain fifth acoustic features; inputting the fifth acoustic features into a full connection layer to obtain a target scenario acoustic model corresponding to each acoustic scenario encoding vector; obtaining a multi-scenario acoustic model according to at least one target scenario acoustic model corresponding to the acoustic scenario encoding vector, the multi-scenario acoustic model comprising at least one acoustic scenario encoding vector and a target scenario acoustic model corresponding to the acoustic scenario encoding vector.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the multi-scenario voice recognition method of any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the multi-scenario voice recognition method of any one of claims 1 to 5.

Images