A recognition model training method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a kind of identification model training method, device, electronic equipment and storage medium, by obtaining speech sample, speech sample includes voice frame, speech sample has sample label corresponding;Through keyword recognition model, acoustic feature extraction is carried out to speech sample, and the acoustic feature information of each frame voice frame in speech sample is obtained;Through keyword recognition model, the acoustic feature information of each frame voice frame is carried out feature fusion processing with the context feature information of associated voice frame, and fusion feature information is obtained;According to fusion feature information, speech content recognition is carried out, and the prediction result corresponding to speech sample is obtained;According to the prediction result of speech sample and sample label, loss weight is determined;Based on loss weight, prediction result and the sample label of speech sample, the network parameters of keyword recognition model are adjusted, to obtain the keyword recognition model after training, can effectively reduce the misrecognition rate of keyword recognition model, improve recognition rate.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to a recognition model training method, apparatus, electronic device, and storage medium. Background Technology

[0002] Command word recognition is a subfield of speech recognition. The recognition model includes an acoustic model and a language model (the language model is generally implicitly included in the decoding network). In command word recognition systems that include language model information, a language model trained on general recognition data will reduce the recognition rate of command words, while a language model trained on command word data will cause the false recognition rate of command words to increase. Existing command word recognition systems usually require developers to adjust the language model based on test results in order to ensure good performance. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and storage medium for training a keyword recognition model. The scheme determines the impact of the training sample on the keyword recognition model's recognition performance based on whether the sample label of the training sample and the predicted text obtained based on the prediction result contain keywords. Based on the impact on the keyword recognition model's recognition performance, the loss weight corresponding to the training sample is determined. By adjusting the training degree of different training samples on the keyword recognition model through different loss weights, the false recognition rate of the keyword recognition model can be effectively reduced and the recognition rate can be improved.

[0004] This application provides a method for training a recognition model, including:

[0005] Acquire speech samples, which include speech frames and have corresponding sample labels;

[0006] The acoustic features of the speech samples are extracted using a keyword recognition model to obtain the acoustic feature information of each speech frame in the speech samples.

[0007] The keyword recognition model is used to perform feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frames to obtain fused feature information.

[0008] Speech content recognition is performed based on the fused feature information to obtain the prediction result corresponding to the speech sample;

[0009] The loss weights are determined based on the prediction results and sample labels of the speech samples;

[0010] The network parameters of the keyword recognition model are adjusted based on the loss weights, the prediction results, and the sample labels of the speech samples to obtain the trained keyword recognition model.

[0011] Accordingly, embodiments of this application also provide a recognition model training device, comprising:

[0012] A sample acquisition unit is used to acquire speech samples, wherein the speech samples include speech frames and each speech sample has a corresponding sample label.

[0013] An acoustic feature extraction unit is used to extract acoustic features from the speech sample using a keyword recognition model, thereby obtaining acoustic feature information for each speech frame in the speech sample.

[0014] The feature fusion unit is used to perform feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frame through the keyword recognition model to obtain fused feature information.

[0015] The content recognition unit is used to perform speech content recognition based on the fused feature information to obtain the prediction result corresponding to the speech sample.

[0016] The weight determination unit is used to determine the loss weights based on the prediction results and sample labels of the speech samples.

[0017] The model training unit is used to adjust the network parameters of the keyword recognition model based on the loss weights, the prediction results, and the sample labels of the speech samples, so as to obtain the trained keyword recognition model.

[0018] In one embodiment, the weight determination unit includes:

[0019] The probability determination subunit is used to determine the predicted probability of each speech frame in the speech sample with respect to the text unit in the preset dictionary based on the prediction results.

[0020] The text determination subunit is used to determine the predicted text corresponding to the speech sample based on the maximum prediction probability among the prediction probabilities of the text units.

[0021] The weight determination subunit is used to determine the loss weight corresponding to the speech sample based on the predicted text and the sample label.

[0022] In one embodiment, the weight determination subunit includes:

[0023] The probability calculation module is used to calculate the maximum probability mean of the speech sample based on the maximum predicted probability corresponding to each speech frame in the speech sample.

[0024] The strategy determination module is used to determine the corresponding weight calculation strategy based on the sample labels of the predicted text and the speech samples;

[0025] The weight calculation module is used to calculate the loss weight based on the weight calculation strategy and the maximum probability mean.

[0026] In one embodiment, the strategy determination module includes:

[0027] The first strategy determination submodule is used to determine the weight calculation strategy as the first weight calculation strategy if the predicted text of the speech sample contains keywords and the sample label of the speech sample does not contain the keywords.

[0028] The second strategy determination submodule is used to determine the weight calculation strategy as the second weight calculation strategy if the predicted text of the speech sample does not contain the keyword, but the sample label of the speech sample contains the keyword.

[0029] The third strategy determination submodule is used to determine the weight calculation strategy as the third weight calculation strategy if the predicted text of the speech sample does not contain keywords and the sample label of the speech sample does not contain the keywords.

[0030] The fourth strategy determination submodule is used to determine the weight calculation strategy as the fourth weight calculation strategy if the predicted text of the speech sample contains keywords and the sample label of the speech sample contains the keywords.

[0031] In one embodiment, the feature fusion unit includes:

[0032] The feature extraction subunit is used to extract context features from the predicted text corresponding to the associated speech frames of each speech frame through the predictive language network to obtain context feature information.

[0033] The fusion processing subunit is used to perform feature fusion processing on the context feature information and the acoustic feature information of the associated speech frame to obtain fused feature information.

[0034] In one embodiment, the model training unit includes:

[0035] The parameter adjustment subunit is used to adjust the network parameters of the keyword recognition model based on the loss weights, the predicted text, and the sample labels of the speech samples, so that the keyword recognition model converges and a candidate keyword recognition model is obtained.

[0036] A network training subunit is used to train the predictive language network based on the speech samples to obtain the trained predictive language network.

[0037] The combination subunit is used to combine the trained keyword recognition model and the trained predictive language network to obtain the trained keyword recognition model.

[0038] In one embodiment, the recognition model training device further includes:

[0039] The speech acquisition unit is used to acquire the speech to be recognized;

[0040] The first extraction unit is used to extract acoustic features from the speech to be recognized using the trained keyword recognition model, so as to obtain the acoustic feature information of the speech to be recognized.

[0041] The second extraction unit is used to extract context features from the speech to be recognized using the trained keyword recognition model, so as to obtain the context feature information of the speech to be recognized.

[0042] The fusion unit is used to perform feature fusion processing on the acoustic feature information of the speech to be recognized and the contextual feature information of the speech to be recognized through the trained keyword recognition model, so as to obtain the fused feature information of the speech to be recognized.

[0043] The recognition unit is used to perform speech content recognition based on the fusion feature information of the speech to be recognized, obtain the text corresponding to the speech to be recognized, and perform corresponding operations based on the text.

[0044] Accordingly, this application also provides an electronic device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any of the recognition model training methods provided in this application.

[0045] Accordingly, embodiments of this application also provide a computer-readable storage medium for storing a computer program, which is loaded by a processor to execute any of the recognition model training methods provided in embodiments of this application.

[0046] This application embodiment acquires speech samples, including speech frames, each with a corresponding sample label; extracts acoustic features from the speech samples using a keyword recognition model to obtain acoustic feature information for each speech frame; fuses the acoustic feature information of each speech frame with the contextual feature information of associated speech frames using the keyword recognition model to obtain fused feature information; performs speech content recognition based on the fused feature information to obtain a prediction result corresponding to the speech sample; determines the loss weight based on the prediction result and sample label of the speech sample; and adjusts the network parameters of the keyword recognition model based on the loss weight, prediction result, and sample label of the speech sample to obtain the trained keyword recognition model.

[0047] This scheme determines the impact of training samples on the keyword recognition model's performance by comparing the sample labels of the training samples with whether the predicted text obtained from the prediction results contains keywords. Based on the impact on the keyword recognition model's performance, it determines the corresponding loss weight for the training samples. By adjusting the training degree of different training samples on the keyword recognition model through different loss weights, it can effectively reduce the false recognition rate of the keyword recognition model and improve the recognition rate. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a scene diagram of the recognition model training method provided in the embodiments of this application;

[0050] Figure 2 This is a flowchart of the recognition model training method provided in the embodiments of this application;

[0051] Figure 3 This is another flowchart of the recognition model training method provided in the embodiments of this application;

[0052] Figure 4 This is a schematic diagram of the recognition model training device provided in the embodiments of this application;

[0053] Figure 5 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] This application provides a method, apparatus, electronic device, and computer-readable storage medium for training a recognition model. The recognition model training apparatus can be integrated into an electronic device, which may be a server or a terminal, etc.

[0056] The terminal may include mobile phones, wearable smart devices, tablets, laptops, personal computers (PCs), and in-vehicle computers, etc.

[0057] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0058] For example, such as Figure 1 As shown, the electronic device acquires speech samples, which include speech frames and corresponding sample labels. It then extracts acoustic features from the speech samples using a keyword recognition model, obtaining the acoustic feature information of each speech frame. The keyword recognition model then fuses the acoustic feature information of each speech frame with the contextual feature information of associated speech frames to obtain fused feature information. Based on the fused feature information, it performs speech content recognition to obtain the prediction result corresponding to the speech sample. The loss weights are determined based on whether the predicted text obtained from the prediction result contains keywords and whether the sample label of the speech sample contains keywords. A loss function is calculated based on the predicted text and the sample label of the speech sample, and the loss function is adjusted using the loss weights. Finally, the network parameters of the keyword recognition model are adjusted based on the adjusted loss function to obtain the trained keyword recognition model.

[0059] This scheme determines the impact of training samples on the keyword recognition model's performance by comparing the sample labels of the training samples with whether the predicted text obtained from the prediction results contains keywords. Based on the impact on the keyword recognition model's performance, it determines the corresponding loss weight for the training samples. By adjusting the training degree of different training samples on the keyword recognition model through different loss weights, it can effectively reduce the false recognition rate of the keyword recognition model and improve the recognition rate.

[0060] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0061] This embodiment will be described from the perspective of a recognition model training device, which can be integrated into an electronic device, such as a server or a terminal.

[0062] This application provides a method for training a recognition model, such as... Figure 2 As shown, the specific process of training this recognition model can be as follows:

[0063] 101. Obtain speech samples. Speech samples include speech frames, and each speech sample has a corresponding sample label.

[0064] The speech samples can include positive speech samples and negative speech samples. The sample label corresponding to the speech sample is the text information corresponding to the speech sample. The sample label corresponding to the positive speech sample contains keywords, while the sample label corresponding to the negative speech sample does not contain keywords.

[0065] Keywords can include command words that instruct the device to perform corresponding operations, such as command words like "A-genie" and "Little A" to wake up the device, as well as command words like "switch to next song" and "pause".

[0066] For example, voice samples can be obtained from databases or blockchains, or optionally, voice samples can be crawled from the Internet.

[0067] 102. The acoustic features of the speech samples are extracted by using a keyword recognition model to obtain the acoustic feature information of each speech frame in the speech samples.

[0068] The keyword recognition model can be a model for keyword recognition, which may include neural networks such as acoustic feature extraction networks (e.g., encoder networks), predictive language networks (e.g., prediction networks), and feature fusion networks (e.g., joint networks). The acoustic feature extraction network extracts acoustic features from speech samples, the predictive language network predicts the output text from the previous text and extracts contextual features, and the feature fusion network fuses contextual and acoustic features. The keyword recognition model can be a model trained on general speech data.

[0069] The acoustic feature information can include feature information extracted from acoustic features of speech samples, such as pitch period, formants, short-time average energy or amplitude, linear prediction coefficient (LPC), perceptually weighted prediction coefficient (PLP), short-time average zero-crossing rate, linear predictive cepstral coefficient (LPCC), autocorrelation function, Mel cepstral coefficient (MFCC), wavelet transform coefficients, empirical mode decomposition coefficients (EMD), and gamma-pass filter coefficients (GFCC).

[0070] For example, the speech sample can be preprocessed, which may include framing, windowing, and pre-emphasis to obtain multiple speech frames of the speech sample. Then, the acoustic features of each speech frame obtained by the speech sample framing can be extracted by the keyword recognition model to obtain the acoustic feature information corresponding to each speech frame in the speech sample.

[0071] 103. Using a keyword recognition model, the acoustic feature information of each speech frame is fused with the contextual feature information of the associated speech frames to obtain fused feature information.

[0072] The associated speech frame can be at least one speech frame that precedes the currently processed speech frame in time. For example, the associated speech frame can be the previous speech frame. When the language sample contains only one speech frame, the associated speech frame can be the speech frame itself.

[0073] The contextual feature information may include feature information obtained by the keyword recognition model based on the output of associated speech frames.

[0074] For example, specifically, for each speech frame of a speech sample, the contextual feature information of its associated speech frames can be obtained. Then, through a keyword recognition model, the contextual feature information and acoustic feature information can be fused by adding, concatenating, or multiplying features to obtain the fused feature information corresponding to each speech frame.

[0075] The contextual feature information of the associated speech frames can be pre-stored in a database or cache, or it can be obtained by learning the contextual information based on the associated speech frames through a speech network. Specifically, the step "using a keyword recognition model to perform feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frames to obtain fused feature information" can include:

[0076] By using a predictive language network to extract contextual features from the predicted text corresponding to the associated speech frames of each speech frame, contextual feature information is obtained.

[0077] The contextual feature information and the acoustic feature information of the associated speech frame are fused to obtain the fused feature information.

[0078] Among them, the predictive language network can be an RNN network used to learn contextual information based on associated speech frames, such as a Prediction Network. The predictive language network can be used to predict the text output by the previous text and extract contextual features.

[0079] For example, the keyword recognition model can be used to recognize the speech content of the associated speech frames to obtain the predicted text of the associated speech frames. The predicted text of the associated speech frames can then be input into a predictive language network. The predictive language network can then extract contextual features from the predicted text of the associated speech frames to obtain the contextual feature information corresponding to the currently processed speech frames.

[0080] The acoustic feature information of the speech frame currently being processed by the keyword recognition model is fused with the context feature information obtained by extracting context features from the associated speech frames to obtain fused feature information.

[0081] Taking speech frame A and speech frame B as examples, with speech frame A preceding speech frame B in time sequence, and speech frame A being a related speech frame to speech frame B, the following detailed explanation is provided:

[0082] The speech content of speech frame A is recognized by the feature fusion network in the keyword recognition model, and the predicted text sequence A corresponding to speech frame A is output (the predicted text sequence A ends with a blank character). In this process, the last non-empty predicted character output in the predicted text sequence A is the predicted character A. Acoustic features of speech frame B are extracted through the acoustic feature extraction network in the keyword model to obtain the acoustic feature information B of speech frame B. Contextual features of predicted character A are extracted through the predictive language network to obtain contextual feature information B. Then, the acoustic feature information B and contextual feature information B are fused through the keyword recognition model. Finally, the fused feature information corresponding to speech frame B is obtained through the feature fusion network.

[0083] It is understandable that the method for recognizing the speech content of speech frame A is the same as the method for recognizing the speech content of speech frame B.

[0084] 104. Based on the fused feature information, perform speech content recognition to obtain the prediction results corresponding to the speech samples.

[0085] For example, specifically, for each speech frame of a speech sample, speech content recognition can be performed based on the fusion feature information corresponding to the speech frame and a preset dictionary to obtain the predicted probability of each speech frame in the speech sample with respect to the text unit in the preset dictionary, and thus obtain the prediction result.

[0086] 105. Determine the loss weights based on the prediction results and sample labels of the speech samples.

[0087] Among them, loss weights can be used to adjust the loss function. By assigning loss weights to the loss weights, the impact of different speech samples on the training of the keyword recognition model can be changed.

[0088] For example, the prediction result can be obtained by calculating the prediction probability of the text unit corresponding to each speech frame in the prediction result, and the predicted text corresponding to the speech sample can be determined by methods such as dynamic path planning, beam search, or greedy search.

[0089] Optionally, the text unit with the highest predicted probability among the text units corresponding to each speech frame in the speech sample can be determined as the predicted text of that speech frame. The predicted text of the speech sample can be obtained based on the predicted text units corresponding to each speech frame, i.e., the step "determine the loss weight based on the prediction result of the speech sample and the sample label" can specifically include:

[0090] The prediction probability of each speech frame in the speech sample with respect to the text unit in the preset dictionary is determined based on the prediction results.

[0091] The predicted text corresponding to the speech sample is determined based on the maximum prediction probability among the prediction probabilities of the text units.

[0092] The loss weights corresponding to the speech samples are determined based on the predicted text and the sample labels.

[0093] Among them, the preset dictionary is a dictionary that can be set in advance. The preset dictionary can include the classes corresponding to the classification processing of fused feature information.

[0094] For example, specifically, the prediction probability of each speech frame in the speech sample is determined based on the prediction result, and the text unit corresponding to the highest prediction probability is determined as the predicted text unit of the speech frame. The predicted text of the speech sample can be obtained based on the predicted text unit corresponding to each speech frame.

[0095] The loss weight is determined based on whether the predicted text and sample label of the speech sample contain keywords. For example, if the predicted text contains keywords but the sample label does not contain keywords, or if the predicted sample does not contain keywords but the sample label contains keywords, it means that the speech sample is difficult to train in the keyword recognition task. The speech sample can be given a larger loss weight to increase the loss function and strengthen the training of the keyword recognition model. In this case, the loss weight can be set to ω1, where ω1 > 1.

[0096] If the predicted text contains keywords and the sample label contains keywords, or if the predicted sample does not contain keywords and the sample label does not contain keywords, it means that the speech sample is relatively easy to train in the keyword recognition task. In this case, the speech sample can be given a smaller loss weight and a smaller loss function to weaken the training of the keyword recognition model. In this case, the loss weight can be set to ω2, where ω2 < 1; or the loss function can be kept unchanged. In this case, the loss weight can be set to ω2, where ω2 = 1.

[0097] Optionally, if the predicted text contains keywords and the sample label contains keywords, the loss function can be kept unchanged, and the loss weight can be set to ω3, ω3 = 1; if the predicted sample does not contain keywords and the sample label does not contain keywords, the speech sample can be given a smaller loss weight, and the loss weight can be set to ω4, ω4 < 1.

[0098] In addition to using preset loss weights, loss weights can also be calculated based on the prediction probability of each speech frame in the speech sample. That is, in one embodiment, the step "determine the loss weights based on the prediction results and sample labels of the speech samples" can specifically include:

[0099] Calculate the maximum probability mean of the speech samples based on the maximum predicted probability corresponding to each speech frame in the speech samples;

[0100] The corresponding weight calculation strategy is determined based on the sample labels of the predicted text and speech samples;

[0101] The loss weight is calculated based on the weight calculation strategy and the maximum probability mean.

[0102] For example, the maximum prediction probability can be determined based on the prediction probability of each speech frame being predicted as a different text unit, and the average value can be calculated based on the maximum prediction probability corresponding to each speech frame to obtain the maximum probability mean.

[0103] If the predicted text contains keywords and the sample label does not contain keywords, or if the predicted sample does not contain keywords and the sample label contains keywords, the corresponding weight calculation strategy is ω = (1 + p), where p is the maximum probability mean and ω is the loss weight.

[0104] If the predicted text contains keywords and the sample label contains keywords, or if the predicted sample does not contain keywords and the sample label does not contain keywords, the corresponding weight calculation strategy is ω = (1-p), where p is the maximum probability mean and ω is the loss weight.

[0105] Optionally, if the predicted text contains keywords and the sample label contains keywords, the corresponding weight calculation strategy is ω = (1-p), where p is the maximum probability mean and ω is the loss weight; if the predicted sample does not contain keywords and the sample label does not contain keywords, the corresponding weight calculation strategy is ω = 1, where ω is the loss weight.

[0106] The loss weights are calculated based on the weighting strategy and the maximum probability mean.

[0107] Optionally, different weight calculation strategies can be determined based on whether the predicted text and sample labels of the speech samples contain keywords. That is, the step "determine the corresponding weight calculation strategy based on the predicted text and sample labels of the speech samples" can specifically include:

[0108] If the predicted text of a speech sample contains keywords, and the sample label of the speech sample does not contain keywords, then the weight calculation strategy is determined to be the first weight calculation strategy.

[0109] If the predicted text of a speech sample does not contain keywords, but the sample label of the speech sample contains keywords, then the weight calculation strategy is determined to be the second weight calculation strategy.

[0110] If the predicted text of the speech sample does not contain keywords, and the sample label of the speech sample does not contain keywords, then the weight calculation strategy is determined to be the third weight calculation strategy.

[0111] If the predicted text of a speech sample contains keywords, and the sample label of the speech sample also contains keywords, then the weight calculation strategy is determined to be the fourth weight calculation strategy.

[0112] For example, if the predicted text of a speech sample contains keywords, but the sample label of the speech sample does not contain keywords, it indicates that this type of negative speech sample is difficult to train in the keyword recognition task. Therefore, more weight is given to this type of negative speech sample, and the weight calculation strategy is determined to be the first weight calculation strategy, which is ω=(1+p). γ .

[0113] If the predicted text of a speech sample does not contain keywords, but the sample label of the speech sample does contain keywords, it indicates that this type of positive speech sample is difficult to train in the keyword recognition task. Therefore, more weight is given to this positive and negative speech sample, and the weight calculation strategy is determined to be the second weight calculation strategy, which is ω=(1+p). α .

[0114] If the predicted text of a speech sample does not contain keywords, and the sample label of the speech sample does not contain keywords, it means that this type of negative speech sample is easier to train in the keyword recognition task and is assigned less weight. Therefore, the weight calculation strategy is determined to be the third weight calculation strategy, and the third weight calculation strategy is ω=(1-p). β .

[0115] If the predicted text of a speech sample contains keywords, and the sample label of the speech sample also contains keywords, it means that this type of positive speech sample is easy to train. No adjustment is made to the loss function. Therefore, the weight calculation strategy is determined to be the fourth weight calculation strategy, and the fourth weight calculation strategy is ω = 1.

[0116] Here, α, β, and γ are hyperparameters, which are parameters set before training the keyword recognition model, not parameters obtained through training. Typically, hyperparameters need to be optimized to select an optimal set for the keyword recognition model, thereby improving learning performance and effectiveness.

[0117] 106. Adjust the network parameters of the keyword recognition model based on the loss weights, prediction results, and sample labels of the speech samples to obtain the trained keyword recognition model.

[0118] For example, a loss function can be calculated based on the sample labels of the speech samples and the prediction results. The loss weights and the loss function are then multiplied to adjust the loss function, resulting in an adjusted loss function. The keyword recognition model can then be trained based on this adjusted loss function to obtain the trained keyword recognition model.

[0119] Optionally, when training the keyword recognition model, the network parameters of the language prediction network in the keyword recognition model can be set not to be updated.

[0120] Optionally, the trained keyword recognition model may include an acoustic feature extraction network and a feature fusion network. Alternatively, the trained keyword recognition model may include an acoustic feature extraction network, a feature fusion network, and a predictive language network.

[0121] In one embodiment, the trained keyword recognition model may include an acoustic feature extraction network, a feature fusion network, and a predictive language network. After the keyword recognition model is trained, the predictive language network can be fine-tuned, and the fine-tuned predictive language network can be combined with the trained keyword recognition model to obtain the trained keyword recognition model. Specifically, the step "adjusting the network parameters of the keyword recognition model based on loss weights, prediction results, and sample labels of speech samples to obtain the trained keyword recognition model" may include:

[0122] The network parameters of the keyword recognition model are adjusted based on the loss weights, prediction results, and sample labels of the speech samples to make the keyword recognition model converge and obtain the candidate keyword recognition model.

[0123] The predictive language network is trained based on speech samples to obtain the trained predictive language network.

[0124] The candidate keyword recognition model and the trained predictive language network are combined to obtain the trained keyword recognition model.

[0125] For example, specifically, a loss function can be calculated based on the sample labels of the speech samples and the prediction results, and the loss function can be adjusted according to the loss weights to obtain the adjusted loss function. The keyword recognition model can then be trained based on the loss function until the keyword recognition model converges to obtain the candidate keyword recognition model. At this point, the predictive language network can be trained with a small learning rate using speech samples to update the network parameters of the predictive language network and obtain the trained predictive language network.

[0126] By combining the candidate keyword model and the trained predictive language network, a trained keyword recognition model can be obtained.

[0127] The trained keyword recognition model can be used to perform speech content recognition on the speech to be recognized and to perform corresponding operations. The specific process may include:

[0128] Acquire the speech to be recognized;

[0129] The trained keyword recognition model is used to extract acoustic features from the speech to be recognized, thereby obtaining the acoustic feature information of the speech to be recognized.

[0130] The trained keyword recognition model extracts contextual features from the speech to be recognized, thus obtaining the contextual feature information of the speech to be recognized.

[0131] The acoustic features of the speech to be recognized are fused with the contextual features of the speech to be recognized by the trained keyword recognition model to obtain the fused feature information of the speech to be recognized.

[0132] Speech content recognition is performed based on the fusion feature information of the speech to be recognized, resulting in the text corresponding to the speech to be recognized, and corresponding operations are performed based on the text.

[0133] The speech to be recognized can be user-input speech collected by a computer.

[0134] For example, the process could involve acquiring the speech to be recognized, extracting acoustic features from the speech using the acoustic feature extraction network of a trained keyword recognition model, obtaining acoustic feature information of the speech to be recognized; extracting contextual features from the speech using the predictive language network of a trained keyword recognition model, obtaining contextual feature information of the speech to be recognized; then fusing the acoustic feature information and contextual feature information using the feature fusion network of a trained keyword recognition model, obtaining fused feature information; finally, recognizing the speech content based on the fused feature information of the speech to be recognized, obtaining the text corresponding to the speech to be recognized. If the text contains keywords, the operation corresponding to the keywords is executed to provide the corresponding service. For example, if the keyword is a wake word, a wake-up operation is executed when the speech to be recognized contains a wake word.

[0135] As can be seen from the above, the electronic device in this application acquires speech samples, which include speech frames and corresponding sample labels; it extracts acoustic features from the speech samples using a keyword recognition model to obtain the acoustic feature information of each speech frame in the speech sample; it then uses the keyword recognition model to perform feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frames to obtain fused feature information; it performs speech content recognition based on the fused feature information to obtain the prediction result corresponding to the speech sample; it determines the loss weight based on the prediction result and sample label of the speech sample; and it adjusts the network parameters of the keyword recognition model based on the loss weight, prediction result, and sample label of the speech sample to obtain the trained keyword recognition model.

[0136] This scheme determines the impact of training samples on the keyword recognition model's performance by comparing the sample labels of the training samples with whether the predicted text obtained from the prediction results contains keywords. Based on the impact on the keyword recognition model's performance, it determines the corresponding loss weight for the training samples. By adjusting the training degree of different training samples on the keyword recognition model through different loss weights, it can effectively reduce the false recognition rate of the keyword recognition model and improve the recognition rate.

[0137] Based on the above embodiments, the following examples will provide further detailed explanations.

[0138] This embodiment will describe the training device for the recognition model from the perspective of the training device. The trained keyword recognition model is a trained command word recognition model. The command word recognition model includes a keyword recognition model and a predictive language network. Specifically, the command word recognition model includes an acoustic feature extraction network, a predictive language network, and a feature fusion network. The training device for this recognition model can be integrated into an electronic device, such as a server.

[0139] This application provides a method for training a recognition model, such as... Figure 3 As shown, the specific process of training this recognition model can be as follows:

[0140] 201. The server acquires speech samples and sets the network parameters of the predictive language network in the command word recognition model to not be updated.

[0141] The command word recognition model can be an RNN-Transducer (RNN-T) model, or a model based on the RNN-T model structure. For example, it can be based on general recognition data, modeled with syllables or phonemes, and trained as an RNN-T-based speech recognition model.

[0142] For example, the server can obtain speech samples from a database or blockchain. Alternatively, it can crawl speech samples from the Internet and set the network parameters of the predictive language network in the command word recognition model not to be updated during backpropagation. This prevents the model from overfitting due to a large number of repeated command words in the positive speech samples, which would increase the false recognition rate.

[0143] 202. The server extracts acoustic features from the speech samples using the acoustic feature extraction network of the command word recognition model, and obtains the acoustic feature information of each speech frame in the speech sample.

[0144] For example, the server can preprocess the speech samples. Preprocessing can include framing, windowing, and pre-emphasis to obtain multiple speech frames of the speech samples. Then, the acoustic features of each speech frame obtained by the command word recognition model can be extracted to obtain the acoustic feature information corresponding to each speech frame in the speech samples.

[0145] 203. The server extracts contextual features from associated speech frames using the predictive language network of the command word recognition model to obtain contextual feature information.

[0146] For example, the server can train a command word recognition model to perform speech content recognition on associated speech frames, obtain the predicted text of the associated speech frames, input the predicted text of the associated speech frames into a predictive language network, and extract contextual features from the predicted text of the associated speech frames through the predictive language network to obtain the contextual feature information corresponding to the currently processed speech frame.

[0147] 204. The server uses the feature fusion network of the command word recognition model to perform feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frames to obtain fused feature information.

[0148] For example, the server can obtain the contextual feature information of the associated speech frames for each speech frame of the speech sample, and perform feature fusion processing by adding, concatenating or multiplying the contextual feature information and acoustic feature information through command word recognition model training, so as to obtain the fused feature information corresponding to each speech frame.

[0149] 205. The server performs speech content recognition based on the fused feature information and obtains the prediction results corresponding to the speech samples.

[0150] For example, the server can perform speech content recognition for each speech frame of a speech sample based on the fusion feature information corresponding to the speech frame and a preset dictionary, and obtain the prediction probability of each speech frame in the speech sample with respect to the text unit in the preset dictionary, thus obtaining the prediction result.

[0151] 206. The server determines the corresponding weight calculation strategy based on the sample labels of the speech samples and the prediction results, and calculates the loss weight based on the weight calculation strategy.

[0152] For example, if the predicted text of a speech sample contains command words, but the sample label of the speech sample does not contain command words, it indicates that this type of negative speech sample is more difficult to train, and more weight is given to this type of negative speech sample. In this case, the server determines the weight calculation strategy as the first weight calculation strategy, and the first weight calculation strategy is ω=(1+p). γ .

[0153] If the predicted text of a speech sample does not contain command words, but the sample label of the speech sample does contain command words, it indicates that this type of positive speech sample is more difficult to train. Therefore, the server assigns more weight to this positive and negative speech sample and determines the weight calculation strategy as the second weight calculation strategy, which is ω=(1+p). α .

[0154] If the predicted text of a speech sample does not contain command words, and the sample label of the speech sample does not contain command words, it means that this type of negative speech sample is easier to train and is assigned fewer weights. In this case, the server determines the weight calculation strategy as the third weight calculation strategy, which is ω = (1-p). β .

[0155] If the predicted text of a speech sample contains command words, and the sample label of the speech sample also contains command words, it indicates that this type of positive speech sample is easy to train. No adjustment is made to the loss function. In this case, the server determines the weight calculation strategy as the fourth weight calculation strategy, and the fourth weight calculation strategy is ω = 1.

[0156] 207. The server trains the command word recognition model based on the loss weights, prediction results, and sample labels of the speech samples until the command word recognition model converges.

[0157] For example, the server can calculate the loss function based on the sample labels of the speech samples and the prediction results, and then multiply the loss weights and the loss function to adjust the loss function, obtaining the adjusted loss function. The command word recognition model can then be trained based on the loss function until the command word recognition model converges.

[0158] 208. When the command word recognition model converges, update the network parameters of the predictive language network of the command word recognition model to obtain the trained command word recognition model.

[0159] For example, in a command word recognition system containing language model information, a language model trained on general recognition data will reduce the recognition rate of command words, while a language model trained on command word data will increase the false recognition rate of command words. Therefore, the language model is also an important part of this type of command word recognition system, affecting the final recognition rate and false recognition rate. When the command word recognition model converges, the server can train the predictive language network of the command word recognition model through speech samples, and update the network parameters of the predictive language network in the command word recognition model with a small learning rate to obtain the trained command word recognition model.

[0160] Optionally, once the trained command word model is obtained, the model can be deployed in smart electronic devices, such as smart speakers and smart robots, so that the smart electronic devices can perform speech content recognition based on the user's input speech to be recognized, and execute corresponding operations based on the recognized command words.

[0161] Optionally, the command word recognition model can also be deployed on a cloud server. The smart electronic device sends the collected speech to be recognized to the cloud server for speech content recognition, and sends a corresponding command to the smart electronic device based on the speech content recognition result to instruct the smart electronic device to perform the corresponding operation.

[0162] As described above, in this embodiment, the server acquires speech samples and sets the network parameters of the predictive language network in the command word recognition model to remain unchanged. Acoustic features are extracted from the speech samples using the acoustic feature extraction network of the command word recognition model to obtain acoustic feature information for each speech frame. Contextual features are extracted from associated speech frames using the predictive language network of the command word recognition model to obtain contextual feature information. The acoustic feature information of each speech frame is fused with the contextual feature information of associated speech frames using the feature fusion network of the command word recognition model to obtain fused feature information. Speech content recognition is performed based on the fused feature information to obtain the prediction result corresponding to the speech sample. A corresponding weight calculation strategy is determined based on the sample label of the speech sample and the prediction result, and a loss weight is calculated based on the weight calculation strategy. The command word recognition model is trained based on the loss weight, the prediction result, and the sample label of the speech sample until the command word recognition model converges. When the command word recognition model converges, the network parameters of the predictive language network of the command word recognition model are updated to obtain the trained command word recognition model.

[0163] This scheme determines the impact of training samples on the keyword recognition model's performance by comparing the sample labels of the training samples with whether the predicted text obtained from the prediction results contains keywords. Based on this impact, it determines the corresponding loss weight for each training sample. By adjusting the loss weight, the training intensity of the keyword recognition model is adjusted for different training samples, effectively reducing the false recognition rate and improving the recognition rate. This scheme can be applied to command word recognition in the field of automatic speech recognition, enhancing the command word recognition model's resistance to false triggering and its recognition capability.

[0164] To facilitate better implementation of the recognition model training method provided in the embodiments of this application, a recognition model training apparatus is also provided in one embodiment. The meanings of the terms used are the same as in the recognition model training method described above, and specific implementation details can be found in the description of the method embodiments.

[0165] The recognition model training device can be integrated into an electronic device, such as... Figure 4 As shown, the recognition model training device may include: a sample acquisition unit 301, an acoustic feature extraction unit 302, a feature fusion unit 303, a content recognition unit 304, a weight determination unit 305, and a model training unit 306, as detailed below:

[0166] (1) Sample acquisition unit 301: used to acquire speech samples, the speech samples include speech frames, and the speech samples correspond to sample tags.

[0167] (2) Acoustic feature extraction unit 302: used to extract acoustic features from the speech sample through the keyword recognition model to obtain the acoustic feature information of each speech frame in the speech sample.

[0168] (3) Feature fusion unit 303: used to perform feature fusion processing on the acoustic feature information of each speech frame and the context feature information of the associated speech frame through the keyword recognition model to obtain fused feature information;

[0169] In one embodiment, the feature fusion unit 303 includes a feature extraction subunit and a fusion processing subunit, specifically:

[0170] The feature extraction subunit is used to extract context features from the predicted text corresponding to the associated speech frames of each speech frame through the predictive language network to obtain context feature information.

[0171] The fusion processing subunit is used to perform feature fusion processing on the context feature information and the acoustic feature information of the associated speech frame to obtain fused feature information.

[0172] (4) Content recognition unit 304: used to perform speech content recognition based on the fused feature information to obtain the prediction result corresponding to the speech sample.

[0173] (5) Weight determination unit 305: used to determine the loss weight based on the prediction result and sample label of the speech sample.

[0174] In one embodiment, the weight determination unit 305 may include a probability determination subunit, a text determination subunit, and a weight determination subunit, specifically:

[0175] The probability determination subunit is used to determine the predicted probability of each speech frame in the speech sample with respect to the text unit in the preset dictionary based on the prediction results.

[0176] The text determination subunit is used to determine the predicted text corresponding to the speech sample based on the maximum prediction probability among the prediction probabilities of the text units.

[0177] The weight determination subunit is used to determine the loss weight corresponding to the speech sample based on the predicted text and the sample label.

[0178] In one embodiment, the weight determination subunit may include a probability calculation module, a policy determination module, and a weight calculation module, specifically:

[0179] The probability calculation module is used to calculate the maximum probability mean of the speech sample based on the maximum predicted probability corresponding to each speech frame in the speech sample.

[0180] The strategy determination module is used to determine the corresponding weight calculation strategy based on the sample labels of the predicted text and the speech samples;

[0181] The weight calculation module is used to calculate the loss weight based on the weight calculation strategy and the maximum probability mean.

[0182] In one embodiment, the strategy determination module may include a first strategy determination submodule, a second strategy determination submodule, a third strategy determination submodule, and a fourth strategy determination submodule, specifically:

[0183] The first strategy determination submodule is used to determine the weight calculation strategy as the first weight calculation strategy if the predicted text of the speech sample contains keywords and the sample label of the speech sample does not contain the keywords.

[0184] The second strategy determination submodule is used to determine the weight calculation strategy as the second weight calculation strategy if the predicted text of the speech sample does not contain the keyword, but the sample label of the speech sample contains the keyword.

[0185] The third strategy determination submodule is used to determine the weight calculation strategy as the third weight calculation strategy if the predicted text of the speech sample does not contain keywords and the sample label of the speech sample does not contain the keywords.

[0186] The fourth strategy determination submodule is used to determine the weight calculation strategy as the fourth weight calculation strategy if the predicted text of the speech sample contains keywords and the sample label of the speech sample contains the keywords.

[0187] (6) Model training unit 306: used to adjust the network parameters of the keyword recognition model based on the loss weights, the prediction results and the sample labels of the speech samples, so as to obtain the trained keyword recognition model.

[0188] In one embodiment, the model training unit 306 may include a parameter tuning subunit, a network training subunit, and a combination subunit:

[0189] The parameter adjustment subunit is used to adjust the network parameters of the keyword recognition model based on the loss weights, the predicted text, and the sample labels of the speech samples, so that the keyword recognition model converges and a candidate keyword recognition model is obtained.

[0190] A network training subunit is used to train the predictive language network based on the speech samples to obtain the trained predictive language network.

[0191] The combination subunit is used to combine the trained keyword recognition model and the trained predictive language network to obtain the trained keyword recognition model.

[0192] In one embodiment, the recognition model training device may further include a speech acquisition unit, a first extraction unit, a second extraction unit, a fusion unit, and a recognition unit, specifically:

[0193] The speech acquisition unit is used to acquire the speech to be recognized;

[0194] The first extraction unit is used to extract acoustic features from the speech to be recognized using the trained keyword recognition model, so as to obtain the acoustic feature information of the speech to be recognized.

[0195] The second extraction unit is used to extract context features from the speech to be recognized using the trained keyword recognition model, so as to obtain the context feature information of the speech to be recognized.

[0196] The fusion unit is used to perform feature fusion processing on the acoustic feature information of the speech to be recognized and the contextual feature information of the speech to be recognized through the trained keyword recognition model, so as to obtain the fused feature information of the speech to be recognized.

[0197] The recognition unit is used to perform speech content recognition based on the fusion feature information of the speech to be recognized, obtain the text corresponding to the speech to be recognized, and perform corresponding operations based on the text.

[0198] The recognition model training device in this embodiment can acquire speech samples through the sample acquisition unit 301. The speech samples include speech frames and corresponding sample labels. The acoustic feature extraction unit 302 extracts acoustic features from the speech samples using a keyword recognition model to obtain the acoustic feature information of each speech frame in the speech samples. The feature fusion unit 303 performs feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frames using the keyword recognition model to obtain fused feature information. The content recognition unit 304 performs speech content recognition based on the fused feature information to obtain the prediction result corresponding to the speech sample. The weight determination unit 305 determines the loss weight based on the prediction result and sample label of the speech sample. Finally, the model training unit 306 adjusts the network parameters of the keyword recognition model based on the loss weight, prediction result, and sample label of the speech sample to obtain the trained keyword recognition model.

[0199] This scheme determines the impact of training samples on the keyword recognition model's performance by comparing the sample labels of the training samples with whether the predicted text obtained from the prediction results contains keywords. Based on the impact on the keyword recognition model's performance, it determines the corresponding loss weight for the training samples. By adjusting the training degree of different training samples on the keyword recognition model through different loss weights, it can effectively reduce the false recognition rate of the keyword recognition model and improve the recognition rate.

[0200] This application also provides an electronic device, which can be a terminal or a server, such as... Figure 5 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0201] The electronic device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0202] The processor 1001 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1002, and by calling data stored in the memory 1002, thereby providing overall monitoring of the electronic device. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and computer programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001.

[0203] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0204] The electronic device also includes a power supply 1003 that supplies power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1003 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0205] The electronic device may also include an input unit 1004, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0206] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1001 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 runs the computer programs stored in the memory 1002 to realize various functions, as follows:

[0207] Acquire speech samples, which include speech frames and corresponding sample labels.

[0208] Acoustic features are extracted from speech samples using a keyword recognition model to obtain acoustic feature information for each speech frame in the speech sample.

[0209] By using a keyword recognition model, the acoustic feature information of each speech frame is fused with the contextual feature information of the associated speech frames to obtain fused feature information.

[0210] Speech content recognition is performed based on fused feature information to obtain the prediction results corresponding to the speech samples;

[0211] The loss weights are determined based on the prediction results and sample labels of the speech samples.

[0212] The network parameters of the keyword recognition model are adjusted based on the loss weights, prediction results, and sample labels of the speech samples to obtain the trained keyword recognition model.

[0213] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0214] As can be seen from the above, the electronic device in this embodiment can acquire speech samples, which include speech frames and corresponding sample labels; extract acoustic features from the speech samples using a keyword recognition model to obtain acoustic feature information for each speech frame in the speech sample; perform feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frames using the keyword recognition model to obtain fused feature information; perform speech content recognition based on the fused feature information to obtain the prediction result corresponding to the speech sample; determine the loss weight based on the prediction result and sample label of the speech sample; and adjust the network parameters of the keyword recognition model based on the loss weight, prediction result, and sample label of the speech sample to obtain the trained keyword recognition model.

[0215] This scheme determines the impact of training samples on the keyword recognition model's performance by comparing the sample labels of the training samples with whether the predicted text obtained from the prediction results contains keywords. Based on the impact on the keyword recognition model's performance, it determines the corresponding loss weight for the training samples. By adjusting the training degree of different training samples on the keyword recognition model through different loss weights, it can effectively reduce the false recognition rate of the keyword recognition model and improve the recognition rate.

[0216] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.

[0217] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0218] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute any of the recognition model training methods provided in embodiments of this application.

[0219] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0220] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0221] Since the computer program stored in the computer-readable storage medium can execute any of the recognition model training methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the recognition model training methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0222] The foregoing has provided a detailed description of a recognition model training method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for training a recognition model, characterized in that, include: Acquire a speech sample, wherein the speech sample includes at least one speech frame and the speech sample has a corresponding sample label; The acoustic features of the speech samples are extracted using a keyword recognition model to obtain the acoustic feature information of each speech frame in the speech samples. The keyword recognition model is used to fuse the acoustic features of each speech frame with the contextual features of the associated speech frame to obtain fused feature information. The associated speech frame is at least one speech frame that is temporally earlier than the currently processed speech frame. The contextual feature information is extracted based on the contextual features of the predicted text corresponding to the associated speech frame. Speech content recognition is performed based on the fused feature information to obtain the prediction result corresponding to the speech sample. The prediction result includes the prediction probability of the text unit corresponding to each speech frame. The text unit with the highest prediction probability corresponding to the speech frame is used as the predicted text of the speech frame. The loss weight is determined based on the prediction result and sample label of the speech sample, wherein determining the loss weight based on the prediction result and sample label of the speech sample includes: determining the loss weight based on whether the predicted text in the speech sample and the sample label contain keywords, wherein the presence of keywords indicates whether keywords are contained. The network parameters of the keyword recognition model are adjusted based on the loss weights, the prediction results, and the sample labels of the speech samples to obtain the trained keyword recognition model.

2. The method according to claim 1, characterized in that, The step of determining the loss weight based on the prediction result and sample label of the speech sample includes: The prediction probability of each speech frame in the speech sample with respect to the text unit in the preset dictionary is determined based on the prediction results. The predicted text corresponding to the speech sample is determined based on the maximum prediction probability among the prediction probabilities of the text units. The loss weight corresponding to the speech sample is determined based on the predicted text and the sample label.

3. The method according to claim 2, characterized in that, The step of determining the loss weight corresponding to the speech sample based on the predicted text and the sample label includes: Calculate the maximum probability mean of the speech samples based on the maximum predicted probability corresponding to each speech frame in the speech samples; The corresponding weight calculation strategy is determined based on the sample labels of the predicted text and the speech samples; The loss weight is calculated based on the weight calculation strategy and the maximum probability mean.

4. The method according to claim 3, characterized in that, The step of determining the corresponding weight calculation strategy based on the keywords contained in the sample labels of the predicted text and the speech samples includes: If the predicted text of the speech sample contains keywords, and the sample label of the speech sample does not contain the keywords, then the weight calculation strategy is determined to be the first weight calculation strategy. If the predicted text of the speech sample does not contain the keyword, but the sample label of the speech sample contains the keyword, then the weight calculation strategy is determined to be the second weight calculation strategy. If the predicted text of the speech sample does not contain keywords, and the sample label of the speech sample does not contain the keywords, then the weight calculation strategy is determined to be the third weight calculation strategy. If the predicted text of the speech sample contains keywords, and the sample label of the speech sample contains the keywords, then the weight calculation strategy is determined to be the fourth weight calculation strategy.

5. The method according to claim 1, characterized in that, The keyword recognition model is used to perform feature fusion processing on the acoustic feature information of each speech frame and the contextual feature information of the associated speech frames to obtain fused feature information, including: By using a predictive language network, contextual feature extraction is performed on the predicted text corresponding to the associated speech frames of each speech frame to obtain contextual feature information. The contextual feature information and the acoustic feature information of the associated speech frame are fused to obtain fused feature information.

6. The method according to claim 5, characterized in that, The step of adjusting the network parameters of the keyword recognition model based on the loss weights, the predicted text, and the sample labels of the speech samples to obtain the trained keyword recognition model includes: The network parameters of the keyword recognition model are adjusted based on the loss weights, the predicted text, and the sample labels of the speech samples to make the keyword recognition model converge and obtain a candidate keyword recognition model. The predictive language network is trained based on the speech samples to obtain the trained predictive language network. The candidate keyword recognition model and the trained predictive language network are combined to obtain the trained keyword recognition model.

7. The method according to claim 1, characterized in that, The method further includes: Acquire the speech to be recognized; The acoustic features of the speech to be identified are extracted by the trained keyword recognition model to obtain the acoustic feature information of the speech to be identified. The trained keyword recognition model is used to extract contextual features from the speech to be recognized, thereby obtaining the contextual feature information of the speech to be recognized. The trained keyword recognition model is used to perform feature fusion processing on the acoustic feature information of the speech to be recognized and the contextual feature information of the speech to be recognized, so as to obtain the fused feature information of the speech to be recognized. Speech content recognition is performed based on the fusion feature information of the speech to be recognized to obtain the text corresponding to the speech to be recognized, and corresponding operations are performed based on the text.

8. A recognition model training device, characterized in that, include: A sample acquisition unit is used to acquire speech samples, wherein the speech samples include speech frames and the speech samples are associated with sample labels. An acoustic feature extraction unit is used to extract acoustic features from the speech sample using a keyword recognition model, thereby obtaining acoustic feature information for each speech frame in the speech sample. The feature fusion unit is used to perform feature fusion processing on the acoustic feature information of each speech frame and the context feature information of the associated speech frame through the keyword recognition model to obtain fused feature information. The associated speech frame is at least one speech frame that is temporally earlier than the currently processed speech frame. The context feature information is extracted based on the context features of the predicted text corresponding to the associated speech frame. The content recognition unit is used to perform speech content recognition based on the fused feature information to obtain the prediction result corresponding to the speech sample. The prediction result includes the prediction probability of the text unit corresponding to each speech frame, and the text unit with the highest prediction probability corresponding to the speech frame is used as the predicted text of the speech frame. The weight determination unit is used to determine the loss weight based on the prediction result and sample label of the speech sample, wherein determining the loss weight based on the prediction result and sample label of the speech sample includes: determining the loss weight based on whether the predicted text in the speech sample and the sample label contain keywords, wherein the presence of keywords indicates whether keywords are contained. The model training unit is used to adjust the network parameters of the keyword recognition model based on the loss weights, the prediction results, and the sample labels of the speech samples, so as to obtain the trained keyword recognition model.

9. An electronic device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the recognition model training method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which is loaded by a processor to execute the recognition model training method according to any one of claims 1 to 7.

Images