A Speech Recognition Method, Device, Equipment and Storage Medium

Abstract

The present invention provides a voice recognition method, apparatus, device and storage medium. The method includes: obtaining the voice to be recognized, and recognizing the voice to be recognized based on a pre-trained voice recognition model. The voice recognition model is obtained through two-stage training. In the first stage, the training is performed with the goal of making the recognition result of the training voice consistent with the text annotated for the training voice. In the second stage, the training is performed with the goal of balancing the text unit error rate and semantic acceptability of the voice recognition result of the training voice. With the voice recognition method provided by the present invention, a voice recognition result with relatively high user acceptability can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition, and in particular, to a speech recognition method, apparatus, device, and storage medium. Background Art

[0002] Speech recognition technology is a technology for recognizing speech into text. The current speech recognition solution is a solution based on a speech recognition model. This solution uses a pre-trained speech recognition model to recognize the speech to be recognized to obtain a recognition result.

[0003] In the current speech recognition solution, the speech recognition model used is usually trained based on the cross-entropy criterion. However, the recognition performance of the speech model trained only based on the cross-entropy criterion is not good, which makes it difficult to obtain a good recognition effect when performing speech recognition based on the trained speech recognition model. Summary of the Invention

[0004] In view of this, the present invention provides a speech recognition method, apparatus, device, and storage medium to solve the problem of poor recognition effect of the current speech recognition solution. The technical solution is as follows:

[0005] A speech recognition method, comprising:

[0006] Obtaining the speech to be recognized;

[0007] Recognizing the speech to be recognized based on a pre-trained speech recognition model;

[0008] Wherein, the speech recognition model is trained through two stages. The first stage is trained with the goal of making the recognition result of the training speech consistent with the text annotated for the training speech. The second stage is trained with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech.

[0009] Optionally, a speech recognition baseline model is obtained in the first stage, and the speech recognition baseline model is trained in the second stage;

[0010] Training the speech recognition baseline model includes:

[0011] Recognizing the training speech based on the speech recognition baseline model to obtain several candidate recognition results of the training speech;

[0012] Determining the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result, wherein the semantic change evaluation index can reflect the semantic change of the corresponding candidate recognition result relative to the text annotated for the training speech;

[0013] Combined with the text unit error rate and semantic change evaluation metrics corresponding to each candidate recognition result, determine the prediction loss of the speech recognition baseline model for each candidate recognition result;

[0014] Update the parameters of the speech recognition baseline model according to the determined prediction loss.

[0015] Optionally, the combining the text unit error rate and semantic change evaluation metrics corresponding to each candidate recognition result to determine the prediction loss of the speech recognition baseline model for each candidate recognition result includes:

[0016] For each candidate recognition result:

[0017] Determine the weight corresponding to the candidate recognition result according to the text unit error rate and semantic change evaluation metrics corresponding to the candidate recognition result;

[0018] Determine the prediction loss of the speech recognition baseline model for the candidate recognition result according to the weight corresponding to the candidate recognition result and the prediction probability corresponding to the candidate recognition result.

[0019] Optionally, the semantic change evaluation metrics include: part-of-speech deviation and / or syntactic category deviation;

[0020] The part-of-speech deviation can reflect the deviation degree of the corresponding candidate recognition result in terms of part-of-speech relative to the labeled text of the training speech;

[0021] The syntactic category deviation can reflect the deviation degree of the corresponding candidate recognition result in terms of syntactic category relative to the labeled text of the training speech.

[0022] Optionally, the determining the weight corresponding to the candidate recognition result according to the text unit error rate and semantic change evaluation metrics corresponding to the candidate recognition result includes:

[0023] Calculate the difference between the text unit error rate corresponding to the candidate recognition result and the average text unit error rate, the difference between the part-of-speech deviation corresponding to the candidate recognition result and the average part-of-speech deviation, and the difference between the syntactic category deviation corresponding to the candidate recognition result and the average syntactic category deviation, where the average word error rate is the average of the text unit error rates corresponding to the several candidate recognition results, the average part-of-speech deviation is the average of the part-of-speech deviations corresponding to the several candidate recognition results, and the average syntactic category deviation is the average of the syntactic category deviations corresponding to the several candidate recognition results;

[0024] Fuse the calculated differences, and use the fusion result as the weight corresponding to the candidate recognition result.

[0025] Optionally, determining the part-of-speech deviation degree corresponding to a candidate recognition result includes:

[0026] Determining the part of speech of each word included in the labeled text of the training speech, and determining the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech;

[0027] Determining the part-of-speech deviation degree corresponding to this candidate recognition result according to the part of speech of each word included in the labeled text of the training speech and the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech.

[0028] Optionally, the determining the part-of-speech deviation degree corresponding to this candidate recognition result according to the part of speech of each word included in the labeled text of the training speech and the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech includes:

[0029] Determining the part-of-speech deviation weight of the candidate recognition result relative to the labeled text of the training speech according to the weight of the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech, where the weight of the part of speech of a word can represent the importance degree of the part of speech of the word;

[0030] Summing the weights of the parts of speech of each word included in the labeled text of the training speech to obtain the sum of part-of-speech weights;

[0031] Determining the part-of-speech deviation degree corresponding to this candidate recognition result according to the sum of part-of-speech weights and the part-of-speech deviation weight of the candidate recognition result relative to the labeled text of the training speech.

[0032] Optionally, determining the syntactic category deviation degree corresponding to a candidate recognition result includes:

[0033] Determining the dependency relationship corresponding to each word in the labeled text of the training speech, and determining the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text;

[0034] Determining the syntactic category deviation degree corresponding to this candidate recognition result according to the dependency relationship corresponding to each word in the labeled text of the training speech and the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text.

[0035] Optionally, the determining the syntactic category deviation degree corresponding to this candidate recognition result according to the dependency relationship corresponding to each word in the labeled text of the training speech and the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text includes:

[0036] Determining the dependency relationship deviation weight of the candidate recognition result relative to the labeled text of the training speech according to the weight of the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text;

[0037] Determine the sum of the weights of the dependency relationships corresponding to each word in the labeled text of the training speech to obtain the dependency relationship weight sum;

[0038] Determine the syntactic category deviation degree corresponding to the candidate recognition result according to the dependency relationship weight sum and the dependency relationship deviation weight of the candidate recognition result relative to the labeled text of the training speech.

[0039] A voice recognition device, comprising: a voice acquisition module and a voice recognition module;

[0040] The voice acquisition module is used to acquire the voice to be recognized;

[0041] The voice recognition module is used to recognize the voice to be recognized based on a pre-trained voice recognition model;

[0042] Wherein, the voice recognition model is obtained through two-stage training. The first stage is trained with the goal of making the recognition result of the training speech consistent with the text labeled by the training speech. The second stage is trained with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech.

[0043] A voice recognition device, comprising: a memory and a processor;

[0044] The memory is used to store programs;

[0045] The processor is used to execute the program to implement each step of the voice recognition method described in any one of the above.

[0046] A readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each step of the voice recognition method described in any one of the above is implemented.

[0047] The voice recognition method, device, equipment and storage medium provided by the present invention first obtain the voice to be recognized, and then recognize the voice to be recognized based on a pre-trained voice recognition model. The voice recognition model is obtained through two-stage training. In the first stage, the training is carried out with the goal of making the recognition result of the training voice consistent with the text annotated for the training voice. In the second stage, the training is carried out with the goal of balancing the text unit error rate and semantic acceptability of the voice recognition result of the training voice. Considering that the recognition performance of the model trained only with the goal of making the recognition result of the training voice consistent with the text annotated for the training voice is poor (the user acceptability of the recognition result obtained based on this model is not high), the present invention proposes to further train on the basis of the model trained with the goal of making the recognition result of the training voice consistent with the text annotated for the training voice, with the goal of balancing the text unit error rate and semantic acceptability of the voice recognition result of the training voice. The voice recognition model trained in this way has better performance. When using it to recognize the voice to be recognized, a recognition result with higher user acceptability can be obtained, and the user experience is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0049] Figure 1 It is a schematic flowchart of the voice recognition method provided by the embodiment of the present invention;

[0050] Figure 2 It is a schematic flowchart of training a voice recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the voice recognition result of the training voice provided by the embodiment of the present invention;

[0051] Figure 3 It is a schematic structural diagram of the voice recognition device provided by the embodiment of the present invention;

[0052] Figure 4 It is a schematic structural diagram of the voice recognition equipment provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] Given that the recognition performance of a speech recognition model trained solely based on the cross-entropy criterion is poor, and that it is difficult to achieve good recognition results when performing speech recognition based on the trained speech recognition model, the inventors of this case conducted research and initially proposed the following approach:

[0055] The speech recognition model is trained through two training phases. In the first phase, a baseline speech recognition model is trained based on the cross-entropy criterion. In the second phase, the baseline model is further trained using the text unit error rate as a constraint. The model obtained in the second phase serves as the final speech recognition model, and the trained speech recognition model is then used to recognize the target speech. Further training the speech recognition baseline model using the text unit error rate as a constraint based on the cross-entropy criterion can improve the performance of the speech recognition model to a certain extent.

[0056] The inventors studied the above-mentioned scheme of obtaining a speech recognition model through two training stages and found that although the above-mentioned scheme has a certain improvement in effect compared with the scheme of obtaining a speech recognition model through training based only on the cross-entropy criterion, there are still some problems, specifically: the text unit error rate can only reflect the statistical information of the text units that are recognized incorrectly in the recognition results, but cannot directly reflect the information at the semantic level of the recognition results. This results in that when the speech recognition model trained with the text unit error rate as the constraint criterion is used for speech recognition, the acceptance of the obtained recognition results is not high.

[0057] For example, the recognition result obtained by using the speech recognition model trained in the above two stages to recognize a speech and the annotated text of the speech are as follows:

[0058] Marked result: My illness will be gone once I soak my feet, and everything will be normal afterwards.

[0059] Identification result: Is it true that the pain disappeared as soon as I soaked my feet, and then everything went back to normal?

[0060] Comparing the above recognition results with the annotated text, we find that the recognition result has one wrong character, "并->病". Although the recognition result only has one wrong character, its impact on the semantics is very large. It can be seen that the recognition result is not acceptable.

[0061] Regarding the problem of the low acceptability of the above recognition results, the inventor thought of increasing the proportion of the language model and selecting the results with a more reasonable language distribution. However, this solution usually causes fluctuations in the recognition results because it is impossible to find unified parameters for the fusion of acoustics and language. The inventor also thought of jointly adjusting the recognition results by combining upstream and downstream tasks, such as combining speech recognition and recognition result translation tasks, and further strengthening the reasonable distribution of speech recognition results with the loss function of the translation task. However, the joint task method requires parallel data for training and has high requirements for data and tasks.

[0062] In view of the infeasibility of the above ideas for solving the problem of low acceptability of recognition results, the inventor continued to conduct in-depth research. Through research, a speech recognition method with better effects was finally proposed. Through this speech recognition method, speech recognition results with higher acceptability can be obtained. In one possible implementation, the hardware architecture involved in the speech recognition method provided by the present invention may include an electronic device with data processing capabilities. Exemplarily, the electronic device may be any electronic product that can perform human-computer interaction with users through voice interaction. For example, a smart phone, a smart speaker, a notebook computer, a tablet computer, a handheld computer, a wearable device, a smart TV, a vehicle-mounted terminal, etc. The electronic device can obtain the speech to be recognized and recognize the speech to be recognized according to the speech recognition method provided by the present invention. In another possible implementation, the hardware architecture involved in the speech recognition method provided by the present invention may include an electronic device and a server. The electronic device may be any electronic product that can perform human-computer interaction with users through voice interaction. The server may be a single server, a server cluster composed of multiple servers, or a cloud computing server center. The server may include a processor, a memory, and a network interface, etc. The electronic device can establish a connection and communicate with the server through a wired communication network or a wireless communication network. The electronic device obtains the speech to be recognized and sends the obtained speech to be recognized to the server through the communication network. The server recognizes the speech to be recognized according to the speech recognition method provided by the present invention.

[0063] Those skilled in the art should understand that the above electronic devices and servers are only examples. Other existing or future possible electronic devices or servers that can be applied to the present invention should also be included in the protection scope of the present invention and are hereby incorporated by reference.

[0064] Next, the speech recognition method provided by the present invention will be introduced through the following embodiments.

[0065] Please refer to Figure 1 , which shows a schematic flowchart of the speech recognition method provided by an embodiment of the present invention. The method may include:

[0066] Step S101: Obtain the speech to be recognized.

[0067] Step S102: Recognize the speech to be recognized based on a pre-trained speech recognition model.

[0068] Among them, the speech recognition model is obtained through two-stage training. The first stage is trained with the goal of making the recognition result of the training speech consistent with the text annotated for the training speech. The second stage is trained with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech.

[0069] It should be noted that the first stage can be trained based on the cross-entropy criterion. The trained model is used as the speech recognition baseline model. After obtaining the speech recognition baseline model, in order to improve the performance of the speech recognition model and make the speech recognition model output a recognition result with a relatively high user acceptability, the present invention further trains the speech recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech.

[0070] The training method based on the cross-entropy criterion is a prior art, and this embodiment will not elaborate on it. This embodiment focuses on introducing the process of training the speech recognition baseline model obtained in the first stage with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech.

[0071] Please refer to Figure 2 , which shows a schematic flowchart of training the speech recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech, and may include:

[0072] Step S201: Recognize the training speech based on the speech recognition baseline model to obtain several candidate recognition results of the training speech.

[0073] Among them, several candidate recognition results of the training speech may be Nbest candidate recognition results.

[0074] Step S202: Determine the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result.

[0075] Among them, the text unit error rate is a statistical index of the text units recognized incorrectly in the corresponding candidate recognition result, which is an index that has no direct association with semantics. The semantic change evaluation index is an index that can reflect the semantic change of the corresponding candidate recognition result relative to the annotated text of the training speech.

[0076] The text unit error rate corresponding to a candidate recognition result is determined according to the candidate recognition result and the labeled text of the training speech. More specifically, it is determined according to the unaligned part between the candidate recognition result and the labeled text of the training speech.

[0077] Optionally, the text unit error rate can be the word error rate WER, and the word error rate WER can be calculated by the following formula:

[0078]

[0079] Where S represents the number of replaced words, D represents the number of deleted words, I represents the number of inserted words, and M represents the total number of words in the labeled text of the training speech.

[0080] When determining the word error rate corresponding to a candidate recognition result, the unaligned part between the candidate recognition result and the labeled text of the training speech can be first determined. Then, for the unaligned part, the number of replaced words, the number of deleted words, and the number of inserted words are counted, and the total number of words in the labeled text of the training speech is also counted. Finally, the word error rate corresponding to the candidate recognition result is calculated in the manner shown in the above formula (1).

[0081] It should be noted that this embodiment does not limit the text unit error rate to the word error rate, and it can also be other, such as the word error rate, etc.

[0082] Optionally, the above semantic change evaluation metrics can include the part-of-speech deviation degree and / or the syntactic category deviation degree. Among them, the part-of-speech deviation degree can reflect the deviation degree of the corresponding candidate recognition result in terms of part of speech relative to the labeled text of the training speech, and the syntactic category deviation degree can reflect the deviation degree of the corresponding candidate recognition result in terms of syntactic category relative to the labeled text of the training speech. The specific determination methods of the part-of-speech deviation degree and the syntactic category deviation degree will be introduced in subsequent embodiments.

[0083] It should be noted that the above part-of-speech deviation degree and syntactic category deviation degree are only examples. The present invention does not limit the semantic change evaluation metrics to the part-of-speech deviation degree and the syntactic category deviation degree, and other metrics that can reflect the semantic change of the corresponding candidate recognition result relative to the labeled text of the training speech can also be used.

[0084] Step S203: Combine the text unit error rate and the semantic change evaluation metric corresponding to each candidate recognition result to determine the prediction loss of the speech recognition baseline model on each candidate recognition result.

[0085] The specific implementation process of step S203 includes: for each candidate recognition result, execute:

[0086] Step a1: Determine the weight corresponding to the candidate recognition result according to the text unit error rate and the semantic change evaluation metric corresponding to the candidate recognition result.

[0087] Specifically, according to the text unit error rate and semantic change evaluation index corresponding to the candidate recognition result, determine the weight corresponding to the candidate recognition result, including:

[0088] Step a11: Calculate the difference between the text unit error rate corresponding to the candidate recognition result and the average text unit error rate, the difference between the part-of-speech deviation degree corresponding to the candidate recognition result and the average part-of-speech deviation degree, and the difference between the syntactic category deviation degree corresponding to the candidate recognition result and the average syntactic category deviation degree.

[0089] Among them, the average character error rate is the average of the text unit error rates corresponding to several candidate recognition results (such as Nbest candidate recognition results), the average part-of-speech deviation degree is the average of the part-of-speech deviation degrees corresponding to several candidate recognition results, and the average syntactic category deviation degree is the average of the syntactic category deviation degrees corresponding to several candidate recognition results.

[0090] For the j-th candidate recognition result among several candidate recognition results, if the text unit error rate corresponding to the j-th candidate recognition result is expressed as W(y j , y*), and the average text unit error rate is expressed as Express the part-of-speech deviation degree corresponding to the j-th candidate recognition result as P(y j , y*), and the average part-of-speech deviation degree is expressed as Express the syntactic category deviation degree corresponding to the j-th candidate recognition result as S(y j , y*), and the average syntactic category deviation degree is expressed as Then three differences can be obtained through step a11 Among them, y j represents the j-th candidate recognition result, and y* represents the labeled text of the training speech.

[0091] Step a12: Integrate the calculated differences, and use the integration result as the weight corresponding to the candidate recognition result.

[0092] Optionally, for the j-th candidate recognition result among several candidate recognition results, the three differences can be integrated through the following formula:

[0093]

[0094] Among them, α and β are balance coefficients.

[0095] It should be noted that this embodiment does not limit the integration of each difference in the manner of formula (2), and other methods can also be used, for example, directly summing each difference.

[0096] Step a2: Determine the prediction loss of the speech recognition baseline model on the candidate recognition result according to the weight corresponding to the candidate recognition result and the prediction probability corresponding to the candidate recognition result.

[0097] Specifically, the weight corresponding to the candidate recognition result can be used to weight the prediction probability corresponding to the candidate recognition result (i.e., multiply the weight corresponding to the candidate recognition result by the prediction probability corresponding to the candidate recognition result), and the obtained result is used as the prediction loss of the speech recognition baseline model on the candidate recognition result.

[0098] The prediction loss of the speech recognition baseline model on the j-th candidate recognition result among several candidate recognition results can be expressed as:

[0099]

[0100] Step S204: Update the parameters of the speech recognition baseline model according to the prediction loss of the speech recognition baseline model on each candidate recognition result.

[0101] Specifically, the prediction losses of the speech recognition baseline model on each candidate recognition result can be summed, and the sum of the losses is used as the prediction loss of the speech recognition baseline model. The parameters of the speech recognition baseline model are updated according to the prediction loss of the speech recognition baseline model. The prediction loss of the speech recognition baseline model can be expressed as:

[0102]

[0103] The speech recognition model adopted in the speech recognition method provided by the embodiment of the present invention is trained in two stages. That is, first, a speech recognition baseline model is trained based on the cross-entropy criterion, and then the speech recognition baseline model is further trained with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition results of the training speech. The speech recognition model obtained through the above two-stage training has good performance. When using it to recognize the speech to be recognized, a recognition result with a relatively high acceptability can be obtained, and the user experience is good. When training the speech recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition results of the training speech, in the embodiment of the present invention, on the basis of the text unit error rate, the part-of-speech deviation degree and syntactic category deviation degree that can reflect the semantic change situation are introduced, and the decoding results of the speech recognition are subjected to discriminative training in combination with the text unit error rate, the part-of-speech deviation degree and the syntactic category deviation degree, which can improve the acceptability of the speech recognition results.

[0104] Next, the determination methods of the part-of-speech deviation degree and the syntactic category deviation degree will be introduced. Since the determination methods of the part-of-speech deviation degree and the syntactic category deviation degree corresponding to each candidate recognition result are the same, in this embodiment, one candidate recognition result is taken as an example to introduce the determination methods of the part-of-speech deviation degree and the syntactic category deviation degree.

[0105] First, the process of determining the part-of-speech deviation degree corresponding to one candidate recognition result will be introduced.

[0106] The part-of-speech deviation degree is a statistical index for part-of-speech deviation. Considering that different parts of speech have certain discriminability in terms of semantic importance, the part-of-speech deviation degree in the present invention focuses on the parts where the part of speech changes after the candidate recognition result of the training speech is aligned with the annotated text of the training speech due to recognition errors, deletion errors, and insertion errors. It should be noted that if a recognition error occurs but the part of speech does not change, it is considered that there is no deviation from the perspective of part-of-speech deviation.

[0107] The process of determining the part-of-speech deviation degree corresponding to one candidate recognition result may include:

[0108] Step b1: Determine the part of speech of each word included in the annotated text of the training speech, and determine the part of speech of the words included in the unaligned part of the candidate recognition result and the annotated text of the training speech.

[0109] When determining the part of speech of each word included in the annotated text of the training speech, first perform word segmentation on the annotated text of the training speech to obtain each word included in the annotated text of the training speech, and then determine the part of speech of each word obtained through word segmentation.

[0110] When determining the part of speech of each word in the unaligned part of the candidate recognition result and the annotated text of the training speech, it is also necessary to perform word segmentation on the candidate recognition result, and then align the word segmentation result of the annotated text of the training speech with the word segmentation result of the candidate recognition result, so as to determine each word in the unaligned part of the candidate recognition result and the annotated text of the training speech, and further determine the part of speech of each word in the aligned part.

[0111] In this embodiment, the parts of speech of the annotated text of the training speech and the candidate recognition result of the training speech can be obtained by using common word segmentation tools.

[0112] Step b2: Determine the part-of-speech deviation degree corresponding to this candidate recognition result according to the part of speech of each word included in the annotated text of the training speech and the part of speech of the words included in the unaligned part of the candidate recognition result and the annotated text of the training speech.

[0113] Specifically, the implementation process of step b2 may include: determining the part-of-speech deviation weight of the candidate recognition result relative to the annotated text of the training speech according to the weight of the part of speech of the words included in the unaligned part between the candidate recognition result and the annotated text of the training speech, where the weight of the part of speech of a word can represent the importance degree of the part of speech of the word; summing up the weights of the parts of speech of all the words included in the annotated text of the training speech to obtain the sum of part-of-speech weights; and determining the part-of-speech deviation degree corresponding to the candidate recognition result according to the sum of part-of-speech weights and the part-of-speech deviation weight of the candidate recognition result relative to the annotated text of the training speech. Specifically, the ratio of the part-of-speech deviation weight of the candidate recognition result relative to the annotated text of the training speech to the sum of part-of-speech weights can be calculated as the part-of-speech deviation degree corresponding to the candidate recognition result.

[0114] In this embodiment, the part-of-speech deviation degree emphasizes the influence of the change in part of speech on the rationality of the grammatical structure through the definition of the weight of part-of-speech importance. The calculation formula of the part-of-speech deviation degree is:

[0115]

[0116] where, W j represents the weight of the part of speech of the j-th word that is unaligned between the candidate recognition result and the annotated text of the training speech, N represents the total number of words included in the annotated text of the training speech, and W i represents the weight of the part of speech of the i-th word included in the annotated text of the training speech.

[0117] To determine the part-of-speech deviation degree, the present invention pre-defines the weights of various parts of speech. When defining the weights of parts of speech, it is mainly based on principles such as the combination principle (the superior grammatical structure is composed of several secondary grammatical structures combined according to a certain level, and the overall fluency of the sentence is the sum of the local fluencies) and the predication center principle (the stronger the possibility of being the predicate of an independent sentence, the greater the impact on the legality of the grammatical structure), etc. Based on the 863 part-of-speech annotation set, the importance of different parts of speech is defined:

[0118]

[0119]

[0120] Next, two specific examples are combined to introduce the process of determining the part-of-speech deviation degree:

[0121] The first example: The parts of speech of the words included in the annotated text of the training speech and a candidate recognition result of the training speech are as follows:

[0122]

[0123]

[0124] Since the part of the candidate recognition result of the training speech that is not aligned with the annotated text of the training speech is pharynx (v) -> smoke (n), the numerator in the above formula (2) is the part-of-speech weight "6", and the denominator is the sum of the part-of-speech weights of each word contained in the annotated text of the training speech (6+4+1+2+3+4+6+6), that is, the part-of-speech deviation corresponding to the candidate recognition result of the training speech is POSD = 6 / (6+4+1+2+3+4+6+6) = 0.1875.

[0125] Second example: The parts of speech of each word in the annotated text of the training speech and the candidate recognition result of the training speech are as follows:

[0126]

[0127] Since the parts of the candidate recognition result that are not aligned with the annotated text of the training speech include: I (r) -> NULL, pharynx (v) -> smoke (n), NULL -> asp, the numerator in the above formula (2) is the sum of the part-of-speech weight 4, the part-of-speech weight 6, and the part-of-speech weight 5, and the denominator is the sum of the part-of-speech weights of each word contained in the annotated text of the training speech (6+4+1+2+3+4+6+6), that is, the part-of-speech deviation POSD corresponding to the candidate recognition result = (4+6+5) / (6+4+1+2+3+4+6+6) = 0.46875.

[0128] Finally, the method of determining the degree of syntactic category deviation is introduced.

[0129] Syntactic category deviation is a statistical indicator of syntactic category deviation. Since sentences are composed of grammatical components combined according to a certain hierarchical structure, and the closeness between syntactic components is inconsistent, once the closely connected grammatical relationship is destroyed, it will be more difficult for users to understand. When recognition errors, deletion errors, or insertion errors cause the syntactic relationship between components to change, the component is said to have experienced syntactic category deviation. It should be noted that if a recognition error occurs but does not cause a category change, it is considered that no syntactic category deviation has occurred.

[0130] The process of determining the part-of-speech deviation corresponding to a candidate recognition result may include:

[0131] Step c1: Determine the dependency relationship corresponding to each word in the annotated text of the training speech, and determine the dependency relationship changed by the candidate recognition result and the words in the unaligned part of the annotated text.

[0132] In this embodiment, the dependency relationship between the annotated text of the training speech and the words in the candidate recognition results can be analyzed using a commonly used sentence category analysis tool.

[0133] Step c2: Determine the syntactic category deviation degree corresponding to the candidate recognition result according to the dependency relationships corresponding to each word in the labeled text of the training speech and the changed dependency relationships of the words in the unaligned part between the candidate recognition result and the labeled text.

[0134] Specifically, the specific implementation process of step c2 includes: determining the dependency deviation weight of the candidate recognition result relative to the labeled text of the training speech according to the weight of the changed dependency relationships of the words in the unaligned part between the candidate recognition result and the labeled text; determining the sum of the weights of the dependency relationships corresponding to each word in the labeled text of the training speech to obtain the sum of dependency weights; and determining the syntactic category deviation degree corresponding to the candidate recognition result according to the sum of dependency weights and the dependency deviation weight of the candidate recognition result relative to the labeled text of the training speech. Specifically, the ratio of the dependency deviation weight of the candidate recognition result relative to the labeled text of the training speech to the sum of dependency weights can be calculated as the syntactic category deviation degree corresponding to the candidate recognition result. Among them, the weight of the dependency relationship can represent the importance degree of the dependency relationship.

[0135] The syntactic category deviation degree in this embodiment emphasizes the influence of the change in the category dependency on the rationality of the grammatical structure. The calculation formula of the syntactic category deviation degree SCD is:

[0136]

[0137] where S j represents the weight of the j-th changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text of the training speech, N represents the total number of words contained in the labeled text of the training speech, and S i represents the weight of the dependency relationship corresponding to the i-th word in the labeled text of the training speech.

[0138] To implement the determination of the syntactic category deviation degree, the present invention pre-defines the weights of various dependency relationships. When defining the weights of various dependency relationships, it is mainly based on the combination principle (the superior grammatical structure is composed of several secondary grammatical structures combined according to a certain hierarchy, and the overall sentence fluency is the sum of the local fluencies), the syntactic hierarchy principle (the lower the hierarchy, the closer the combination grammatical relationship of the categories, and the greater the impact on the acceptability of the grammatical structure), and the predication center principle (the stronger the predication function of the dependency object of the category, the greater the impact of the relationship between the categories on the acceptability of the grammatical structure), etc., to define the importance of various dependency relationships:

[0139]

[0140]

[0141] The following uses a specific example to illustrate the method for determining the syntactic category deviation degree:

[0142] Exemplarily, the labeled text of the training speech is ": Those who praise me have swallowed them all", and a candidate recognition result of the training speech is "Those who praise me have smoked them all". Among them, the dependency relationships corresponding to each word in the labeled text of the training speech are as follows:

[0143] Praise(1,7,'FOB') me(2,1,'VOB')'s(3,1,'RAD') all(4,7,'ADV') are(5,7,'ADV') swallowed(6,5,'POB') by(7,0,'HED') them(8,7,'CMP')

[0144] Among them, in (i, j, dependency relationship label), i represents the ranking of the corresponding word in the sentence, j represents the ranking of the dependency object of the corresponding word, and the dependency relationship label represents the type of dependency relationship.

[0145] Comparing the candidate recognition result of the training speech with the labeled text of the training speech, it is found that the words in the unaligned part between the candidate recognition result of the training speech and the labeled text of the training speech are "swallowed - smoked", and the changes in the dependency relationships involved include "HED–>Root" and "CMP–>HED". It can be seen that among the dependency relationships corresponding to each word included in the labeled text, "HED" and "CMP" are changed. Therefore, the numerator in formula (6) is the sum of the weight "2" of "HED" and the weight "6" of "CMP", and the denominator is the sum of the weights of the dependency relationships "FOB", "VOB", "RAD", "ADV", "ADV", "POB", "HED", "CMP" corresponding to each word included in the labeled text of the training speech (5 + 9 + 6 + 7 + 7 + 6 + 2 + 11), that is, the syntactic category deviation degree SCD corresponding to the candidate recognition result = (2 + 11) / (5 + 9 + 6 + 7 + 7 + 6 + 2 + 11)≈0.245.

[0146] The part-of-speech deviation degree and syntactic category deviation degree in this embodiment emphasize the semantic changes of the candidate recognition result of the training speech relative to the labeled text of the training speech from the aspects of part of speech and syntactic category dependence.

[0147] The embodiment of the present invention also provides a speech recognition device. The speech recognition device provided by the embodiment of the present invention will be described below. The speech recognition device described below can be correspondingly referred to the speech recognition method described above.

[0148] Please refer to Figure 3 , which shows a schematic structural diagram of the speech recognition device provided by the embodiment of the present invention, and may include: a speech acquisition module 301 and a speech recognition module 302.

[0149] The voice acquisition module 301 is configured to acquire the voice to be recognized.

[0150] The voice recognition module 302 is configured to recognize the voice to be recognized based on a pre-trained voice recognition model.

[0151] Wherein, the voice recognition model is obtained through two-stage training. The first stage is trained with the goal of making the recognition result of the training voice consistent with the text annotated by the training voice. The second stage is trained with the goal of balancing the text unit error rate and semantic acceptability of the voice recognition result of the training voice.

[0152] Optionally, the voice recognition device provided in this embodiment may further include: a model training module.

[0153] The model training module includes: a first training sub-module and a second training sub-module.

[0154] The first training sub-module is configured to train a voice recognition baseline model with the goal of making the recognition result of the training voice consistent with the text annotated by the training voice. <tmp>id="18" name="18"

[0155] The second training sub-module is configured to train the voice recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the voice recognition result of the training voice to obtain the final voice recognition model.

[0156] Optionally, when the second training sub-module trains the voice recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the voice recognition result of the training voice, it is specifically configured to:

[0157] Recognize the training voice based on the voice recognition baseline model to obtain several candidate recognition results of the training voice;

[0158] Determine the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result, wherein the semantic change evaluation index can reflect the semantic change of the corresponding candidate recognition result relative to the text annotated by the training voice;

[0159] Combine the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result to determine the prediction loss of the voice recognition baseline model on each candidate recognition result;

[0160] Update the parameters of the voice recognition baseline model according to the determined prediction loss.

[0161] Optionally, when the second training sub-module combines the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result to determine the prediction loss of the voice recognition baseline model on each candidate recognition result, it is specifically configured to:

[0162] For each candidate recognition result:

[0163] Determine the weight corresponding to the candidate recognition result according to the text unit error rate and semantic change evaluation index corresponding to the candidate recognition result;

[0164] Determine the prediction loss of the speech recognition baseline model on the candidate recognition result according to the weight corresponding to the candidate recognition result and the prediction probability corresponding to the candidate recognition result.

[0165] Optionally, the second training sub-module includes, in the semantic change evaluation index: part-of-speech deviation and / or syntactic category deviation.

[0166] Among them, the part-of-speech deviation can reflect the deviation degree of the corresponding candidate recognition result in terms of part of speech relative to the annotated text of the training speech, and the syntactic category deviation can reflect the deviation degree of the corresponding candidate recognition result in terms of syntactic category relative to the annotated text of the training speech.

[0167] Optionally, when the second training sub-module determines the weight corresponding to the candidate recognition result according to the text unit error rate and semantic change evaluation index corresponding to the candidate recognition result, it is specifically used for:

[0168] Calculate the difference between the text unit error rate corresponding to the candidate recognition result and the average text unit error rate, the difference between the part-of-speech deviation corresponding to the candidate recognition result and the average part-of-speech deviation, and the difference between the syntactic category deviation corresponding to the candidate recognition result and the average syntactic category deviation. Among them, the average word error rate is the average of the text unit error rates corresponding to the several candidate recognition results, the average part-of-speech deviation is the average of the part-of-speech deviations corresponding to the several candidate recognition results, and the average syntactic category deviation is the average of the syntactic category deviations corresponding to the several candidate recognition results;

[0169] Fuse the calculated differences, and use the fusion result as the weight corresponding to the candidate recognition result.

[0170] Optionally, when the second training sub-module determines the part-of-speech deviation corresponding to a candidate recognition result, it is specifically used for:

[0171] Determine the part of speech of each word included in the annotated text of the training speech, and determine the part of speech of the words included in the unaligned part of the candidate recognition result and the annotated text of the training speech;

[0172] Determine the part-of-speech deviation corresponding to the candidate recognition result according to the part of speech of each word included in the annotated text of the training speech and the part of speech of the words included in the unaligned part of the candidate recognition result and the annotated text of the training speech.

[0173] Optionally, when determining the part-of-speech deviation degree corresponding to a candidate recognition result according to the part-of-speech of each word included in the labeled text of the training speech and the part-of-speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech, the second training sub-module is specifically configured to:

[0174] Determine the part-of-speech deviation weight of the candidate recognition result relative to the labeled text of the training speech according to the weight of the part-of-speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech, where the weight of the part-of-speech of a word can represent the importance degree of the part-of-speech of the word;

[0175] Sum up the weights of the part-of-speech of each word included in the labeled text of the training speech to obtain the sum of part-of-speech weights;

[0176] Determine the part-of-speech deviation degree corresponding to the candidate recognition result according to the sum of part-of-speech weights and the part-of-speech deviation weight of the candidate recognition result relative to the labeled text of the training speech.

[0177] Optionally, when determining the syntactic category deviation degree corresponding to a candidate recognition result, the second training sub-module is specifically configured to:

[0178] Determine the dependency relationship corresponding to each word in the labeled text of the training speech, and determine the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text;

[0179] Determine the syntactic category deviation degree corresponding to the candidate recognition result according to the dependency relationship corresponding to each word in the labeled text of the training speech and the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text.

[0180] Optionally, when determining the syntactic category deviation degree corresponding to a candidate recognition result according to the dependency relationship corresponding to each word in the labeled text of the training speech and the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text, the second training sub-module is specifically configured to:

[0181] Determine the dependency relationship deviation weight of the candidate recognition result relative to the labeled text of the training speech according to the weight of the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text;

[0182] Determine the sum of the weights of the dependency relationships corresponding to each word in the labeled text of the training speech to obtain the sum of dependency relationship weights;

[0183] Determine the syntactic category deviation degree corresponding to the candidate recognition result according to the sum of dependency relationship weights and the dependency relationship deviation weight of the candidate recognition result relative to the labeled text of the training speech.

[0184] The speech recognition model adopted in the speech recognition device provided by the embodiments of the present invention is trained in two stages. That is, first, a speech recognition baseline model is trained with the goal of making the recognition result of the training speech consistent with the text annotated for the training speech. Then, with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech, the speech recognition baseline model is further trained. The speech recognition model obtained through the above two-stage training has good performance. When using it to recognize the speech to be recognized, a recognition result with a relatively high acceptability can be obtained, and the user experience is good. When training the speech recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech, the embodiments of the present invention introduce the part-of-speech deviation degree and syntactic category deviation degree that can reflect semantic changes on the basis of the text unit error rate, and perform discriminative training on the decoding result of speech recognition by combining the text unit error rate, part-of-speech deviation degree, and syntactic category deviation degree, which can improve the acceptability of the speech recognition result.

[0185] Embodiments of the present invention also provide a speech recognition device. Please refer to Figure 4 , which shows a schematic structural diagram of the speech recognition device. The speech recognition device may include: at least one processor 401, at least one communication interface 402, at least one memory 403, and at least one communication bus 404;

[0186] In the embodiments of the present invention, the number of the processor 401, communication interface 402, memory 403, and communication bus 404 is at least one, and the processor 401, communication interface 402, and memory 403 complete communication with each other through the communication bus 404;

[0187] The processor 401 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0188] The memory 403 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0189] Among them, the memory stores a program, and the processor can call the program stored in the memory. The program is used for:

[0190] Obtain the speech to be recognized;

[0191] Recognize the speech to be recognized based on a pre-trained speech recognition model;

[0192] Among them, the speech recognition model is obtained through two-stage training. The first stage is trained with the goal of making the recognition result of the training speech consistent with the text annotated for the training speech. The second stage is trained with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech.

[0193] Optionally, the refinement function and expansion function of the program can be referred to the above description.

[0194] An embodiment of the present invention further provides a readable storage medium, which can store a program suitable for execution by a processor. The program is used for:

[0195] Obtain the speech to be recognized;

[0196] Recognize the speech to be recognized based on a pre-trained speech recognition model;

[0197] Among them, the speech recognition model is obtained through two-stage training. The first stage is trained with the goal of making the recognition result of the training speech consistent with the text annotated for the training speech. The second stage is trained with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech.

[0198] Optionally, the refinement function and expansion function of the program can be referred to the above description.

[0199] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0200] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same and similar parts among the embodiments can be referred to each other.

[0201] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech recognition method, characterized in that, including: obtaining the speech to be recognized; recognizing the speech to be recognized based on a pre-trained speech recognition model; wherein, the speech recognition model is obtained through two-stage training. In the first stage, a speech recognition baseline model is trained with the goal of making the recognition result of the training speech consistent with the text annotated for the training speech. In the second stage, the speech recognition baseline model is trained with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition result of the training speech; training the speech recognition baseline model includes: recognizing the training speech based on the speech recognition baseline model to obtain several candidate recognition results of the training speech; determining the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result, wherein the semantic change evaluation index can reflect the semantic change of the corresponding candidate recognition result relative to the text annotated for the training speech; combining the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result to determine the prediction loss of the speech recognition baseline model for each candidate recognition result; updating the parameters of the speech recognition baseline model according to the determined prediction loss.

2. The speech recognition method according to claim 1, wherein The combining the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result to determine the prediction loss of the speech recognition baseline model for each candidate recognition result includes: 0For each candidate recognition result: 1determining the weight corresponding to the candidate recognition result according to the text unit error rate and semantic change evaluation index corresponding to the candidate recognition result; 2determining the prediction loss of the speech recognition baseline model for the candidate recognition result according to the weight corresponding to the candidate recognition result and the prediction probability corresponding to the candidate recognition result.

3. The speech recognition method according to claim 1, wherein 3The semantic change evaluation index includes: part-of-speech deviation and / or syntactic category deviation; 4The part-of-speech deviation can reflect the deviation degree of the corresponding candidate recognition result in terms of part of speech relative to the text annotated for the training speech; 5The syntactic category deviation can reflect the deviation degree of the corresponding candidate recognition result in terms of syntactic category relative to the text annotated for the training speech.

4. The speech recognition method according to claim 2, wherein 6The determining the weight corresponding to the candidate recognition result according to the text unit error rate and semantic change evaluation index corresponding to the candidate recognition result includes: 7calculating the difference between the text unit error rate corresponding to the candidate recognition result and the average text unit error rate, the difference between the part-of-speech deviation corresponding to the candidate recognition result and the average part-of-speech deviation, and the difference between the syntactic category deviation corresponding to the candidate recognition result and the average syntactic category deviation, wherein the average text unit error rate is the average of the text unit error rates respectively corresponding to the several candidate recognition results, the average part-of-speech deviation is the average of the part-of-speech deviations respectively corresponding to the several candidate recognition results, and the average syntactic category deviation is the average of the syntactic category deviations respectively corresponding to the several candidate recognition results; 8fusing the calculated differences, and taking the fusion result as the weight corresponding to the candidate recognition result.

5. The voice recognition method according to claim 3, wherein 9Determining the part-of-speech deviation corresponding to a candidate recognition result includes: Determine the part of speech of each word included in the labeled text of the training speech, and determine the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech; Determine the part-of-speech deviation degree corresponding to this candidate recognition result according to the part of speech of each word included in the labeled text of the training speech and the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech.

6. The voice recognition method according to claim 5, wherein The determining the part-of-speech deviation degree corresponding to this candidate recognition result according to the part of speech of each word included in the labeled text of the training speech and the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech includes: Determine the part-of-speech deviation weight of this candidate recognition result relative to the labeled text of the training speech according to the weight of the part of speech of the words included in the unaligned part between the candidate recognition result and the labeled text of the training speech, where the weight of the part of speech of a word can characterize the importance degree of the part of speech of this word; Sum up the weights of the parts of speech of each word included in the labeled text of the training speech to obtain the sum of part-of-speech weights; Determine the part-of-speech deviation degree corresponding to this candidate recognition result according to the sum of part-of-speech weights and the part-of-speech deviation weight of this candidate recognition result relative to the labeled text of the training speech.

7. The voice recognition method according to claim 3, wherein Determine the syntactic category deviation degree corresponding to a candidate recognition result, including: Determine the dependency relationship corresponding to each word in the labeled text of the training speech, and determine the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text; Determine the syntactic category deviation degree corresponding to this candidate recognition result according to the dependency relationship corresponding to each word in the labeled text of the training speech and the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text.

8. The speech recognition method according to claim 7, characterized in that, The determining the syntactic category deviation degree corresponding to this candidate recognition result according to the dependency relationship corresponding to each word in the labeled text of the training speech and the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text includes: Determine the dependency relationship deviation weight of this candidate recognition result relative to the labeled text of the training speech according to the weight of the changed dependency relationship of the words in the unaligned part between the candidate recognition result and the labeled text; Determine the sum of the weights of the dependency relationships corresponding to each word in the labeled text of the training speech to obtain the sum of dependency relationship weights; Determine the syntactic category deviation degree corresponding to this candidate recognition result according to the sum of dependency relationship weights and the dependency relationship deviation weight of this candidate recognition result relative to the labeled text of the training speech.

9. A voice recognition device, characterized in that, including: A model training module, a speech acquisition module, and a speech recognition module; The model training module includes: a first training sub-module and a second training sub-module; The speech acquisition module is used to acquire the speech to be recognized; The speech recognition module is used to recognize the speech to be recognized based on a pre-trained speech recognition model. Among them, the speech recognition model is obtained through two-stage training. In the first stage, the first training sub-module trains to obtain a speech recognition baseline model with the goal of making the recognition result of the training speech consistent with the text annotated for the training speech. In the second stage, the second training sub-module trains the speech recognition baseline model with the goal of balancing the text unit error rate and semantic acceptability of the speech recognition results of the training speech to obtain the final speech recognition model. The second training sub-module trains the speech recognition baseline model, including: Recognize the training speech based on the speech recognition baseline model to obtain several candidate recognition results of the training speech. Determine the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result, where the semantic change evaluation index can reflect the semantic change of the corresponding candidate recognition result relative to the text annotated for the training speech. Combine the text unit error rate and semantic change evaluation index corresponding to each candidate recognition result to determine the prediction loss of the speech recognition baseline model for each candidate recognition result. Update the parameters of the speech recognition baseline model according to the determined prediction loss.

10. A voice recognition device, characterized in that, including: A memory and a processor; The memory is used to store programs. The processor is used to execute the program to implement each step of the speech recognition method described in any one of claims 1 to 8.

11. A readable storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, each step of the speech recognition method described in any one of claims 1 to 8 is implemented.

Images