Speech recognition model training method, speech recognition method and related device

Abstract

The invention provides a voice recognition model training method, a voice recognition method and a related device, and the method comprises the steps: carrying out the phoneme feature extraction of a first sample voice, and obtaining the phoneme feature of the first sample voice; according to the time sequence information of the phoneme features, attention processing is carried out on the phoneme features, and phoneme time sequence features are obtained; decoding the phoneme time sequence characteristics to obtain a first text; determining a first loss value according to the phoneme feature and a phoneme label of the first sample voice; determining a second loss value according to the first text and a first text label of the first sample voice; and training the speech recognition model according to the first loss value and the second loss value. According to the embodiment of the invention, the recognition accuracy of the speech recognition model can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a training method for a speech recognition model, a speech recognition method and related devices. Background Art

[0002] In recent years, speech recognition technology has developed rapidly and has been widely used. Speech recognition technology is usually implemented based on a speech recognition model. In the related art, in order to simplify the training task of the speech recognition model, an end-to-end training method is usually adopted. This training method uses a training set including a large amount of speech-text parallel corpus to supervise the speech recognition model so that the speech recognition model directly learns the mapping from speech to text. However, the recognition accuracy of the speech recognition model obtained by end-to-end training is not high. Summary of the invention

[0003] The present invention provides a training method for a speech recognition model, a speech recognition method and related devices.

[0004] In a first aspect, the present disclosure provides a method for training a speech recognition model, the method comprising: extracting phoneme features from a first sample speech to obtain phoneme features of the first sample speech, the phoneme features being used to represent pronunciation phonemes corresponding to each audio frame of the first sample speech; performing attention processing on the phoneme features based on timing information of the phoneme features to obtain phoneme timing features; decoding the phoneme timing features to obtain a first text; determining a first loss value based on the phoneme features and the phoneme label of the first sample speech; determining a second loss value based on the first text and the first text label of the first sample speech; and training the speech recognition model based on the first loss value and the second loss value.

[0005] In a second aspect, the present disclosure provides a speech recognition method, which comprises: performing speech recognition on a speech to be recognized through a speech recognition model to obtain a target text; wherein the speech recognition model is trained according to the training method of the above-mentioned speech recognition model.

[0006] In a third aspect, the present disclosure provides a training device for a speech recognition model, the device comprising: a first extraction module, used to extract phoneme features from a first sample speech to obtain the phoneme features of the first sample speech, wherein the phoneme features are used to represent the pronunciation phonemes corresponding to each audio frame of the first sample speech; a second extraction module, used to perform attention processing on the phoneme features according to the timing information of the phoneme features to obtain phoneme timing features; a decoding module, used to decode the phoneme timing features to obtain a first text; a first determination module, used to determine a first loss value according to the phoneme features and the phoneme label of the first sample speech; a second determination module, used to determine a second loss value according to the first text and the first text label of the first sample speech; and a training module, used to train the speech recognition model according to the first loss value and the second loss value.

[0007] In a fourth aspect, the present disclosure provides a speech recognition device, comprising: a speech recognition module, used to perform speech recognition on a speech to be recognized through a speech recognition model to obtain a target text; wherein the speech recognition model is trained according to the training method of the above-mentioned speech recognition model.

[0008] In a fifth aspect, the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned speech recognition model training method or speech recognition method.

[0009] In a sixth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned speech recognition model training method or speech recognition method.

[0010] In the seventh aspect, the present disclosure provides a computer program product, which includes a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned speech recognition model training method or speech recognition method.

[0011] The training method of the speech recognition model of the embodiment of the present disclosure can extract the phoneme features of the first sample speech to obtain the phoneme features of the first sample speech, and perform attention processing on the phoneme features according to the timing information of the phoneme features to obtain the phoneme timing features; then decode the phoneme timing features to obtain the first text; then determine the first loss value according to the phoneme features and the phoneme labels of the first sample speech, and determine the second loss value according to the first text and the first text label of the first sample speech; then train the speech recognition model according to the first loss value and the second loss value, so that the speech recognition model can be trained according to the first loss value (determined by the phoneme features and the phoneme labels of the first sample speech) and the second loss value (determined by the first text and the first text label of the first sample speech). Through this training method, the speech recognition model can learn the mapping from speech to text and the corresponding relationship between audio frames and phonemes, which not only improves the convergence speed of the speech recognition model, but also improves the recognition accuracy of the speech recognition model.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0014] Figure 1 A schematic diagram of a speech recognition model provided in an embodiment of the present disclosure;

[0015] Figure 2 A flowchart of a method for training a speech recognition model provided in an embodiment of the present disclosure;

[0016] Figure 3 A schematic diagram of a training process of a method for training a speech recognition model provided in an embodiment of the present disclosure;

[0017] Figure 4 A schematic diagram of a verification process of a training method for a speech recognition model provided in an embodiment of the present disclosure;

[0018] Figure 5 A block diagram of a speech recognition model training device provided in an embodiment of the present disclosure;

[0019] Figure 6 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0021] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0022] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0023] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0024] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0025] In the related art, the training of speech recognition models is usually carried out in an end-to-end manner, in which the speech recognition model cannot realize the mandatory alignment of audio frames and phonemes, thus affecting the recognition accuracy of the speech recognition model.

[0026] In addition, since the speech recognition model is trained in an end-to-end manner, only the mapping from speech to text is learned. When the amount of training data is small or the number of layers of the speech recognition model is too large, it is difficult to learn the regular distribution of data features, which makes it difficult for the speech recognition model to converge.

[0027] In view of this, an embodiment of the present disclosure provides a method for training a speech recognition model, the method comprising: extracting phoneme features of a first sample speech to obtain phoneme features of the first sample speech, the phoneme features being used to represent the pronunciation phonemes corresponding to each audio frame of the first sample speech; performing attention processing on the phoneme features according to the timing information of the phoneme features to obtain phoneme timing features; decoding the phoneme timing features to obtain a first text; determining a first loss value according to the phoneme features and the phoneme label of the first sample speech; determining a second loss value according to the first text and the first text label of the first sample speech; and training the speech recognition model according to the first loss value and the second loss value.

[0028] The training method of the speech recognition model of the embodiment of the present disclosure can extract the phoneme features of the first sample speech to obtain the phoneme features of the first sample speech, and perform attention processing on the phoneme features according to the timing information of the phoneme features to obtain the phoneme timing features; then decode the phoneme timing features to obtain the first text; then determine the first loss value according to the phoneme features and the phoneme labels of the first sample speech, and determine the second loss value according to the first text and the first text label of the first sample speech; then train the speech recognition model according to the first loss value and the second loss value, so that the speech recognition model can be trained according to the first loss value (determined by the phoneme features and the phoneme labels of the first sample speech) and the second loss value (determined by the first text and the first text label of the first sample speech). Through this training method, the speech recognition model can learn the mapping from speech to text and the corresponding relationship between audio frames and phonemes, which not only improves the convergence speed of the speech recognition model, but also improves the recognition accuracy of the speech recognition model.

[0029] The training method of the speech recognition model of the embodiment of the present disclosure can be executed by an electronic device such as a server. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. The present disclosure does not limit the specific type of the server. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory.

[0030] The speech recognition model of the embodiment of the present disclosure may be an encoder-decoder architecture. The speech recognition model includes an encoder and a decoder, which can perform speech recognition on the input speech (i.e., audio) to obtain the corresponding text. The encoder can be used to extract the phoneme features of the input speech to obtain the phoneme features, and perform attention processing on the phoneme features according to the timing information of the phoneme features to obtain the phoneme timing features. The decoder can be used to decode the timing phoneme features output by the encoder to obtain the text corresponding to the speech.

[0031] In some possible implementations, the encoder may include M (M is a positive integer) layers. The first N (N is a positive integer and N < M) layers of the encoder can be used to extract phoneme features from the input speech to obtain phoneme features. The N+1 to M layers of the encoder can be used to perform attention processing on the phoneme features according to the timing information of the phoneme features to obtain phoneme timing features. Based on this, the first N layers of the encoder can be regarded as a phoneme feature extraction network, and the N+1 to M layers of the encoder can be regarded as a timing feature extraction network. Among them, the phoneme feature extraction network can be implemented by CNN (Convolutional Neural Networks) and the like, and the timing feature extraction network can be implemented by a neural network based on the attention mechanism (Attention).

[0032] It should be noted that those skilled in the art can determine the specific implementation of the phoneme feature extraction network and the time series feature extraction network according to actual conditions, and the present disclosure does not limit this.

[0033] Figure 1 A schematic diagram of a speech recognition model provided by an embodiment of the present disclosure. Figure 1 The speech recognition model 100 includes an encoder 110 and a decoder 120 , and the encoder 110 includes a phoneme feature extraction network 111 and a time series feature extraction network 112 .

[0034] The phoneme feature extraction network 111 can be used to extract phoneme features from the speech input to the speech recognition model 100 to obtain phoneme features, and the phoneme features can be used to represent the pronunciation phonemes corresponding to each audio frame of the speech. In other words, the speech input to the speech recognition model is first processed by the phoneme feature extraction network 111 to obtain phoneme features.

[0035] The time series feature extraction network 112 can be used to perform attention processing on the phoneme features according to the time series information of the phoneme features (output of the phoneme feature extraction network 111) to obtain the phoneme time series features. It can be considered that the phoneme time series features are the time series features of the speech added on the basis of the phoneme features.

[0036] The decoder 120 can be used to decode the phoneme timing features (the output of the timing feature extraction network 112) to obtain the recognized text, and the recognized text is the output of the speech recognition model 100.

[0037] In some possible implementations, before training the speech recognition model, a training set may be established. The training set includes a plurality of first sample voices and a phoneme label and a first text label of each first sample voice. The phoneme label of the first sample voice may be used to represent the standard pronunciation phoneme corresponding to each audio frame of the first sample voice; and the first text label may be used to represent the standard text corresponding to the first sample voice.

[0038] In some possible implementations, when establishing a training set, a large number of speech segments are first collected, and a speech segment is regarded as a first sample speech; then, for any first sample speech, the following processing is performed:

[0039] Step 1: Determine the text corresponding to the first sample speech by manual recognition, intelligent recognition + manual proofreading, etc., and use the determined text as the first text label of the first sample speech;

[0040] Step 2: extract frequency domain features of the first sample speech to obtain corresponding speech features; for example, a set of bandpass filters can be set from dense to sparse according to the size of the critical bandwidth within a preset frequency band (including from low frequency to high frequency), and then the first sample speech is filtered to obtain its MFCC (Mel Frequency Cepstrum Coefficient) feature, and the MFCC feature is used as the speech feature of the first sample speech;

[0041] Step three, the speech features of the first sample speech are processed by a trained model (such as a hybrid speech recognition model Hybrid) that realizes forced alignment of audio frames and pronunciation phonemes to obtain the phoneme label of the first sample speech; for example, assuming that the first sample speech has 60 audio frames, and its first text label is "The weather is really good today", after the speech features of the first sample speech are processed by the hybrid speech recognition model Hybrid, the pronunciation phonemes corresponding to each audio frame of the first sample speech are obtained: the pronunciation phonemes corresponding to the 1st to 10th audio frames are all "jin", the pronunciation phonemes corresponding to the 11th to 20th audio frames are all "tian", the pronunciation phonemes corresponding to the 21st to 30th audio frames are all "tian", the pronunciation phonemes corresponding to the 31st to 40th audio frames are all "qi", the pronunciation phonemes corresponding to the 41st to 50th audio frames are all "zhen", and the pronunciation phonemes corresponding to the 51st to 60th audio frames are all "hao", and then the pronunciation phonemes corresponding to the above audio frames are determined as the phoneme labels of the first sample speech;

[0042] Step 4: Add the first sample speech and its first text label and phoneme label to the training set.

[0043] In some possible implementations, after the training set is established in the above manner, the training set can be used to train the speech recognition model.

[0044] Figure 2 A flowchart of a method for training a speech recognition model provided by an embodiment of the present disclosure. Figure 2 , the method comprising:

[0045] Step S201 , extracting phoneme features from a first sample speech to obtain phoneme features of the first sample speech, wherein the phoneme features are used to represent pronunciation phonemes corresponding to each audio frame of the first sample speech.

[0046] Step S202, performing attention processing on the phoneme feature according to the time sequence information of the phoneme feature to obtain a phoneme time sequence feature;

[0047] Step S203, decoding the phoneme time sequence feature to obtain a first text;

[0048] Step S204, determining a first loss value according to the phoneme feature and the phoneme label of the first sample speech;

[0049] Step S205, determining a second loss value according to the first text and the first text label of the first sample speech;

[0050] Step S206: training the speech recognition model according to the first loss value and the second loss value.

[0051] In some possible implementations, in step S201, phoneme features may be extracted from the first sample speech in the training set to obtain the phoneme features of the first sample speech. The phoneme features may be used to represent the pronunciation phonemes corresponding to each audio frame of the first sample speech.

[0052] In some possible implementations, the phoneme features of the first sample speech may be directly extracted to obtain its phoneme features. Figure 1 When the speech recognition model 100 shown is trained, the first sample speech can be input into the speech recognition model 100, and the phoneme features of the first sample speech are extracted through the phoneme feature extraction network 111 in the encoder 110 of the speech recognition model 100 to obtain the phoneme features of the first sample speech.

[0053] In some possible implementations, when performing phoneme feature extraction on the first sample speech, frequency domain feature extraction can be performed on the first sample speech first to obtain speech features of the first sample speech (such as MFCC features), and then phoneme feature extraction can be performed on the speech features of the first sample speech to obtain phoneme features of the first sample speech.

[0054] It should be noted that those skilled in the art may determine the specific implementation method of phoneme feature extraction according to actual conditions, and the present disclosure does not limit this.

[0055] In some possible implementations, in step S202, attention processing may be performed on the phoneme features according to the time sequence information of the phoneme features to obtain the phoneme time sequence features. Figure 1 When the speech recognition model 100 shown is trained, the phoneme features output by the phoneme feature extraction network 111 can be input into the time series feature extraction network 112 for attention processing to obtain the phoneme time series features of the first sample speech. The phoneme time series features here are high-dimensional vectors that characterize the phoneme features and time series features of the first sample speech.

[0056] In some possible implementations, after obtaining the phoneme time sequence features of the first sample speech, the phoneme time sequence features of the first sample speech may be decoded in step S203 to obtain the first text. Decoding is the process of converting a high-dimensional vector representing the phoneme time sequence features into text. Figure 1 When the speech recognition model 100 shown is trained, the phoneme timing features output by the timing feature extraction network 112 can be input into the decoder 120. The decoder 120 decodes the input phoneme timing features, that is, converts the input phoneme timing features (represented as high-dimensional vectors) into text, thereby obtaining a first text of the first sample speech. The first text is also used as the output of the speech recognition model 100.

[0057] In some possible implementations, in step S204, a first loss value may be determined based on the phoneme feature and the phoneme label of the first sample speech. That is, the first loss value may be used to represent the difference between the phoneme feature of the first sample speech and the phoneme label, and therefore, the first loss value may be regarded as a loss value of the phoneme feature extraction process. The first loss value may be calculated by a cross-entropy loss function (CeLoss).

[0058] The first loss value may be calculated, for example, by representing the phoneme label of the first sample speech as a one-hot vector, and then using the phoneme features of the first sample speech and the one-hot vector of the phoneme label as inputs of a cross-entropy loss function, and obtaining the first loss value by calling the cross-entropy loss function.

[0059] In some possible implementations, in step S205, a second loss value may be determined based on the first text and the first text label of the first sample speech. That is, the second loss value may be used to represent the difference between the first text and the first text label, and therefore, the second loss value may be regarded as the loss value of the entire speech recognition process. The second loss value may be calculated by a connectionist temporal classification loss function (CTCLoss). That is, the first text and the first text label may be used as inputs to the connectionist temporal classification loss function, and the second loss value may be obtained by calling the connectionist temporal classification loss function.

[0060] In some possible implementations, the speech recognition model may be trained in step S206 according to the first loss value and the second loss value. When training the speech recognition model, the parameters of the speech recognition model may be adjusted according to the first loss value and the second loss value, respectively: first adjusting the parameters of the speech recognition model according to the first loss value, and then adjusting the parameters of the speech recognition model according to the second loss value; or, first adjusting the parameters of the speech recognition model according to the second loss value, and then adjusting the parameters of the speech recognition model according to the first loss value. When training the speech recognition model, the parameters of the speech recognition model may also be adjusted according to the sum of the first loss value and the second loss value.

[0061] The training method of the speech recognition model of the embodiment of the present disclosure can train the speech recognition model according to a first loss value (determined by phoneme features and the phoneme label of the first sample speech) and a second loss value (determined by the first text and the first text label of the first sample speech), so that the speech recognition model can learn both the mapping from speech to text and the correspondence between audio frames and phonemes, which not only improves the convergence speed of the speech recognition model, but also improves the recognition accuracy of the speech recognition model.

[0062] The following is a detailed description of the training method of the speech recognition model according to the embodiment of the present disclosure.

[0063] As described above, in step S204, the first loss value may be determined according to the phoneme feature and the phoneme label of the first sample speech. In some possible implementations, the step S204 may include: smoothing the phoneme label of the first sample speech to obtain the smoothed phoneme label; and determining the first loss value according to the smoothed phoneme label and the phoneme feature.

[0064] When determining the first loss value, the phoneme label of the first sample speech can be smoothed first. For example, assuming that there are a total of five phonemes, "jin", "tian", "qi", "zhen", and "hao", and the phoneme label corresponding to a certain audio frame is the pronunciation phoneme "qi", the phoneme label is represented as a one-hot vector [0,0,1,0,0], and the phoneme label [0,0,1,0,0] can be smoothed according to the preset smoothing rules (here the example is: reduce 1 to 0.92, and increase the four 0s to 0.02 respectively), and the smoothed phoneme label [0.02, 0.02, 0.92, 0.02, 0.02] is obtained. After that, the first loss value can be determined according to the smoothed phoneme label and phoneme features through the cross entropy loss function.

[0065] It should be noted that those skilled in the art may determine the specific rules for the above-mentioned smoothing process according to actual conditions, and the present disclosure does not limit this.

[0066] In an embodiment of the present disclosure, when determining the first loss value, the phoneme labels of the first sample speech can be smoothed first, and then the first loss value can be determined based on the smoothed phoneme labels and phoneme features, so that the smoothed phoneme labels can be used when determining the first loss value to reduce the supervision intensity of the phoneme labels during the training process of the speech recognition model, thereby reducing the overfitting of the phoneme labels during the training process of the speech recognition model.

[0067] As mentioned above, in step S206, the speech recognition model may be trained according to the first loss value and the second loss value. In some possible implementations, step S206 may include: training the speech recognition model according to the first weight coefficient, the second weight coefficient, the first loss value, and the second loss value.

[0068] Among them, the first weight coefficient is the weight coefficient of the first loss value. The first weight coefficient can be used to indicate the supervision intensity of the phoneme label on the speech recognition model during the training process. The larger the first weight coefficient is, the greater the supervision intensity of the phoneme label on the speech recognition model during the training process. The smaller the first weight coefficient is, the smaller the supervision intensity of the phoneme label on the speech recognition model during the training process.

[0069] The second weight coefficient is a weight coefficient of the second loss value. The second weight coefficient can be used to indicate the supervision strength of the first text label on the speech recognition model during the training process. The larger the second weight coefficient, the greater the supervision strength of the first text label on the speech recognition model during the training process. The smaller the second weight coefficient, the smaller the supervision strength of the first text label on the speech recognition model during the training process.

[0070] The value range of the first weight coefficient and the second weight coefficient is (0-1), and the sum of the first weight coefficient and the second weight coefficient is 1. For example, the first weight coefficient is 0.3, and the second weight coefficient is 0.7. For another example, the first weight coefficient is 0.45, and the second weight coefficient is 0.55. Those skilled in the art can set the specific values ​​of the first weight coefficient and the second weight coefficient according to actual conditions, and the present disclosure does not limit this.

[0071] When training a speech recognition model, the total loss value of the speech recognition model can be determined based on the first weight coefficient, the second weight coefficient, the first loss value and the second loss value, and then the parameters of the speech recognition model can be adjusted based on the total loss value.

[0072] In some possible implementations, the total loss value of the speech recognition model can be calculated by the following formula (1):

[0073] L=λ×L1+ (1-λ) ×L2 (1)

[0074] In formula (1), L represents the total loss value of the speech recognition model, L1 represents the first loss value, L2 represents the second loss value, λ represents the first weight coefficient, and (1-λ) represents the second weight coefficient.

[0075] In the embodiments of the present disclosure, the speech recognition model can be trained according to the first weight coefficient, the second weight coefficient, the first loss value and the second loss value, so that the weights of the first loss value and the second loss value can be considered when training the speech recognition model, thereby improving the training accuracy of the speech recognition model and further improving the convergence speed of the speech recognition model.

[0076] In some possible implementations, training the speech recognition model may include multiple training rounds. When the number of training rounds is greater than or equal to a preset round threshold, the speech recognition model may be verified using a validation set. The round threshold may be, for example, 50, 100, etc. Those skilled in the art may determine the specific value of the round threshold according to actual conditions, and the present disclosure does not limit this.

[0077] The verification set includes a plurality of second sample voices and a second text label of each second sample voice. The second sample voices and their second text labels are obtained in a similar manner to the first sample voice and their first text label, which will not be described in detail here.

[0078] In some possible implementations, the speech recognition model training method of the embodiment of the present disclosure may also include: when the training round is greater than or equal to a preset round threshold, inputting a second sample speech into the speech recognition model for speech recognition to obtain a second text; determining the recognition error rate of the second text based on the second text label of the second sample speech; and when the recognition error rate is less than a preset threshold, obtaining a trained speech recognition model.

[0079] When the training round of the speech recognition model is greater than or equal to the preset round threshold, the second sample speech in the verification set can be input into the speech recognition model for speech recognition, that is, the second sample speech is input into the speech recognition model, and is processed by the phoneme feature extraction network, the time series feature extraction network and the decoder in sequence to obtain the second text.

[0080] After obtaining the second text, the recognition error rate of the second text can be determined according to the second text label of the second sample speech in the validation set. The recognition error rate may include word error rate (WER), etc. For example, the second text and the second text label of the second sample speech can be input into a trained error judgment model for processing (including judgment of text correctness, statistics, etc.) to obtain the recognition error rate of the second text.

[0081] Then, it can be determined whether the recognition error rate of the second text is less than a preset threshold. If the recognition error rate of the second text is less than the preset threshold, the trained speech recognition model is obtained. That is, if the recognition error rate of the second text is less than the preset threshold, it can be considered that the speech recognition model of the current round has been verified and the speech recognition model of the current round is used as the trained speech recognition model.

[0082] In an embodiment of the present disclosure, when the training round of the speech recognition model is greater than or equal to a preset round threshold, the second sample speech is input into the speech recognition model for speech recognition to obtain a second text, and the recognition error rate of the second text is determined based on the second text label of the second sample speech; when the recognition error rate is less than the preset threshold, a trained speech recognition model is obtained, so that when the training round is greater than or equal to the preset round threshold, the speech recognition model can be verified by the second sample speech, and when the recognition error rate is less than the preset threshold, the trained speech recognition model can be obtained.

[0083] In some possible implementation manners, the method for training a speech recognition model according to the embodiments of the present disclosure may further include: when the recognition error rate is greater than or equal to the preset threshold, determining the proportion of homophone errors in the text with recognition errors according to the second text label and the second text of the second sample speech; when the proportion of homophone errors is greater than or equal to the preset homophone error proportion threshold, reducing the first weight coefficient by a preset step length and increasing the second weight coefficient by the preset step length to obtain updated first and second weight coefficients;

[0084] The training of the speech recognition model according to the first weight coefficient, the second weight coefficient, the first loss value, and the second loss value includes: training the speech recognition model according to the updated first weight coefficient, second weight coefficient, the first loss value, and the second loss value.

[0085] When the recognition error rate of the second text is greater than or equal to the preset threshold, it can be considered that the verification of the speech recognition model in the current round fails. In this case, the proportion of homophone errors in the text with recognition errors can be determined according to the second text label and the second text of the second sample speech in the verification set. For example, after performing speech recognition on the second sample speech (whose second text label is "Today is Wednesday") through the speech recognition model, the obtained second text is "Jin Tian is Wednesday". There are two texts with recognition errors in this second text, where "今 (jīn)" is recognized as "金 (jīn)" and "天 (tiān)" is recognized as "添 (tiān)". Both of these two errors are homophone errors. Therefore, the proportion of homophone errors is 100%. After determining the proportion of homophone errors, the proportion of homophone errors can be compared with the preset homophone error proportion threshold (such as 50%, which can be set according to the actual situation).

[0086] When the proportion of homophone errors is greater than or equal to the homophone error proportion threshold, it can be considered that the phoneme modeling ability of the speech recognition model is too strong. And the reason for the overly strong phoneme modeling ability is the excessive supervision intensity of the phoneme labels. In order to reduce the supervision intensity of the phoneme labels, the first weight coefficient can be reduced by a preset step length, and the second weight coefficient can be increased by the preset step length to obtain updated first and second weight coefficients. Among them, the preset step length can be, for example, numerical values such as 0.01 and 0.05. Those skilled in the art can set the specific value of the preset step length according to the actual situation, and the present disclosure does not limit this.

[0087] Then, the speech recognition model can be trained for the next training round according to the updated first weight coefficient, the second weight coefficient, the first loss value, and the second loss value, so as to reduce the supervision intensity of the phoneme labels during the training process, thereby reducing the phoneme modeling ability of the speech recognition model. Among them, the next training round can refer to the first round of training that is restarted after updating the first weight coefficient, the second weight coefficient and clearing the previous training round. For example, no matter which round the current training round is, the next training round is the first round. The next training round can also be the training of the next training round of the current training round after updating the first weight coefficient and the second weight coefficient. For example, the current training round is the 10th round (using the first weight coefficient and the second weight coefficient before the update), and the next training round is the 11th round (using the first weight coefficient and the second weight coefficient after the update). The present disclosure does not limit the specific method for determining the next training round.

[0088] In an embodiment of the present disclosure, when the speech recognition model verification fails (i.e., the recognition error rate of the second text is greater than or equal to a preset threshold) and the proportion of homophone errors is greater than or equal to the threshold of the proportion of homophone errors, it is considered that the phoneme modeling capability of the speech recognition model is too strong, and the first weight coefficient can be reduced by a preset step size, and the second weight coefficient can be increased by a preset step size. Then, the speech recognition model is trained based on the updated first weight coefficient, the second weight coefficient, the first loss value and the second loss value, so that the supervision intensity of the phoneme labels can be reduced during the training process, thereby reducing the phoneme modeling capability of the speech recognition model.

[0089] In some possible implementations, the training method of the speech recognition model of the embodiment of the present disclosure may further include: when the homophone error ratio is less than the homophone error ratio threshold, increasing the first weight coefficient by the preset step length and decreasing the second weight coefficient by the preset step length to obtain updated first weight coefficient and second weight coefficient;

[0090] The training of the speech recognition model according to the first weight coefficient, the second weight coefficient, the first loss value and the second loss value includes: training the speech recognition model according to the updated first weight coefficient, the second weight coefficient, the first loss value and the second loss value.

[0091] When the speech recognition model fails to pass the verification and the homophone error ratio is less than the homophone error ratio threshold, it can be considered that the phoneme modeling ability of the speech recognition model is weak. The weak phoneme modeling ability is due to the fact that the supervision strength of the phoneme label is too low. In order to improve the supervision strength of the phoneme label, the first weight coefficient can be increased by a preset step size, and the second weight coefficient can be reduced by a preset step size to obtain the updated first weight coefficient and the second weight coefficient, and the updated first weight coefficient and the second weight coefficient can be obtained.

[0092] Then, the speech recognition model can be trained for the next training round based on the updated first weight coefficient, second weight coefficient, first loss value and second loss value, so as to improve the supervision strength of the phoneme labels during the training process, thereby improving the phoneme modeling capability of the speech recognition model.

[0093] In an embodiment of the present disclosure, when the speech recognition model verification fails and the proportion of homophone errors is less than the threshold of the proportion of homophone errors, it is considered that the phoneme modeling ability of the speech recognition model is weak, and the first weight coefficient can be increased by a preset step size, and the second weight coefficient can be reduced by a preset step size. The speech recognition model is trained based on the updated first weight coefficient, the second weight coefficient, the first loss value and the second loss value, so that the supervision strength of the phoneme labels can be improved during the training process, thereby improving the phoneme modeling ability of the speech recognition model.

[0094] The following will combine Figure 3 , Figure 4 The training process and verification process of the speech recognition model training method provided in the embodiment of the present disclosure are exemplarily described.

[0095] Figure 3 A schematic diagram of a training process of a speech recognition model training method provided in an embodiment of the present disclosure. Figure 3 The speech recognition model 300 includes an encoder 310 and a decoder 320. The encoder 310 includes a phoneme feature extraction network 311 and a time series feature extraction network 312.

[0096] According to the training method of the speech recognition model of the embodiment of the present disclosure, the process of training the speech recognition model 300 can be exemplified as follows:

[0097] Step 1: input the first sample speech 350 in the training set into the speech recognition model 300, and the phoneme feature extraction network 311 in the encoder 310 extracts the phoneme features of the first sample speech 350 to obtain the phoneme features 351 of the first sample speech;

[0098] Step 2: Input the phoneme feature 351 into the time series feature extraction network 312. The time series feature extraction network 312 performs attention processing on the phoneme feature 351 according to the time series information of the phoneme feature 351 to obtain the phoneme time series feature 352. The phoneme time series feature 352 is the output of the encoder 310.

[0099] Step 3: input the phoneme time sequence feature 352 into the decoder 320, and the decoder 320 decodes the phoneme time sequence feature to obtain a first text 353;

[0100] Step 4: Determine a first loss value 355 according to the phoneme feature 351 and the phoneme label 354 of the first sample speech through a cross entropy loss function;

[0101] Step 5, according to the first text 353 and the first text label 356 of the first sample speech, determine the second loss value 357 through the connectionist time classification loss function;

[0102] Step 6, determining a total loss value 360 ​​according to the first loss value 355, the second loss 357, the first weight coefficient 358 and the second weight coefficient 359;

[0103] Step seven, according to the total loss value 360, adjust the parameters of the speech recognition model 300.

[0104] Figure 4 A schematic diagram of the verification process of a speech recognition model training method provided in an embodiment of the present disclosure. Figure 4 According to the training method of the speech recognition model of the embodiment of the present disclosure, an example of the process of verifying the speech recognition model through the verification set is as follows:

[0105] Step S401, when the training round of the speech recognition model is greater than or equal to a preset round threshold, inputting a second sample speech in the verification set into the speech recognition model for speech recognition to obtain a second text;

[0106] Step S402, inputting the second text and the second text label of the second sample speech into the error judgment model for processing to obtain a recognition error rate;

[0107] Step S403, determining whether the recognition error rate is less than a preset threshold;

[0108] When the recognition error rate is less than the preset threshold, step S404 is executed to obtain a trained speech recognition model;

[0109] When the recognition error rate is greater than or equal to the preset threshold, step S405 is executed to determine the proportion of homophone errors in the text with recognition errors;

[0110] Step S406, determining whether the homophone error ratio is greater than or equal to a preset homophone error ratio threshold;

[0111] When the homophone error ratio is greater than or equal to a preset homophone error ratio threshold, executing steps S407 and S408: step S407, reducing the first weight coefficient by a preset step length, and increasing the second weight coefficient by a preset step length, to obtain updated first weight coefficients and second weight coefficients; step S408, training a speech recognition model according to the updated first weight coefficient, the second weight coefficient, the first loss value, and the second loss value;

[0112] When the error ratio of homophones is less than the preset threshold value of the error ratio of homophones, execute steps S409 and S410: Step S409, increase the first weight coefficient by a preset step size, and decrease the second weight coefficient by a preset step size to obtain updated first weight coefficient and second weight coefficient; Step S410, train the speech recognition model according to the updated first weight coefficient, second weight coefficient, first loss value and second loss value.

[0113] The training method of the speech recognition model of the embodiment of the present disclosure adds auxiliary supervision of phoneme labels on the basis of end-to-end supervised training of the speech recognition model using the first text label, so that the speech recognition model can learn the alignment information of the audio frame and the phoneme. Therefore, the speech recognition model trained by the training method of the speech recognition model of the embodiment of the present disclosure has the ability to characterize the correspondence between the audio frame and the phoneme, thereby not only improving the convergence speed of the speech recognition model, but also improving the recognition accuracy of the speech recognition model.

[0114] The embodiment of the present disclosure also provides a speech recognition method. The speech recognition method comprises: performing speech recognition on the speech to be recognized through a speech recognition model to obtain a target text. The speech recognition model is trained according to the training method described in the embodiment of the present disclosure. During speech recognition, the speech recognition model trained according to the training method described in the embodiment of the present disclosure can improve the accuracy of speech recognition.

[0115] The speech recognition method can be performed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory.

[0116] The speech recognition method in the embodiments of the present disclosure can be applied to scenarios such as intelligent customer service, voice assistant, speech recognition input method, voice search, smart home, smart driving, and smart robots. It should be noted that the present disclosure does not limit the specific application scenarios of the speech recognition method.

[0117] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0118] In addition, the present disclosure also provides a training device, an electronic device, and a computer-readable storage medium for a speech recognition model, all of which can be used to implement any training method for a speech recognition model provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0119] The present disclosure also provides a speech recognition device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any speech recognition method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0120] Figure 5 A block diagram of a speech recognition model training device provided in an embodiment of the present disclosure.

[0121] Reference Figure 5 , the present disclosure provides a training device for a speech recognition model, the device comprising:

[0122] A first extraction module 51 is used to extract phoneme features from the first sample speech to obtain phoneme features of the first sample speech, where the phoneme features are used to represent pronunciation phonemes corresponding to each audio frame of the first sample speech;

[0123] A second extraction module 52 is used to perform attention processing on the phoneme feature according to the time sequence information of the phoneme feature to obtain a phoneme time sequence feature;

[0124] A decoding module 53, used for decoding the phoneme time sequence feature to obtain a first text;

[0125] A first determination module 54, configured to determine a first loss value according to the phoneme feature and the phoneme label of the first sample speech;

[0126] A second determination module 55, configured to determine a second loss value according to the first text and the first text label of the first sample speech;

[0127] The training module 56 is used to train the speech recognition model according to the first loss value and the second loss value.

[0128] In some possible implementations, training the speech recognition model includes multiple training rounds, and the device also includes: a recognition module, used to input a second sample speech into the speech recognition model for speech recognition to obtain a second text when the training round is greater than or equal to a preset round threshold; a recognition error rate determination module, used to determine the recognition error rate of the second text based on the second text label of the second sample speech; and a third determination module, used to obtain the trained speech recognition model when the recognition error rate is less than a preset threshold.

[0129] In some possible implementations, the training module 56 includes: a training submodule, used to train the speech recognition model based on a first weight coefficient, a second weight coefficient, the first loss value and the second loss value, wherein the first weight coefficient is the weight coefficient of the first loss value, and the second weight coefficient is the weight coefficient of the second loss value.

[0130] In some possible implementations, the device further includes: a homophone error ratio module, used to determine the homophone error ratio in the text with recognition errors based on the second text label of the second sample speech and the second text when the recognition error rate is greater than or equal to the preset threshold; a first adjustment module, used to reduce the first weight coefficient by a preset step length and increase the second weight coefficient by the preset step length when the homophone error ratio is greater than or equal to the preset homophone error ratio threshold, to obtain updated first weight coefficients and second weight coefficients,

[0131] The training submodule is used to train the speech recognition model according to the updated first weight coefficient, the second weight coefficient, the first loss value and the second loss value.

[0132] In some possible implementations, the device further includes: a second adjustment module, configured to increase the first weight coefficient by the preset step length and decrease the second weight coefficient by the preset step length when the homophone error ratio is less than the homophone error ratio threshold, to obtain updated first weight coefficient and second weight coefficient;

[0133] The training submodule is used to train the speech recognition model according to the updated first weight coefficient, the second weight coefficient, the first loss value and the second loss value.

[0134] In some possible implementations, the first determination module 54 includes: a smoothing processing submodule, used to smooth the phoneme label of the first sample speech to obtain the smoothed phoneme label; a loss determination submodule, used to determine the first loss value based on the smoothed phoneme label and the phoneme feature.

[0135] An embodiment of the present disclosure also provides a speech recognition device, which includes: a speech recognition module, which is used to perform speech recognition on a speech to be recognized through a speech recognition model to obtain a target text; wherein the speech recognition model is trained according to the training method described in the embodiment of the present disclosure.

[0136] The training device of the speech recognition module and each module in the speech recognition device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0137] Figure 6 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0138] Reference Figure 6 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702, and one or more I / O interfaces 703, connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by at least one processor 701, and the one or more computer programs are executed by at least one processor 701 so that the at least one processor 701 can execute the above-mentioned speech recognition model training method or speech recognition method.

[0139] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0140] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned speech recognition model training method or speech recognition method when executed by a processor / processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0141] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned speech recognition model training method or speech recognition method.

[0142] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0143] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0144] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0145] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0146] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0147] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0148] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0149] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0150] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0151] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A method for training a speech recognition model, characterized in that: The method comprises: Extracting phoneme features from the first sample speech to obtain phoneme features of the first sample speech, wherein the phoneme features are used to represent pronunciation phonemes corresponding to each audio frame of the first sample speech; According to the time sequence information of the phoneme feature, attention processing is performed on the phoneme feature to obtain a phoneme time sequence feature; Decoding the phoneme time sequence feature to obtain a first text; Determining a first loss value according to the phoneme feature and the phoneme label of the first sample speech; Determining a second loss value according to the first text and the first text label of the first sample speech; The speech recognition model is trained according to the first loss value and the second loss value.

2. The method according to claim 1, characterized in that Training the speech recognition model includes multiple training rounds, and the method further includes: When the training round is greater than or equal to a preset round threshold, inputting a second sample speech into the speech recognition model for speech recognition to obtain a second text; Determining a recognition error rate of the second text according to a second text label of the second sample speech; When the recognition error rate is less than a preset threshold, a trained speech recognition model is obtained.

3. The method according to claim 2, characterized in that The step of training the speech recognition model according to the first loss value and the second loss value comprises: The speech recognition model is trained according to a first weight coefficient, a second weight coefficient, the first loss value and the second loss value, wherein the first weight coefficient is the weight coefficient of the first loss value and the second weight coefficient is the weight coefficient of the second loss value.

4. The method according to claim 3, characterized in that The method further comprises: When the recognition error rate is greater than or equal to the preset threshold, determining the proportion of homophone errors in the text with recognition errors according to the second text label of the second sample speech and the second text; When the homophone error ratio is greater than or equal to a preset homophone error ratio threshold, the first weight coefficient is reduced by a preset step length, and the second weight coefficient is increased by the preset step length to obtain updated first weight coefficients and second weight coefficients, The step of training the speech recognition model according to the first weight coefficient, the second weight coefficient, the first loss value, and the second loss value includes: The speech recognition model is trained according to the updated first weight coefficient, the second weight coefficient, the first loss value and the second loss value.

5. The method according to claim 4, characterized in that The method further comprises: When the homophone error ratio is less than the homophone error ratio threshold, increasing the first weight coefficient by the preset step length and decreasing the second weight coefficient by the preset step length to obtain updated first weight coefficient and second weight coefficient; The step of training the speech recognition model according to the first weight coefficient, the second weight coefficient, the first loss value, and the second loss value includes: The speech recognition model is trained according to the updated first weight coefficient, the second weight coefficient, the first loss value and the second loss value.

6. The method according to any one of claims 1 to 5, characterized in that The determining a first loss value according to the phoneme feature and the phoneme label of the first sample speech includes: Smoothing the phoneme labels of the first sample speech to obtain smoothed phoneme labels; A first loss value is determined according to the smoothed phoneme label and the phoneme feature.

7. A speech recognition method, characterized in that: include: Through the speech recognition model, the speech to be recognized is recognized to obtain the target text; Wherein, the speech recognition model is trained according to the method according to any one of claims 1-6.

8. A training device for a speech recognition model, characterized in that: The device comprises: A first extraction module is used to extract phoneme features from the first sample speech to obtain phoneme features of the first sample speech, where the phoneme features are used to represent pronunciation phonemes corresponding to each audio frame of the first sample speech; A second extraction module is used to perform attention processing on the phoneme feature according to the time sequence information of the phoneme feature to obtain a phoneme time sequence feature; A decoding module, used for decoding the phoneme time sequence feature to obtain a first text; A first determination module, configured to determine a first loss value according to the phoneme feature and the phoneme label of the first sample speech; A second determination module, configured to determine a second loss value according to the first text and a first text label of the first sample speech; A training module is used to train the speech recognition model according to the first loss value and the second loss value.

9. A speech recognition device, characterized in that: The device comprises: A speech recognition module is used to perform speech recognition on the speech to be recognized through a speech recognition model to obtain a target text; Wherein, the speech recognition model is trained according to the method according to any one of claims 1-6.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor so that the at least one processor can execute the training method of the speech recognition model as described in any one of claims 1-6, or execute the speech recognition method as described in claim 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the training method of the speech recognition model as described in any one of claims 1 to 6, or executes the speech recognition method as described in claim 7.

12. A computer program product, characterized in that It includes a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the training method of the speech recognition model as described in any one of claims 1 to 6, or executes the speech recognition method as described in claim 7.

Images