Speech recognition model training method, speech recognition method and apparatus

Abstract

The application relates to a speech recognition model training method, a speech recognition method, a device, computer equipment, a storage medium and a computer program product. The application relates to artificial intelligence technology. The method comprises the following steps: acquiring a general speech recognition model as an initial speech recognition model; based on a first speech training data set corresponding to an accent scene and a third speech training data set corresponding to a general scene, performing model training on the initial speech recognition model to obtain an intermediate speech recognition model; inputting a second speech training data set corresponding to the accent scene and a fourth speech training data set corresponding to the general scene into the intermediate speech recognition model to calculate a target loss; updating model parameters of the initial speech recognition model based on the target loss to obtain an updated speech recognition model; and performing cyclic iteration on the updated speech recognition model as the initial speech recognition model until a convergence condition is met, so that a target speech recognition model is obtained. The method can improve the accuracy of speech recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a speech recognition model training method, speech recognition method, device, computer equipment, storage medium and computer program product. Background Technology

[0002] With the development of artificial intelligence technology, speech recognition technology has emerged, which is used to convert speech into text, bringing great convenience to people's daily lives.

[0003] In traditional techniques, to train a speech recognition model applicable to general speech data and accent data, the initial speech recognition model is typically adjusted directly based on a training set containing limited general and accent-specific speech data. However, directly adjusting the initial speech recognition model based on the training set only allows the model to perform speech recognition on speech data similar to the training set; it fails to enable the model to effectively learn knowledge of speech recognition in different scenarios, resulting in low recognition accuracy for the target speech recognition model. Summary of the Invention

[0004] Therefore, it is necessary to provide a speech recognition model training method, speech recognition method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of speech recognition in order to address the above-mentioned technical problems.

[0005] This application provides a method for training a speech recognition model. The method includes:

[0006] Obtain a general speech recognition model corresponding to a common scenario, and use the general speech recognition model as the initial speech recognition model;

[0007] Obtain the first and second speech training datasets corresponding to the accent scenario, and obtain the third and fourth speech training datasets corresponding to the general scenario; each speech training dataset includes training speech and corresponding label text, wherein the label text is obtained by converting the training speech into text.

[0008] Based on the first speech training dataset and the third speech training dataset, the initial speech recognition model is trained to obtain an intermediate speech recognition model.

[0009] The second speech training dataset and the fourth speech training dataset are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text corresponding to the training speech and the label text, the target loss is obtained.

[0010] The model parameters of the initial speech recognition model are updated based on the target loss to obtain an updated speech recognition model. The updated speech recognition model is then used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and obtaining the third and fourth speech training datasets corresponding to the general scenario are executed until the convergence condition is met, thus obtaining the target speech recognition model. The target speech recognition model is used to recognize speech to obtain the corresponding text.

[0011] This application also provides a speech recognition model training device. The device includes:

[0012] The model acquisition module is used to acquire a general speech recognition model corresponding to a general scenario, and use the general speech recognition model as the initial speech recognition model.

[0013] The training dataset acquisition module is used to acquire the first and second speech training datasets corresponding to the accent scenario, and the third and fourth speech training datasets corresponding to the general scenario. Each speech training dataset includes training speech and the label text corresponding to the training speech, wherein the label text is obtained by converting the training speech into text.

[0014] The first model training module is used to train the initial speech recognition model based on the first speech training dataset and the third speech training dataset to obtain an intermediate speech recognition model.

[0015] The second model training module is used to input the second and fourth speech training datasets into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text and the labeled text corresponding to the training speech, a target loss is obtained. The model parameters of the initial speech recognition model are updated based on the target loss to obtain an updated speech recognition model. The updated speech recognition model is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario and obtaining the third and fourth speech training datasets corresponding to the general scenario are executed until the convergence condition is met to obtain the target speech recognition model. The target speech recognition model is used to recognize speech to obtain the corresponding text.

[0016] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps described in the speech recognition model training method above.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps described in the speech recognition model training method above.

[0018] A computer program product includes a computer program that, when executed by a processor, implements the steps described in the speech recognition model training method above.

[0019] The aforementioned speech recognition model training method, apparatus, computer equipment, storage medium, and computer program product acquire a general speech recognition model corresponding to a common scenario, using this model as the initial speech recognition model; acquire a first and second speech training dataset corresponding to an accent scenario, and acquire a third and fourth speech training dataset corresponding to a common scenario; each speech training dataset includes training speech and corresponding label text, the label text being obtained by converting the training speech into text; based on the first and third speech training datasets, the initial speech recognition model is trained to obtain an intermediate speech recognition model; the second and fourth speech training datasets are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech; based on the difference between the predicted text corresponding to the training speech and the label text, a target loss is obtained; the model parameters of the initial speech recognition model are updated based on the target loss to obtain an updated speech recognition model; the updated speech recognition model is used as the initial speech recognition model; the steps of acquiring the first and second speech training datasets corresponding to accent scenarios, and acquiring the third and fourth speech training datasets corresponding to a common scenario are repeated until the convergence condition is met, resulting in a target speech recognition model; the target speech recognition model is used to recognize speech to obtain the corresponding text. In this way, a general speech recognition model corresponding to common scenarios is used as the initial speech recognition model. By training the general speech recognition model, the performance of the model for accent scenarios is improved without deteriorating its performance for common scenarios, thus achieving accent domain extension. A meta-learning approach is used for model training. First, based on the first and third speech training datasets, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The intermediate speech recognition model learns speech recognition knowledge applicable to the first and third speech training datasets. Then, the second and fourth speech training datasets are input into the intermediate speech recognition model to calculate the target loss. The target loss reflects the effectiveness of the knowledge learned by the intermediate speech recognition model in applying to the second and fourth speech training datasets, and whether the knowledge learned by the intermediate speech recognition model is applicable to the second and fourth speech training datasets. Based on the target loss, the model parameters of the initial speech recognition model are updated to obtain an updated speech recognition model. This allows the updated speech recognition model to learn the variation rules between different speech scenarios and to learn the general knowledge between different speech recognition scenarios, so that the knowledge learned by the updated speech recognition model can be applied to new speech training datasets.Through multiple iterations, the final trained target speech recognition model can learn speech recognition methods for different scenarios. The target speech recognition model is applicable to both general scenarios and various accent scenarios. It has high speech recognition accuracy for general scenarios and also high speech recognition accuracy for various accent scenarios.

[0020] This application provides a speech recognition method. The method includes:

[0021] Acquire the speech to be recognized;

[0022] The speech to be recognized is input into the target speech recognition model to obtain the recognized text corresponding to the speech to be recognized;

[0023] The training process of the target speech recognition model includes the following steps:

[0024] A general speech recognition model corresponding to a common scenario is obtained and used as the initial speech recognition model. A first and second speech training dataset corresponding to an accent scenario are obtained, as well as a third and fourth speech training dataset corresponding to the common scenario. Each speech training dataset includes training speech and corresponding label text, where the label text is obtained by converting the training speech into text. Based on the first and third speech training datasets, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The second and fourth speech training datasets are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text and the label text, a target loss is obtained. The model parameters of the initial speech recognition model are updated based on the target loss to obtain an updated speech recognition model. The updated speech recognition model is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario and the third and fourth speech training datasets corresponding to the common scenario are repeated until the convergence condition is met, resulting in the target speech recognition model.

[0025] This application also provides a speech recognition device. The device includes:

[0026] The speech acquisition module is used to acquire the speech to be recognized;

[0027] The speech recognition module is used to input the speech to be recognized into the target speech recognition model to obtain the recognized text corresponding to the speech to be recognized.

[0028] The training process of the target speech recognition model includes the following steps:

[0029] A general speech recognition model corresponding to a common scenario is obtained and used as the initial speech recognition model. A first and second speech training dataset corresponding to an accent scenario are obtained, as well as a third and fourth speech training dataset corresponding to the common scenario. Each speech training dataset includes training speech and corresponding label text, where the label text is obtained by converting the training speech into text. Based on the first and third speech training datasets, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The second and fourth speech training datasets are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text and the label text, a target loss is obtained. The model parameters of the initial speech recognition model are updated based on the target loss to obtain an updated speech recognition model. The updated speech recognition model is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario and the third and fourth speech training datasets corresponding to the common scenario are repeated until the convergence condition is met, resulting in the target speech recognition model.

[0030] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps described in the speech recognition method above.

[0031] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps described in the speech recognition method above.

[0032] A computer program product includes a computer program that, when executed by a processor, implements the steps described in the speech recognition method above.

[0033] The aforementioned speech recognition methods, devices, computer equipment, storage media, and computer program products acquire the speech to be recognized, input the speech into a target speech recognition model, and obtain the recognized text corresponding to the speech. The target speech recognition model has high speech recognition accuracy for both general and accented scenarios, and speech recognition based on the target speech recognition model can effectively improve speech recognition accuracy. By using a general speech recognition model corresponding to general scenarios as the initial speech recognition model, and training the general speech recognition model, the performance of the model for accented scenarios can be improved without deteriorating its performance for general scenarios, thus achieving accent domain extension. A meta-learning approach is employed for model training. First, an initial speech recognition model is trained using a first and a third speech training dataset to obtain an intermediate speech recognition model. This intermediate model learns speech recognition knowledge applicable to the first and third training datasets. Then, a second and a fourth training dataset are input into the intermediate model to calculate the target loss. The target loss reflects the effectiveness of the intermediate model's learned knowledge applied to the second and fourth training datasets. Based on the target loss, the model parameters of the initial speech recognition model are updated to obtain an updated speech recognition model. This updated model learns the variation patterns of speech in different scenarios and acquires general knowledge applicable to different speech recognition scenarios, enabling it to quickly learn relevant knowledge when faced with new speech training datasets. Through multiple iterations, the final trained target speech recognition model learns methods for speech recognition in different scenarios. This target speech recognition model is applicable to both general scenarios and various accent scenarios, exhibiting high accuracy in general scenarios and also demonstrating high accuracy in various accent scenarios. Attached Figure Description

[0034] Figure 1 This is a diagram illustrating the speech recognition model training method and the application environment of the speech recognition method in one embodiment.

[0035] Figure 2 This is a flowchart illustrating a speech recognition model training method in one embodiment;

[0036] Figure 3 This is a flowchart illustrating the speech recognition model training method in another embodiment;

[0037] Figure 4 This is a flowchart illustrating the speech recognition model training method in another embodiment;

[0038] Figure 5 This is a flowchart illustrating the speech recognition model training method in another embodiment;

[0039] Figure 6 This is a schematic diagram of the structure of a speech recognition model in one embodiment;

[0040] Figure 7 This is a flowchart illustrating a speech recognition method in one embodiment;

[0041] Figure 8 This is a structural block diagram of a speech recognition model training device in one embodiment;

[0042] Figure 9 This is a structural block diagram of a speech recognition device in one embodiment;

[0043] Figure 10 This is an internal structural diagram of a computer device in one embodiment;

[0044] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] The solutions provided in this application involve artificial intelligence technologies such as speech technology, machine learning, and natural language processing, which are specifically illustrated through the following embodiments:

[0047] The speech recognition model training method and speech recognition method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other servers. Terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server, a server cluster consisting of multiple servers, or a cloud server.

[0048] Both the terminal and the server can be used independently to execute the speech recognition model training method and the speech recognition method provided in the embodiments of this application.

[0049] For example, the server obtains a general speech recognition model corresponding to a common scenario and uses it as the initial speech recognition model. The server obtains a first and a second speech training dataset corresponding to the accent scenario, and a third and a fourth speech training dataset corresponding to the common scenario. Each speech training dataset includes training speech and corresponding label text, where the label text is obtained by converting the training speech into text. Based on the first and third speech training datasets, the server trains the initial speech recognition model to obtain an intermediate speech recognition model. The second and fourth speech training datasets are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text and the label text, the target loss is obtained. The server updates the model parameters of the initial speech recognition model based on the target loss to obtain an updated speech recognition model. This updated speech recognition model is used as the initial speech recognition model. The server then returns to the steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and the third and fourth speech training datasets corresponding to the common scenario, until the convergence condition is met, resulting in the target speech recognition model.

[0050] Subsequently, the server can acquire the speech to be recognized, input the speech to be recognized into the target speech recognition model, and obtain the recognized text corresponding to the speech to be recognized.

[0051] The terminal and server can also work together to execute the speech recognition model training method and speech recognition method provided in the embodiments of this application.

[0052] For example, the server obtains a general speech recognition model corresponding to a common scenario and uses it as the initial speech recognition model. The server then obtains a first and a second speech training dataset corresponding to the accent scenario from the terminal, as well as a third and a fourth speech training dataset corresponding to the common scenario. Based on the first and third speech training datasets, the server trains the initial speech recognition model to obtain an intermediate speech recognition model. The second and fourth speech training datasets are then input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text and the labeled text, the target loss is obtained. The server updates the model parameters of the initial speech recognition model based on the target loss to obtain an updated speech recognition model. This updated model is used as the initial speech recognition model. The server then returns to the steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and the third and fourth speech training datasets corresponding to the common scenario, until the convergence condition is met, resulting in the target speech recognition model.

[0053] Subsequently, the server can send the target speech recognition model to the terminal, which then inputs the speech to be recognized into the target speech recognition model to obtain the recognized text. The terminal can then display the recognized text.

[0054] In one embodiment, such as Figure 2 As shown, a method for training a speech recognition model is provided. Taking the application of this method to a computer device as an example, the computer device can be a terminal or a server. The method can be executed independently by the terminal or server, or it can be implemented through interaction between the terminal and the server. (Reference) Figure 2 The speech recognition model training method includes the following steps:

[0055] Step S202: Obtain the general speech recognition model corresponding to the general scenario, and use the general speech recognition model as the initial speech recognition model.

[0056] In this context, "universal scenario" refers to a speech context that is universally applicable across multiple regions. Speech in a universal scenario is suitable for multiple regions and is naturally understandable to users in those regions. Universal scenario speech is typically delivered in a standard, universal language. For example, in Mandarin Chinese, a universal scenario refers to a Mandarin-speaking scenario.

[0057] Compared to general speech contexts, accent-based speech contexts refer to speech scenarios specific to a particular region or group. Speech in accent-based contexts typically carries the regional characteristics of that area and is usually easier for users in that specific region to understand. There are two main types of accents: one is speech with personal, regional linguistic features, and the other is an accent that naturally forms due to the current inability of humans to consciously and precisely control their vocal organs. Taking Mandarin Chinese as an example, accents arise from variations in standard Mandarin, such as changes in initials, finals, and tones, as well as variations in intonation and rhythm. Accent-based speech contexts can correspond to various regional accents, such as Sichuan accent, Shandong accent, and Guangdong accent.

[0058] A general-purpose speech recognition model refers to a speech recognition model applicable to general scenarios. Speech recognition models are used to convert speech into corresponding text. General-purpose speech recognition models are primarily used to convert speech belonging to general scenarios into corresponding text; that is, they have a certain level of accuracy when recognizing speech belonging to general scenarios. For example, a general-purpose speech recognition model has an accuracy greater than a preset threshold when recognizing speech belonging to general scenarios. It can be understood that a general-purpose speech recognition model can also be applied to accented scenarios to some extent, but its accuracy in general scenarios is higher than its accuracy in accented scenarios. A general-purpose speech recognition model can be a pre-trained model.

[0059] Specifically, the computer device can obtain a general speech recognition model corresponding to a general scenario locally or from other devices. Based on the general speech recognition model, a series of model training processes are performed on the general speech recognition model to finally obtain a target speech recognition model with high speech recognition accuracy in both general scenarios and accent scenarios.

[0060] Step S204: Obtain the first and second speech training datasets corresponding to the accent scenario, and obtain the third and fourth speech training datasets corresponding to the general scenario; each speech training dataset includes training speech and the label text corresponding to the training speech, and the label text is obtained by converting the training speech into text.

[0061] The speech training dataset corresponding to an accent scenario refers to a training set consisting of speech data from that accent scenario and corresponding labeled text. The speech training dataset corresponding to an accent scenario can include speech data from at least one accent scenario; for example, it can include speech data for accents A1, A2, and A3. The first and second speech training datasets corresponding to the accent scenarios are different datasets, containing different speech data.

[0062] The general scenario-specific speech training dataset refers to a training set consisting of speech data from the general scenario and corresponding labeled text. The third and fourth speech training datasets for the general scenario are different datasets, each containing different speech data.

[0063] Each speech training dataset is used for model training, and the speech data in each dataset is called training speech. Each training speech has a corresponding labeled text, which is obtained by converting the training speech into text. The labeled text is used to describe the semantic content expressed by the training speech in text form. The labeled text can be considered as the real text corresponding to the training speech, and is used as training labels and supervision signals during model training.

[0064] Step S206: Based on the first speech training dataset and the third speech training dataset, train the initial speech recognition model to obtain an intermediate speech recognition model.

[0065] Specifically, in the process of training the target speech recognition model based on the general speech recognition model, the computer device can first use the general speech recognition model as the initial speech recognition model, and select a portion of general scenario training data and accent scenario training data as the training set to adjust the model parameters of the initial speech recognition model. That is, the initial speech recognition model is trained based on the first speech training dataset and the third speech training dataset.

[0066] In one embodiment, the first speech training dataset and the third speech training dataset are input into the initial speech recognition model. After data processing by the model, the predicted text corresponding to each training speech in the first speech training dataset and the third speech training dataset is obtained. The training loss is generated based on the difference between the label text and the predicted text corresponding to the same training speech. The model parameters of the initial speech recognition model are updated based on the training loss to obtain the intermediate speech recognition model.

[0067] The initial speech recognition model can be trained using a combination of unsupervised and supervised training methods.

[0068] Step S208: Input the second speech training dataset and the fourth speech training dataset into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text corresponding to the training speech and the label text, obtain the target loss.

[0069] In this context, the predicted text refers to the text output by the model after processing the input data, representing the model's prediction result. The target loss characterizes the difference between the predicted text and the labeled text corresponding to the training speech in the second and fourth speech training datasets. The smaller the target loss, the closer the predicted text and the corresponding labeled text are, indicating a more accurate prediction and a more accurate prediction result from the model.

[0070] Specifically, after training an initial speech recognition model using the first and third speech training datasets to obtain an intermediate speech recognition model, the computer inputs the second and fourth speech training datasets into the intermediate speech recognition model. Through data processing, the model obtains the predicted text corresponding to each training speech in the second and fourth training datasets. A target loss is generated based on the difference between the label text and the predicted text corresponding to the same training speech. The target loss reflects the effectiveness of applying the knowledge learned by the intermediate speech recognition model to the second and fourth speech training datasets.

[0071] It is understandable that the target loss can be calculated using a custom formula or algorithm, or using various loss functions. The calculation process for the training loss is similar to that for the target loss.

[0072] In one embodiment, the speech recognition model can be a real-time speech recognition model, which simultaneously acquires speech segments and identifies the corresponding text segments during speech acquisition. The predicted text output by the model can then include the predicted text segments corresponding to each individual speech segment. Alternatively, the speech recognition model can be a non-real-time speech recognition model, which performs overall recognition of the complete speech to obtain the corresponding overall text. The predicted text output by this model can then include the overall text obtained from the overall recognition of the complete speech. Of course, the speech recognition model can also be a combination of real-time and non-real-time methods. In this case, the predicted text output by the model can include the predicted text segments corresponding to each individual speech segment as well as the overall text.

[0073] If the predicted text includes the predicted text segments corresponding to each individual speech segment, the target loss includes the loss based on the difference between the labeled text segment and the predicted text segment corresponding to the same speech segment. If the predicted text includes the entire text, the target loss includes the loss based on the difference between the labeled text segment and the entire text corresponding to the same training speech segment.

[0074] In one embodiment, the training speech is input into the speech recognition model. First, feature encoding is performed to obtain encoded features, and then feature decoding is performed on the encoded features to obtain the predicted text. During feature decoding, the labeled text corresponding to the training speech can also be further referenced.

[0075] Step S210: Update the model parameters of the initial speech recognition model based on the target loss to obtain the updated speech recognition model. Use the updated speech recognition model as the initial speech recognition model. Return to the steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and obtaining the third and fourth speech training datasets corresponding to the general scenario, until the convergence condition is met to obtain the target speech recognition model. The target speech recognition model is used to recognize speech to obtain the corresponding text.

[0076] The target speech recognition model refers to the trained speech recognition model. Convergence criteria determine whether the model has reached convergence. Convergence criteria include, but are not limited to, at least one of the following: the target loss is less than a preset loss value, the number of model iterations is greater than a preset number of iterations, or the rate of change of the target loss is less than a preset rate of change. The smaller the target loss, the more the knowledge learned by the model from the first and third speech training datasets is applicable to the second and fourth speech training datasets. In other words, the model gradually learns methods for speech recognition in different scenarios and the patterns of change between different speech scenarios, thus enabling the model to maintain high performance even when facing new datasets.

[0077] Specifically, after calculating the target loss, the computer device performs backpropagation based on the target loss to update the model parameters of the initial speech recognition model, obtaining an intermediate speech recognition model. Then, the updated speech recognition model is used as the initial speech recognition model. The process of obtaining the first and second speech training datasets corresponding to accent scenarios, and the third and fourth speech training datasets corresponding to general scenarios, is executed for iterative model training until the convergence condition is met, ultimately training the target speech recognition model. The target speech recognition model exhibits high recognition accuracy in both general and accent scenarios.

[0078] After training to obtain the target speech recognition model, the computer device can convert any speech in a general scenario or accent scenario into corresponding text based on the target speech recognition model.

[0079] In the first round of model iteration, a general speech recognition model is used as the initial speech recognition model. Based on the first and third speech training datasets, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The second and fourth speech training datasets are then input into the intermediate speech recognition model to calculate the target loss. The model parameters of the initial speech recognition model are updated based on the target loss to obtain an updated speech recognition model. In the second round of model iteration, the updated speech recognition model from the previous round is used as the initial speech recognition model. The first and second speech training datasets corresponding to accent scenarios are re-acquired, as are the third and fourth speech training datasets corresponding to general scenarios. Based on the new first and third speech training datasets, a new initial speech recognition model is trained to obtain a new intermediate speech recognition model. The new second and fourth speech training datasets are then input into the new intermediate speech recognition model to calculate a new target loss. The model parameters of the new initial speech recognition model are updated based on the new target loss to obtain a new updated speech recognition model. In the third round of model iteration, the updated speech recognition model from the previous round is used as the initial speech recognition model, and the data processing process of each single round of model iteration is repeated to obtain a new updated speech recognition model. This process continues until the convergence condition is met, resulting in the target speech recognition model. For example, if the target loss is less than a preset threshold in a certain round of model iteration, the initial speech recognition model used in that round of model iteration is taken as the target speech recognition model; if the preset number of iterations is twenty, the updated speech recognition model obtained in the twenty-first round of model iteration is taken as the initial speech recognition model.

[0080] It's understandable that the training sets used in different rounds of model iteration are not entirely the same. For example, the first speech training dataset used in the first round of model iteration is different from the first speech training dataset used in the second round, and the third speech training dataset used in the first round is different from the third speech training dataset used in the second round. The training sets used in different rounds of model iteration can be completely different, or they can contain repeated training speech.

[0081] In one embodiment, gradient descent can be used to update model parameters. Meta-learning enables the model to learn how to learn. Meta-learning learns classification methods by learning from some categories and observing whether the learned parameters are effective for other categories, and it learns prediction methods by learning from some data and observing whether the learned parameters are effective for other data. Assuming the current parameters of the model are W0, a portion of general data plus accent data (i.e., the first and third speech training datasets) is selected as the training set, and another portion of general data plus accent data (the second and fourth speech training datasets) is selected as the support set. The data in the training set are processed through the model to calculate the training loss, and the gradient G0 is calculated based on the training loss. The model parameters W0 are updated to Wm based on the gradient G0. The data in the support set are processed through the model Wm to calculate the target loss, and the gradient G is calculated based on the target loss, where G is the gradient of the learning method. The model parameters are then updated to W1 based on the model W0. The W1 parameter is the initial parameter W0 of the model for the next iteration. The model is iterated in this way until the convergence condition is met, and the target parameters of the model are obtained, thus obtaining the final target speech recognition model.

[0082] In one embodiment, for the same language, there are typically corresponding general scenarios and accent scenarios. For example, Chinese has corresponding general scenarios and accent scenarios, and English has corresponding general scenarios and accent scenarios. For different languages, corresponding target speech recognition models can be trained separately. For example, based on the training dataset of the general scenarios and accent scenarios corresponding to Chinese, the general speech recognition model for the general scenarios corresponding to Chinese is trained using the training method of this application to obtain the target speech recognition model for Chinese.

[0083] In one embodiment, the training datasets corresponding to the general scenario and the accent scenario can also be language-mixed datasets. The training speech in a language-mixed dataset includes at least two languages, but the dominant language in the training speech is the same. For example, the training speech in the training dataset corresponding to the general scenario and the accent scenario of Chinese can be mixed Chinese-English speech, which refers to speech containing both Chinese and English. However, the dominant language in the mixed Chinese-English speech corresponding to the general scenario and the accent scenario of Chinese is Chinese; that is, most of the content in the mixed Chinese-English speech corresponding to the general scenario and the accent scenario of Chinese is spoken in Chinese.

[0084] In the above speech recognition model training method, a general speech recognition model corresponding to a common scenario is obtained and used as the initial speech recognition model; a first speech training dataset and a second speech training dataset corresponding to an accent scenario are obtained, as well as a third and fourth speech training dataset corresponding to a common scenario; each speech training dataset includes training speech and corresponding label text, where the label text is obtained by converting the training speech into text; based on the first and third speech training datasets, the initial speech recognition model is trained to obtain an intermediate speech recognition model; the second and fourth speech training datasets are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech; based on the difference between the predicted text corresponding to the training speech and the label text, the target loss is obtained; based on the target loss, the model parameters of the initial speech recognition model are updated to obtain an updated speech recognition model, which is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to an accent scenario and the third and fourth speech training datasets corresponding to a common scenario are repeated until the convergence condition is met to obtain the target speech recognition model; the target speech recognition model is used to recognize speech to obtain the corresponding text. In this way, a general speech recognition model corresponding to common scenarios is used as the initial speech recognition model. By training the general speech recognition model, the performance of the model for accent scenarios is improved without deteriorating its performance for common scenarios, thus achieving accent domain extension. A meta-learning approach is used for model training. First, based on the first and third speech training datasets, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The intermediate speech recognition model learns speech recognition knowledge applicable to the first and third speech training datasets. Then, the second and fourth speech training datasets are input into the intermediate speech recognition model to calculate the target loss. The target loss reflects the effectiveness of the knowledge learned by the intermediate speech recognition model in applying to the second and fourth speech training datasets, and whether the knowledge learned by the intermediate speech recognition model is applicable to the second and fourth speech training datasets. Based on the target loss, the model parameters of the initial speech recognition model are updated to obtain an updated speech recognition model. This allows the updated speech recognition model to learn the variation rules between different speech scenarios and to learn the general knowledge between different speech recognition scenarios, so that the knowledge learned by the updated speech recognition model can be applied to new speech training datasets. Through multiple iterations, the final trained target speech recognition model can learn speech recognition methods for different scenarios. The target speech recognition model is applicable to both general scenarios and various accent scenarios. It has high speech recognition accuracy for general scenarios and also high speech recognition accuracy for various accent scenarios.

[0085] Common training methods typically involve training an initial model on a first training set to obtain a first model, then training the first model on a second training set to obtain a second model, and so on, until the training termination condition is met to obtain the target model. This training method fails to allow the model to effectively learn how to handle new datasets; it only ensures high accuracy with the training set, but cannot guarantee high accuracy with new datasets. However, the training method proposed in this application allows the model to learn speech recognition methods for different scenarios, resulting in high accuracy with both the training set and new datasets. In this application, an initial speech recognition model is trained based on a first speech training dataset and a third speech training dataset to obtain an intermediate speech recognition model. The second and fourth speech training datasets are then input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. The target loss is obtained based on the difference between the predicted text corresponding to the training speech and the label text. The model parameters of the initial speech recognition model are updated based on the target loss, rather than updating the intermediate speech recognition model based on the target loss. This allows the model to learn knowledge that is general across different training datasets, rather than just learning knowledge specific to a particular training dataset. This enables the model to learn model parameters that are also applicable when facing new datasets.

[0086] In one embodiment, such as Figure 3 As shown, step S208 includes:

[0087] Step S302: In the intermediate speech recognition model, the training speech is segmented and encoded to obtain the speech coding features corresponding to multiple speech segments of the training speech. The speech coding features are then decoded to obtain the first predicted text segment corresponding to each speech segment.

[0088] Segmented encoding refers to feature encoding based on speech segments. The training speech can be divided into multiple segments; for example, a segment might consist of 500ms of speech data. If the training speech is 3 seconds long, the corresponding segments would be six ordered segments, each containing 500ms of speech data. Feature encoding is then applied to each segment separately to obtain its corresponding speech encoding features. Feature encoding converts the speech data into feature vectors that are easy for the model to process. Feature decoding is used to decode the text data from these feature vectors.

[0089] The first predicted text segment is obtained by directly decoding the speech coding features.

[0090] Specifically, the speech recognition model can be a real-time speech recognition model. The computer device inputs the second speech training dataset and the fourth speech training dataset into the intermediate speech recognition model. In the intermediate speech recognition model, the training speech is feature-encoded in units of speech segments to obtain speech coding features corresponding to multiple speech segments of the training speech. Each speech coding feature is then decoded to obtain the first predicted text segment corresponding to each speech segment.

[0091] Step S304: Based on the labeled text corresponding to the training speech, perform feature decoding on each speech encoding feature to obtain the second predicted text segment corresponding to each speech segment.

[0092] The second predicted text segment is obtained by decoding the speech coding features based on the labeled text.

[0093] Specifically, besides directly decoding the speech coding features to obtain the predicted text fragments, the speech coding features can be further decoded by referring to the label text to obtain the predicted text fragments. When generating the word sequence used as the predicted text fragments, the computer device refers to the relevant information in the label text, which makes the generated predicted text fragments have a certain degree of reliability and accuracy, avoiding the influence of completely wrong prediction results on the model during training. When decoding the speech coding features to generate a word in the predicted text fragment, the historical part of the label text can be referenced to guide the generation of the next word, which can improve the accuracy of the generated words, and thus improve the accuracy of the generated predicted text fragments.

[0094] In one embodiment, when decoding to generate a word sequence for use as a predicted text fragment, each word in the word sequence may reference its corresponding historical portion in the label text. For example, when decoding to generate the t-th word, the previous t-1 words in the label text may be referenced. Alternatively, when decoding to generate a word sequence for use as a predicted text fragment, some words in the word sequence may reference their corresponding historical portions in the label text, while other words may reference previously decoded words. For example, when decoding to generate the t-th word, a probability p may be used to select whether to reference its corresponding historical portion in the label text or to reference currently decoded words.

[0095] Step S306: A first loss is obtained based on the difference between the first predicted text segment and the labeled text segment corresponding to the same speech segment, and a second loss is obtained based on the difference between the second predicted text segment and the labeled text segment corresponding to the same speech segment.

[0096] Step S308: Obtain the target loss based on the first loss and the second loss.

[0097] In this context, the tag text segment corresponding to a speech segment refers to the text segment in the tag text that matches the content narrated by the speech segment. For example, if a speech segment includes 500ms of speech data, the tag text segment corresponding to this speech segment is the text segment in the tag text that corresponds to these 500ms.

[0098] Specifically, the computer device derives a first loss based on the difference between the first predicted text segment and the labeled text segment corresponding to the same speech segment. This can be achieved by calculating a first sub-loss based on the difference between the first predicted text segment and the labeled text segment corresponding to the same speech segment, obtaining the first sub-loss for each speech segment, and then deriving the first loss based on these first sub-losses. The first loss guides the model to obtain more accurate predicted text segments during real-time and segmented recognition. Similarly, the computer device derives a second loss based on the difference between the second predicted text segment and the labeled text segment corresponding to the same speech segment. The second loss assists in guiding the model to obtain more accurate predicted text segments during real-time and segmented recognition, accelerating model convergence. Decoding speech coding features based on labeled text can correct model predictions during training, preventing further error amplification during word sequence generation. Feature decoding based on labeled text can significantly accelerate model convergence, making model training faster and smoother. Finally, the computer device obtains a target loss based on the first and second losses. For example, the target loss can be obtained by calculating the sum of the first and second losses, or by weighted summing the first and second losses.

[0099] In the above embodiments, in the intermediate speech recognition model, the training speech is segmented and encoded to obtain speech coding features corresponding to multiple speech segments of the training speech. Each speech coding feature is then decoded to obtain a first predicted text segment corresponding to each speech segment. Based on the tag text corresponding to the training speech, each speech coding feature is decoded to obtain a second predicted text segment corresponding to each speech segment. A first loss is obtained based on the difference between the first predicted text segment and the tag text segment corresponding to the same speech segment, and a second loss is obtained based on the difference between the second predicted text segment and the tag text segment corresponding to the same speech segment. A target loss is obtained based on the first and second losses. Thus, the first loss helps improve the model's recognition accuracy when performing real-time and segmented speech recognition, the second loss helps improve the model's convergence speed, and the target loss obtained based on the first and second losses helps improve the model's training quality during training.

[0100] In one embodiment, the training speech is segmented and encoded to obtain speech coding features corresponding to multiple speech segments of the training speech, including:

[0101] Feature extraction is performed on the training speech to obtain initial speech features corresponding to multiple speech segments of the training speech; the initial speech features are downsampled to obtain target speech features; the target speech features are then encoded to obtain speech encoded features corresponding to each speech segment.

[0102] Downsampling refers to deleting or fusing a portion of features from the initial speech features to improve model training efficiency. Speech exhibits short-term consistency, meaning the waveforms of several frames of speech may be completely identical. During model training, repetitive information leads to low training efficiency. Therefore, downsampling can improve training efficiency. Downsampling can be achieved by convolutional processing of the initial speech features. The initial speech features include speech sub-features corresponding to multiple speech frames of a speech segment. Downsampling can involve deleting or fusing a portion of these speech sub-features from the initial speech features. Alternatively, downsampling can generate target speech features by taking one frame of data from the initial speech features every preset number of frames, for example, taking one frame of data every four frames. Another approach is to divide the initial speech features into multiple sets of speech sub-features and take the average of each set to generate the target speech features, for example, taking the average of four frames from the initial speech features.

[0103] Specifically, during segmented coding, the computer device can first convert the training speech into speech segments into initial speech features that are easy to process, then downsample the initial speech features to obtain target speech features, and then perform feature coding on the target speech features to obtain speech coding features corresponding to multiple speech segments of the training speech.

[0104] In one embodiment, the initial speech features can be FBank features (FilterBank, features based on filter banks).

[0105] In the above embodiments, feature extraction is performed on the training speech to obtain initial speech features corresponding to multiple speech segments of the training speech; the initial speech features are downsampled to obtain target speech features; and the target speech features are feature encoded to obtain speech encoded features corresponding to each speech segment. Converting speech segments into speech features can improve the efficiency of model data processing, and downsampling can improve the efficiency of model training.

[0106] In one embodiment, such as Figure 4 As shown, the target loss is obtained based on the first loss and the second loss, including:

[0107] Step S402: Concatenate the speech coding features corresponding to each speech segment to obtain the initial coding features.

[0108] Specifically, the speech recognition model can be a combination of real-time and non-real-time speech recognition models. The computer device can concatenate the speech coding features corresponding to each speech segment to obtain initial coding features, which are used to represent the initial coding features corresponding to the training speech.

[0109] Step S404: Perform feature encoding on the initial encoded features to obtain target encoded features, and perform feature decoding on the target encoded features to obtain the third predicted text.

[0110] Specifically, the computer device can further encode the initial encoded features to obtain target encoded features. These target encoded features represent the final encoded features corresponding to the training speech, and they better represent the acoustic and semantic information of the training speech. Then, the target encoded features are decoded to obtain the third predicted text corresponding to the training speech. The third predicted text represents the overall text obtained by performing holistic recognition of the training speech.

[0111] Step S406: The third loss is obtained based on the difference between the third predicted text and the label text corresponding to the same training speech.

[0112] Step S408: Based on the first loss, the second loss, and the third loss, the target loss is obtained.

[0113] Specifically, the computer device can derive a third loss based on the difference between the third predicted text and the label text corresponding to the same training speech. This third loss can be calculated by sub-calculating the difference between the third predicted text and the label text corresponding to the same training speech, resulting in a third sub-loss for each training speech. The third loss is then used to guide the model to obtain more accurate predicted text during non-real-time and overall recognition. Finally, the computer device derives the target loss based on the first, second, and third losses.

[0114] In the above embodiments, the speech coding features corresponding to each speech segment are concatenated to obtain initial coding features; the initial coding features are then feature-encoded to obtain target coding features; the target coding features are then feature-decoded to obtain the third predicted text; a third loss is obtained based on the difference between the third predicted text and the label text corresponding to the same training speech; and a target loss is obtained based on the first loss, second loss, and third loss. The third loss helps improve the recognition accuracy of the model when performing overall speech recognition, and the target loss obtained based on the first loss, second loss, and third loss helps improve the model training quality during model training.

[0115] In one embodiment, such as Figure 5As shown, based on the first loss, the second loss, and the third loss, the target loss is obtained, including:

[0116] Step S502: Based on the labeled text corresponding to the training speech, the target encoded features are decoded to obtain the fourth predicted text.

[0117] Specifically, besides directly decoding the target encoded features to obtain the predicted text, the target encoded features can also be further decoded by referring to the label text. When generating word sequences to serve as predicted text, computer devices refer to relevant information from the label text, ensuring the generated predicted text has a certain degree of reliability and accuracy, and preventing the model from being affected by completely erroneous predictions during training. Decoding the target encoded features based on the label text can correct the model's predictions during training, preventing further amplification of errors during word sequence generation. Feature decoding based on the label text can also greatly accelerate the model's convergence speed, making the model training process faster and smoother.

[0118] When decoding target encoded features to generate a word in the predicted text, the historical part of the label text can be referenced to guide the generation of the next word, which can improve the accuracy of the generated words and thus improve the accuracy of the generated predicted text.

[0119] In one embodiment, when decoding to generate a word sequence for prediction text, each word in the word sequence may reference its corresponding historical portion in the label text. For example, when decoding to generate the t-th word, the previous t-1 words in the label text may be referenced. Alternatively, when decoding to generate a word sequence for prediction text, some words in the word sequence may reference their corresponding historical portions in the label text, while others may reference previously decoded words. For example, when decoding to generate the t-th word, a probability p may be used to select whether to reference its corresponding historical portion in the label text or to reference currently decoded words.

[0120] Step S504: The fourth loss is obtained based on the difference between the fourth predicted text and the label text corresponding to the same training speech.

[0121] Step S506: Based on the first loss, second loss, third loss, and fourth loss, the target loss is obtained.

[0122] Specifically, similar to the third loss, the computer device obtains the fourth loss based on the difference between the fourth predicted text and the label text corresponding to the same training speech, and finally obtains the target loss based on the first loss, the second loss, the third loss and the fourth loss.

[0123] In the above embodiments, based on the labeled text corresponding to the training speech, feature decoding is performed on the target encoded features to obtain the fourth predicted text; the fourth loss is obtained based on the difference between the fourth predicted text and the labeled text corresponding to the same training speech; and the target loss is obtained based on the first loss, second loss, third loss, and fourth loss. Thus, the fourth loss helps improve the model's convergence speed, and the target loss obtained based on the first loss, second loss, third loss, and fourth loss helps improve the model's training quality during training.

[0124] In one embodiment, based on the labeled text corresponding to the training speech, feature decoding is performed on each speech encoding feature to obtain the second predicted text segment corresponding to each speech segment, including:

[0125] From the current label text corresponding to the current training speech, obtain the current label text segment corresponding to the current speech segment of the current training speech; determine the target position from each word position corresponding to the current label text segment; determine the reference position matching the target position from the current label text, and obtain the label words before the reference position as reference words; based on the reference words, perform feature decoding on the speech coding features corresponding to the current speech segment to obtain the predicted word corresponding to the target position; obtain the next word position in the current label text segment as the target position, and return to the steps of determining the reference position matching the target position from the current label text, and obtaining the label words before the reference position as reference words, until the termination condition is met, resulting in multiple predicted words; based on each predicted word, obtain the second predicted text segment corresponding to the current speech segment.

[0126] The tag text consists of multiple ordered tag words. For example, if the tag text is "Hello everyone", then the ordered tag words would be "Da-Jia-Hao".

[0127] The current training speech refers to the training speech being processed, which can be any training speech from the second and fourth speech training datasets. The current label text refers to the label text corresponding to the current training speech. The current speech segment refers to the speech segment currently being processed within the current training speech, and can be any speech segment from multiple speech segments corresponding to the current training speech. The current label text segment refers to the label text segment corresponding to the current speech segment.

[0128] The target position is determined from the positions of each word in the current tag text fragment. The target position represents the location of a specific word within the current tag text fragment. The reference position is also determined from the positions of each word in the current tag text fragment. The reference position that matches the target position refers to the word position of the tag word corresponding to the target position within the current tag text. For example, if the current tag text is "I'm already waiting for the elevator downstairs," and the current tag text fragment is "Waiting for the elevator downstairs," if we take the first word position in the current tag text fragment as the target position, and the tag word corresponding to the target position is "building," then the reference position that matches the target position is the word position of "building" within the current tag text.

[0129] Specifically, when decoding the speech coding features of a speech segment based on the labeled text, the predicted words used to compose the predicted text segment can be generated sequentially. When decoding a predicted word, it is necessary to refer to the historical part before the corresponding position in the labeled text. By referring to the historical part in the labeled text, a more accurate predicted word can be decoded.

[0130] For the feature decoding process of the speech coding features corresponding to the current speech segment, the computer device can select the first word position as the target position from the word positions corresponding to the current labeled text segment of the current speech segment. It then determines a reference position matching the target position from the current labeled text, and retrieves the label words preceding the reference position from the current labeled text as reference words. If no label words exist before the reference position in the current labeled text, a preset character is retrieved as a reference word. Based on the reference words, the speech coding features corresponding to the current speech segment are decoded to obtain the predicted word corresponding to the target position, i.e., the predicted word corresponding to the first word position. The computer device then selects the second word position as the new target position from the word positions corresponding to the current labeled text segment. It again determines a reference position matching the target position from the current labeled text, and retrieves the label words preceding the reference position from the current labeled text as new reference words. Based on the reference words, the speech coding features corresponding to the current speech segment are decoded to obtain the predicted word corresponding to the new target position, i.e., the predicted word corresponding to the second word position. This process continues until the termination condition is met, resulting in multiple predicted words. Finally, the predicted words are arranged in order to obtain the second predicted text segment corresponding to the current speech segment.

[0131] The termination condition can be either that the total number of predicted words exceeds a preset number. If the total number of predicted words exceeds the preset number, decoding stops, and the decoded predicted words are arranged in order to obtain the second predicted text segment. Alternatively, the termination condition can be that a predicted word representing an end symbol is decoded. If a predicted word representing an end symbol is decoded, decoding stops, and the decoded predicted words are arranged in order to obtain the second predicted text segment.

[0132] If no tag word exists before the reference position in the current tag text, a preset character is used as the reference word. The preset character can be set according to actual needs; for example, the start character can be used as the preset character.

[0133] It's understandable that during model training, the process of decoding speech coding features based on labeled text can be sequential, generating each predicted word in turn. Alternatively, it can be parallel, generating each predicted word in parallel. Since the labeled text is known, parallel processing can quickly yield each predicted word during model training, thus rapidly obtaining the predicted text fragment. When the trained model is applied, since the text corresponding to the speech to be recognized is unknown, the feature decoding process can be sequential, decoding each word sequentially based on the encoded features of the speech to be recognized to assemble the final recognized text.

[0134] In one embodiment, reference words can be text-encoded to obtain reference text features, speech encoding features and reference text features can be fused to obtain predicted text features, and finally the predicted text features can be decoded to obtain the predicted word corresponding to the target position.

[0135] In one embodiment, the process of decoding the target encoded features based on the labeled text corresponding to the training speech is similar to the process of decoding the speech encoded features based on the labeled text corresponding to the training speech. The target position is determined from each word position in the current labeled text corresponding to the current training speech. The labeled words preceding the target position are obtained as reference words. Based on the reference words, the target encoded features corresponding to the current training speech are decoded to obtain the predicted word corresponding to the target position. The next word position in the current labeled text is obtained as the target position. The process of obtaining the labeled words preceding the target position as reference words is repeated until the termination condition is met, resulting in multiple predicted words. Based on these predicted words, the fourth predicted text corresponding to the current training speech is obtained.

[0136] In the above embodiments, predictive words are generated to form predictive text fragments. When generating predictive words, the label words preceding the corresponding positions in the label text are referenced. This can guide the model to output more accurate predictive words based on the forward correct answer, avoiding further amplification of errors during the word sequence generation process. This can greatly accelerate the convergence speed of the model and make the model training process faster and more stable.

[0137] In one embodiment, the initial speech recognition model includes an encoder and a decoder, the encoder being used for feature encoding and the decoder being used for feature decoding, the encoder including a start layer, an intermediate layer and an end layer.

[0138] The updated speech recognition model is obtained by updating the model parameters of the initial speech recognition model based on the target loss, including:

[0139] Keeping the model parameters of the intermediate layers in the encoder unchanged, the model parameters of the start layer, end layer and decoder in the encoder are updated based on the target loss to obtain the updated speech recognition model.

[0140] The initial speech recognition model includes an encoder and a decoder. The encoder decodes features and outputs encoded features, while the decoder decodes features and outputs predicted text. The encoder consists of multiple network layers, specifically a start layer, intermediate layers, and an end layer. The start layer represents the first network layer in the encoder, the end layer represents the last network layer, and the intermediate layers are the network layers located between the start and end layers.

[0141] Specifically, to improve the stability and speed of model training, only specific layers are trained on top of the meta-learning training method, while the intermediate layers are fixed. When updating the model parameters of the initial speech recognition model based on the target loss, the model parameters of the intermediate layers in the encoder can be kept unchanged, and the model parameters of the start layer, end layer, and decoder in the encoder can be updated based on the target loss, thus obtaining the updated speech recognition model.

[0142] The parameters learned in different layers of a model have different meanings. In the encoder, the layers closest to the input data primarily process the acoustic features of the speech signal, learning its acoustic information. The layers in the middle of the encoder, however, mainly learn the contextual information of the speech signal. The main difference between accented and standard speech lies in intonation and pronunciation. Since the initial speech recognition model is already suitable for general scenarios, keeping most of the middle layers unchanged won't significantly affect speech recognition performance. The main focus is on training the parameters of the initial layers to enable the model to extract more text-related features during the acoustic transformation of speech information into features. Because the features change, the encoder's output features will also change slightly, so the parameters of the last few layers are adaptively adjusted.

[0143] In one embodiment, the initial speech recognition model may include at least two encoders and at least two decoders. Different encoders are used for different feature encoding methods, and different decoders are used for different feature decoding methods. When adjusting the model parameters, the model parameters of the intermediate layers in each encoder can be kept unchanged, and the model parameters of the start layer, end layer in each encoder, and each decoder can be updated based on the target loss to obtain the updated speech recognition model.

[0144] In the above embodiments, keeping the model parameters of the intermediate layers in the encoder unchanged, and updating the model parameters of the start layer, end layer and decoder in the encoder based on the target loss, an updated speech recognition model is obtained, which can ensure the stability of model training.

[0145] In one embodiment, obtaining a general speech recognition model corresponding to a general scenario includes:

[0146] Obtain the fifth speech training dataset corresponding to the general scenario; based on the fifth speech training dataset, train the candidate speech recognition model to obtain the general speech recognition model.

[0147] Here, the candidate speech recognition model refers to the initialized speech recognition model, which can be a randomly initialized model or a model with manually set initialization parameters. The fifth speech training dataset refers to the training set consisting of speech data from a general scenario and corresponding labeled text. The fifth speech training dataset can be the same as or different from the third and fourth speech training datasets.

[0148] Specifically, a general speech recognition model can be trained based on a speech training dataset corresponding to a general scenario. A computer device can obtain a fifth speech training dataset corresponding to a general scenario, and based on this dataset, train candidate speech recognition models to obtain the general speech recognition model.

[0149] It is understandable that candidate speech recognition models can be trained using various supervised training methods or a combination of unsupervised and supervised training methods.

[0150] In one embodiment, a candidate speech recognition model is used as the first speech recognition model. A general training subset is obtained by sampling the fifth speech training dataset. The general training subset is input into the first speech recognition model. After data processing by the model, the predicted text corresponding to each training speech in the general training subset is obtained. A model loss is generated based on the difference between the label text and the predicted text corresponding to the same training speech. The model parameters of the first speech recognition model are updated based on the model loss to obtain the second speech recognition model. The second speech recognition model is used as the first speech recognition model. The process of sampling the fifth speech training dataset to obtain the general training subset is repeated until the convergence condition is met to obtain an intermediate speech recognition model.

[0151] In the above embodiments, a fifth speech training dataset corresponding to a general scenario is obtained; based on the fifth speech training dataset, the candidate speech recognition model is trained to obtain a general speech recognition model, which ensures that the general speech recognition model has a certain degree of recognition accuracy. Subsequently, training the general speech recognition model based on meta-learning can improve the model's performance in accent scenarios without deteriorating the model's performance in general scenarios.

[0152] In one embodiment, the initial speech recognition model includes an encoder and a decoder, the encoder including a local encoder and a global encoder, and the decoder including a basic decoder, a local decoder, and a global decoder.

[0153] The local encoder is used to perform feature encoding on speech segments to obtain the corresponding speech coding features. In other words, the local encoder is used for segmented encoding.

[0154] The initial encoded features of each speech segment belonging to the same speech can be obtained by concatenating the speech coding features. A global encoder is used to perform feature encoding on these initial encoded features to obtain the target encoded features. In other words, the global encoder performs further feature encoding on the initial encoded features for overall encoding. The global encoder is connected to the local encoder. The initial encoded features of each speech segment belonging to the same speech, output from the local encoder, are concatenated to obtain the initial encoded features of the speech. These initial encoded features are then input into the global encoder for feature encoding to obtain the target encoded features of the speech.

[0155] The base decoder is used to decode the speech coding features output by the local encoder. The base decoder is connected to the local encoder. The speech coding features corresponding to the speech segments output by the local encoder are input into the base decoder for feature decoding to obtain the predicted text segments corresponding to the speech segments. The base decoder is also used to decode the target coding data output by the global encoder. The base decoder is also connected to the global encoder. The target coding features corresponding to the speech output by the global encoder are input into the base decoder for feature decoding to obtain the predicted text corresponding to the speech.

[0156] The local decoder is used to decode the speech coding features corresponding to the speech segments of the training speech based on the labeled text segments corresponding to the speech segments. The local encoder is connected to the local encoder; the speech coding features corresponding to the speech segments output by the local encoder are input into the local encoder, and the labeled text segments corresponding to the speech segments are also input into the local encoder. In the local decoder, the speech coding features are decoded based on the labeled text segments to obtain the second predicted text segment corresponding to the speech segment.

[0157] The global decoder is used to decode the target encoded features corresponding to the training speech based on the labeled text corresponding to the training speech. The global decoder is connected to the global encoder. The target encoded features corresponding to the training speech output by the global encoder are input into the global decoder, and the labeled text corresponding to the speech is also input into the global decoder. In the global decoder, the target encoded features are decoded based on the labeled text to obtain the fourth predicted text corresponding to the speech.

[0158] It is understandable that local and global decoders are used to guide model training and improve the model's convergence speed during training. After model training is complete, local and global decoders do not participate in prediction; accurate predictions of text fragments and text can be obtained based on the local encoder, global encoder, and basic decoder.

[0159] In one embodiment, a local encoder is used to encode speech segments in real time to obtain speech coding features corresponding to the speech segments, and a basic decoder is used to decode the output data of the local encoder in real time. For each speech segment acquired, the local encoder encodes the speech segments in real time to obtain speech coding features corresponding to the speech segments. These speech coding features are then input into the basic decoder in real time, and the basic decoder decodes the speech coding features in real time to obtain the predicted text segment corresponding to the speech segment. After acquiring the last speech segment of a speech, the speech coding features corresponding to each speech segment belonging to the same speech segment output by the local encoder are concatenated to obtain the initial coding features corresponding to the speech. These initial coding features are then input into the global encoder for feature encoding to obtain the target coding features corresponding to the speech. The target coding features are then input into the basic decoder, and the basic decoder decodes the target coding features to obtain the predicted text corresponding to the speech.

[0160] For the same speech, each time a predicted text segment is decoded, that segment can be displayed. After decoding the predicted text, that text can also be displayed. Alternatively, both predicted text segments and the predicted text itself can be displayed, or the predicted text can overlay the predicted text segments.

[0161] In one specific embodiment, reference Figure 6 The initial speech recognition model includes a speech input layer (which can be called Audio Input), a local encoder, a global encoder, a basic decoder, a local decoder, a global decoder, and a text input layer (which can be called Text Input).

[0162] During model training, training speech and its corresponding labeled text are input into the model. The labeled text is input into the text input layer for text recognition to obtain a sequence of labeled words. The training speech is then input into the speech input layer, where each speech segment is converted into initial speech features. These initial speech features are input into the local encoder, where a downsampling layer is used to downsample the initial speech features to obtain target speech features. These target speech features are then input into the first encoding layer, where they are encoded through multiple sequentially connected encoding layers to obtain the speech encoded features corresponding to each speech segment of the training speech. The speech encoded features are then input into the base decoder, where feature decoding is performed to obtain the first predicted text segment corresponding to each speech segment. Finally, the speech encoded features and the labeled word sequence are input into the local decoder, where feature decoding is performed based on the labeled word sequence to obtain the second predicted text segment corresponding to each speech segment.

[0163] The speech coding features corresponding to various speech segments belonging to the same training speech, obtained from the local encoder, are concatenated to obtain the initial coding features corresponding to the training speech. These initial coding features are then input into the global encoder for feature encoding to obtain the target coding features corresponding to the training speech. The target coding features are then input into the basic decoder for feature decoding to obtain the third predicted text corresponding to the training speech. Finally, the target coding features and the tag word sequence are input into the global decoder, where feature decoding based on the tag word sequence is performed to obtain the fourth predicted text corresponding to the training speech.

[0164] For the training data, the speech training dataset corresponding to the general scenario is labeled G, and the speech training dataset corresponding to the accent scenario is labeled A. For each accent i, it is labeled Ai, so the speech training dataset corresponding to the accent scenario is A = {A1, A2, ..., An}. The data in G and A are divided into training set and support set, namely Gtra, Gsup, Atra, and Asup.

[0165] The loss is calculated using the following formula for training:

[0166] L=λ1L ctc-streaming +λ2L ctc-nonstreaming +λ3L aed-streaming +λ4L aed-nonstreaming

[0167] The local encoder can be called a Causal Encoder, the global encoder can be called a Non-Causal Encoder, the basic decoder can be a CTC Decoder, the local decoder can be called a Streaming Attention Decoder, and the global decoder can be called an Attention Decoder. ctc-streaming The first loss is calculated based on the difference between the labeled text segment corresponding to the speech segment of the training speech and the first predicted text segment. The first predicted text segment is the data output by the Causal Encoder input into the CTC Decoder, and the data output by the CTC Decoder. ctc-nonstreaming The third loss is calculated based on the difference between the labeled text corresponding to the training speech and the third predicted text. The third predicted text is the data output by the Non-Causal Encoder and the output of the CTC Decoder. aed-streamingThe second loss is calculated based on the difference between the labeled text segment corresponding to the training speech and the second predicted text segment. The second predicted text segment is obtained by inputting the output data of the Causal Encoder and the output data of the Text Input into the Streaming Attention Decoder, and the output data of the Streaming Attention Decoder. aed-nonstreaming The fourth loss is calculated based on the difference between the labeled text corresponding to the training speech and the fourth predicted text. The fourth predicted text is the output data of the Non-Causal Encoder and the output data of the TextInput, which are input into the Attention Decoder, and the output data of the Attention Decoder. L represents the total loss. λ1, λ2, λ3, and λ4 represent the loss weights, which can be set according to actual needs.

[0168] In the first iteration, model f is a speech recognition model suitable for general scenarios.

[0169] For each iteration, some sample batches from Gtra and Atra constitute Itra, and some sample batches from Gsup and Asup constitute Isup. The initial parameters of model f are θ. First, the gradient is calculated on Itra and then updated on the model as follows (1):

[0170]

[0171] Where α represents the learning rate. This represents the total loss calculated for the model f with input parameter θ in the Itra class. Indicates based on The calculated gradient. θ′ i This represents the updated parameters of model f.

[0172] Alternatively, it can be divided into several parts and passed through the model sequentially to update the model's parameters. This also represents the model's learning of the recognition data in the training set. After training the model based on the training set, our meta-learning objective is Equation (2):

[0173]

[0174] The goal of meta-learning is to input Isup with parameters θ′ i The model f calculates the minimum total loss. p( / ) represents the general data and accent data used during training.

[0175] After training the model using the training set, calculate Isup at f θ′ The loss and gradient are used to learn the speech recognition method, and the model parameters are finally updated as follows:

[0176]

[0177] Where β represents the learning rate. This represents the total loss calculated for the model f with input parameter θ′ in the Itra class. Indicates based on The calculated gradient. Based on... The parameter θ is updated to obtain the initial parameters of the model for the next iteration.

[0178] Because meta-learning training methods often lead to unstable model training compared to more common methods, we train only specific layers based on meta-learning, while fixing most of the intermediate layers. (Reference) Figure 6 In the model, the parameters of the network layers indicated by the dashed boxes are fixed, and only the parameters of other parts are trained. This can make the prediction results of sensitive data in general data change less, and can also improve the training speed.

[0179] After model training is complete, the speech to be recognized is input into the trained model during application. The speech is then input into the speech input layer, where it is converted into initial speech features. These initial features are input into the local encoder, where a downsampling layer further downsamples the initial speech features to obtain target speech features. These target speech features are then input into the first coding layer, where they are segmented and encoded through multiple sequentially connected coding layers to obtain the speech coding features corresponding to each speech segment of the speech to be recognized. These speech coding features are then input into the base decoder, where feature decoding yields the corresponding recognized text segments. The speech coding features obtained from the local encoder are concatenated to obtain the initial coding features corresponding to the speech to be recognized. These initial coding features are then input into the global encoder for feature encoding to obtain the target coding features corresponding to the speech to be recognized. Finally, the target coding features are input into the base decoder for feature decoding to obtain the recognized text corresponding to the speech to be recognized.

[0180] It's understandable that the model can be applied in a sampling-as-you-go manner. Each acquired speech segment is input into the trained model. The speech segment to be recognized is input into the speech input layer, where it is converted into initial speech features. These initial features are then input into the local encoder, where a downsampling layer further downsamples the initial speech features to obtain target speech features. These target speech features are then input into the first encoding layer, and through multiple sequentially connected encoding layers, feature encoding is performed to obtain the speech encoded features corresponding to the speech segment to be recognized. These speech encoded features are then input into the base decoder, where feature decoding yields the corresponding recognized text segment. The model outputs the recognized text segment for each speech segment it obtains. When the speech encoded features corresponding to the last speech segment belonging to the same speech segment are obtained, the speech encoded features of all the speech segments belonging to the same speech segment processed by the local encoder are concatenated to obtain the initial encoded features corresponding to the speech segment to be recognized. These initial encoded features are then input into the global encoder for feature encoding to obtain the target encoded features corresponding to the speech segment to be recognized. Finally, the target encoded features are input into the base decoder for feature decoding to obtain the recognized text corresponding to the speech segment to be recognized. The model will recognize the text and output it.

[0181] In one embodiment, such as Figure 7 As shown, a speech recognition method is provided. Taking the application of this method to a computer device as an example, the computer device can be a terminal or a server. The method can be executed independently by the terminal or server, or it can be implemented through interaction between the terminal and the server. (Reference) Figure 2 The speech recognition method includes the following steps:

[0182] Step S702: Obtain the speech to be recognized.

[0183] Step S704: Input the speech to be recognized into the target speech recognition model to obtain the recognized text corresponding to the speech to be recognized.

[0184] The speech to be recognized refers to the speech to be recognized. This speech can be of various types, such as voice input by a user using an input method's speech recognition function, voice from a video, or voice during a phone call. The speech to be recognized can be captured in real-time or pre-captured. The recognized text is the text obtained by performing speech recognition on the speech to be recognized.

[0185] The training process of the target speech recognition model includes the following steps:

[0186] Obtain a general speech recognition model corresponding to a common scenario, and use this model as the initial speech recognition model. Obtain the first and second speech training datasets corresponding to the accent scenario, and the third and fourth speech training datasets corresponding to the common scenario. Each speech training dataset includes training speech and corresponding label text, where the label text is obtained by converting the training speech into text. Based on the first and third speech training datasets, train the initial speech recognition model to obtain an intermediate speech recognition model. Input the second and fourth speech training datasets into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text and the label text, obtain the target loss. Update the model parameters of the initial speech recognition model based on the target loss to obtain an updated speech recognition model. Use the updated speech recognition model as the initial speech recognition model, and return to the steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and the third and fourth speech training datasets corresponding to the common scenario, until the convergence condition is met, and obtain the target speech recognition model.

[0187] It is understood that the specific training process of the target speech recognition model can be referred to the contents of the various embodiments of the aforementioned speech recognition model training method, and will not be repeated here.

[0188] Specifically, the computer device can pre-train a target speech recognition model. The computer device can acquire the speech to be recognized locally or from other devices, input the speech to be recognized into the pre-trained target speech recognition model, and through data processing by the target speech recognition model, the target speech recognition model outputs the recognized text corresponding to the speech to be recognized.

[0189] It can be understood that if the target speech recognition model is a real-time speech recognition model, then the recognized text output by the target speech recognition model includes the recognized text segments corresponding to each segment of the speech to be recognized. If the target speech recognition model is a non-real-time speech recognition model, then the recognized text output by the target speech recognition model includes the overall text obtained by recognizing the speech as a whole. If the target speech recognition model is a speech recognition model that combines real-time and non-real-time methods, then the recognized text output by the target speech recognition model includes both the individual recognized text segments and the overall text.

[0190] After obtaining the text corresponding to the speech to be recognized, the computer device can display the recognized text or send the recognized text to the terminal for display.

[0191] The aforementioned speech recognition method acquires the speech to be recognized, inputs it into a target speech recognition model, and obtains the corresponding recognized text. The target speech recognition model exhibits high accuracy in both general and accent-based scenarios, and using it for speech recognition can effectively improve accuracy. By using a general speech recognition model corresponding to general scenarios as the initial model, training this model improves its performance in accent-based scenarios without deteriorating its performance in general scenarios, thus achieving accent-domain extension. A meta-learning approach is employed for model training. First, an initial speech recognition model is trained using a first and a third speech training dataset to obtain an intermediate speech recognition model. This intermediate model learns speech recognition knowledge applicable to the first and third training datasets. Then, a second and a fourth training dataset are input into the intermediate model to calculate the target loss. The target loss reflects the effectiveness of the intermediate model's learned knowledge applied to the second and fourth training datasets. Based on the target loss, the model parameters of the initial speech recognition model are updated to obtain an updated speech recognition model. This updated model learns the variation patterns of speech in different scenarios and acquires general knowledge applicable to different speech recognition scenarios, enabling it to quickly learn relevant knowledge when faced with new speech training datasets. Through multiple iterations, the final trained target speech recognition model learns methods for speech recognition in different scenarios. This target speech recognition model is applicable to both general scenarios and various accent scenarios, exhibiting high accuracy in general scenarios and also demonstrating high accuracy in various accent scenarios.

[0192] In one embodiment, the target speech recognition model includes an encoder and a decoder, wherein the encoder includes a local encoder and a global encoder;

[0193] Speech recognition methods also include:

[0194] During the process of acquiring the speech to be recognized, each acquired speech segment is input into the target speech recognition model. The local encoder and decoder obtain the corresponding recognized text segment in real time. The recognized text segments corresponding to each speech segment are displayed in sequence. After the speech acquisition is completed, the output data of each local encoder are concatenated and input into the global encoder. The global encoder and decoder obtain the overall recognized text corresponding to the speech to be recognized. The overall recognized text is displayed to cover each recognized text segment.

[0195] Specifically, the target speech recognition model is a combination of real-time and non-real-time speech recognition. During the acquisition of the speech to be recognized, each acquired speech segment is input into the target speech recognition model. A local encoder performs feature encoding on the speech segment in real time, and a decoder performs feature decoding on the encoded result in real time, thus obtaining the corresponding recognized text segment. In the process of acquiring the speech to be recognized, a pre-defined length of speech can be considered as a single speech segment. Each acquired segment is input into the target speech recognition model for speech recognition, yielding the corresponding recognized text segment. Once a recognized text segment is obtained, it can be displayed. During the acquisition of the speech to be recognized, the recognized text segments corresponding to each speech segment are displayed sequentially.

[0196] After the speech to be recognized is acquired, the feature encoding results corresponding to each speech segment belonging to the speech to be recognized, output by the local encoder, are concatenated and input into the global encoder. The global encoder performs further feature encoding, and the encoding result of the global encoder is input into the decoder for feature decoding, thereby obtaining the overall recognized text corresponding to the speech to be recognized. It can be understood that recognizing the overall text has higher accuracy than recognizing individual text segments. Therefore, when displaying the overall recognized text, it can be presented by overlaying the individual recognized text segments.

[0197] In the above embodiments, during the process of collecting the speech to be recognized, recognition is performed simultaneously to improve recognition efficiency. After the collection is completed, overall recognition is performed to improve recognition accuracy. This recognition method takes into account both recognition delay and recognition effect.

[0198] In one specific embodiment, the speech recognition model training method and language recognition method of this application can be applied to speech recognition scenarios in input methods. Users can enable the voice input function of the input method on the terminal and speak into the terminal's microphone. The terminal will automatically use the target speech recognition model trained by the method of this application to perform speech recognition and obtain the corresponding text, displaying the recognition result on the terminal. The terminal can perform recognition locally or send the speech to a server for recognition.

[0199] Accents in speech differ from standard Mandarin in tone and pronunciation. To extend the input method to include accent-specific features—that is, to improve performance in general scenarios without compromising performance—we introduce meta-learning techniques for optimization. Meta-learning enables the model to learn methods for speech recognition in different scenarios, rather than just learning general and accent-specific speech recognition tasks (i.e., traditional training methods). This more fundamental learning approach results in performance improvements over traditional training methods for accent-specific tasks.

[0200] 1. Model Training

[0201] The specific scheme for each iteration during training is as follows:

[0202] The current parameters of the 1-1 model are W0. A portion of general data plus accent data is selected as the training set, and another portion of general data plus accent data is selected as the support set.

[0203] 1-2. After passing the data in the training set through the model, calculate the gradient G0 and update the model parameters to Wm = W0 + G0. After passing the data in the support set through the model Wm, calculate the gradient G, where G is the gradient of the learning method. Therefore, update the model parameters to W1 (i.e., W0 + G) based on the model W0.

[0204] 1-3. The W1 parameter is the initial model parameter W0 for the next iteration.

[0205] Furthermore, because meta-learning makes model training less stable than traditional training methods, we only train specific layers and fix most of the intermediate layers based on meta-learning.

[0206] In the first iteration, a speech recognition model suitable for general scenarios was used as the initial model. After multiple iterations, a speech recognition model was finally trained that not only maintained good recognition performance in general scenarios but also demonstrated excellent performance in accent-based scenarios. To achieve both high recognition rates and fast first-word output in input method speech recognition scenarios, a cascaded model can be used. A cascaded model includes a local encoder, a global encoder, a basic decoder, a local decoder, and a global decoder. The cascaded model first recognizes short speech segments, quickly outputting words, and then re-recognizes longer, complete speech segments, refreshing the first-output words.

[0207] 2. Model Application

[0208] When a user inputs voice data using an input method, short speech segments are first input into the model. The output data of the local encoder is then processed through feature decoding to produce streaming results for on-screen display. This means short speech segments are recognized quickly, and the words are displayed rapidly. When the user finishes speaking, all the outputs of the last layer of the local encoder are concatenated and fed into the global encoder for forward computation. The output data of the global encoder is then processed again through feature decoding to refresh and overwrite the previous results. This involves re-recognizing the longer, complete speech segment and updating the previously output words. This approach balances recognition latency with recognition accuracy.

[0209] The model trained according to this application was tested, and the results show that, based on Mandarin, the training method of this application for accent domain expansion has the following improvements: 1. The WER (Word Error Rate) of the accent test set is reduced by 2%-5% compared to the traditional training method. 2. The WER of the Mandarin test set is reduced by 1%-2% compared to the traditional training method. 3. Due to training on specific layers, the model training is more stable (the model is 2%-5% better on the general test set) and the training speed is faster (approximately 20% faster).

[0210] It is understood that the speech recognition model training method and language recognition method of this application can also be applied to other speech recognition scenarios, such as speech recognition in video conferencing; speech recognition in voice calls; and so on.

[0211] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0212] Based on the same inventive concept, this application also provides a speech recognition model training device for implementing the speech recognition model training method described above, and a speech recognition device for implementing the speech recognition method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more speech recognition model training device embodiments provided below can be found in the limitations of the speech recognition model training method above, and the specific limitations of one or more speech recognition device embodiments can be found in the limitations of the speech recognition method above, and will not be repeated here.

[0213] In one embodiment, such as Figure 8 As shown, a speech recognition model training device is provided, including: a model acquisition module 802, a training dataset acquisition module 804, a first model training module 806, and a second model training module 808, wherein:

[0214] The model acquisition module 802 is used to acquire a general speech recognition model corresponding to a general scenario and use the general speech recognition model as the initial speech recognition model.

[0215] The training dataset acquisition module 804 is used to acquire the first and second speech training datasets corresponding to the accent scenarios, and the third and fourth speech training datasets corresponding to the general scenarios. Each speech training dataset includes training speech and the label text corresponding to the training speech. The label text is obtained by converting the training speech into text.

[0216] The first model training module 806 is used to train the initial speech recognition model based on the first speech training dataset and the third speech training dataset to obtain an intermediate speech recognition model.

[0217] The second model training module 808 is used to input the second and fourth speech training datasets into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text corresponding to the training speech and the label text, the target loss is obtained. The model parameters of the initial speech recognition model are updated based on the target loss to obtain the updated speech recognition model. The updated speech recognition model is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario and obtaining the third and fourth speech training datasets corresponding to the general scenario are executed until the convergence condition is met to obtain the target speech recognition model. The target speech recognition model is used to recognize speech to obtain the corresponding text.

[0218] The aforementioned speech recognition model training device uses a general speech recognition model corresponding to common scenarios as the initial speech recognition model. By training this general speech recognition model, it improves the model's performance in accent-specific scenarios without deteriorating its performance in general scenarios, thus achieving accent domain expansion. A meta-learning approach is employed for model training. First, based on a first and a third speech training dataset, the initial speech recognition model is trained to obtain an intermediate speech recognition model. This intermediate model learns speech recognition knowledge applicable to the first and third speech training datasets. Then, the second and a fourth speech training datasets are input into the intermediate speech recognition model to calculate the target loss. The target loss reflects the effectiveness of the knowledge learned by the intermediate speech recognition model when applied to the second and fourth speech training datasets, and whether the learned knowledge is applicable to these datasets. Based on the target loss, the model parameters of the initial speech recognition model are updated to obtain an updated speech recognition model. This updated model learns the variation patterns between different speech scenarios and the general knowledge between different speech recognition scenarios, enabling the knowledge learned by the updated speech recognition model to be applicable to new speech training datasets. Through multiple iterations, the final trained target speech recognition model can learn speech recognition methods for different scenarios. The target speech recognition model is applicable to both general scenarios and various accent scenarios. It has high speech recognition accuracy for general scenarios and also has high speech recognition accuracy for various accent scenarios.

[0219] In one embodiment, the second model training module is further configured to: segment and encode the training speech in the intermediate speech recognition model to obtain speech coding features corresponding to multiple speech segments of the training speech; decode each speech coding feature to obtain a first predicted text segment corresponding to each speech segment; decode each speech coding feature based on the label text corresponding to the training speech to obtain a second predicted text segment corresponding to each speech segment; obtain a first loss based on the difference between the first predicted text segment and the label text segment corresponding to the same speech segment; obtain a second loss based on the difference between the second predicted text segment and the label text segment corresponding to the same speech segment; and obtain a target loss based on the first loss and the second loss.

[0220] In one embodiment, the second model training module is further configured to extract features from the training speech to obtain initial speech features corresponding to multiple speech segments of the training speech; downsample the initial speech features to obtain target speech features; and encode the target speech features to obtain speech coding features corresponding to each speech segment.

[0221] In one embodiment, the second model training module is further configured to concatenate the speech coding features corresponding to each speech segment to obtain initial coding features; perform feature encoding on the initial coding features to obtain target coding features; perform feature decoding on the target coding features to obtain third predicted text; obtain a third loss based on the difference between the third predicted text and the label text corresponding to the same training speech; and obtain a target loss based on the first loss, the second loss, and the third loss.

[0222] In one embodiment, the second model training module is further configured to perform feature decoding on the target encoded features based on the label text corresponding to the training speech to obtain the fourth predicted text; obtain the fourth loss based on the difference between the fourth predicted text and the label text corresponding to the same training speech; and obtain the target loss based on the first loss, the second loss, the third loss and the fourth loss.

[0223] In one embodiment, the second model training module is further configured to: obtain the current label text segment corresponding to the current speech segment of the current training speech from the current label text corresponding to the current training speech; determine the target position from each word position corresponding to the current label text segment; determine a reference position matching the target position from the current label text, and obtain the label words before the reference position as reference words; perform feature decoding on the speech coding features corresponding to the current speech segment based on the reference words to obtain the predicted words corresponding to the target position; obtain the next word position in the current label text segment as the target position, and return to the steps of determining the reference position matching the target position from the current label text and obtaining the label words before the reference position as reference words, until the termination condition is met, resulting in multiple predicted words; and obtain the second predicted text segment corresponding to the current speech segment based on each predicted word.

[0224] In one embodiment, the initial speech recognition model includes an encoder and a decoder. The encoder is used for feature encoding, and the decoder is used for feature decoding. The encoder includes a start layer, an intermediate layer, and an end layer. The second model training module is also used to keep the model parameters of the intermediate layer in the encoder unchanged, and update the model parameters of the start layer, end layer, and decoder in the encoder based on the target loss to obtain an updated speech recognition model.

[0225] In one embodiment, the model acquisition module is further used to acquire a fifth speech training dataset corresponding to a general scenario; and to train the candidate speech recognition model based on the fifth speech training dataset to obtain a general speech recognition model.

[0226] In one embodiment, the initial speech recognition model includes an encoder and a decoder. The encoder includes a local encoder and a global encoder, and the decoder includes a basic decoder, a local decoder, and a global decoder. The local encoder is used to encode features of speech segments to obtain speech coding features corresponding to the speech segments. The global encoder is used to encode the initial coding features corresponding to the speech segments to obtain target coding features. The initial coding features corresponding to the speech segments are obtained based on the speech coding features corresponding to each speech segment belonging to the same speech. The basic decoder is used to decode the speech coding features output by the local encoder and also to decode the target coding features output by the global encoder. The local decoder is used to decode the speech coding features corresponding to the speech segments of the training speech based on the labeled text segments corresponding to the speech segments. The global decoder is used to decode the target coding features corresponding to the training speech based on the labeled text segments corresponding to the training speech.

[0227] In one embodiment, such as Figure 9 As shown, a speech recognition device is provided, including: a speech acquisition module 902 and a speech recognition module 904, wherein:

[0228] The voice acquisition module 902 is used to acquire the voice to be recognized.

[0229] The speech recognition module 904 is used to input the speech to be recognized into the target speech recognition model to obtain the recognized text corresponding to the speech to be recognized.

[0230] The training process of the target speech recognition model includes the following steps:

[0231] The process involves: obtaining a general speech recognition model for a common scenario and using it as the initial speech recognition model; obtaining a first and second speech training dataset for the accent scenario, and a third and fourth speech training dataset for the common scenario; each speech training dataset includes training speech and corresponding label text, where the label text is obtained by converting the training speech into text; training the initial speech recognition model using the first and third speech training datasets to obtain an intermediate speech recognition model; inputting the second and fourth speech training datasets into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech, and obtaining the target loss based on the difference between the predicted text and the label text; updating the model parameters of the initial speech recognition model based on the target loss to obtain an updated speech recognition model, using the updated speech recognition model as the initial speech recognition model, and repeating the steps of obtaining the first and second speech training datasets for the accent scenario and the third and fourth speech training datasets for the common scenario until the convergence condition is met to obtain the target speech recognition model.

[0232] The aforementioned speech recognition device features a target speech recognition model that demonstrates high accuracy in both general and accent-based scenarios. Using this target model for speech recognition can effectively improve overall accuracy. A general speech recognition model corresponding to the general scenario is used as the initial model. Training this model improves performance in accent-based scenarios without compromising its performance in general scenarios, thus achieving accent domain expansion. Meta-learning is employed for model training. First, an intermediate speech recognition model is trained using a first and a third speech training dataset. This intermediate model learns speech recognition knowledge applicable to the first and third datasets. Then, a second and a fourth speech training dataset are input into the intermediate model to calculate the target loss. This target loss reflects the effectiveness of the intermediate model's learned knowledge applied to the second and fourth datasets. The parameters of the initial speech recognition model are updated based on the target loss to obtain an updated speech recognition model. This updated model learns the variation patterns between different speech scenarios and the general knowledge shared by different speech recognition scenarios, enabling it to quickly learn relevant knowledge when faced with new speech training datasets. Through multiple iterations, the final trained target speech recognition model can learn speech recognition methods for different scenarios. The target speech recognition model is applicable to both general scenarios and various accent scenarios. It has high speech recognition accuracy for general scenarios and also has high speech recognition accuracy for various accent scenarios.

[0233] In one embodiment, the target speech recognition model includes an encoder and a decoder, the encoder including a local encoder and a global encoder. The speech recognition device is also used for:

[0234] During the process of acquiring the speech to be recognized, each acquired speech segment is input into the target speech recognition model. The local encoder and decoder obtain the corresponding recognized text segment in real time. The recognized text segments corresponding to each speech segment are displayed in sequence. After the speech acquisition is completed, the output data of each local encoder are concatenated and input into the global encoder. The global encoder and decoder obtain the overall recognized text corresponding to the speech to be recognized. The overall recognized text is displayed to cover each recognized text segment.

[0235] The various modules in the aforementioned speech recognition model training device and speech recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0236] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores speech training datasets and speech recognition models corresponding to accent scenarios and general scenarios. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a speech recognition model training method and a speech recognition method.

[0237] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a speech recognition model training method and a speech recognition method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0238] Those skilled in the art will understand that Figure 10 , 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0239] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0240] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0241] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0242] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0243] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0244] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0245] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training a speech recognition model, characterized in that, The method includes: Obtain a general speech recognition model corresponding to a common scenario, and use the general speech recognition model as the initial speech recognition model; the initial speech recognition model includes an encoder and a decoder. Obtain the first and second speech training datasets corresponding to the accent scenario, and obtain the third and fourth speech training datasets corresponding to the general scenario; each speech training dataset includes training speech and the corresponding label text, wherein the label text is obtained by converting the training speech into text. Based on the first speech training dataset and the third speech training dataset, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The second speech training dataset and the fourth speech training dataset are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text corresponding to the training speech and the label text, the target loss is obtained. Keeping the model parameters of the intermediate layers in the encoder unchanged, the model parameters of the start layer, end layer and decoder in the encoder are updated based on the target loss to obtain an updated speech recognition model; the start layer is used to learn the acoustic information of the speech signal, the intermediate layer is used to learn the contextual information of the speech signal, and the end layer is used for the feature output of the encoder. The updated speech recognition model is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and obtaining the third and fourth speech training datasets corresponding to the general scenario are executed until the convergence condition is met, and the target speech recognition model is obtained. The target speech recognition model is used to recognize speech to obtain the corresponding text.

2. The method according to claim 1, characterized in that, The step of inputting the second and fourth speech training datasets into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech, and obtaining the target loss based on the difference between the predicted text corresponding to the training speech and the labeled text, includes: In the intermediate speech recognition model, the training speech is segmented and encoded to obtain speech coding features corresponding to multiple speech segments of the training speech. Each speech coding feature is then decoded to obtain the first predicted text segment corresponding to each speech segment. Based on the labeled text corresponding to the training speech, feature decoding is performed on each speech coding feature to obtain the second predicted text segment corresponding to each speech segment. A first loss is obtained based on the difference between the first predicted text segment and the labeled text segment corresponding to the same speech segment, and a second loss is obtained based on the difference between the second predicted text segment and the labeled text segment corresponding to the same speech segment; The target loss is obtained based on the first loss and the second loss.

3. The method according to claim 2, characterized in that, The step of segmenting and encoding the training speech to obtain speech coding features corresponding to multiple speech segments of the training speech includes: Feature extraction is performed on the training speech to obtain the initial speech features corresponding to multiple speech segments of the training speech; The initial speech features are downsampled to obtain the target speech features; The target speech features are encoded to obtain the speech coding features corresponding to each speech segment.

4. The method according to claim 2, characterized in that, The process of obtaining the target loss based on the first loss and the second loss includes: The speech coding features corresponding to each speech segment are concatenated to obtain the initial coding features; The initial encoded features are feature-encoded to obtain target encoded features, and the target encoded features are feature-decoded to obtain the third predicted text; The third loss is derived from the difference between the third predicted text and the label text corresponding to the same training speech. Based on the first loss, the second loss, and the third loss, the target loss is obtained.

5. The method according to claim 4, characterized in that, The process of obtaining the target loss based on the first loss, the second loss, and the third loss includes: Based on the labeled text corresponding to the training speech, the target encoded features are decoded to obtain the fourth predicted text; The fourth loss is derived from the difference between the fourth predicted text and the label text corresponding to the same training speech. The target loss is obtained based on the first loss, the second loss, the third loss, and the fourth loss.

6. The method according to claim 2, characterized in that, The step of decoding the speech coding features based on the labeled text corresponding to the training speech to obtain the second predicted text segments corresponding to each speech segment includes: From the current label text corresponding to the current training speech, obtain the current label text segment corresponding to the current speech segment of the current training speech, and determine the target position from the word positions corresponding to the current label text segment; From the current tag text, determine a reference position that matches the target position, and obtain the tag words before the reference position as reference words; Based on reference words, the speech coding features corresponding to the current speech segment are decoded to obtain the predicted words corresponding to the target position; The process of obtaining the next word position in the current tag text segment as the target position, returning to the step of determining the reference position that matches the target position from the current tag text, and obtaining the tag word before the reference position as the reference word, continues until the termination condition is met, resulting in multiple predicted words. The second predicted text segment corresponding to the current speech segment is obtained based on each predicted word.

7. The method according to claim 1, characterized in that, The process of obtaining a general speech recognition model corresponding to a general scenario includes: Obtain the fifth speech training dataset corresponding to the general scenario; Based on the fifth speech training dataset, candidate speech recognition models are trained to obtain a general speech recognition model.

8. The method according to any one of claims 1 to 7, characterized in that, The encoder includes a local encoder and a global encoder, and the decoder includes a basic decoder, a local decoder, and a global decoder; The local encoder is used to perform feature encoding on speech segments to obtain speech coding features corresponding to the speech segments. The global encoder is used to perform feature encoding on the initial coding features corresponding to the speech to obtain target coding features. The initial coding features corresponding to the speech are obtained based on the speech coding features corresponding to each speech segment belonging to the same speech. The basic decoder is used to perform feature decoding on the speech coding features output by the local encoder, and also to perform feature decoding on the target coding features output by the global encoder. The local decoder is used to perform feature decoding on the speech coding features corresponding to the speech segments of the training speech based on the labeled text segments corresponding to the speech segments of the training speech. The global decoder is used to perform feature decoding on the target coding features corresponding to the training speech based on the labeled text corresponding to the training speech.

9. A speech recognition method, characterized in that, The method includes: Acquire the speech to be recognized; The speech to be recognized is input into the target speech recognition model to obtain the recognized text corresponding to the speech to be recognized; The training process of the target speech recognition model includes the following steps: Obtain a general speech recognition model corresponding to a common scenario, and use the general speech recognition model as the initial speech recognition model; the initial speech recognition model includes an encoder and a decoder. Obtain the first and second speech training datasets corresponding to the accent scenario, and obtain the third and fourth speech training datasets corresponding to the general scenario; each speech training dataset includes training speech and the corresponding label text, wherein the label text is obtained by converting the training speech into text. Based on the first speech training dataset and the third speech training dataset, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The second speech training dataset and the fourth speech training dataset are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text corresponding to the training speech and the label text, the target loss is obtained. Keeping the model parameters of the intermediate layers in the encoder unchanged, the model parameters of the start layer, end layer and decoder in the encoder are updated based on the target loss to obtain an updated speech recognition model; the start layer is used to learn the acoustic information of the speech signal, the intermediate layer is used to learn the contextual information of the speech signal, and the end layer is used for the feature output of the encoder. The updated speech recognition model is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and obtaining the third and fourth speech training datasets corresponding to the general scenario are executed until the convergence condition is met, and the target speech recognition model is obtained.

10. The method according to claim 9, characterized in that, The target speech recognition model includes an encoder and a decoder, and the encoder includes a local encoder and a global encoder. The method further includes: During the process of collecting the speech to be recognized, each speech segment to be recognized is input into the target speech recognition model, and the corresponding recognized text segment is obtained in real time through the local encoder and decoder. The corresponding recognized text segments for each speech segment to be recognized are displayed sequentially. After the speech to be recognized is collected, the output data of each local encoder are concatenated and input into the global encoder. The global encoder and decoder then obtain the overall recognized text corresponding to the speech to be recognized. The overall recognized text is displayed to cover each individual recognized text fragment.

11. A speech recognition model training device, characterized in that, The device includes: The model acquisition module is used to acquire a general speech recognition model corresponding to a general scenario, and use the general speech recognition model as the initial speech recognition model; the initial speech recognition model includes an encoder and a decoder. The training dataset acquisition module is used to acquire the first and second speech training datasets corresponding to the accent scenario, and the third and fourth speech training datasets corresponding to the general scenario. Each speech training dataset includes training speech and the label text corresponding to the training speech, wherein the label text is obtained by converting the training speech into text. The first model training module is used to train the initial speech recognition model based on the first speech training dataset and the third speech training dataset to obtain an intermediate speech recognition model. The second model training module is used to input the second and fourth speech training datasets into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text and the labeled text corresponding to the training speech, a target loss is obtained. Keeping the model parameters of the intermediate layers in the encoder unchanged, the model parameters of the start layer, end layer, and decoder in the encoder are updated based on the target loss to obtain an updated speech recognition model. The start layer is used to learn the acoustic information of the speech signal, the intermediate layer is used to learn the contextual information of the speech signal, and the end layer is used for the feature output of the encoder. Using the updated speech recognition model as the initial speech recognition model, the steps of obtaining the first and second speech training datasets corresponding to the accent scene, and obtaining the third and fourth speech training datasets corresponding to the general scene are executed until the convergence condition is met to obtain the target speech recognition model. The target speech recognition model is used to recognize speech to obtain the corresponding text.

12. The speech recognition model training device according to claim 11, characterized in that, The second model training module is further configured to, in the intermediate speech recognition model, segment and encode the training speech to obtain speech coding features corresponding to multiple speech segments of the training speech, decode each speech coding feature to obtain a first predicted text segment corresponding to each speech segment; based on the label text corresponding to the training speech, decode each speech coding feature to obtain a second predicted text segment corresponding to each speech segment; obtain a first loss based on the difference between the first predicted text segment and the label text segment corresponding to the same speech segment, obtain a second loss based on the difference between the second predicted text segment and the label text segment corresponding to the same speech segment; and obtain a target loss based on the first loss and the second loss.

13. The speech recognition model training device according to claim 12, characterized in that, The second model training module is also used to extract features from the training speech to obtain initial speech features corresponding to multiple speech segments of the training speech. The initial speech features are downsampled to obtain target speech features; the target speech features are then encoded to obtain speech coding features corresponding to each speech segment.

14. The speech recognition model training device according to claim 12, characterized in that, The second model training module is further configured to concatenate the speech coding features corresponding to each speech segment to obtain initial coding features; perform feature encoding on the initial coding features to obtain target coding features; perform feature decoding on the target coding features to obtain third predicted text; obtain a third loss based on the difference between the third predicted text and the label text corresponding to the same training speech; and obtain a target loss based on the first loss, the second loss, and the third loss.

15. The speech recognition model training device according to claim 14, characterized in that, The second model training module is further used to perform feature decoding on the target encoded features based on the label text corresponding to the training speech to obtain the fourth predicted text; to obtain the fourth loss based on the difference between the fourth predicted text and the label text corresponding to the same training speech; and to obtain the target loss based on the first loss, the second loss, the third loss and the fourth loss.

16. The speech recognition model training device according to claim 12, characterized in that, The second model training module is further configured to obtain the current label text segment corresponding to the current speech segment of the current training speech from the current label text corresponding to the current training speech, determine the target position from each word position corresponding to the current label text segment, determine the reference position matching the target position from the current label text, and obtain the label words before the reference position as reference words; Based on reference words, the speech coding features corresponding to the current speech segment are decoded to obtain the predicted words corresponding to the target position; The next word position in the current labeled text segment is obtained as the target position. The steps of determining the reference position that matches the target position from the current labeled text and obtaining the labeled word before the reference position as the reference word are executed until the termination condition is met, resulting in multiple predicted words. Based on each predicted word, the second predicted text segment corresponding to the current speech segment is obtained.

17. The speech recognition model training device according to claim 11, characterized in that, The model acquisition module is also used to acquire the fifth speech training dataset corresponding to the general scenario; based on the fifth speech training dataset, the candidate speech recognition model is trained to obtain the general speech recognition model.

18. The speech recognition model training apparatus according to any one of claims 11 to 17, characterized in that, The encoder includes a local encoder and a global encoder, and the decoder includes a basic decoder, a local decoder, and a global decoder; The local encoder is used to perform feature encoding on speech segments to obtain speech coding features corresponding to the speech segments. The global encoder is used to perform feature encoding on the initial coding features corresponding to the speech to obtain target coding features. The initial coding features corresponding to the speech are obtained based on the speech coding features corresponding to each speech segment belonging to the same speech. The basic decoder is used to perform feature decoding on the speech coding features output by the local encoder, and also to perform feature decoding on the target coding features output by the global encoder. The local decoder is used to perform feature decoding on the speech coding features corresponding to the speech segments of the training speech based on the labeled text segments corresponding to the speech segments of the training speech. The global decoder is used to perform feature decoding on the target coding features corresponding to the training speech based on the labeled text corresponding to the training speech.

19. A voice recognition device, characterized in that, The device includes: The speech acquisition module is used to acquire the speech to be recognized; The speech recognition module is used to input the speech to be recognized into the target speech recognition model to obtain the recognized text corresponding to the speech to be recognized. The training process of the target speech recognition model includes the following steps: Obtain a general speech recognition model corresponding to a common scenario, and use the general speech recognition model as the initial speech recognition model; the initial speech recognition model includes an encoder and a decoder. Obtain the first and second speech training datasets corresponding to the accent scenario, and obtain the third and fourth speech training datasets corresponding to the general scenario; each speech training dataset includes training speech and the corresponding label text, wherein the label text is obtained by converting the training speech into text. Based on the first speech training dataset and the third speech training dataset, the initial speech recognition model is trained to obtain an intermediate speech recognition model. The second speech training dataset and the fourth speech training dataset are input into the intermediate speech recognition model to obtain the predicted text corresponding to the training speech. Based on the difference between the predicted text corresponding to the training speech and the label text, the target loss is obtained. Keeping the model parameters of the intermediate layers in the encoder unchanged, the model parameters of the start layer, end layer and decoder in the encoder are updated based on the target loss to obtain an updated speech recognition model; the start layer is used to learn the acoustic information of the speech signal, the intermediate layer is used to learn the contextual information of the speech signal, and the end layer is used for the feature output of the encoder. The updated speech recognition model is used as the initial speech recognition model. The steps of obtaining the first and second speech training datasets corresponding to the accent scenario, and obtaining the third and fourth speech training datasets corresponding to the general scenario are executed until the convergence condition is met, and the target speech recognition model is obtained.

20. The speech recognition device according to claim 19, characterized in that, The target speech recognition model includes an encoder and a decoder, and the encoder includes a local encoder and a global encoder. The speech recognition module is also used to input the target speech recognition model into each speech segment acquired during the acquisition of speech to be recognized, and obtain the corresponding recognition text segment in real time through the local encoder and decoder; sequentially display the recognition text segments corresponding to each speech segment to be recognized; after the acquisition of speech to be recognized is completed, the output data of each local encoder are concatenated and input into the global encoder, and the overall recognition text corresponding to the speech to be recognized is obtained through the global encoder and decoder; and the overall recognition text is displayed to cover each recognition text segment.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images