Speech recognition method, speech recognition model training method, device, and medium

Abstract

This application discloses a speech recognition method, a speech recognition model training method, an apparatus, and a medium. The method includes acquiring a speech to be recognized and acquiring a trained speech recognition model, the speech recognition model including an encoding network and a decoding network. At each stage of encoding the speech to be recognized using the encoding network, the speech to be recognized is first classified by a target speech attribute to obtain a predicted attribute category to which the speech to be recognized belongs, then encoded based on the predicted attribute category of the target speech attribute, obtaining a first encoding feature, and decoding the first encoding feature based on the decoding network to obtain a recognized text of the speech to be recognized. The speech recognition model is adjusted based on at least a first loss, the first loss characterizing the difference between a preset attribute category of a sample speech label with the target speech attribute and the attribute category of a sample recognized by the speech recognition model. In this way, the application can improve speech recognition accuracy while reducing costs.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority from a Chinese patent application entitled "Speech Recognition Method, Speech Recognition Model Training Method, Apparatus, and Medium" filed on April 25, 2023, with application number CN202310460643.3, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a speech recognition method, a method, an apparatus and a medium for training a speech recognition model. [Background technology]

[0003] Automatic speech recognition (ASR), also known as speech recognition, refers to the process by which a computer processor receives and processes speech signals, converting them into textual information that is understandable to humans. This technology has been widely applied in mobile phone voice assistants, input method software, car navigation systems, and various AI wearable devices, and has significant application value. Mixture of Experts (MoE) is currently a popular field in deep learning. Deep learning models can expand the number of model parameters while maintaining the original level of computational complexity, significantly improving the overall effectiveness of the model.

[0004] In the training process of a conventional speech recognition model based on a mixture of experts, samples are randomly assigned to different experts for processing, and the training process is unsupervised, meaning that model developers cannot clearly determine the characteristics of the samples assigned to each expert, nor can they clearly determine the number of experts that need to be configured. As a result, a large number of samples and experts are often required for training, resulting in very high training costs. Furthermore, because unsupervised training is used, samples are randomly assigned to each expert for processing, and the accuracy of the features obtained by randomly assigning samples to each expert for processing is lower than when samples are randomly assigned to experts with corresponding attributes for processing, resulting in low speech recognition accuracy of the speech recognition model. Summary of the Invention

[0005] The main technical problem that the present application aims to solve is to provide a speech recognition method, a speech recognition model training method, an apparatus, and a medium that can improve the accuracy of speech recognition while reducing costs.

[0006] In order to solve the above technical problems, a first aspect of the present application provides a speech recognition method, the method including the steps of: acquiring a recognition-target speech; and acquiring a trained speech recognition model, where the speech recognition model includes an encoding network and a decoding network; and at each stage of encoding the recognition-target speech using the encoding network, first classifying the recognition-target speech by a target speech attribute to obtain a predicted attribute category to which the recognition-target speech belongs, and then encoding based on the predicted attribute category of the target speech attribute to obtain first encoding features; and decoding the first encoding features based on the decoding network to obtain recognized text of the recognition-target speech, where the speech recognition model is adjusted based on at least a first loss, and the first loss characterizes, with the target speech attribute, the difference between a preset attribute category of a sample speech label and a sample attribute category recognized by the speech recognition model and obtained.

[0007] In order to solve the above technical problems, a second aspect of the present application provides a training method for a speech recognition model, the method including: a step of obtaining a sample speech; at each stage of encoding the sample speech using an encoding network of the speech recognition model, first classifying the sample speech by a target speech attribute to obtain a sample attribute category to which the sample speech belongs, and then encoding based on the sample attribute category of the target speech attribute to obtain a first sample encoding feature; a step of decoding the first sample encoding feature using a decoding network of the speech recognition model to obtain a recognized text of the sample speech; a step of determining a first loss based on the difference between the sample attribute category to which the sample speech belongs and the preset attribute category of the sample speech label, and determining a recognition loss based on the difference between the recognized text of the sample speech and the preset text of the sample speech label; and a step of adjusting network parameters of the speech recognition model based on at least the first loss and the recognition loss.

[0008] To solve the above technical problem, a third aspect of the present application provides an electronic device, the device including a memory and a processor coupled to each other, wherein program instructions are stored in the memory, and the processor is used to execute the program instructions stored in the memory, thereby realizing the method provided by the first or second aspect.

[0009] To solve the above technical problem, a fourth aspect of the present application provides a computer-readable storage medium, the computer-readable storage medium being used to store program instructions, and the program instructions being executed to realize the method provided by the first or second aspect.

[0010] The beneficial effects of the present application are as follows: Different from the state of the art, the present application adjusts the network parameters of the speech recognition model based on at least a first loss during the training phase of the speech recognition model, and the first loss characterizes the difference between the preset attribute categories of the sample speech labels and the attribute categories of the samples recognized by the speech recognition model in the target speech attribute, that is, the present application can first determine the target speech attribute and the preset attribute categories during the training phase, that is, the model developer can clarify the characteristics of the samples assigned to each expert and the number of experts that need to be set, and can use a supervised method to train the speech recognition model using fewer samples and experts, thereby realizing cost reduction.

[0011] Furthermore, at each stage of encoding the speech to be recognized using the encoding network of the trained speech recognition model, the speech to be recognized can be first classified by the target speech attribute to obtain a predicted attribute category to which the speech to be recognized belongs, then encoded based on the predicted attribute category of the target speech attribute to obtain first encoded features, and finally decoded based on the decoding network of the trained speech recognition model to obtain recognized text of the speech to be recognized. In the encoding process, the predicted attribute category is first determined, and then encoded based on the predicted attribute category to obtain more accurate first encoded features, further improving the speech recognition accuracy of the speech recognition model. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a flowchart of a first embodiment of a speech recognition method provided by the present application; [Figure 2] 1 is a frame schematic diagram of one embodiment of a coding network provided by the present application; [Figure 3] 4 is a flowchart of a second embodiment of a speech recognition method provided by the present application; [Figure 4]FIG. 2 is a schematic frame diagram of another embodiment of a coding network provided by the present application; [Figure 5] 1 is a flowchart of a first embodiment of a method for training a speech recognition model provided by the present application; [Figure 6] 1 is a flowchart of a second embodiment of a method for training a speech recognition model provided by the present application. [Figure 7] 1 is a frame schematic diagram of an embodiment of a speech recognition device provided by the present application; [Figure 8] 1 is a frame schematic diagram of an embodiment of a speech recognition model training device provided by the present application; [Figure 9] 1 is a frame structure schematic diagram of an embodiment of an electronic device provided by the present application; [Figure 10] 1 is a frame schematic diagram of one embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, the technical solutions of the embodiments of the present application will be clearly and completely described with reference to the drawings of the embodiments of the present application. It should be understood that the described embodiments are only a part of the embodiments of the present application, but not all of them. Based on the embodiments of the present application, all other embodiments that a person skilled in the art can obtain without inventive efforts fall within the scope of protection of the present application.

[0014] In the embodiments of the present application, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood as indicating or implying their relative importance or the number of technical features shown. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one feature.

[0015] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described with reference to the embodiment may be included in at least one embodiment of the present application. The appearance of such a combination in various places in the specification does not necessarily refer to the same embodiment, nor does it mean that the embodiments are mutually exclusive, independent, or alternative embodiments. As will be understood by those skilled in the art, both explicitly and implicitly, the embodiments described herein can be combined with other embodiments.

[0016] 1 and 2, Fig. 1 is a flowchart of a first embodiment of a speech recognition method provided by the present application, and Fig. 2 is a frame schematic diagram of an embodiment of a coding network provided by the present application. The method includes the following steps:

[0017] S11: Obtain a speech to be recognized, and obtain a trained speech recognition model.

[0018] In one embodiment, the speech recognition model may include an encoding network and a decoding network. For example, the speech recognition model is a model based on a transformer or a conformer (convolution-augmented transformer). The encoding network may include multiple first network blocks connected in series. The multiple first network blocks may be associated with target speech attributes. The target speech attributes may include language, phonemes, visual field, importance, etc. The target speech attributes may be set by a user and are not specifically limited herein. It should be understood that the multiple first network blocks may be associated with the same target speech attribute or different target speech attributes. Each of the multiple first network blocks is used to perform a different stage of encoding. In a specific embodiment, the multiple first network blocks may be divided into at least one network group, and each first network block in the same network group is associated with the same target speech attribute. Assume that the speech recognition model is a transformer model including 12 first network blocks. The shallower layers of the model often contain more phoneme information, while the deeper layers contain more speech information. Based on this, the first network blocks in the shallow layer can be associated with phonemes, for example, 1 to 3 first network blocks are divided into a first grid group, and the first grid group is associated with phonemes, 4 to 5 first network blocks are divided into a second grid group, and the first grid group is associated with importance, 6 to 8 first network blocks are divided into a third grid group, and the third grid group is associated with the visual field of interest, and 9 to 12 first network blocks are divided into a fourth grid group, and the fourth grid group is associated with language.

[0019] The decoding network can also include multiple second network blocks connected in series. Similarly, the second network blocks are associated with the same target speech attribute, and the multiple second network blocks may be associated with the same target speech attribute or different target speech attributes. The multiple second network blocks are used to perform each stage of any decoding time. It is understood that the multiple second network blocks may be completely the same as or different from the multiple first network blocks.

[0020] S12: At each stage of encoding the speech to be recognized using the encoding network, the speech to be recognized is first classified by the target speech attribute to obtain the predicted attribute category to which the speech to be recognized belongs, and then encoded based on the predicted attribute category of the target speech attribute to obtain the first encoding feature.

[0021] In one embodiment, the encoding network includes a plurality of sequentially connected first network blocks, each of which is used to perform a respective step of encoding speech to be recognized. For example, the speech recognition model is a transformer-based model, and the transformer may include a plurality of transformer blocks (i.e., first network blocks). Each of the first network blocks may include a first classification layer and a plurality of first expert layers. The first classification layer is used to classify speech based on a target speech attribute. Specifically, the first classification layer may obtain a first probability that the speech to be recognized belongs to each of the predetermined attribute categories of the target speech attribute through classification. Here, each of the predetermined attribute categories may be set according to the target speech attribute. For example, if the target speech attribute is language, each of the predetermined attribute categories for the language may include Chinese, English, etc.; if the target speech attribute is phonemes, each of the predetermined attribute categories for the language may include one-tone final, two-tone final, three-tone final, four-tone final, etc. For another example, if the target speech attribute is a visual field, each preset attribute category for a language can include a long visual field and a short visual field; if the target speech attribute is importance, each preset attribute category for a language can include an important frame and an unimportant frame. Each of the multiple first expert layers corresponds one-to-one to each preset attribute category for the target speech attribute. For example, if the target speech attribute is language and includes two preset attribute categories, Chinese and English, a first expert layer can be set corresponding to each preset attribute category, such as a Chinese expert layer and an English expert layer. Each of the first expert layers is used to perform coding based on the predicted attribute category for the target speech attribute.

[0022] Specifically, a first network block corresponding to the current stage can be selected as the first target network block. For example, if the first network block corresponding to the current stage is the first first network block, the first first network block can be selected as the first target network block. Furthermore, if the first network block corresponding to the current stage is the last first network block, the last first network block can be selected as the first target network block. Classification is performed using a first classification layer in the first target network block, and a first probability that the target speech belongs to each preset attribute category with the target speech attribute is obtained. A predicted attribute category to which the target speech belongs is determined based on the first probability that the target speech belongs to each preset attribute category. Next, a first expert layer corresponding to the predicted attribute category in the first target network block is selected as the first target expert layer, and encoding is performed using the first target expert layer to obtain a first encoded feature.

[0023] In one embodiment, if the first network block corresponding to the current stage is the first first network block in the encoding network, the first classification layer of the first first network block is used to classify the initial features of the speech to be recognized, and a first probability that the speech to be recognized belongs to each preset attribute category of the target speech attribute is obtained. The preset attribute category corresponding to the largest first probability is selected as the predicted attribute category to which the speech to be recognized belongs. Furthermore, a first expert layer corresponding to the predicted attribute category in the first first network block is selected as the first target expert layer. The initial features of the speech to be recognized are encoded using the first target expert layer to obtain first encoded features. Here, the speech recognition model further includes an embedding layer and an attention layer, and after the embedding layer and the attention layer process the speech to be recognized, the initial features of the speech to be recognized are obtained. The first coding feature obtained by the first first network block is input into the second first network block, so that the second first network block performs the same steps as the first first network block and obtains the first coding feature output by the second first network block, and this is repeated until the first coding feature output by the last first network block is obtained, and the first coding feature output by the last first network block is used as the first coding feature finally output by the coding network, or the first coding feature output by the last first network block is subjected to residual processing and normalization operations before obtaining the first coding feature finally output by the coding network.

[0024] In another embodiment, for a first network block corresponding to each stage, a first classification layer in the first network block is used to classify the initial features of the speech to be recognized, a first probability that the speech to be recognized belongs to each preset attribute category based on the target speech attribute is obtained, and the preset attribute category corresponding to the largest first probability is selected as the predicted attribute category to which the speech to be recognized belongs. A first expert layer corresponding to the predicted attribute category in the first network block is selected, and the initial features of the speech to be recognized are used to encode the first encoded features. The first encoded features obtained in each first network block are fused to obtain the first encoded features finally output by the encoding network.

[0025] In a specific embodiment, as shown in Figure 2, the coding network includes a self-attention layer, a first residual calculation and normalization layer, a first network block, and a second residual calculation and normalization layer, which are connected in sequence. After the initial features of the speech to be recognized are coded by the coding network, the first coded features can be obtained. Here, the first network block includes a first classification layer and a plurality of first expert layers.

[0026] S13: Decode the first encoded feature based on the decoding network to obtain the recognized text of the speech to be recognized.

[0027] In one embodiment, the decoding network may employ an autoregressive decoder, a transformer decoder, or an attention-based decoder.

[0028] In a specific embodiment, the decoding network uses a Transformer model decoder, and the decoding network includes an attention layer, an interactive attention processing layer, a feedforward neural network, a fully connected layer, and a normalization layer connected in sequence. The inputs of the decoding network include the decoded characters obtained at each previous decoding time point and the first encoding features output by the encoding network. The decoding network uses the attention layer to perform attention processing on the features of the decoded characters obtained at each previous decoding time point to obtain a first feature vector. The first feature vector and the first encoding features are used to perform interactive attention processing to obtain a second feature vector. The second feature vector passes through the feedforward neural network, the fully connected layer, and the normalization layer in sequence to obtain the decoded character at the current time point.

[0029] In another specific embodiment, the decoding network further includes a plurality of second network blocks connected in sequence, and the second network blocks may be placed before the interactive attention layer, that is, after using a plurality of second network blocks to process the features of the decoded characters obtained at each previous decoding point, a third feature vector is obtained, and interactive attention processing is performed using the third feature vector and the first encoding feature to obtain a second feature vector.

[0030] Specifically, each of the second network blocks is associated with a target speech attribute. As can be understood, each second network block may be associated with the same target speech attribute or different target speech attributes. The second network block includes a second classification layer that performs classification based on the target speech attribute and a second expert layer that has a one-to-one correspondence with each preset attribute category of the target speech attribute, and the second expert layer is used to perform decoding based on each preset attribute category of the target speech attribute. Different second network blocks are used to perform different stages at any given decoding time.

[0031] At any decoding time point, a second network block corresponding to the current stage at that decoding time point is selected as a second target network block. For example, at the first stage of the second decoding time point, the second network block corresponding to the first stage is the first second network block in the decoding network, and the first second network block is designated as the second target network block. A second classification layer in the second target network block is used to classify the decoded characters obtained by decoding at each previous decoding time point, and a second probability that each decoded character belongs to each predetermined attribute category is obtained. For example, at the first decoding time point, the decoded characters obtained by decoding at each previous decoding time point include only the start character. For example, at the second decoding time point, the decoded characters obtained by decoding at each previous decoding time point include the decoded character obtained at the first decoding time point and the start character. A second expert layer for decoding the decoded characters in the second target network block is determined based on the second probability that each decoded character belongs to each predetermined attribute category, and designated as the second target expert layer. In one embodiment, a second network layer corresponding to a predetermined attribute category with the largest second probability is selected as the second target expert layer. In another embodiment, any one of the second expert layers corresponding to the preset attribute categories corresponding to each second probability greater than the preset probability may be selected as the second target expert layer. The second target expert layer is used to decode the decoded character to obtain the first decoding feature. If the decoding network includes only one second network block, the second network block outputs the first decoding feature, and then performs decoding based on the first encoding feature and the first decoding feature to obtain the decoded character at the current decoding point.When the decoding network includes multiple second network blocks, after the second network block corresponding to the current stage outputs the first decoding feature, the first decoding feature is input to the second network block corresponding to the next stage, so that the second network block corresponding to the next stage performs the same steps as the second network block corresponding to the current stage, obtains the first decoding feature output by the second network block corresponding to the next stage, and repeats this until the second network block corresponding to the last stage, performs decoding based on the first decoding feature and the first encoding feature output by the second network block corresponding to the last stage, and obtains the decoded character at the current decoding point.

[0032] Using the above method, the decoded characters can be obtained at each decoding point and then combined to obtain the recognized text.By installing multiple second network blocks in the decoding network, the overall scale of the speech recognition model can be greatly expanded, further improving the computational efficiency of the model.

[0033] In the above embodiment, the speech recognition model is adjusted based on at least a first loss, which characterizes the difference between the preset attribute categories of the sample speech labels and the sample attribute categories recognized by the speech recognition model using the target speech attribute. Specifically, during the training process, each first classification layer in the encoding network can obtain sample attribute categories obtained by classification using the target speech attribute. The first loss is determined based on the difference between the sample attribute categories and the preset attribute categories of the sample speech labels, and the network parameters of the speech recognition model are adjusted based on at least the first loss. In another embodiment, the speech recognition model is also adjusted based on a second loss, which is determined based on the following factors: the proportion of sample characters in the sample text labeled by the sample speech that belong to each preset attribute category and the average probability that the sample characters belong to each preset attribute category using the target speech attribute. The training process of the speech recognition model is not described in detail here; see below for a detailed description.

[0034] Using the above method, speech recognition can be performed to obtain recognized text, and during the training phase, the process of sample attribute categories recognized and obtained by the speech recognition model is trained in a supervised manner, so that the model developer can clarify the sample characteristics assigned to each expert and the number of experts that need to be set, and the speech recognition model can be trained using fewer samples and experts, thereby reducing costs.Furthermore, the trained speech recognition model is used to perform speech recognition, and during the encoding process, predicted attribute categories are first determined, and then encoding is performed based on the predicted attribute categories, resulting in a first encoding feature with a higher accuracy rate, further improving the accuracy rate of speech recognition of the speech recognition model.

[0035] 3 and 4, Fig. 3 is a flowchart of a second embodiment of the speech recognition method provided by the present application, and Fig. 4 is a frame schematic diagram of another embodiment of the coding network provided by the present application. The method includes the following steps:

[0036] S31: Acquire a speech to be recognized, and acquire a trained speech recognition model.

[0037] Here, the speech recognition model includes an encoding network and a decoding network.

[0038] S32: Select the first network block corresponding to the current stage as the first target network block.

[0039] S33: Classify using the first classification layer in the first target network block, and obtain a first probability that the speech to be recognized belongs to each preset attribute category with the target speech attribute.

[0040] S34: Determine a predicted attribute category to which the recognition target speech belongs based on a first probability that the recognition target speech belongs to each of the predetermined attribute categories.

[0041] S35: Select a first expert layer corresponding to the predicted attribute category in the first target network block as a first target expert layer.

[0042] S36: Perform encoding using the first target expert layer to obtain first encoded features.

[0043] Steps (S31) to (S36) can refer to the first embodiment of the speech recognition method provided by the present application, and will not be described again here.

[0044] S37: Perform coding using the shared expert layer to obtain second coding features.

[0045] Furthermore, as shown in Figure 4, the coding network can further include a shared expert layer. Specifically, a shared expert layer can be arranged in each first network block. The shared expert layer is used to perform coding processing on the initial features of the speech to be recognized input to the first network block or the first coding features output by the previous first network block to obtain second coding features.

[0046] S38: The first encoding feature and the second encoding feature are fused to obtain the first encoding feature finally output by the first target network block.

[0047] The first coding feature output by the first target expert layer and the second coding feature output by the shared expert layer in the first network block corresponding to the current stage are fused to obtain the first coding feature finally output by the first target network block, where the fusion of the first coding feature and the second coding feature can be achieved by adding or concatenating the first coding feature and the second coding feature.

[0048] S39: Decode the first encoded feature based on the decoding network to obtain the recognized text of the speech to be recognized.

[0049] The detailed embodiment of step (S39) may refer to step (S13) of the first embodiment of the speech recognition method provided by the present application, and will not be described again here.

[0050] As can be seen, a shared expert layer can also be added to the decoding network. Specifically, a shared expert layer can be configured in each second network block. The shared expert layer is used to perform decoding processing on the initial features of the decoded character input to the second network block or the second decoded features output by the previous second network block to obtain the second decoded features. The first decoded features output by the second target expert layer in the second network block corresponding to the current stage are fused with the second decoded features output by the shared expert layer to obtain the first decoded features finally output by the second target network block. Here, the step of encoding the shared expert layer can be performed simultaneously with the step of encoding the first target expert layer, or can be performed before the step of encoding the first target expert layer.

[0051] In this embodiment, the model developer presets target speech attributes, preset attribute categories, and expert layers that correspond one-to-one to the preset attribute categories, so that a certain expert layer is assigned with less speech data that needs to be processed, resulting in the speech recognition model having insufficient fitting ability to the preset attribute categories of the speech data.Therefore, a shared expert layer is added to the encoding network and / or decoding network, so that each speech data entering the network block passes through the shared expert layer, and the shared expert layer sees each feature entering the network block, so that it has strong fitting ability, thereby compensating for the poor fitting ability caused by uneven allocation of speech data.

[0052] Referring to FIG. 5, FIG. 5 is a flowchart of a first embodiment of a method for training a language identification model provided in the present application, which includes the following steps:

[0053] S51: Acquire a sample voice.

[0054] S52: At each stage of encoding a sample voice using the encoding network of the speech recognition model, first classify the sample voice by the target voice attribute to obtain the sample attribute category to which the sample voice belongs, and then encode it based on the sample attribute category of the target voice attribute to obtain the first sample encoding feature.

[0055] S53: Using the decoding network of the speech recognition model, decode the coded features of the first sample to obtain the recognized text of the sample speech.

[0056] Here, the process by which the encoding network encodes the sample speech is the same as the process by which the encoding network encodes the speech to be recognized, and the process by which the decoding network decodes the first sample encoding feature is the same as the process by which the decoding network decodes the first encoding feature, and therefore will not be repeated here.

[0057] During the training phase of the speech recognition model, each sample speech is labeled with a preset attribute category. As can be seen, one sample speech may be labeled with multiple preset attribute categories, such as Chinese, one-tone vowels, etc. In one embodiment, a first classification layer in a first network block of the encoding network classifies target speech attributes to obtain sample attribute categories for the sample speech. As can be seen, some first classification layers may be configured with learning parameters, i.e., network parameters of the speech recognition model, while some first classification layers may not be configured with learning parameters. For example, the language classification layer and phoneme classification layer may be configured with learning parameters, while the attention field classification layer and importance classification layer may not be configured with learning parameters.

[0058] For example, if the target speech attribute is language, the first classification layer in the first network block related to language may be a language classification layer, that is, the first classification layer is used to determine the language of the sample speech, and the calculation process of the language classification layer is as follows (Equation 1):

[0059] JPEG2026505793000002.jpg27170

[0060] JPEG2026505793000003.jpg76170

[0061] For example, the target speech attribute is a phoneme. The first classification layer in the first network block related to the phoneme may be a phoneme classification layer, that is, the first classification layer is used to determine the phoneme of the sample speech, and the calculation process of the phoneme classification layer is as follows (Equation 2):

[0062] JPEG2026505793000004.jpg28170

[0063] JPEG2026505793000005.jpg76170

[0064] For example, if the target audio attribute is a field of attention, the first classification layer in the first network block associated with the field of attention may be a field of attention classification layer, i.e., the first classification layer is used to determine the attention of a certain frame of a sample audio to other frames. Specifically, a self-attention module is included in the Transformer-based model, and the attention of a certain frame to other frames can be calculated based on the score value calculated by the self-attention module. The attention of a certain frame within a short field of view can be obtained by calculating the sum of the attention scores of the certain frame to its neighboring frames, and an attention threshold is set to distinguish the length of the field of view of a certain frame. A frame with a long field of view contains some unique information and can calculate a high score compared to distant frames, while a frame with a short field of view always contains only its own information and has a small calculated score. Therefore, an attention threshold can be set, and it is determined whether the attention of a frame is greater than the attention threshold. If the attention is greater than the threshold, the frame is considered to have a long attention field, and the frame is assigned to the long-field expert layer for processing; if the attention is less than the threshold, the frame is considered to have a short attention field, and the frame is assigned to the short-field expert layer for processing.

[0065] For example, if the target speech attribute is importance, the first classification layer in the first network block associated with importance may be an importance classification layer, i.e., the first classification layer is used to determine the importance of a frame of a sample speech. Specifically, the sum of the attention scores of a speech frame to each other speech frame within a predetermined range can be calculated. If the sum of the attention scores is greater than a predetermined importance threshold, the speech frame is considered to be an important frame and is assigned to the important frame expert layer for processing; if the attention score is less than the predetermined importance threshold, the speech frame is considered to be an unimportant frame and is assigned to the unimportant frame expert layer for processing.

[0066] S54: Determine a first loss based on the difference between the sample attribute category to which the sample speech belongs and the preset attribute category of the sample speech label, and determine a recognition loss based on the difference between the recognized text of the sample speech and the preset text of the sample speech label.

[0067] For each first classification layer with the learning parameters set, a first loss is determined by using the difference between the sample attribute category to which the sample speech obtained by the classification belongs and the preset attribute category of the sample speech label, and the first loss may specifically be a cross-entropy loss. As can be understood, if the encoding network includes multiple first classification layers with learning parameters, multiple first losses will be obtained.

[0068] S55: Adjusting network parameters of a speech recognition model based on at least the first loss and the recognition loss.

[0069] The sum of the multiple first losses and the recognition loss is defined as a total loss, and the network parameters of the speech recognition model are adjusted based on the total loss.

[0070] Referring to FIG. 6, FIG. 6 is a flowchart of a second embodiment of a speech recognition model training method provided by the present application, which includes the following steps:

[0071] S61: Acquire a sample audio.

[0072] S62: At each stage of encoding a sample voice using the encoding network of the speech recognition model, first classify the sample voice by the target voice attribute to obtain the sample attribute category to which the sample voice belongs, and then encode it based on the sample attribute category of the target voice attribute to obtain the first sample encoding feature.

[0073] S63: Decode the first sample encoded feature using the decoding network of the speech recognition model to obtain the recognized text of the sample speech.

[0074] Here, the process of the encoding network encoding the sample speech is the same as the process of the encoding network encoding the speech to be recognized, and the process of the decoding network decoding the first sample encoding feature is the same as the process of the decoding network decoding the first encoding feature, so they will not be described again here.Specific embodiments of steps (S61) to (S63) can refer to steps (S51) to (S53) in the first embodiment of the speech recognition model training method provided by the present application, and descriptions thereof will be omitted here.

[0075] S64: Obtain the proportion of sample characters in the sample text labeled by the sample voice that belong to each of the preset attribute categories, and the average probability that the sample characters belong to each of the preset attribute categories.

[0076] In one embodiment, the decoding network further includes a plurality of second network blocks connected in series, and a third loss function is set to constrain the assignment of each decoded character to each second expert layer in the second network block so as to make the decoded characters assigned to each second expert layer as uniform as possible. The third loss function is shown in the following equation (3).

[0077] JPEG2026505793000006.jpg21170

[0078] JPEG2026505793000007.jpg47170

[0079] In one embodiment, the sample characters in the sample text labeled with the sample audio may belong to the same training set. The total number of sample characters in the training set and a first number of sample characters belonging to each preset attribute category are obtained, and the ratio of the first number to the total number is used as the proportion of the sample characters in the sample text labeled with the sample audio that belong to each preset attribute category. For example, if the total number of sample characters is 10 and the first number of sample characters belonging to preset attribute category A is 3, the proportion of the sample characters that belong to preset attribute category A is 3 / 10. The probability that each sample character belongs to each preset attribute category is obtained, and the probabilities that each sample character belongs to the same preset attribute category are averaged to obtain an average probability that the sample characters belong to the preset attribute category. For example, the probability that sample character 1 belongs to preset attribute category A is 1 / 3, and the probability that sample character 2 belongs to preset attribute category A is 1 / 4. By averaging these two probabilities, the average probability that the sample characters belong to preset attribute category A is obtained.

[0080] S65: Determine a first loss based on the difference between the sample attribute category to which the sample voice belongs and the preset attribute category of the sample voice label, and determine a recognition loss based on the difference between the recognized text of the sample voice and the preset text of the sample voice label, and determine a second loss based on the proportion and average probability.

[0081] In one embodiment, the step of determining the second loss based on the proportion and the average probability includes obtaining a probability parameter that is a product of a proportion of each sample character belonging to the same preset attribute category and an average probability of each sample character belonging to the same preset attribute category, and obtaining a second loss that is a product of the sum of each first parameter and the number of sample attribute categories.

[0082] S66: Adjust network parameters of the speech recognition model based on the first loss, the second loss, and the recognition loss.

[0083] In one embodiment, the first loss, the second loss, and the recognition loss may be summed to obtain a total loss, and the network parameters of the speech recognition model are adjusted based on the total loss.

[0084] During the training process of the speech recognition model, the encoding network side of the speech recognition model is trained using a supervised method. That is, for each first classification layer with set learning parameters, the first loss is determined based on the difference between the sample attribute category to which the sample speech obtained by classification belongs and the preset attribute category of the sample speech label. The decoding network side of the speech recognition model is trained using an unsupervised method with constraints. That is, each second network block included in the decoding network is classified by the target speech attribute to obtain the sample attribute category to which the sample speech belongs, and decoding is performed based on the sample attribute category for the target speech attribute. In this process, a third loss function is introduced to ensure that the number of sample speeches assigned to each second expert layer is as uniform as possible. Furthermore, the recognition loss is determined based on the difference between the recognized text of the sample speech and the preset text of the sample speech label. Finally, the parameters of the speech recognition model are adjusted based on the recognition loss, the first loss, and the second loss. Through this training, the speech recognition model can maintain good text recognition results even when a small number of second expert layers are set.

[0085] Further, FIG. 7 is a schematic frame diagram of an embodiment of a speech recognition device provided by the present application. The speech recognition device 70 includes a first acquisition module 71, a second encoding module 72, and a first decoding module 73. The first acquisition module 71 is used to acquire speech to be recognized and obtain a trained speech recognition model. Here, the speech recognition model includes an encoding network and a decoding network. At each stage of encoding the speech to be recognized using the encoding network, the first encoding module 72 is used to first classify the speech to be recognized by target speech attributes to obtain a predicted attribute category to which the speech to be recognized belongs, and then encode the speech to be recognized based on the predicted attribute category of the target speech attributes to obtain first encoded features. The first decoding module 73 is used to decode the first encoded features using the decoding network to obtain recognized text of the speech to be recognized. Here, the speech recognition model is adjusted based on at least a first loss, which characterizes the difference between the predetermined attribute category of the sample speech label and the sample attribute category recognized by the speech recognition model using the target speech attribute.

[0086] In the above method, during the training phase of the speech recognition model, the network parameters of the speech recognition model are adjusted based on at least a first loss, where the first loss is a target speech attribute that characterizes the difference between the preset attribute category of the sample speech label and the attribute category of the sample obtained by recognition by the speech recognition model. That is, during the training phase, the present application can first determine the target speech attribute and the preset attribute category, that is, the model developer can clarify the characteristics of the samples assigned to each expert and the number of experts that need to be set, and a supervised method can be used to train the speech recognition model using fewer samples and experts, thereby reducing costs.

[0087] Furthermore, at each stage of encoding the speech to be recognized using the encoding network of the trained speech recognition model, the speech to be recognized is first classified by the target speech attribute to obtain the predicted attribute category to which the speech to be recognized belongs, then encoded based on the predicted attribute category of the target speech attribute to obtain first encoded features, and finally the first encoded features are decoded based on the decoding network of the trained speech recognition model to obtain recognized text of the speech to be recognized. In the encoding process, the predicted attribute category is first determined, and then encoded based on the predicted attribute category to obtain first encoded features with high accuracy, further improving the speech recognition accuracy of the speech recognition model.

[0088] In one embodiment, the encoding network includes a plurality of first networks connected in series and associated with a target speech attribute, each of which is used to perform a different stage of encoding. The first network block associated with the target speech attribute includes a first classification layer for classification based on the target speech attribute and a first expert layer having a one-to-one correspondence with each preset attribute category of the target speech attribute. Each of the first expert layers is used to encode based on the predicted attribute category of the target speech attribute.

[0089] In one embodiment, before classifying by target speech attributes to obtain a predicted attribute category to which the speech to be recognized belongs, the method further includes using a selection module of the speech recognition device 70 to select a first network block corresponding to the current stage as a first target network block.

[0090] The step of classifying the speech to be recognized by the target speech attribute to obtain a predicted attribute category to which the speech to be recognized belongs includes: classifying the speech to be recognized by using a first classification layer in a first target network block to obtain a first probability that the speech to be recognized belongs to each predetermined attribute category based on the target speech attribute; and determining a predicted attribute category to which the speech to be recognized belongs based on the first probability that the speech to be recognized belongs to each predetermined attribute category.

[0091] The step of performing encoding based on the predicted attribute category in the target speech attribute and obtaining a first encoding feature includes: selecting a first expert layer corresponding to the predicted attribute category in the first target network block as a first target expert layer; and performing encoding using the first target expert layer to obtain a first encoding feature.

[0092] In one embodiment, the method further includes, in response to the first network block corresponding to the current stage being not the last first network block in the encoding network, inputting the first encoding feature output by the first network block corresponding to the current stage to a next first network block until the first network block corresponding to the current stage is the last first network block in the encoding network, and selecting the first encoding feature output by the last first network block as the first encoding feature finally output by the encoding network.

[0093] Thus, by installing multiple first network blocks, the overall scale of the speech recognition model can be greatly expanded and the speech recognition effect of the speech recognition model can be improved.

[0094] In one embodiment, the first network block further includes a shared expert layer. After using the first target expert layer to perform encoding and obtain first encoded features, the method further includes using the shared expert layer to perform encoding and obtain second encoded features, and fusing the first encoded features and the second encoded features to obtain first encoded features finally output by the first target network block. This step may also be performed by the first encoding module 72.

[0095] Therefore, the problem of insufficient sample fitting ability of speech recognition models can be solved by sharing the expert layer.

[0096] In one embodiment, the plurality of first network blocks are divided into at least one network group, and each first network block in the same network group is associated with the same target voice attribute.

[0097] In one embodiment, the decoding network includes a plurality of second network blocks connected in series and associated with target voice attributes, and the second network blocks associated with the target voice attributes include a second classification layer for classifying by the target voice attributes, and a second expert layer having a one-to-one correspondence with each preset attribute category in the target voice attributes, and each second expert layer is used for decoding based on the preset attribute category in the target voice attributes.

[0098] In one embodiment, the identified text is obtained by combining decoded characters obtained by decoding at each decoding time point, and each stage of any decoding time point is performed by a different second network block. At any decoding time point, the method includes: selecting a second network block corresponding to a current stage at the decoding time point as a second target network block; using a second classification layer in the second target network block to classify the decoded characters obtained by decoding at each previous decoding time point, and obtaining a second probability that each decoded character belongs to each predetermined attribute category; determining a second expert layer for decoding the decoded characters in the second target network block according to the second probability that each decoded character belongs to each predetermined attribute category, and determining the second expert layer as the second target expert layer; decoding the decoded characters using the second target expert layer to obtain first decoding features; performing decoding based on the first encoding features and the first decoding features to obtain a decoded character at the decoding time point.

[0099] Therefore, installing multiple second network blocks in the decoding network can further improve the speech recognition effect of the speech recognition model.

[0100] In one embodiment, the speech recognition model is further adjusted based on a second loss, and the second loss is determined based on the following factors: a proportion of sample characters in a sample text labeled by a sample voice, for a target speech attribute, that belong to each of the preset attribute categories, and an average probability that the sample characters belong to each of the preset attribute categories.

[0101] Therefore, by adjusting the parameters of the speech recognition model using the second loss, the number of sample characters belonging to each preset attribute category can be made uniform.

[0102] Referring to Figure 8, Figure 8 is a schematic diagram of an embodiment of a speech recognition model training device provided by the present application. The speech recognition model training device 80 includes a second acquisition module 81, a second encoding module 82, a second decoding module 83, a loss determination module 84, and an adjustment module 85. The second acquisition module 81 is used to acquire sample speech. The second encoding module 82 is used to first classify the sample speech by target speech attributes to obtain the sample attribute category to which the sample speech belongs, and then encode the sample attribute category based on the target speech attribute to obtain first sample encoding features. The second decoding module 83 is used to decode the first sample encoding features using the decoding network of the speech recognition model to obtain recognized text of the sample speech. The loss determination module 84 is used to determine a first loss based on the difference between the sample attribute category to which the sample speech belongs and the preset attribute category of the sample speech label, and to determine a recognition loss based on the difference between the recognized text of the sample speech and the preset text of the sample speech label. The tuning module 85 is used to tune network parameters of the speech recognition model based on at least the first loss and the recognition loss.

[0103] In one embodiment, the second acquisition module 81 is further configured to acquire the proportion of sample characters in the sample text labeled by the sample speech that belong to each of the preset attribute categories and the average probability that the sample characters belong to each of the preset attribute categories. The loss determination module 84 is further configured to determine a second loss based on the proportions and the average probability. The adjustment module 85 is configured to adjust network parameters of the speech recognition model based on the first loss, the second loss, and the recognition loss.

[0104] In one embodiment, determining the second loss based on the proportion and the average probability includes obtaining a product of a proportion of each sample character belonging to the same preset attribute category and an average probability of each sample character belonging to the same preset attribute category, and determining the product as a probability parameter; and determining the second loss as a product of the sum of the probability parameters and the number of the preset attribute categories.

[0105] The above method introduces a third loss into the decoding network side, thereby making the number of sample characters assigned to each second expert layer in the decoding network uniform.

[0106] Please refer to FIG. 9, which is a frame structure schematic diagram of one embodiment of an electronic device provided by the present application.

[0107] The electronic device 90 includes a memory 91 and a processor 92 coupled to each other. The memory 91 stores program instructions, and the processor 92 is used to execute the program instructions stored in the memory 91, thereby realizing the steps of any one of the above method embodiments. In one specific implementation scenario, the electronic device 90 can include, but is not limited to, a microcomputer and a server. Furthermore, the electronic device 90 can also include, but is not limited to, a mobile device such as a notebook computer or a tablet computer.

[0108] Specifically, the processor 92 is used to control itself and the memory 91 to perform any of the steps of the above-described method embodiments. The processor 92 may also be referred to as a CPU (Central Processing Unit). The processor 92 may be an integrated circuit chip having signal processing capabilities. The processor 92 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or any conventional processor, etc. Furthermore, the processor 92 is implemented by an integrated circuit chip.

[0109] Reference is now made to FIG. 10, which is a schematic diagram of an embodiment of a computer-readable medium provided by the present application.

[0110] A computer readable storage medium 100 stores program instructions 101. When executed by a processor, the program instructions 101 are used to implement the steps of the method embodiments described above.

[0111] The computer-readable storage medium 100 may specifically be a medium capable of storing a computer program, such as a U-disk, a portable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a server that stores the computer program. The server may transmit the stored computer program to another device for execution, or may automatically execute the stored computer program.

[0112] The above description of each embodiment is intended to highlight the differences between each embodiment, and the same or similar parts thereof may be referred to each other by reference, and will not be repeated herein for the sake of brevity.

[0113] In the embodiments provided by the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely exemplary, and the division into modules or units is merely a division of logical functions. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into other systems, or some features may be ignored or not implemented. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be some interfaces, indirect couplings, or communication connections between devices or units, which may be electrical, mechanical, or other types.

[0114] The units described as separate elements may or may not be physically separated, and the elements shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. To achieve the objectives of this embodiment, some or all of the units may be selected according to actual needs.

[0115] Furthermore, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated units may be realized in the form of hardware or in the form of software functional units.

[0116] The integrated unit may be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as a standalone product. Based on this understanding, the essence or conventional contribution of the technical solution of the present application, or all or part of the technical solution, may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes a plurality of instructions for a computer device (such as a personal computer, a server, a network device, etc.) or a processor to perform all or part of the steps of the methods of various embodiments of the present application. The storage medium may include a medium capable of storing program code, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk.

[0117] If the technical solution of this application relates to personal information, the product applying this technical solution shall clearly notify the individual of the personal information processing rules and obtain their voluntary consent before processing the personal information. If the technical solution of this application relates to sensitive personal information, the product applying this technical solution shall obtain the individual's consent before processing the sensitive personal information, thereby meeting the requirement of "explicit consent." For example, a personal information collection device such as a camera may be equipped with a clear and conspicuous sign informing the individual that they have entered the personal information collection range and that their personal information will be collected. If an individual voluntarily enters the collection range, they are deemed to have consented to the collection of their personal information. Alternatively, if a device processing personal information uses clear signs / information to notify the individual of the personal information processing rules, the individual's permission may be obtained by means of a pop-up window or by requesting the individual to upload their personal information. Here, the personal information processing rules may include information such as the personal information controller, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0118] The above are embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by utilizing the contents of the specification and drawings of the present application, or any directly or indirectly applicable to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. obtaining a speech to be recognized and obtaining a trained speech recognition model, wherein the speech recognition model includes an encoding network and a decoding network; In each step of encoding the speech to be recognized using the encoding network, first, classifying the speech to be recognized by a target speech attribute to obtain a predicted attribute category to which the speech to be recognized belongs, and then encoding the speech based on the predicted attribute category for the target speech attribute to obtain a first encoding feature; A speech recognition method comprising: a step of decoding the first encoding features based on the decoding network to obtain recognized text of the speech to be recognized, wherein the speech recognition model is adjusted based on at least a first loss, the first loss characterizing differences in the target speech attributes between preset attribute categories of sample speech labels and sample attribute categories recognized and obtained by the speech recognition model.

2. 2. The speech recognition method of claim 1, wherein the encoding network includes a plurality of first network blocks connected in series and associated with the target speech attribute, each used to perform a different step of coding; the first network blocks associated with the target speech attribute include a first classification layer for classification based on the target speech attribute and a first expert layer each having a one-to-one correspondence with each preset attribute category of the target speech attribute; and the first expert layer is used to perform coding based on the predicted attribute category of the target speech attribute.

3. Before classifying the speech to be recognized by the target speech attribute to obtain a predicted attribute category to which the speech to be recognized belongs, the speech recognition method further includes a step of selecting a first network block corresponding to a current stage as a first target network block; The step of classifying the speech to be recognized by the target speech attribute and obtaining a predicted attribute category to which the speech to be recognized belongs includes: classifying the speech using the first classification layer in the first target network block, and obtaining a first probability that the speech to be recognized belongs to each of the predetermined attribute categories in the target speech attribute; determining a predicted attribute category to which the recognition-target speech belongs based on a first probability that the recognition-target speech belongs to each of the predetermined attribute categories; The step of performing encoding based on the predicted attribute category of the target speech attribute to obtain a first encoding feature includes: selecting a first expert layer corresponding to the predicted attribute category in the first target network block as a first target expert layer; 3. The speech recognition method of claim 2, further comprising the step of: performing encoding using the first target expert layer to obtain the first encoded features.

4. The speech recognition method includes: In response to the first network block corresponding to the current stage not being the last one of the first network blocks in the encoding network, inputting the first encoding feature output by the first network block corresponding to the current stage to a next first network block until the first network block corresponding to the current stage is the last one of the first network blocks in the encoding network; selecting the first coding feature output by the last first network block as the first coding feature finally output by the coding network; 4. The speech recognition method according to claim 2, further comprising:

5. wherein the first network block further includes a shared expert layer; After performing encoding using the first target expert layer and obtaining the first encoding features, the speech recognition method includes: performing coding using the shared expert layer to obtain second coding features; fusing the first encoding feature and the second encoding feature to obtain a first encoding feature finally output by the first target network block; 4. The speech recognition method of claim 3, comprising:

6. The speech recognition method according to claim 2 , wherein a plurality of the first network blocks are divided into at least one network group, and each of the first network blocks in the same network group is associated with the same target speech attribute.

7. 2. The speech recognition method of claim 1, wherein the decoding network includes a plurality of second network blocks connected in series and associated with the target speech attribute, and the second network blocks associated with the target speech attribute include a second classification layer for classification based on the target speech attribute and a second expert layer having a one-to-one correspondence with each preset attribute category in the target speech attribute, and the second expert layer is respectively used for decoding based on the preset attribute categories in the target speech attribute.

8. The recognized text is obtained by combining decoded characters respectively decoded at each decoding time point, and each step of any one of the decoding time points is performed by a different second network block, and at any one of the decoding time points, the speech recognition method includes: selecting a second network block corresponding to the current stage at the time of decoding as a second target network block; using a second classification layer in the second target network block to classify the decoded characters obtained by decoding at each of the previous decoding times, and obtain a second probability that each of the decoded characters belongs to each of the predetermined attribute categories; determining a second expert layer as a second target expert layer for decoding the decoded characters in the second target network block according to a second probability that the decoded characters belong to each of the predetermined attribute categories; Decoding the decoded character using the second target expert layer to obtain a first decoding feature; decoding the character string based on the first encoding characteristics and the first decoding characteristics to obtain a decoded character string at the time of decoding; 8. The speech recognition method of claim 7, comprising:

9. 2. The speech recognition method of claim 1, wherein the speech recognition model is adjusted based on a second loss, and the second loss is determined based on factors including a proportion of sample characters in the sample text of the sample speech label that belong to each of the predetermined attribute categories for the target speech attribute, and an average probability that the sample characters belong to each of the predetermined attribute categories.

10. obtaining a sample audio; In each step of encoding the sample voice using the encoding network of the speech recognition model, first classify the sample voice by a target voice attribute to obtain a sample attribute category to which the sample voice belongs, then encode the sample voice according to the sample attribute category in the target voice attribute to obtain a first sample encoding feature; Decoding the first sample coding feature using a decoding network of the speech recognition model to obtain a recognized text of the sample speech; determining a first loss based on the difference between the sample attribute category to which the sample voice belongs and the preset attribute category of the sample voice label, and determining a recognition loss based on the difference between the recognized text of the sample voice and the preset text of the sample voice label; adjusting network parameters of the speech recognition model based on at least the first loss and the recognition loss; A method for training a speech recognition model, including:

11. How to train a speech recognition model: Obtaining a proportion of sample characters in the sample text of the sample voice label that belong to each of the preset attribute categories, and an average probability that the sample characters belong to the preset attribute categories; determining a second loss based on the ratio and the average probability; adjusting parameters of the speech recognition model based on at least the first loss and the recognition loss, The method for training a speech recognition model of claim 10, further comprising adjusting network parameters of the speech recognition model based on the first loss, the second loss, and the recognition loss.

12. determining a second loss based on the ratio and the average probability, obtaining a product of a proportion of each of the sample characters belonging to the same predetermined attribute category and an average probability that each of the sample characters belongs to the same predetermined attribute category, and setting the product as a probability parameter; and setting a product of the sum of the probability parameters and a predetermined number of attribute categories as a second loss.

13. An electronic device including a memory and a processor, An electronic device, wherein the memory stores program instructions and the processor is used to execute the program instructions, thereby implementing a speech recognition method according to any one of claims 1 to 9 or a method for training a speech recognition model according to any one of claims 10 to 12.

14. A computer-readable storage medium used to store program instructions, which are executed by a processor to thereby perform the speech recognition method of any one of claims 1 to 9 or the method for training a speech recognition model of any one of claims 10 to 12.

Images