Speech recognition model training, speech recognition method, apparatus, device, and medium
By constructing a label set of candidate pronunciations for polyphonic characters and calculating the loss value by aggregating probabilities, the accuracy and robustness problems of traditional speech recognition models in polyphonic character scenarios are solved, achieving more efficient speech recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- IFLYTEK CO LTD
- Filing Date
- 2026-03-30
- Publication Date
- 2026-06-09
AI Technical Summary
In existing technologies, CTC-based speech recognition models have difficulty accurately processing the acoustic representation of polyphonic characters in polyphonic character scenarios, resulting in a decrease in recognition accuracy. Furthermore, the hard labels generated by traditional G2P tools are prone to causing the model to fit incorrect alignment paths and cannot adapt to user accent deviations.
By constructing a label set containing all candidate pronunciations of polyphonic characters and using the sum of the predicted probabilities of the candidate pronunciations to obtain the aggregate probability to calculate the loss value, end-to-end soft label training is achieved, allowing the model to automatically learn the most reasonable pronunciation path during the training process.
It significantly improves the robustness and accuracy of speech recognition models in polyphonic character scenarios, avoids training interference caused by G2P annotation errors and user accent deviations, and improves the model's fault tolerance and recognition performance.
Smart Images
Figure CN122177115A_ABST
Abstract
Claims
1. A method for training a speech recognition model, characterized in that, include: Acquire sample speech data and the corresponding text sequence of the sample speech data; Determine the pronunciation label of each character in the text sequence; The pronunciation labels corresponding to the polyphonic characters in the text sequence are a set of labels consisting of at least two candidate pronunciations corresponding to the polyphonic characters; The sample speech data is input into the initial speech recognition model to obtain the predicted probability of the candidate pronunciation for each character; The predicted probabilities of all candidate pronunciations for each character are fused to obtain the aggregated probability of the corresponding character, and the loss value is determined based on the aggregated probability of all characters in the text sequence. The parameters of the initial speech recognition model are updated using the loss value to obtain the speech recognition model.
2. The speech recognition model training method according to claim 1, characterized in that, Determining the pronunciation label of each character in the text sequence includes: Traverse the text sequence, and for each character in the text sequence, query all corresponding candidate pronunciations in a preset pronunciation dictionary; If the query result is that any character corresponds to a single candidate pronunciation, then the single candidate pronunciation is used as the pronunciation tag of the character. If any character in the query result corresponds to at least two candidate pronunciations, then the set of tags consisting of the at least two candidate pronunciations is taken as the pronunciation tag of the character.
3. The speech recognition model training method according to claim 2, characterized in that, The pronunciation dictionary stores the mapping relationship between characters and candidate pronunciations, and the mapping relationship corresponding to the polyphonic characters includes all candidate pronunciations of the polyphonic characters in different contexts.
4. The speech recognition model training method according to any one of claims 1 to 3, characterized in that, The step of inputting the sample speech data into the initial speech recognition model to obtain the predicted probability of each character's corresponding candidate pronunciation includes: The sample speech data is input into the initial speech recognition model to obtain the predicted probability distribution sequence of each character, wherein the predicted probability distribution sequence of each character includes the predicted probability distribution of all candidate pronunciations of the corresponding character; A target pronunciation sequence is constructed based on the pronunciation tags of each character in the text sequence, and non-pronunciation symbols are inserted into the target pronunciation sequence to generate an extended pronunciation sequence; Based on the predicted probability distribution sequence of each character, alignment path calculation is performed on the extended pronunciation sequence, and the predicted probability of the candidate pronunciation corresponding to each character is extracted during the calculation process.
5. The speech recognition model training method according to claim 4, characterized in that, The step of inserting non-pronounceable symbols into the target pronunciation sequence to generate an extended pronunciation sequence includes: The non-pronunciation symbols are inserted at the beginning and end positions of the target pronunciation sequence and between each pronunciation tag to generate the extended pronunciation sequence.
6. The speech recognition model training method according to claim 4, characterized in that, Determining the loss value based on the aggregated probabilities of all characters in the text sequence includes: Based on the extended pronunciation sequence, an aligned path grid is constructed; For each node in the alignment path grid, the aggregation probability is used as the emission probability to calculate the forward probability of reaching each node and the backward probability of starting from each node. The loss value is calculated based on the forward probability and backward probability of each node.
7. The speech recognition model training method according to any one of claims 1 to 3, characterized in that, The process of fusing the predicted probabilities of all candidate pronunciations for each character to obtain the aggregate probability of the corresponding character includes: The predicted probabilities of all candidate pronunciations of the polyphonic character are summed to obtain the aggregate probability of the polyphonic character. The predicted probability of the candidate pronunciation of a monosyllabic character is used as the aggregate probability of the monosyllabic character.
8. The speech recognition model training method according to any one of claims 1 to 3, characterized in that, The step of updating the parameters of the initial speech recognition model using the loss value to obtain the speech recognition model includes: The parameter gradient of the initial speech recognition model is calculated based on the loss value; wherein, in the process of calculating the parameter gradient, for the pronunciation label corresponding to the polyphonic character, the partial derivative of the loss value with respect to the aggregation probability of the polyphonic character is passed to the output node corresponding to each candidate pronunciation of the polyphonic character to participate in the calculation of the parameter gradient; The parameters of the initial speech recognition model are updated using the parameter gradient to obtain the speech recognition model.
9. A speech recognition method, characterized in that, include: Acquire the speech data to be recognized; The speech data to be recognized is input into the speech recognition model to obtain the speech recognition result; The speech recognition model is trained based on the speech recognition model training method according to any one of claims 1 to 8.
10. A speech recognition model training device, characterized in that, include: The first acquisition module is used to acquire sample speech data and the text sequence corresponding to the sample speech data; A determination module is used to determine the pronunciation label of each character in the text sequence; The pronunciation labels corresponding to the polyphonic characters in the text sequence are a set of labels consisting of at least two candidate pronunciations corresponding to the polyphonic characters; The prediction module is used to input the sample speech data into the initial speech recognition model to obtain the predicted probability of the candidate pronunciation corresponding to each character; The fusion module is used to fuse the predicted probabilities of all candidate pronunciations for each character to obtain the aggregated probability of the corresponding character, and to determine the loss value based on the aggregated probabilities of all characters in the text sequence. An update module is used to update the parameters of the initial speech recognition model using the loss value, thereby obtaining a speech recognition model.
11. A voice recognition device, characterized in that, include: The second acquisition module is used to acquire the speech data to be recognized; The recognition module is used to input the speech data to be recognized into the speech recognition model to obtain the speech recognition result; The speech recognition model is trained based on the speech recognition model training method according to any one of claims 1 to 8.
12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the speech recognition model training method as described in any one of claims 1 to 8, or implements the speech recognition method as described in claim 9.
13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the speech recognition model training method as described in any one of claims 1 to 8, or implements the speech recognition method as described in claim 9.