Speech recognition model training method and device, equipment and readable storage medium
By fuzzing the text-annotated data and introducing intent classification information into the intelligent customer service speech recognition model, the training method of the speech recognition model was optimized, the accuracy of intent classification was improved, and the problem of insufficient intent classification accuracy in the existing technology was solved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-06
- Publication Date
- 2026-03-13
AI Technical Summary
Existing intelligent customer service voice recognition technology is insufficient in terms of the accuracy of intent classification, resulting in the problem that the voice can be understood but not comprehended.
By blurring the text-annotated data and ignoring text irrelevant to the intent classification, speech features are extracted, and intent classification information is introduced during training. The speech recognition model is then optimized using the cross-entropy loss function.
It improves the accuracy of the recognition results generated by the speech recognition model for intent classification, making it more focused on keywords and enhancing the accuracy of intent classification.
Smart Images
Figure CN121662029A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and more specifically to a method, apparatus, device, and readable storage medium for training a speech recognition model. Background Technology
[0002] Speech recognition is the technology of converting a given audio recording of a human voice into text, and it has wide applications in the field of intelligent customer service. However, in the entire intelligent customer service process, speech recognition is only one key technology at the beginning of the process. Subsequent processes also include classifying intents based on speech recognition results and jumping to specified dialogue nodes based on intent classification results.
[0003] Since the accuracy of intent classification based on speech recognition results has a significant impact on the service quality provided by intelligent customer service, current intelligent customer service speech recognition technology typically focuses on improving the accuracy of the recognition model, neglecting the influence of intent classification information. This often leads to situations where intelligent customer service representatives can understand the spoken words but cannot comprehend them. Therefore, there is an urgent need for a training method for speech recognition models that can overcome these shortcomings. Summary of the Invention
[0004] The purpose of this invention is to provide a training method, apparatus, device, and readable storage medium for a speech recognition model. By fuzzing the text annotation data, some text that is irrelevant to intent classification is ignored, making the speech recognition model more focused on keywords. Furthermore, the information on intent classification is introduced during the training process, thereby improving the accuracy of the recognition results generated by the speech recognition model for intent classification.
[0005] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for training a speech recognition model, the method comprising: Acquire speech training data and corresponding annotation data; the annotation data includes text annotation data and intent annotation data. The text annotation data is blurred, and the speech features of the speech training data are extracted. The speech recognition model is trained based on the speech features, text annotation data, and intent annotation data until the speech recognition model converges.
[0006] In some embodiments, training a speech recognition model based on the speech features, text annotation data, and intent annotation data until the speech recognition model converges includes: The speech features are input into the text recognition layer of the speech recognition model to obtain the text recognition result; The text recognition result is input into the intent recognition layer of the speech recognition model to obtain the intent recognition result; The speech recognition model is trained based on the text recognition results, intent recognition results, text annotation data, and intent annotation data until the speech recognition model converges.
[0007] In some embodiments, a speech recognition model is trained based on the text recognition results, intent recognition results, text annotation data, and intent annotation data until the speech recognition model converges, including: The cross-entropy loss between the text recognition result and the text annotation data is calculated using the cross-entropy loss function to obtain the text recognition result loss. The cross-entropy loss between the intent recognition result and the intent annotation data is calculated using the cross-entropy loss function to obtain the intent recognition result loss; The speech recognition model is trained based on the text recognition result loss and the intent recognition result loss until the speech recognition model converges.
[0008] In some embodiments, a speech recognition model is trained based on the text recognition result loss and the intent recognition result loss until the speech recognition model converges, including: The loss of the intent recognition result is normalized to obtain the normalized loss weight; The training loss of the speech recognition model is calculated based on the normalized loss weights and the text recognition result loss. The speech recognition model is trained until the training loss is less than a preset loss threshold.
[0009] In some embodiments, extracting speech features from the speech training data includes: Extract the temporal features from the speech training data; the temporal features include short-time energy and zero-crossing rate. Extract the frequency domain features from the speech training data; the frequency domain features include Mel frequency cepstral coefficients; The time-domain features and the frequency-domain features are used as the speech features of the speech training data.
[0010] In some embodiments, the text annotation data is blurred, including: Remove text from the text annotation data that is irrelevant to intent recognition.
[0011] Secondly, the present invention also provides a training apparatus for a speech recognition model, the apparatus comprising: The data acquisition module is used to acquire speech training data and the corresponding annotation data; the annotation data includes text annotation data and intent annotation data; The data processing module is used to perform fuzzing processing on the text annotation data and extract the speech features of the speech training data; The model training module is used to train a speech recognition model based on the speech features, text annotation data, and intent annotation data until the speech recognition model converges.
[0012] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the training method of the speech recognition model provided in the first aspect.
[0013] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the training method for the speech recognition model provided in the first aspect.
[0014] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the training method for the speech recognition model provided in the first aspect.
[0015] The beneficial effects of this invention are as follows: The training method for the speech recognition model in this invention first obtains speech training data and corresponding labeled data; the labeled data includes text labeled data and intent labeled data; the text labeled data is fuzzified, and speech features of the speech training data are extracted; the speech recognition model is trained based on the speech features, text labeled data, and intent labeled data until the speech recognition model converges. By fuzzifying the text labeled data, some text that is irrelevant to intent classification is ignored, making the speech recognition model more focused on keywords, and the information of intent classification is introduced during the training process, improving the accuracy of the recognition results generated by the speech recognition model for intent classification.
[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating a training method for a speech recognition model according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating another method for training a speech recognition model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a training device for a speech recognition model according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a training device for another speech recognition model according to an embodiment of the present invention; Figure 5 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation
[0018] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. It should be noted that references to "an embodiment," "embodiment," "example embodiment," etc., in this specification refer to the described embodiment including specific features, structures, or characteristics; however, not every embodiment must include these specific features, structures, or characteristics. Furthermore, such expressions do not refer to the same embodiment. Moreover, when describing specific features, structures, or characteristics in conjunction with embodiments, whether or not explicitly described, it is indicated that incorporating such features, structures, or characteristics into other embodiments is within the knowledge scope of those skilled in the art.
[0019] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0020] In some embodiments, such as Figure 1 As shown, a method for training a speech recognition model is provided, the specific method including: S101, obtain the speech training data and the corresponding labeled data.
[0021] The labeled data includes text labeled data and intent labeled data. The speech training data is the speech used for model training, the text labeled data is the text corresponding to the speech training data, and the intent labeled data is the intent corresponding to the speech training data.
[0022] Specifically, when training a speech recognition model, you can directly obtain speech training data and corresponding labeled data from the database, or obtain manually uploaded speech training data and corresponding labeled data.
[0023] S102, blur the text annotation data and extract the speech features from the speech training data.
[0024] Optionally, text irrelevant to intent recognition can be removed from the text annotation data; temporal features of the speech training data can be extracted, including short-time energy and zero-crossing rate; frequency features of the speech training data can be extracted, including Mel-frequency cepstral coefficients; and the temporal and frequency features can be used as speech features of the speech training data.
[0025] Specifically, the influence of each character in the text annotation data on intent recognition can be calculated. Deleting characters with an influence of 0 completes the fuzzification process of the text annotation data. Then, the short-time energy and zero-crossing rate of the speech training data are calculated. Short-time energy reflects speech intensity, and zero-crossing rate reflects pitch changes or voiced / unvoiced characteristics. Next, frequency domain features of the speech training data are extracted. The frequency domain feature library includes Mel-frequency cepstral coefficients, Mel-frequency cepstral energy spectrum, linear prediction cepstral coefficients, and perceptual linear prediction, etc. To extract speech features from the speech training data more comprehensively, higher-level features such as spectral centroid, spectral bandwidth, spectral flux, fundamental frequency, pitch, and formants are also extracted and incorporated into the speech features of the speech training data. S103 trains a speech recognition model based on speech features, text annotation data, and intent annotation data until the speech recognition model converges.
[0026] Optionally, a speech recognition model is trained based on speech features, text annotation data, and intent annotation data until the speech recognition model converges, including: inputting speech features into the text recognition layer of the speech recognition model to obtain text recognition results; inputting the text recognition results into the intent recognition layer of the speech recognition model to obtain intent recognition results; and training the speech recognition model based on the text recognition results, intent recognition results, text annotation data, and intent annotation data until the speech recognition model converges.
[0027] Optionally, a speech recognition model is trained based on the text recognition results, intent recognition results, text annotation data, and intent annotation data until the speech recognition model converges. This includes: calculating the cross-entropy loss between the text recognition results and the text annotation data using the cross-entropy loss function to obtain the text recognition result loss; calculating the cross-entropy loss between the intent recognition results and the intent annotation data using the cross-entropy loss function to obtain the intent recognition result loss; and training the speech recognition model based on the text recognition result loss and the intent recognition result loss until the speech recognition model converges.
[0028] Optionally, the speech recognition model is trained based on the text recognition result loss and the intent recognition result loss until the speech recognition model converges, including: normalizing the intent recognition result loss to obtain normalized loss weights; calculating the training loss of the speech recognition model based on the normalized loss weights and the text recognition result loss; and training the speech recognition model until the training loss is less than a preset loss threshold.
[0029] For example, speech features are , The number of speech features is [number], and the text annotation data is [number]. The intended labeled data is First, the speech features are input into the text recognition layer of the speech recognition model to obtain the text recognition result. The text recognition result is then input into the intent recognition layer of the speech recognition model to obtain the intent recognition result. ,in , For the number of intentions, For the first The predicted probability of each intent is calculated, and then the cross-entropy loss function is used to calculate the cross-entropy loss between the text recognition result and the text annotation data, resulting in the text recognition result loss as follows: The cross-entropy loss function is used to calculate the cross-entropy loss between the intent recognition result and the intent annotation data, resulting in the intent recognition result loss as follows: The formula for calculating the cross entropy loss is as follows: ; in, For the first The predicted value of each element.
[0030] Then, the loss of the intent recognition result is normalized to obtain the normalized loss weight. The specific calculation formula is as follows: ; Here, ε is a local minimum.
[0031] Finally, the training loss of the speech recognition model is calculated based on the normalized loss weights and the text recognition result loss. .
[0032] The training method for the speech recognition model in the above embodiments first obtains speech training data and corresponding labeled data. The labeled data includes text labeled data and intent labeled data. The text labeled data is then blurred, and speech features of the speech training data are extracted. The speech recognition model is trained based on the speech features, text labeled data, and intent labeled data until the speech recognition model converges. By blurring the text labeled data, some text that is irrelevant to intent classification is ignored, allowing the speech recognition model to focus more on keywords. Furthermore, the inclusion of intent classification information during training improves the accuracy of the recognition results generated by the speech recognition model for intent classification.
[0033] The speech recognition model needs to be trained until the training loss is less than a preset loss threshold to achieve convergence.
[0034] To more comprehensively demonstrate this solution, this embodiment presents an optional method for training a speech recognition model, such as... Figure 2 As shown: S201, Obtain the speech training data and the corresponding labeled data.
[0035] The labeled data includes text labeled data and intent labeled data.
[0036] S202, Remove text from the text annotation data that is irrelevant to intent recognition.
[0037] S203, extract the temporal features of the speech training data.
[0038] Among them, the time-domain characteristics include short-time energy and zero-crossing rate.
[0039] S204, extract the frequency domain features of the speech training data.
[0040] Among them, the frequency domain features include Mel frequency cepstral coefficients.
[0041] S205 uses time-domain features and frequency-domain features as speech features in speech training data.
[0042] S206, input the speech features into the text recognition layer of the speech recognition model to obtain the text recognition result.
[0043] S207, input the text recognition result into the intent recognition layer of the speech recognition model to obtain the intent recognition result.
[0044] S208. The cross-entropy loss function is used to calculate the cross-entropy loss between the text recognition result and the text annotation data to obtain the text recognition result loss.
[0045] S209, calculate the cross-entropy loss between the intent recognition result and the intent annotation data using the cross-entropy loss function, and obtain the intent recognition result loss.
[0046] S210, normalize the loss of the intent recognition result to obtain the normalized loss weight.
[0047] S211, calculate the training loss of the speech recognition model based on the normalized loss weights and the text recognition result loss.
[0048] S212, train the speech recognition model until the training loss is less than the preset loss threshold.
[0049] The specific processes of S201-S212 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.
[0050] Based on the same inventive concept, this application also provides a speech recognition model training apparatus for implementing the speech recognition model training method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more speech recognition model training apparatus embodiments provided below can be found in the limitations of the speech recognition model training method described above, and will not be repeated here.
[0051] In one embodiment, such as Figure 3 As shown, a training device for a speech recognition model is provided, the device comprising: The data acquisition module 30 is used to acquire speech training data and the annotation data corresponding to the speech training data; the annotation data includes text annotation data and intent annotation data; The data processing module 31 is used to perform fuzzing processing on the text annotation data and extract the speech features of the speech training data; The model training module 32 is used to train a speech recognition model based on the speech features, text annotation data and intent annotation data until the speech recognition model converges.
[0052] In another embodiment, such as Figure 4 As shown above, Figure 3 The model training module 32 in the middle includes: The first recognition unit 320 is used to input the speech features into the text recognition layer of the speech recognition model to obtain the text recognition result; The second recognition unit 321 is used to input the text recognition result into the intent recognition layer of the speech recognition model to obtain the intent recognition result; The model training unit 322 is used to train a speech recognition model based on the text recognition results, intent recognition results, text annotation data and intent annotation data until the speech recognition model converges.
[0053] In another embodiment, the above Figure 4 The model training unit 322 includes: The first calculation subunit uses the cross-entropy loss function to calculate the cross-entropy loss between the text recognition result and the text annotation data, and obtains the text recognition result loss. The second calculation subunit uses the cross-entropy loss function to calculate the cross-entropy loss between the intent recognition result and the intent annotation data, and obtains the intent recognition result loss. The third computational subunit trains a speech recognition model based on the text recognition result loss and the intent recognition result loss until the speech recognition model converges.
[0054] In another embodiment, the third calculation subunit in the above embodiment is specifically used for: normalizing the intention recognition result loss to obtain normalized loss weights; calculating the training loss of the speech recognition model based on the normalized loss weights and the text recognition result loss; and training the speech recognition model until the training loss is less than a preset loss threshold.
[0055] In another embodiment, the above Figure 3 The data processing module 31 is specifically used for: removing text from the text annotation data that is irrelevant to intent recognition; extracting time-domain features from the speech training data; the time-domain features include short-time energy and zero-crossing rate; extracting frequency-domain features from the speech training data; the frequency-domain features include Mel-frequency cepstral coefficients; and using the time-domain features and the frequency-domain features as the speech features of the speech training data.
[0056] This application also provides an electronic device, in some embodiments, referring to... Figure 5 As shown, the electronic device 700 includes an input unit 710, a memory 720, a processor 730, and an output unit 740. The memory 720 stores program instructions that can be executed on the processor 730. The processor 730 can execute training methods and / or technical solutions based on the speech recognition model in the foregoing embodiments by calling the program instructions. The electronic device 700 can be a mobile terminal device such as a mobile phone or a computer.
[0057] Furthermore, embodiments of this application also provide a computer-readable storage medium for storing a computer program that executes a training method for a speech recognition model. For example, computer program instructions, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. The program instructions that invoke the methods of this application may be stored in a fixed or removable storage medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in a storage medium that operates according to the program instructions.
[0058] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.
[0059] The technical features of the above embodiments can be arbitrarily integrated. For the sake of brevity, not all possible integrations of the technical features in the above embodiments are described. However, as long as the integration of these technical features does not contradict each other, they should be considered to be within the scope of this specification.
[0060] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.
Claims
1. A method for training a speech recognition model, characterized in that, The method includes: Acquire speech training data and corresponding annotation data; the annotation data includes text annotation data and intent annotation data. The text annotation data is blurred, and the speech features of the speech training data are extracted. The speech recognition model is trained based on the speech features, text annotation data, and intent annotation data until the speech recognition model converges.
2. The training method for the speech recognition model as described in claim 1, characterized in that, Training a speech recognition model based on the aforementioned speech features, text annotation data, and intent annotation data until the speech recognition model converges includes: The speech features are input into the text recognition layer of the speech recognition model to obtain the text recognition result; The text recognition result is input into the intent recognition layer of the speech recognition model to obtain the intent recognition result; The speech recognition model is trained based on the text recognition results, intent recognition results, text annotation data, and intent annotation data until the speech recognition model converges.
3. The training method for the speech recognition model as described in claim 2, characterized in that, Training a speech recognition model based on the text recognition results, intent recognition results, text annotation data, and intent annotation data until the speech recognition model converges includes: The cross-entropy loss between the text recognition result and the text annotation data is calculated using the cross-entropy loss function to obtain the text recognition result loss. The cross-entropy loss between the intent recognition result and the intent annotation data is calculated using the cross-entropy loss function to obtain the intent recognition result loss; The speech recognition model is trained based on the text recognition result loss and the intent recognition result loss until the speech recognition model converges.
4. The training method for the speech recognition model as described in claim 3, characterized in that, Training a speech recognition model based on the text recognition result loss and the intent recognition result loss until the speech recognition model converges includes: The loss of the intent recognition result is normalized to obtain the normalized loss weight; The training loss of the speech recognition model is calculated based on the normalized loss weights and the text recognition result loss. The speech recognition model is trained until the training loss is less than a preset loss threshold.
5. The training method for the speech recognition model as described in claim 1, characterized in that, Extracting speech features from the speech training data includes: Extract the temporal features from the speech training data; the temporal features include short-time energy and zero-crossing rate. Extract the frequency domain features from the speech training data; the frequency domain features include Mel frequency cepstral coefficients; The time-domain features and the frequency-domain features are used as the speech features of the speech training data.
6. The training method for the speech recognition model as described in claim 1, characterized in that, The text annotation data is blurred, including: Remove text from the text annotation data that is irrelevant to intent recognition.
7. A training device for a speech recognition model, characterized in that, The device includes: The data acquisition module is used to acquire speech training data and the corresponding annotation data; the annotation data includes text annotation data and intent annotation data; The data processing module is used to perform fuzzing processing on the text annotation data and extract the speech features of the speech training data; The model training module is used to train a speech recognition model based on the speech features, text annotation data, and intent annotation data until the speech recognition model converges.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the training method of the speech recognition model according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the training method for the speech recognition model according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the training method for the speech recognition model according to any one of claims 1 to 6.