Data calibration method, device, equipment and medium
By dividing the transcribed text into sub-words based on pronunciation units and matching them with the predicted phonemes of the phoneme recognition model, the problem of phoneme recognition accuracy in complex acoustic environments by end-to-end models is solved, the recognition accuracy is improved and the system deployment is simplified, making it suitable for resource-constrained scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-01-15
- Publication Date
- 2026-05-12
AI Technical Summary
Existing end-to-end deep learning models suffer from insufficient accuracy in phoneme recognition under conditions of low signal-to-noise ratio, fast speech, or non-standard pronunciation, resulting in errors in the output phoneme sequence and affecting the performance and reliability of downstream speech processing tasks.
By acquiring the target audio and transcribed text, the transcribed text is divided into sub-words according to pronunciation units using a preset mapping table, and matched with the predicted phonemes of the phoneme recognition model. The matching results are then calibrated to form a more accurate phoneme sequence.
It improves the accuracy of phoneme recognition, reduces the phoneme error rate, simplifies system deployment complexity, adapts to scenarios with limited end-side resources, and provides a more reliable acoustic representation basis for downstream speech tasks.
Smart Images
Figure CN122024734A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a data calibration method, apparatus, device, and medium. Background Technology
[0002] Phoneme recognition, as a fundamental step in speech processing, aims to convert continuous speech signals into discrete phoneme sequences. Currently, end-to-end deep learning models are widely used for phoneme recognition to provide a more refined acoustic representation for downstream speech processing tasks such as speech recognition, speaker verification, and speech emotion analysis. For example, in scenarios with high security and compliance requirements, such as financial insurance, automatic quality inspection of customer service recordings, voiceprint authentication, or fraud risk speech analysis all rely on the accuracy of front-end phoneme recognition. However, due to complex acoustic interference in practical applications, differences in user dialects and accents, and the coarticulation and ambiguity of the speech signal itself, the accuracy of existing end-to-end models in phoneme recognition, especially under conditions of low signal-to-noise ratio, fast speech, or non-standard pronunciation, still has significant limitations. This leads to errors such as insertion, deletion, or obfuscation in the output phoneme sequence, which directly impairs and restricts the performance and reliability of downstream speech processing tasks.
[0003] Therefore, how to perform calibration on the data after phoneme recognition to improve the accuracy of phoneme recognition has become an urgent problem to be solved. Summary of the Invention
[0004] This invention provides a data calibration method, apparatus, device, and medium to address the problem of how to perform data calibration after phoneme recognition in order to improve the accuracy of phoneme recognition.
[0005] Firstly, a data calibration method includes: Obtain the target audio and the corresponding transcribed text. Then, use a target phoneme recognition model to perform phoneme recognition on the target audio to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio. The transcribed text is divided into sub-words based on the pronunciation units. The mapping phoneme of each sub-word is determined according to a preset mapping table, which includes the mapping relationship between each sub-word and its corresponding mapping phoneme. For any audio frame, the audio frame is matched with each sub-word according to the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapped phoneme of each sub-word to obtain the matching result. Based on the matching result, the target sub-word corresponding to the audio frame is determined. Based on the mapped phonemes of the target subword, the predicted phonemes of the audio frame are calibrated to obtain the calibration results of the predicted phonemes of the audio frame. The calibration results of the predicted phonemes of all audio frames are then fused to form the final calibration result.
[0006] Secondly, a data calibration device is provided, comprising: The acquisition module is used to acquire the target audio and the transcribed text corresponding to the target audio, and to perform phoneme recognition on the target audio through the target phoneme recognition model to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio. The mapping module is used to divide the transcribed text into sub-words based on pronunciation units, and to determine the mapping phoneme of each sub-word according to a preset mapping table, wherein the preset mapping table includes the mapping relationship between each sub-word and the corresponding mapping phoneme; The matching module is used to match any audio frame with each sub-word based on the predicted phonemes of the audio frame, the prediction probabilities of the corresponding predicted phonemes, and the mapped phonemes of each sub-word, to obtain the matching results, and to determine the target sub-word corresponding to the audio frame based on the matching results. The calibration module is used to calibrate the predicted phonemes of the audio frame according to the mapped phonemes of the target subword, obtain the calibration result of the predicted phonemes of the audio frame, and fuse the calibration results of the predicted phonemes of all audio frames to form the final calibration result.
[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-described data calibration method when executing the computer program.
[0008] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data calibration method.
[0009] The above data calibration method involves acquiring the target audio and its corresponding transcribed text, performing phoneme recognition on the target audio using a target phoneme recognition model to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio, dividing the transcribed text into sub-words based on pronunciation units, determining the mapped phoneme of each sub-word according to a preset mapping table, matching each audio frame with each sub-word based on the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapped phoneme of each sub-word to obtain matching results, determining the target sub-word corresponding to the audio frame based on the matching results, calibrating the predicted phonemes of the audio frame based on the mapped phonemes of the target sub-word to obtain the calibration results of the predicted phonemes of the audio frame, calibrating the predicted phonemes of the audio frame based on the mapped phonemes of the target sub-word to obtain the calibration results of the predicted phonemes of the audio frame, and finally fusing the calibration results of the predicted phonemes of all audio frames to form the final calibration result.
[0010] This process involves dividing the transcribed text of the target audio into sub-words based on pronunciation units and mapping them to phoneme sequences. This serves as a calibration benchmark. Combined with the frame-by-frame prediction results output by the phoneme recognition model, precise matching and calibration are performed at the frame level. This process effectively utilizes prior text information to correct the prediction bias of the phoneme recognition model, reducing the phoneme error rate and improving recognition accuracy. At the same time, since the calibration process does not require the introduction of external models and relies solely on internal mapping and matching mechanisms, it not only simplifies the system deployment complexity but also enables it to adapt to scenarios with limited edge resources. This provides a more reliable and efficient acoustic representation foundation for downstream speech tasks such as speech recognition, speaker verification, or speech emotion analysis. Attached Figure Description
[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 This is a schematic diagram of an application environment for a data calibration method according to an embodiment of the present invention; Figure 2 This is a flowchart of a data calibration method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a data calibration device in one embodiment of the present invention; Figure 4 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation
[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0014] The data calibration method provided in this embodiment of the invention can be applied to, for example... Figure 1 The application environment shown. Specifically, this data calibration method is applied in a fintech platform, which includes, for example, [examples of such platforms]. Figure 1 The diagram illustrates a client and server that communicate over a network to address the problem of improving the accuracy of phoneme recognition. The client, also known as the user terminal, is the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0015] For example, in intelligent insurance customer service scenarios, clear and accurate voice interaction is the foundation of efficient service. Currently, deep learning-based phoneme recognition technology can automatically convert customer speech into phoneme sequences, providing key features for understanding content, verifying identity, and even judging emotions. However, in actual calls, background noise, customer accents, or changes in speech rate often cause deviations in model predictions, affecting subsequent processing results. To address this issue, the data calibration method proposed in this invention can be seamlessly integrated into existing customer service voice processing workflows. In practice, the system first acquires the customer service call recording and its corresponding accurate transcribed text (e.g., from preliminary transcription or dialogue scripts). Then, the target phoneme recognition model analyzes the audio frame by frame, outputting the predicted phonemes and their probabilities for each frame. Simultaneously, the system segments the transcribed text into sub-words (e.g., "insurance" and "claims") according to pronunciation units and maps them to standard phoneme sequences using preset rules, forming a reliable calibration reference. For each audio frame, the system compares its predicted phonemes and probabilities with the mapped phonemes of each sub-word in the text. A dynamic matching algorithm determines the most likely target sub-word for that frame (e.g., matching a syllable of the word "claims"). Then, using the standard phonemes of this target sub-word, the model's original predictions are calibrated and corrected; for example, vague, low-confidence phonemes are replaced with explicit phonemes guided by the text. Finally, the calibration results of all frames are fused to generate a more accurate phoneme sequence aligned with the text content. This process fully leverages the structured or predictable nature of dialogue in customer service scenarios. By using prior textual information, it effectively corrects model recognition errors in complex acoustic environments. The entire calibration mechanism is fully embedded, requiring no additional complex models, resulting in high computational efficiency. It can be deployed locally within the customer service system or on edge devices, significantly improving the robustness of phoneme recognition. This provides a higher-quality acoustic foundation for subsequent tasks such as policy inquiry speech recognition, customer voiceprint verification, and sentiment analysis, thereby helping insurance customer service achieve a more intelligent and reliable human-computer interaction experience.
[0016] In one embodiment, such as Figure 2 As shown, a data calibration method is provided, which is applied to Figure 1 The server in the middle includes the following steps: Step S201: Obtain the target audio and the corresponding transcribed text. Using the target phoneme recognition model, perform phoneme recognition on the target audio to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio.
[0017] Step S202: Divide the transcribed text into sub-words according to the pronunciation units, and determine the mapped phoneme of each sub-word according to the preset mapping table.
[0018] In this embodiment, the target audio can refer to the speech data to be identified, the transcribed text can refer to the text description corresponding to the target audio content, the target phoneme recognition model can refer to a trained end-to-end phoneme recognition neural network model that can identify phoneme sequences and their temporal distribution from the target audio, the predicted phoneme can refer to the phoneme category corresponding to each audio frame in the target audio predicted by the target phoneme recognition model, and the prediction probability can refer to the confidence score of the target phoneme recognition model for each audio frame belonging to the corresponding phoneme category. The pronunciation unit can refer to the basic unit of a pronounceable segment, the preset mapping table includes the mapping relationship between each subword and the corresponding mapped phoneme, and the mapped phoneme can refer to one or more possible phonemes pre-assigned to each subword in the preset mapping table.
[0019] Specifically, the target audio can be input into the encoder of the target phoneme recognition model, and the encoder can extract features from the target audio to obtain the acoustic features corresponding to the target audio. The acoustic features are then input into the decoder of the target phoneme recognition model, and the decoder can decode to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame. Based on the pronunciation unit, the transcribed text is divided into sub-words, and the mapping phoneme corresponding to each sub-word is determined from a preset mapping table.
[0020] For example, the predicted phoneme and its corresponding prediction probability for the i-th audio frame obtained by phoneme recognition of the target audio using the target phoneme recognition model can be: / tʃ / =0.6, / k / =0.2, / a / =0.1, / b / =0.05. For the transcribed text character, the transcribed text is divided according to the pronunciation unit, and the resulting sub-words can be ["ch","a", "r", "ac", "t", "er"]. Combining with the preset mapping table, it can be determined that the mapped phonemes corresponding to the sub-word "ch" can be { / tʃ / , / k / }, and so on for other sub-words.
[0021] Among them, acoustic features can refer to feature vectors extracted from target audio that characterize the acoustic properties of speech. The encoder is used to extract acoustic features from the target audio, and the decoder is used to decode the extracted acoustic features into the probability distribution of each audio frame across all phonemes.
[0022] Step S203: For any audio frame, match the audio frame with each sub-word according to the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapped phoneme of each sub-word to obtain the matching result. Based on the matching result, determine the target sub-word corresponding to the audio frame.
[0023] In this embodiment, the matching result can refer to the result of matching the audio frame and the sub-word based on the predicted phonemes, the prediction probability of the corresponding predicted phonemes, and the mapped phonemes of the sub-words. The target sub-word can refer to the sub-word selected from all sub-words according to the matching result that matches the acoustic content of the audio frame.
[0024] Specifically, for any given word, based on the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapped phoneme of the word, the prediction probability score of the target phoneme recognition model for the mapped phoneme of the word on the audio frame is calculated. The prediction probability scores of all words are used as matching results, and the word corresponding to the highest prediction probability score in the matching results is determined as the target word corresponding to the audio frame.
[0025] In calculating the prediction probability score of the target phoneme recognition model for the mapped phonemes of a word on an audio frame, each mapped phoneme of the word can be compared with each predicted phoneme of the audio frame. If there is a mapped phoneme in the word that matches the predicted phoneme of the audio frame, then the mapped phoneme in the word that matches the comparison is determined to be the target mapped phoneme, and the predicted phoneme in the audio frame that matches the corresponding target mapped phoneme is determined to be the target predicted phoneme. The prediction probabilities of the target predicted phonemes corresponding to all target mapped phonemes in the word are summed to obtain the prediction probability score of the target phoneme recognition model for the mapped phonemes of the word on the audio frame. If there is no mapped phoneme in the word that matches the predicted phoneme of the audio frame, then the prediction probability score of the target phoneme recognition model for the mapped phonemes of the word on the audio frame is determined to be zero.
[0026] For example, the predicted phonemes and their corresponding prediction probabilities for the i-th audio frame predicted by the target phoneme recognition model can be: / tʃ / =0.6, / k / =0.2, / a / =0.1, / b / =0.05, and the total of other phonemes = 0.05. The mapped phonemes of the word "ch" can be: { / tʃ / , / k / }. By comparing the consistency of each mapped phoneme of this word with each predicted phoneme of the i-th audio frame, we can determine that the target predicted phonemes in the i-th audio frame that match the mapped phonemes of this word are / tʃ / and / k / . The target mapped phonemes of this word are / tʃ / and / k / . Therefore, the prediction probability score of the target phoneme recognition model for the mapped phonemes of this word in the i-th audio frame can be obtained as: the prediction probability of / tʃ / + the prediction probability of / k / = 0.6 + 0.2 = 0.8.
[0027] Step S204: Based on the mapped phonemes of the target word, calibrate the predicted phonemes of the audio frame to obtain the calibration results of the predicted phonemes of the audio frame, and fuse the calibration results of the predicted phonemes of all audio frames to form the final calibration result.
[0028] In this embodiment, the calibration result can refer to the phonemes after the predicted phonemes of the audio frame have been calibrated.
[0029] Specifically, based on the prediction probability of the predicted phoneme corresponding to the audio frame, the predicted phoneme with the highest prediction probability to be calibrated can be determined from the predicted phonemes of the audio frame. If the predicted phoneme to be calibrated is not a target predicted phoneme that matches the corresponding target mapping phoneme in the audio frame and the target sub-word, then the target predicted phoneme with the highest prediction probability is determined from the target predicted phonemes that match the corresponding target mapping phonemes in the audio frame and the target sub-word as the calibration result of the predicted phoneme of the audio frame. If the predicted phoneme to be calibrated is a target predicted phoneme that matches the corresponding target mapping phoneme in the audio frame and the target sub-word, then the predicted phoneme to be calibrated is determined as the calibration result of the predicted phoneme of the audio frame.
[0030] For example, the predicted phoneme and its prediction probability for the i-th audio frame predicted by the target phoneme recognition model can be: / tʃ / =0.2, / k / =0.1, / a / =0.6, / b / =0.05, and the total of other phonemes = 0.05. We can determine that the phoneme to be calibrated with the highest prediction probability is / a / . The target word corresponding to the i-th audio frame is "ch", and its mapped phonemes are { / tʃ / , / k / }. The target predicted phonemes in the i-th audio frame that match the target word are / tʃ / and / k / . The phoneme to be calibrated, / a / , is not a target predicted phoneme. Therefore, we select / tʃ / , which has the highest prediction probability, from the target predicted phonemes / tʃ / and / k / as the calibration result for the predicted phoneme of the i-th audio frame. That is, the final predicted phoneme corresponding to the i-th audio frame is / tʃ / .
[0031] In this embodiment, the transcribed text of the target audio is divided into sub-words based on pronunciation units and mapped to phoneme sequences. This serves as a calibration benchmark. Combined with the frame-by-frame prediction results output by the phoneme recognition model, precise matching and calibration are performed at the frame level. This process effectively utilizes prior text information to correct the prediction bias of the phoneme recognition model, reducing the phoneme error rate and improving recognition accuracy. Furthermore, since the calibration process does not require the introduction of an external model and relies solely on the internal mapping and matching mechanism, it simplifies system deployment complexity and enables it to adapt to resource-constrained scenarios. This provides a more reliable and efficient acoustic representation foundation for downstream speech tasks such as speech recognition, speaker verification, or speech emotion analysis.
[0032] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0033] In one embodiment, a data calibration device is provided, which corresponds one-to-one with the data calibration methods described in the above embodiments. For example... Figure 3 As shown, the data calibration device includes an acquisition module 31, a mapping module 32, a matching module 33, and a calibration module 34. Detailed descriptions of each functional module are as follows: The acquisition module 31 is used to acquire the target audio and the transcribed text corresponding to the target audio, and to perform phoneme recognition on the target audio through the target phoneme recognition model to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio. The mapping module 32 is used to divide the transcribed text into sub-words according to the pronunciation units, and to determine the mapping phoneme of each sub-word according to a preset mapping table, wherein the preset mapping table includes the mapping relationship between each sub-word and the corresponding mapping phoneme; The matching module 33 is used to match the audio frame with each sub-word according to the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapped phoneme of each sub-word for any audio frame, to obtain the matching result, and to determine the target sub-word corresponding to the audio frame according to the matching result. The calibration module 34 is used to calibrate the predicted phonemes of the audio frame according to the mapped phonemes of the target subword, obtain the calibration result of the predicted phonemes of the audio frame, and fuse the calibration results of the predicted phonemes of all audio frames to form the final calibration result.
[0034] Optionally, the acquisition module 31 mentioned above includes: The feature extraction unit is used to input the target audio into the encoder of the target phoneme recognition model, and extract features from the target audio through the encoder to obtain the acoustic features corresponding to the target audio. The phoneme prediction unit is used to input the acoustic features into the decoder of the target phoneme recognition model, and to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame through the decoder.
[0035] Optionally, the matching module 33 mentioned above includes: The first calculation unit is used to calculate, for any sub-word, a prediction probability score representing the target phoneme recognition model's prediction probability of the sub-word's mapping phoneme on the audio frame, based on the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapping phoneme of the sub-word. The first determining unit is used to determine the predicted probability scores of all sub-words as the matching results, and to determine the sub-word corresponding to the highest predicted probability score from the matching results as the target sub-word corresponding to the audio frame.
[0036] Optionally, the first computing unit mentioned above includes: The second determining subunit is used to perform consistency comparison between each mapped phoneme of the sub-word and each predicted phoneme of the audio frame. If there is a mapped phoneme in the sub-word that matches the predicted phoneme in the audio frame, then the mapped phoneme in the sub-word that matches the comparison is determined to be the target mapped phoneme, and the predicted phoneme in the audio frame that matches the corresponding target mapped phoneme is determined to be the target predicted phoneme. The second calculation subunit is used to sum the prediction probabilities of the target predicted phonemes corresponding to all target mapped phonemes in the sub-word to obtain a prediction probability score representing the target phoneme recognition model for the mapped phonemes of the sub-word on the audio frame.
[0037] Optionally, the first computing unit mentioned above further includes: The third determining subunit is used to determine that if there is no mapping phoneme in the mapping phonemes of the sub-word that matches the predicted phonemes of the audio frame, then the prediction probability score of the target phoneme recognition model for the mapping phonemes of the sub-word on the audio frame is zero.
[0038] Optionally, the calibration module 34 includes: The fourth determining unit is used to determine the predicted phoneme with the highest prediction probability from the predicted phonemes of the audio frame based on the prediction probability of the predicted phoneme corresponding to the audio frame. The fifth determining unit is used to determine the target predicted phoneme with the highest prediction probability as the calibration result of the predicted phoneme of the audio frame if the predicted phoneme to be calibrated is not a target predicted phoneme that matches the corresponding target mapped phoneme in the audio frame and the target subword.
[0039] Optionally, the aforementioned calibration module 34 further includes: The sixth determining unit is used to determine the calibration result of the predicted phoneme of the audio frame if the predicted phoneme to be calibrated is the target predicted phoneme that matches the corresponding target mapped phoneme in the audio frame and the target subword.
[0040] Specific limitations regarding the data calibration device can be found in the limitations of the data calibration method described above, and will not be repeated here. Each module in the aforementioned data calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.
[0041] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores target audio and its corresponding transcribed text. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a data calibration method.
[0042] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the data calibration method described in the above embodiment, for example... Figure 2 S201-S204, as shown, will not be described again here to avoid repetition. Alternatively, the processor may implement the functions of each module / unit in this embodiment of the data calibration device when executing the computer program, for example, Figure 3 The functions of the acquisition module 31, mapping module 32, matching module 33, and calibration module 34 shown are not described again here to avoid repetition.
[0043] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the data calibration method described in the above embodiment, for example... Figure 2 S201-S204, as shown, will not be described again here to avoid repetition. Alternatively, the processor may implement the functions of each module / unit in this embodiment of the data calibration device when executing the computer program, for example, Figure 3 The functions of the acquisition module 31, mapping module 32, matching module 33, and calibration module 34 shown are not described again here to avoid repetition.
[0044] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0045] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0046] If any software tools or components not belonging to this company appear in the embodiments of this invention, they are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this invention is authorized (with knowledge and consent) by the relevant parties or fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with the relevant laws and regulations of the relevant countries and regions, and do not violate public order and good morals.
[0047] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A data calibration method, characterized in that, include: Obtain the target audio and the corresponding transcribed text. Then, use a target phoneme recognition model to perform phoneme recognition on the target audio to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio. The transcribed text is divided into sub-words based on the pronunciation units. The mapping phoneme of each sub-word is determined according to a preset mapping table, which includes the mapping relationship between each sub-word and its corresponding mapping phoneme. For any audio frame, the audio frame is matched with each sub-word according to the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapped phoneme of each sub-word to obtain the matching result. Based on the matching result, the target sub-word corresponding to the audio frame is determined. Based on the mapped phonemes of the target subword, the predicted phonemes of the audio frame are calibrated to obtain the calibration results of the predicted phonemes of the audio frame. The calibration results of the predicted phonemes of all audio frames are then fused to form the final calibration result.
2. The data calibration method according to claim 1, characterized in that, The step of performing phoneme recognition on the target audio using a target phoneme recognition model to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio includes: The target audio is input into the encoder of the target phoneme recognition model, and the encoder extracts features from the target audio to obtain the acoustic features corresponding to the target audio. The acoustic features are input into the decoder of the target phoneme recognition model, and the decoder decodes to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame.
3. The data calibration method according to claim 1, characterized in that, For any given audio frame, based on the predicted phonemes of the audio frame, the prediction probabilities of the corresponding predicted phonemes, and the mapped phonemes of each sub-word, the audio frame is matched with each sub-word to obtain a matching result. Based on the matching result, the target sub-word corresponding to the audio frame is determined, including: For any sub-word, a prediction probability score representing the target phoneme recognition model for the sub-word's mapping phoneme on the audio frame is calculated based on the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapping phoneme of the sub-word. The predicted probability scores of all sub-words are used as the matching results, and the sub-word with the highest predicted probability score is determined from the matching results as the target sub-word corresponding to the audio frame.
4. The data calibration method according to claim 3, characterized in that, For any given word, based on the predicted phoneme of the audio frame, the prediction probability of the corresponding predicted phoneme, and the mapped phoneme of the word, a prediction probability score representing the target phoneme recognition model's prediction probability of the mapped phoneme of the word on the audio frame is calculated, including: Each mapped phoneme of the sub-word is compared with each predicted phoneme of the audio frame. If there is a mapped phoneme in the sub-word that matches the predicted phoneme in the audio frame, then the mapped phoneme in the sub-word that matches the comparison is determined to be the target mapped phoneme, and the predicted phoneme in the audio frame that matches the corresponding target mapped phoneme is determined to be the target predicted phoneme. The predicted probabilities of the target predicted phonemes corresponding to all target mapped phonemes in the sub-word are summed to obtain the prediction probability score representing the target phoneme recognition model for the mapped phonemes of the sub-word in the audio frame.
5. The data calibration method according to claim 4, characterized in that, After performing a consistency comparison between each mapped phoneme of the subword and each predicted phoneme of the audio frame, the method further includes: If none of the mapped phonemes of the sub-word matches the predicted phonemes of the audio frame, then determine... The target phoneme recognition model's prediction probability score for the mapped phoneme of the word on the audio frame is zero.
6. The data calibration method according to claim 4, characterized in that, The step of calibrating the predicted phonemes of the audio frame based on the mapped phonemes of the target subword to obtain the calibration result of the predicted phonemes of the audio frame includes: Based on the prediction probability of the predicted phoneme corresponding to the audio frame, determine the predicted phoneme with the highest prediction probability to be calibrated from the predicted phonemes of the audio frame. If the predicted phoneme to be calibrated is not a target predicted phoneme that matches the corresponding target mapped phoneme in the audio frame and the target subword, then the target predicted phoneme with the highest prediction probability is determined from the target predicted phonemes that match the corresponding target mapped phonemes in the audio frame and the target subword as the calibration result of the predicted phoneme of the audio frame.
7. The data calibration method according to claim 6, characterized in that, After determining the prediction phoneme with the highest prediction probability from the predicted phonemes of the audio frame, the method further includes: If the predicted phoneme to be calibrated is the target predicted phoneme that matches the corresponding target mapped phoneme in the audio frame and the target subword, then the predicted phoneme to be calibrated is determined to be the calibration result of the predicted phoneme of the audio frame.
8. A data calibration device, characterized in that, include: The acquisition module is used to acquire the target audio and the transcribed text corresponding to the target audio, and to perform phoneme recognition on the target audio through the target phoneme recognition model to obtain the predicted phoneme and the prediction probability of the corresponding predicted phoneme for each audio frame in the target audio. The mapping module is used to divide the transcribed text into sub-words based on pronunciation units, and to determine the mapping phoneme of each sub-word according to a preset mapping table, wherein the preset mapping table includes the mapping relationship between each sub-word and the corresponding mapping phoneme; The matching module is used to match any audio frame with each sub-word based on the predicted phonemes of the audio frame, the prediction probabilities of the corresponding predicted phonemes, and the mapped phonemes of each sub-word, to obtain the matching results, and to determine the target sub-word corresponding to the audio frame based on the matching results. The calibration module is used to calibrate the predicted phonemes of the audio frame according to the mapped phonemes of the target subword, obtain the calibration result of the predicted phonemes of the audio frame, and fuse the calibration results of the predicted phonemes of all audio frames to form the final calibration result.
9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data calibration method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the data calibration method as described in any one of claims 1 to 7.