Homophone error correction method and device and storage medium

Abstract

The invention provides a homophone error correction method and device and a storage medium, and belongs to the technical field of character error correction, and the method comprises the steps: importing to-be-corrected voice data, original voice data and real text data; performing voice recognition on the to-be-corrected voice data and the original voice data to obtain to-be-corrected text features and original text features; constructing a training model, and performing model analysis on the training model according to the original text features and the real text data to obtain a homophone error correction model; and performing error correction analysis on the to-be-corrected text features through the homophone error correction model to obtain a homophone error correction result. The method improves the context association capability, does not need to depend on shallow text matching, improves the adaptability of a low-resource scene and a dynamic scene, fully considers the association of voice acoustic features, improves the error-tolerant rate of a rare homophone combination, and also can remarkably improve the homophone error correction accuracy.

Description

A homophone error correction method, device, and storage medium Technical Field

[0001] The present invention mainly relates to the technical field of text error correction, and particularly relates to a homophone error correction method, device, and storage medium. Background Art

[0002] With the popularization of speech recognition in fields such as consumer electronics and enterprise services, "from 'capable of recognition' to 'accurate recognition'" has become the industry upgrade direction. In the early stage, ASR pursued "meeting the recognition accuracy standard" (such as above 95%), but homophone errors are "the fish that slipped through the net of the accuracy standard", directly affecting the user experience; in low-resource scenarios (such as dialects, technical terms), the homophone problem is more prominent - homophone errors in professional fields (such as "dosage" vs "trick") can lead to decision-making risks in scenarios such as medical and financial fields.

[0003] The existing homophone error correction methods mainly have the following problems: 1. Weak context association ability, relying on shallow text matching: In the intelligent customer service scenario, when the user says "Help me check the details of this month's water and electricity bills", if the speech recognition misoutputs the homophone "clarity" of "details", the existing model can only judge that the collocation probability of "water and electricity bills + clarity" is low, but cannot accurately recognize that "details (data details)" is the correct expression that conforms to the semantics based on the scenario intention of "querying expense data", resulting in the correction accuracy being limited by the coverage of the preset rules.

[0004] 2. Overly relying on labeled data and having poor adaptability to low-resource scenarios: In low-resource scenarios such as dialect speech recognition (such as "chicken wing" and "airfoil" in Cantonese), emerging field terms (such as "last hit" and "get last hit" in e-sports), etc., the labeled data is scarce or updated laggingly, resulting in the model being unable to effectively learn the scenario differences of homophones, presenting problems such as "no data available for training, difficult to update with available data", and it is difficult to meet the voice interaction needs of personalized or niche fields.

[0005] 3. Not considering the relevance of speech acoustic features, and the error correction logic is fragmented: Existing solutions mostly focus on text layer correction after speech recognition, ignoring the acoustic feature differences of homophones in the speech input stage (such as the tone difference between "buy (mǎi)" and "sell (mài)", and the pronunciation weight difference between "mountain (shān)" and "delete (shān)"). For example, when the user says "mai a book" with an accent, the speech recognition first misrecognizes "buy" as "sell", and the subsequent text correction model may not be able to judge the semantic contradiction of "sell a book" due to not associating with the tone feature of the original speech ("buy" is the third tone, "sell" is the fourth tone), relying only on the text context, resulting in the fragmentation of the technical logic between the correction link and the speech input link, and missing the key information to improve the correction accuracy based on acoustic features.

[0006] 4. Insufficient adaptability to dynamic scenarios and difficulty in handling semantic ambiguity: In scenarios of multi-scenario switching or semantic ambiguity, the existing technologies cannot dynamically adjust the judgment weights of homophones. For example, in a navigation scenario, when a user says "turn right at the intersection ahead", if it is recognized as the homophone "right spread", the existing model can correct it based on the "navigation + intersection" scenario. However, when the user is in the car and simultaneously conducts "music on-demand" (such as "play the song 'Right Spread'") and "navigation" interactions, the model cannot dynamically distinguish "turn right" from "right spread" according to the real-time scenario intention (whether it is a music requirement or a navigation requirement), which easily leads to scenario confusion and correction errors. This problem is particularly prominent in scenarios such as intelligent cockpits with multi-task concurrency.

[0007] 5. Limited model generalization ability and low tolerance for rare homophone combinations: The existing technologies train models with high-frequency homophone pairs and have good correction effects on daily common combinations (such as "they" and "她们"), but have extremely low tolerance for rare homophone combinations (such as the professional terms "threshold" and "阈植", the place names "Bozhou (bózhōu)" and "毫州 (háo zhōu)"). Due to the low frequency of occurrence of such rare combinations in the training data, the model has not learned their semantic differences and scenario associations, resulting in ineffective correction after misrecognition of homophones in scenarios such as government services (such as place name queries) and scientific research exchanges (such as professional term interactions), directly affecting the professionalism and reliability of voice interaction. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a homophone error correction method, device, and storage medium in view of the deficiencies of the existing technologies.

[0009] The technical solution of the present invention for solving the above technical problems is as follows: A homophone error correction method includes the following steps: Importing multiple voice data to be corrected, multiple original voice data, and real text data corresponding to each of the original voice data; respectively performing speech recognition on each of the voice data to be corrected and the original voice data corresponding to each of the original voice data to obtain text features to be corrected corresponding to each of the voice data to be corrected and original text features corresponding to each of the original voice data; constructing a training model, and performing model analysis on the training model according to all the original text features and all the real text data to obtain a homophone error correction model; performing error correction analysis on all the text features to be corrected through the homophone error correction model to obtain homophone error correction results.

[0010] Another technical solution of the present invention to solve the above-mentioned technical problem is as follows: a homophone correction device, comprising: an import module, used to import multiple speech data to be corrected, multiple original speech data, and real text data corresponding to each of the original speech data; a speech recognition module, used to perform speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data, to obtain text features to be corrected corresponding to each of the speech data to be corrected and original text features corresponding to each of the original speech data; a model analysis module, used to construct a training model, and perform model analysis on the training model based on all the original text features and all the real text data to obtain a homophone correction model; and a correction result acquisition module, used to perform correction analysis on all the text features to be corrected through the homophone correction model to obtain a homophone correction result.

[0011] Based on the above-mentioned homophone correction method, the present invention also provides a homophone correction system.

[0012] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a homophone correction system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the homophone correction method as described above.

[0013] Based on the above-mentioned homophone correction method, the present invention also provides a computer-readable storage medium.

[0014] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the homophone correction method as described above.

[0015] The beneficial effects of this invention are as follows: by obtaining the text features to be corrected and the original text features through speech recognition of the speech data to be corrected and the original speech data, a homophone correction model is obtained by model analysis of the training model based on the original text features and real text data, and the homophone correction result is obtained by analyzing the correction of the text features to be corrected through the homophone correction model. This improves the ability to associate context, does not rely on shallow text matching, improves the adaptability of low-resource scenarios and dynamic scenarios, fully considers the correlation of speech acoustic features, improves the error tolerance rate for rare homophone combinations, and can also significantly improve the accuracy of homophone correction. Attached Figure Description

[0016] Figure 1 is a flowchart illustrating the homophone correction method provided in an embodiment of the present invention; Figure 2 is a comparative learning diagram illustrating the homophone correction method provided in an embodiment of the present invention; Figure 3 is a structural diagram illustrating the training model of the homophone correction method provided in an embodiment of the present invention; Figure 4 is a block diagram illustrating the homophone correction device provided in an embodiment of the present invention. Detailed Implementation

[0017] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0018] Figure 1 is a flowchart illustrating a homophone correction method provided in an embodiment of the present invention.

[0019] As shown in Figure 1, a homophone correction method includes the following steps: S1: Import multiple speech data to be corrected, multiple original speech data, and real text data corresponding to each of the original speech data; S2: Perform speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data to obtain the text features to be corrected and the original text features corresponding to each of the original speech data; S3: Construct a training model, and perform model analysis on the training model based on all the original text features and all the real text data to obtain a homophone correction model; S4: Perform correction analysis on all the text features to be corrected through the homophone correction model to obtain the homophone correction result.

[0020] It should be understood that voice-text pairs (i.e., raw voice data and actual text data) are collected in intelligent customer service scenarios.

[0021] In the above embodiments, the text features to be corrected and the original text features are obtained through speech recognition of the speech data to be corrected and the original speech data. Based on the original text features and real text data, the model analysis of the trained model is used to obtain the homophone correction model. The homophone correction result is obtained by analyzing the correction of the text features to be corrected through the homophone correction model. This improves the ability to associate context, does not rely on shallow text matching, improves the adaptability of low-resource scenarios and dynamic scenarios, fully considers the correlation of speech acoustic features, improves the error tolerance rate for rare homophone combinations, and can also significantly improve the accuracy of homophone correction.

[0022] Optionally, as an embodiment of the present invention, the process of performing speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data to obtain the text features to be corrected corresponding to each of the speech data to be corrected and the original text features corresponding to each of the original speech data includes: performing speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data using a pre-built Paraformer model to obtain the text features to be corrected corresponding to each of the speech data to be corrected and the original text features corresponding to each of the original speech data.

[0023] It should be understood that ASR training or fine-tuning is performed using an existing Paraformer model (i.e., a pre-built Paraformer model) to obtain basic ASR capabilities (i.e., the text features to be corrected and the original text features), which provides n-best decoding or bundle candidates.

[0024] In the above embodiments, speech recognition is performed on each speech data to be corrected and the original speech data to obtain the text features to be corrected and the original text features, which improves the ability to associate contexts, does not rely on shallow text matching, and improves the adaptability to low-resource scenarios and dynamic scenarios.

[0025] Optionally, as an embodiment of the present invention, the training model includes an acoustic encoder, a PLM encoder, an acoustic projection head, and a text projection head. The process of performing model analysis on the training model based on all the original text features and all the real text data to obtain a homophone correction model includes: S31: generating homophone replacement rules using the pypinyin pinyin tool; S32: encoding each of the original text features using the acoustic encoder to obtain original acoustic features corresponding to each of the original speech data; S33: encoding each of the original acoustic features using the PLM encoder to obtain positive candidate text features corresponding to each of the original speech data; S34: performing text replacement on each of the original text features according to the homophone replacement rules to obtain a homophone correction model corresponding to each of the original speech data. S35: Map each of the original acoustic features using the acoustic projection head to obtain mapped acoustic features corresponding to each of the original speech data; S36: Map each of the original text features using the text projection head to obtain mapped text features corresponding to each of the original speech data; S37: Perform loss value analysis on all the positive candidate text features, all the negative candidate text features, all the mapped acoustic features, all the mapped text features, and all the real text data to obtain a target loss value; S38: Determine whether the target loss value is less than or equal to a preset loss threshold. If not, update the parameters of the training model according to the target loss value, and return to S31 after the update; if yes, use the training model as a homophone correction model.

[0026] It should be understood that, based on the output candidates (n-best or N bundles) of Paraformer (i.e., the original text features), a candidate generator is designed to produce a set of homophone candidates (based on pinyin, edit distance, or pronunciation dictionary) (i.e., the mapped text features).

[0027] Specifically, based on ASR output and text labeled by human / language models (i.e., original text features and real text data), positive samples (correct words / phrases) (i.e. positive candidate text features) and negative samples (incorrect phrases generated by homophone substitution) (i.e. negative candidate text features) are generated.

[0028] Specifically, each candidate sentence (i.e., the original text features) is processed by Paraformer's encoder or a separate semantic encoder (e.g., BERT / PLM) (i.e., the PLM encoder) to obtain a semantic vector (i.e., the positive candidate text features). At the same time, the corresponding acoustic embedding (i.e., the original acoustic features) is extracted through the acoustic module / encoder. The acoustic embedding (i.e., the original acoustic features) and the semantic embedding of the candidate text (i.e., the original text features) are mapped to a shared vector space through a small projection head (i.e., the acoustic projection head and the text projection head).

[0029] Specifically, for each training sample (i.e., the original text features), the acoustic embedding is obtained. (Taken from Paraformerencoder or standalone acoustic encoder) (i.e., raw acoustic features), and semantic embeddings of the correct candidate text. (PLM encoding) (i.e., positive candidate text features), and construct semantic embeddings of several negative candidate texts based on homophone substitution. (i.e., negative candidate text features).

[0030] via projection head (i.e., acoustic projection head and text projection head) are mapped to a shared low-dimensional space, as follows: .

[0031] In the above embodiments, a homophone correction model is obtained by performing model analysis on the trained model based on all original text features and all real text data. This improves the ability to associate contexts, eliminates the need to rely on shallow text matching, enhances adaptability to low-resource and dynamic scenarios, fully considers the correlation of speech acoustic features, improves the error tolerance rate for rare homophone combinations, and can also significantly improve the accuracy of homophone correction.

[0032] Optionally, as an embodiment of the present invention, the process of S37 includes: collecting all the positive candidate text features to obtain a positive candidate text feature set, collecting all the mapped acoustic features to obtain a mapped acoustic feature set; calculating the contrast loss value by applying a first formula to the positive candidate text feature set, the mapped acoustic feature set, and all the negative candidate text features, wherein the first formula is: ,in, To compare the loss values, For the set of acoustic features after mapping, This is the set of positive candidate text features. For preset temperature parameters, For similarity function, It is an exponential function. For the first The negative candidate text features corresponding to each original speech data are analyzed; cross-entropy is calculated on all the mapped text features and all the real text data to obtain the cross-entropy loss value; the target loss value is obtained by calculating the contrast loss value and the cross-entropy loss value using the second formula, which is: ,in, For the target loss value, For balance coefficient, To compare the loss values, This represents the cross-entropy loss value.

[0033] It should be understood that using contrastive loss (e.g., InfoNCE) makes correct candidates closer to the acoustic vector and incorrect candidates farther away.

[0034] Specifically, the training process simultaneously minimizes the contrastive loss and the language model or reordering loss (e.g., cross-entropy, ranking loss).

[0035] Specifically, the InfoNCE loss (i.e., the contrastive loss value) is used as follows: Simultaneously, language model reordering loss or cross-entropy is added. (i.e., cross-entropy loss value), the total loss (i.e., the target loss value) is as follows: .

[0036] In the above embodiments, loss value analysis is performed on all positive candidate text features, all negative candidate text features, all mapped acoustic features, all mapped text features, and all real text data to obtain the target loss value. This fully considers the correlation of speech acoustic features, improves the error tolerance rate for rare homophone combinations, and can also significantly improve the accuracy of homophone error correction.

[0037] Optionally, as an embodiment of the present invention, the process of performing error correction analysis on all the text features to be corrected using the homophone correction model to obtain homophone correction results includes: performing error correction processing on each of the text features to be corrected using the homophone correction model to obtain the corrected acoustic features and the corrected text features corresponding to each of the speech data to be corrected; calculating the similarity score corresponding to each of the text features to be corrected and the corrected acoustic features corresponding to each of the speech data to be corrected using a third formula, wherein the third formula is: ,in, For the first The similarity score corresponding to each piece of speech data to be corrected. For the first The corrected acoustic features corresponding to each piece of speech data to be corrected. For the first The text features corresponding to each piece of speech data to be corrected. The similarity scores are sorted in descending order, and all corrected text features corresponding to the sorted similarity scores are used as homophone correction results.

[0038] It should be understood that a well-trained bias-correcting discriminator can score and reorder the Paraformer's n-best candidates (i.e., the corrected text features) during the inference phase, and finally output the optimal text.

[0039] Specifically, for a new speech sample, Paraformer provides n-best candidates (i.e., text features after error correction), the error correction module calculates the similarity score between each candidate and the acoustic embedding, reorders them, and outputs the final text.

[0040] In the above embodiments, the homophone correction model is used to perform error correction analysis on all text features to be corrected, and the homophone correction results are obtained. This fully considers the correlation of speech acoustic features, improves the error tolerance rate for rare homophone combinations, and can also significantly improve the accuracy of homophone correction.

[0041] Optionally, as another embodiment of the present invention, as shown in Figure 2, contrastive learning is a self-supervised learning method that aims to extract useful feature representations from unlabeled data by having the model learn the similarities and differences between data samples. Unlike traditional supervised learning, which relies on a large amount of labeled data, contrastive learning can be trained using unlabeled data, greatly reducing the need for labeled data and broadening the sources and applications of data. Its core idea is to construct positive and negative sample pairs so that during training, the model brings the features of positive sample pairs closer together in the feature space and pushes the features of negative sample pairs further apart. For example, in the image domain, image pairs obtained by performing different data augmentation operations (such as cropping, rotating, scaling, etc.) on the same image can be used as positive sample pairs because they essentially represent the same object or scene; while different images are used as negative sample pairs. By minimizing the distance between positive sample pairs (such as cosine distance, Euclidean distance, etc.) while maximizing the distance between negative sample pairs, the model can learn the intrinsic structure and feature representation of the data, as shown in Figure 2.

[0042] In small - sample image recognition tasks, traditional supervised learning requires a large amount of labeled data, while in actual scenarios, the labeling cost is high and the efficiency is low. Contrastive learning enhances the model's feature representation ability under unsupervised or weakly supervised conditions by constructing positive and negative sample pairs. Taking SimCLR (Simple Contrastive Learning) as an example, it adopts a two - stage training strategy. In the first stage, image variants are generated through data augmentation, and in the second stage, the cosine similarity between feature maps is calculated. Experiments on the ImageNet - 1K dataset show that using only 10% of the labeled samples, SimCLR can achieve 92% of the performance of the baseline model. MoCo (Momentum Contrastive Learning) proposes a dynamic contrastive objective. By maintaining a dynamically updated queue, new samples are compared with samples in the queue whose similarity is lower than the threshold. In the 5 - way 1 - shot experiment on the CIFAR - 100 dataset, the MoCov2 model improves the accuracy by 15.7% compared with traditional methods. These cases fully demonstrate that contrastive learning can effectively improve the model performance and reduce the dependence on large - scale labeled data in the field of image recognition, especially in small - sample learning scenarios.

[0043] When dealing with homophone problems, contrastive learning has unique advantages. Since homophones have the same acoustic features in speech, traditional speech recognition methods are difficult to distinguish. Contrastive learning can let the model learn the semantic and context feature differences of homophones in different contexts by constructing positive and negative sample pairs containing homophones. For example, for the two homophones "formula" and "company", the text containing "formula" and being semantically related can be used as a positive sample pair, and the text containing "company" and being semantically related can be used as another positive sample pair, while "formula" and "company" are respectively combined with text that is not semantically related to form negative sample pairs. During the training process of the model, it will gradually learn the semantic features of "formula" and "company" in different contexts, so as to more accurately distinguish them during speech recognition.

[0044] Currently, existing ASR systems (such as the end - to - end model based on Paraformer) perform well in general speech recognition tasks, but are prone to being affected by acoustic ambiguity when dealing with Chinese homophones (such as "wang / wang / wang" etc.). Especially when there are accents, noise or uncertain candidates generated by the ASR decoding tendency, homophone errors can cause misjudgment of business logic or a decline in user experience. Common solution strategies include relying on stronger language models, post - processing rules or context - based error - correction models, but these methods often cannot fully utilize the cross - information between acoustics and semantics or perform discriminative learning on difficult cases.

[0045] Therefore, the present invention constructs positive and negative sample pairs by jointly using acoustic representations and semantic representations, thereby enhancing the model's ability to distinguish homophones at the discriminative level, and further improving the ASR back - end correction effect.

[0046] Optionally, as another embodiment of the present invention, in terms of system architecture, a lightweight online module is designed to ensure the low latency requirements of conversational intelligent customer service scenarios and support conversation context caching and incremental updates.

[0047] Optionally, as another embodiment of the present invention, the key technical point of the present invention is: on the basis of the original speech recognition model, the semantic information is enhanced by adding contrast loss to distinguish homophones; a homophone training set is automatically constructed through code, and positive and negative sample data can be generated autonomously during the training process to expand the data range; the original speech recognition architecture is not changed, that is, the efficiency and latency remain unchanged on the basis of the original architecture.

[0048] Alternatively, as another embodiment of the present invention, compared with the prior art, the present invention has the following significant advantages: compared with post-processing methods that rely solely on LM or rules, the present invention significantly improves the accuracy of homophone correction through acoustic-semantic alignment discriminative learning; and the module design can be seamlessly integrated with the existing Paraformer model, supporting offline training and online inference.

[0049] Optionally, as another embodiment of the present invention, as shown in FIG3, the main modules of the present invention include: Encoder: is an autoregressive encoder, such as SAN-M, Conformer, etc.

[0050] Predictor: The predictor consists of two convolutional layers, and the output is a floating-point weight α, ranging from 0 to 1. The accumulated weight α is used to predict the number of tokens; predict the number of target characters, extract the acoustic representation corresponding to the target characters, and optimize the target MAE.

[0051] Sampler: Employs Glancing LM to enhance non-autoregressive context modeling capabilities, based on semantic embeddings of sound and character embeddings. Decoder: Similar in structure to the autoregressive model, employing unidirectional / bidirectional networks.

[0052] Loss fucntiong: cross-entroy loss (CE), mean absolute error (MAE), and MWERloss, InfoNCE loss.

[0053] Figure 4 is a block diagram of a homophone correction device provided in an embodiment of the present invention.

[0054] Optionally, as another embodiment of the present invention, as shown in FIG4, a homophone correction device includes: an import module for importing multiple speech data to be corrected, multiple original speech data, and real text data corresponding to each of the original speech data; a speech recognition module for performing speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data to obtain text features to be corrected and original text features corresponding to each of the original speech data; a model analysis module for constructing a training model and performing model analysis on the training model based on all the original text features and all the real text data to obtain a homophone correction model; and a correction result acquisition module for performing correction analysis on all the text features to be corrected through the homophone correction model to obtain a homophone correction result.

[0055] Optionally, as an embodiment of the present invention, the speech recognition module is specifically used to: perform speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data using a pre-built Paraformer model, so as to obtain the text features to be corrected corresponding to each of the speech data to be corrected and the original text features corresponding to each of the original speech data.

[0056] Optionally, as an embodiment of the present invention, the training model includes an acoustic encoder, a PLM encoder, an acoustic projection head, and a text projection head. The model analysis module is specifically used for: S31: generating homophone replacement rules using the pypinyin pinyin tool; S32: encoding each of the original text features using the acoustic encoder to obtain the original acoustic features corresponding to each of the original speech data; S33: encoding each of the original acoustic features using the PLM encoder to obtain the positive candidate text features corresponding to each of the original speech data; S34: performing text replacement on each of the original text features according to the homophone replacement rules to obtain the negative candidate text features corresponding to each of the original speech data; S35: using the acoustic... The projection head maps each of the original acoustic features to obtain mapped acoustic features corresponding to each of the original speech data; S36: The text projection head maps each of the original text features to obtain mapped text features corresponding to each of the original speech data; S37: Loss analysis is performed on all positive candidate text features, all negative candidate text features, all mapped acoustic features, all mapped text features, and all real text data to obtain a target loss value; S38: It is determined whether the target loss value is less than or equal to a preset loss threshold. If not, the parameters of the training model are updated according to the target loss value, and the process returns to S31 after the update; if yes, the training model is used as a homophone correction model.

[0057] Optionally, another embodiment of the present invention provides a homophone correction system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the homophone correction method as described above. This system can be a computer or similar system.

[0058] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the homophone correction method as described above.

[0059] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0060] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0061] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0062] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0063] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0064] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for correcting homophones, characterized in that, The method includes the following steps: importing multiple speech data to be corrected, multiple original speech data, and real text data corresponding to each of the original speech data; performing speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data to obtain the text features to be corrected and the original text features corresponding to each of the original speech data; constructing a training model, performing model analysis on the training model based on all the original text features and all the real text data to obtain a homophone correction model; and performing correction analysis on all the text features to be corrected through the homophone correction model to obtain the homophone correction result.

2. The homophone correction method according to claim 1, characterized in that, The process of performing speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data to obtain the text features to be corrected corresponding to each of the speech data to be corrected and the original text features corresponding to each of the original speech data includes: performing speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data using a pre-built Paraformer model to obtain the text features to be corrected corresponding to each of the speech data to be corrected and the original text features corresponding to each of the original speech data.

3. The homophone correction method according to claim 1, characterized in that, The training model includes an acoustic encoder, a PLM encoder, an acoustic projection head, and a text projection head. The process of performing model analysis on the training model based on all the original text features and all the real text data to obtain a homophone correction model includes: S31: generating homophone replacement rules using the pypinyin pinyin tool; S32: encoding each of the original text features using the acoustic encoder to obtain the original acoustic features corresponding to each of the original speech data; S33: encoding each of the original acoustic features using the PLM encoder to obtain the positive candidate text features corresponding to each of the original speech data; S34: performing text replacement on each of the original text features according to the homophone replacement rules to obtain the negative candidate text corresponding to each of the original speech data. Features; S35: Map each of the original acoustic features using the acoustic projection head to obtain mapped acoustic features corresponding to each of the original speech data; S36: Map each of the original text features using the text projection head to obtain mapped text features corresponding to each of the original speech data; S37: Perform loss value analysis on all positive candidate text features, all negative candidate text features, all mapped acoustic features, all mapped text features, and all real text data to obtain a target loss value; S38: Determine whether the target loss value is less than or equal to a preset loss threshold. If not, update the parameters of the training model according to the target loss value, and return to S31 after the update; if yes, use the training model as a homophone correction model.

4. The homophone correction method according to claim 3, characterized in that, The process S37 includes: aggregating all positive candidate text features to obtain a positive candidate text feature set, and aggregating all mapped acoustic features to obtain a mapped acoustic feature set; calculating the contrast loss value by applying a first formula to the positive candidate text feature set, the mapped acoustic feature set, and all negative candidate text features, wherein the first formula is: ,in, To compare the loss values, For the set of acoustic features after mapping, This is the set of positive candidate text features. For preset temperature parameters, For similarity function, It is an exponential function. For the first The negative candidate text features corresponding to each original speech data are analyzed; cross-entropy is calculated on all the mapped text features and all the real text data to obtain the cross-entropy loss value; the target loss value is obtained by calculating the contrast loss value and the cross-entropy loss value using the second formula, which is: ,in, For the target loss value, For balance coefficient, To compare the loss values, This represents the cross-entropy loss value.

5. The homophone correction method according to claim 1, characterized in that, The process of performing error correction analysis on all the text features to be corrected using the homophone correction model to obtain homophone correction results includes: performing error correction processing on each of the text features to be corrected using the homophone correction model to obtain the corrected acoustic features and the corrected text features corresponding to each of the speech data to be corrected; calculating the similarity score corresponding to each of the text features to be corrected and the corrected acoustic features corresponding to each of the speech data to be corrected using a third formula, wherein the third formula is: ,in, For the first The similarity score corresponding to each piece of speech data to be corrected. For the first The corrected acoustic features corresponding to each piece of speech data to be corrected. For the first The text features corresponding to each piece of speech data to be corrected. The similarity scores are sorted in descending order, and all corrected text features corresponding to the sorted similarity scores are used as homophone correction results.

6. A homophone correction device, characterized in that, include: The import module is used to import multiple voice data to be corrected, multiple original voice data, and real text data corresponding to each of the original voice data. The speech recognition module is used to perform speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data, to obtain the text features to be corrected corresponding to each of the speech data to be corrected and the original text features corresponding to each of the original speech data. The model analysis module is used to construct a training model and perform model analysis on the training model based on all the original text features and all the real text data to obtain a homophone correction model. The correction result acquisition module is used to perform correction analysis on all the text features to be corrected through the homophone correction model to obtain homophone correction results.

7. The homophone correction device according to claim 6, characterized in that, The speech recognition module is specifically used to: perform speech recognition on each of the speech data to be corrected and the original speech data corresponding to each of the original speech data using a pre-built Paraformer model, so as to obtain the text features to be corrected corresponding to each of the speech data to be corrected and the original text features corresponding to each of the original speech data.

8. The homophone correction device according to claim 6, characterized in that, The training model includes an acoustic encoder, a PLM encoder, an acoustic projection head, and a text projection head. The model analysis module is specifically used for: S31: generating homophone replacement rules using the pypinyin pinyin tool; S32: encoding each of the original text features using the acoustic encoder to obtain the original acoustic features corresponding to each of the original speech data; S33: encoding each of the original acoustic features using the PLM encoder to obtain positive candidate text features corresponding to each of the original speech data; S34: performing text replacement on each of the original text features according to the homophone replacement rules to obtain negative candidate text features corresponding to each of the original speech data; S35: using the acoustic projection head to... The original acoustic features are mapped to obtain mapped acoustic features corresponding to each of the original speech data; S36: The original text features are mapped using the text projection head to obtain mapped text features corresponding to each of the original speech data; S37: Loss analysis is performed on all positive candidate text features, all negative candidate text features, all mapped acoustic features, all mapped text features, and all real text data to obtain a target loss value; S38: It is determined whether the target loss value is less than or equal to a preset loss threshold. If not, the parameters of the training model are updated according to the target loss value, and the process is returned to S31; if yes, the training model is used as a homophone correction model.

9. A homophone correction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the homophone correction method as described in any one of claims 1 to 5.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the homophone correction method as described in any one of claims 1 to 5.

Images