Text error correction method and device

Abstract

The present application discloses a text error correction method and device. Based on the speech recognition text output by the speech recognition system, the speech recognition text is recognized to obtain the error information, text semantics and domain information of the speech recognition text respectively; based on the error information, text semantics and domain information, the error correction position of the speech recognition text is obtained through the position submodel in the text error correction model; the error text at the error correction position is corrected through the error correction submodel in the text error correction model to obtain the corrected speech recognition text. This method can greatly improve the accuracy of the error correction position of the speech recognition text obtained by the error correction submodel by integrating the error information, text semantics and domain information of the speech recognition text; by correcting the error text at the accurate error position in the text, the correctness of the speech recognition text after the final correction can be greatly improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of speech recognition technology, and in particular to a text error correction method and device. Background Art

[0002] In recent years, the effect of Automatic Speech Recognition (ASR) has been very stable, and the word error rate of speech recognition has been decreasing every year. The application of ASR technology in some fields has achieved good results, such as telephone calls, human-computer interaction, automatic dictation, etc. However, the problem of automatic transcription of speech for any speaker and in any environment is far from being solved. Due to the influence of multiple factors such as environmental noise, voice quality, dialect and speaking style, as well as the vocabulary of the ASR system, errors in ASR are inevitable. For example, unexpected erroneous texts such as homophones, similar-sounding words, and typos lead to a series of errors such as part-of-speech tagging and dependency analysis. Therefore, under the bottleneck of speech recognition technology, text error correction has become a very critical link. Good error correction can greatly improve the quality of output text.

[0003] Text correction for speech recognition is a natural language processing technology that corrects erroneous content in text after speech recognition. It specifically includes corrections for typos, grammatical errors, and semantic errors in specific scenarios. Text correction technology can reduce the error rate of speech recognition. Existing text correction methods mainly include: methods based on Chinese word segmentation, N-gram language model, linguistic rules and confusion word library, methods based on statistical machine translation and neural machine translation, methods based on hash strings and dependency syntax, methods based on deep learning, etc. The accuracy and effectiveness of these methods vary, but most of them are used in general fields for input errors and human errors. Summary of the invention

[0004] The present application provides a text error correction method and device for accurately correcting a speech recognition text output by a speech recognition system, thereby outputting a highly accurate speech recognition text to a user.

[0005] In a first aspect, an embodiment of the present application provides a text error correction method, the method comprising: based on a speech recognition text output by a speech recognition system, recognizing the speech recognition text, and respectively obtaining error information of the speech recognition text, text semantics of the speech recognition text, and domain information of the speech recognition text; based on the error information, the text semantics, and the domain information, obtaining the error correction position of the speech recognition text through a position sub-model in a text error correction model; correcting the erroneous text at the error correction position through the error correction sub-model in the text error correction model to obtain the corrected speech recognition text; the text error correction model is obtained by training a first loss value of the position sub-model and a second loss value of the error correction sub-model.

[0006] In the above scheme, by respectively identifying the error information, text semantics and domain information of a speech recognition text output by the speech recognition system, the error information, text semantics and domain information of the speech recognition text obtained can be used to obtain the error correction position of the speech recognition text through the position submodel in the text error correction model, and the error text at the determined error correction position can be corrected through the error correction submodel in the text error correction model, so that the corrected speech recognition text can be obtained. In this method, by integrating the error information, text semantics and domain information of the speech recognition text, the accuracy of the error correction position of the speech recognition text obtained by the error correction submodel can be greatly improved; then, by correcting the error text at the accurate error position in the text, the correctness of the speech recognition text after the final correction can be greatly improved, thereby improving the user experience in the speech-to-text scenario.

[0007] In one possible implementation method, the speech recognition text is recognized to obtain error information of the speech recognition text, including: the speech recognition text is recognized through an error learning module to obtain error information of the speech recognition text; the error learning module is used to compare the original text and the error correction text in the historical samples, and generate and store various error information.

[0008] In the above scheme, given that the error learning module obtains and stores the error correction position and error correction information of each original text by comparing the original text and the error correction text in the historical samples, the error information of the speech recognition text can be obtained by using the error learning module to recognize the speech recognition text. The obtained error information can be subsequently used by the text error correction model, and ultimately the speech recognition text can generate a highly accurate corrected speech recognition text after the error correction processing of the text error correction model, thereby improving the user experience in the speech-to-text scenario.

[0009] In one possible implementation method, the speech recognition text is recognized to obtain the text semantics of the speech recognition text, including: recognizing the speech recognition text through a text semantics acquisition module to obtain the text semantics of the speech recognition text; the text semantics acquisition module is obtained by learning word vectors for original texts in historical samples and performing semantic learning based on the word vectors.

[0010] In the above scheme, given that the text semantics acquisition module is obtained by learning word vectors through the original text in historical samples and semantic learning based on the word vectors, the text semantics of the speech recognition text can be obtained by using the text semantics acquisition module to acquire the text semantics of the speech recognition text. The obtained text semantics can be subsequently used by the text error correction model, and ultimately the speech recognition text can generate highly accurate corrected speech recognition text after error correction processing by the text error correction model, thereby improving the user experience in the speech-to-text scenario.

[0011] In one possible implementation method, the speech recognition text is recognized to obtain domain information of the speech recognition text, including: recognizing the speech recognition text through a domain information acquisition module to obtain domain information of the speech recognition text; the domain information acquisition module performs domain learning on word vectors of original texts in historical samples, and sets domain weights for domain word vectors of the original text based on the domain.

[0012] In the above scheme, given that the domain information acquisition module is obtained by performing domain learning on the word vectors of the original text in the historical samples and setting domain weights for the domain word vectors of the original text based on the domain, the domain information of the speech recognition text can be obtained by using the domain information acquisition module to acquire the domain information of the speech recognition text. The obtained domain information can be subsequently used by the text error correction model, and ultimately the speech recognition text can be corrected by the text error correction model to generate a highly accurate corrected speech recognition text, thereby improving the user experience in the speech-to-text scenario.

[0013] In a possible implementation method, the text error correction model is trained in the following manner, including: for the original text and the corrected text in the historical sample, obtaining the error information of the original text through the error learning module; the error information includes the error correction position and the error correction information; for the original text in the historical sample, obtaining the text semantics of the original text through the text semantic acquisition module; for the original text in the historical sample, obtaining the domain information of the original text through the domain information acquisition module; using the error information of the original text, the text semantics of the original text and the domain information of the original text as input values ​​of the text error correction model, using the error correction position of the error information as the label value of the position sub-model, and using the error correction information of the error information as the label value of the error correction sub-model, to train the text error correction model.

[0014] In the above scheme, for the original text and error-corrected text in the historical samples, the error information of the original text can be obtained through the error learning module, the text semantics of the original text can be obtained through the text semantics acquisition module, and the domain information of the original text can be obtained through the domain information acquisition module. Then, the error information, text semantics and domain information of the original text can be input into the initial text error correction model, and the error correction position in the error information is used as the label value of the position sub-model during training, and the error correction information in the error information is used as the label value of the error correction sub-model during training, so as to train the initial text error correction model. The text error correction model trained in this way can be used to accurately identify the position where the text error occurs in a speech recognition text and the correct text information corresponding to the error position, thereby improving the user experience in the speech-to-text scenario.

[0015] In one possible implementation method, the error correction submodel includes a first submodel based on a generation mode, a second submodel based on a determination mode, and an evaluation submodel; the error correction submodel in the text error correction model is used to correct the erroneous text at the error correction position to obtain a corrected speech recognition text, including: correcting the erroneous text at the error correction position through the first submodel to obtain a first error correction result; correcting the erroneous text at the error correction position through the second submodel to obtain a second error correction result; evaluating the first error correction result and the second error correction result through the evaluation submodel, and using the error correction result that meets the evaluation result as the corrected speech recognition text.

[0016] In the above scheme, for a speech recognition text, after the position sub-model in the text error correction model confirms the position where the text error occurs, the error correction sub-model in the text error correction model can be used to correct the erroneous text at the error position. In order to improve the accuracy of text error correction in this application, it is proposed that the erroneous text at the error position can be corrected according to the first sub-model based on the generation model and the second sub-model based on the judgment mode, and the first error correction result and the second error correction result are generated respectively. For the two error correction results, the evaluation sub-model can be further used in the embodiment of this application to evaluate the two error correction results, and the error correction result that meets the evaluation result is used as the speech recognition result after correction. In this way, by integrating the error correction results of the text error correction model of at least two modes, the accuracy of text error correction can be greatly improved, and the user experience in the speech-to-text scenario is improved.

[0017] In a possible implementation method, the first error correction result and the second error correction result are evaluated, and the error correction result that meets the evaluation result is used as the speech recognition text after error correction, including: determining a first score based on the word semantic distance between the first error correction result and each word vector in the speech recognition text and the sentence semantic distance between the first error correction result and the speech recognition text; determining a second score based on the word semantic distance between the second error correction result and each word vector in the speech recognition text and the sentence semantic distance between the second error correction result and the speech recognition text; and using the error correction result with a lower score between the first score and the second score as the speech recognition text after error correction.

[0018] In the above scheme, when the evaluation sub-model is used to screen the first error correction result and the second error correction result, the word semantic distance and sentence semantic distance between the error correction result and each word vector in the speech recognition text can be calculated for any error correction result, and the score corresponding to the current error correction result can be generated. Finally, the error correction result with the lower score among the two scores can be used as the corrected speech recognition text. In this method, by comprehensively considering the meaning of words and the semantic coherence at the sentence level, when confirming a good error correction result, the determined error correction result will be infinitely close to the correct text, that is, the accuracy of the speech recognition text correction process is improved, and the user experience in the speech-to-text scenario is improved.

[0019] In a possible implementation method, the speech recognition system is any one of various speech recognition systems.

[0020] In the above scheme, since the error learning module can be used to learn and record the error information of the speech recognition text, the present application can be connected to any speech recognition system among various speech recognition systems, that is, the method of the present application can achieve the effect of working in collaboration with multiple speech recognition systems.

[0021] In a second aspect, an embodiment of the present application provides a text correction device, which includes: an acquisition unit, which is used to recognize the speech recognition text based on the speech recognition text output by the speech recognition system, and obtain error information of the speech recognition text, text semantics of the speech recognition text, and domain information of the speech recognition text; an error correction unit, which is used to obtain the correction position of the speech recognition text based on the error information, the text semantics, and the domain information through a position sub-model in a text error correction model; correct the erroneous text at the correction position through the error correction sub-model in the text error correction model to obtain the corrected speech recognition text; the text error correction model is obtained by training a first loss value of the position sub-model and a second loss value of the error correction sub-model.

[0022] In a third aspect, an embodiment of the present application provides a computing device, including:

[0023] A memory for storing program instructions;

[0024] The processor is used to call the program instructions stored in the memory and execute any implementation method of the first aspect according to the obtained program.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute any implementation method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 A schematic diagram of a text error correction method provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a BERT pre-training model provided in an embodiment of the present application;

[0029] Figure 3A schematic diagram of the BiLSTM model structure provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of a training network for a position sub-model provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of an error correction subunit based on a generation mode provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of a collaborative working mechanism provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of a text error correction device provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0036] In view of the problem that the current speech recognition system is prone to errors in the text converted from speech, the embodiment of the present application can provide a text error correction method. Figure 1 FIG. 1 is a schematic diagram of a text error correction method provided by an embodiment of the present application, the method comprising the following steps:

[0037] Step 101, based on the speech recognition text output by the speech recognition system, the speech recognition text is recognized, and error information of the speech recognition text, text semantics of the speech recognition text, and domain information of the speech recognition text are obtained respectively;

[0038] Step 102, based on the error information, the text semantics and the domain information, obtain the error correction position of the speech recognition text through the position sub-model in the text error correction model; correct the erroneous text at the error correction position through the error correction sub-model in the text error correction model to obtain the corrected speech recognition text.

[0039] The text error correction model is obtained by training the first loss value of the position sub-model and the second loss value of the error correction sub-model.

[0040] In the above scheme, by respectively identifying the error information, text semantics and domain information of a speech recognition text output by the speech recognition system, the error information, text semantics and domain information of the speech recognition text obtained can be used to obtain the error correction position of the speech recognition text through the position submodel in the text error correction model, and the error text at the determined error correction position can be corrected through the error correction submodel in the text error correction model, so that the corrected speech recognition text can be obtained. In this method, by integrating the error information, text semantics and domain information of the speech recognition text, the accuracy of the error correction position of the speech recognition text obtained by the error correction submodel can be greatly improved; then, by correcting the error text at the accurate error position in the text, the correctness of the speech recognition text after the final correction can be greatly improved, thereby improving the user experience in the speech-to-text scenario.

[0041] The following will explain the above steps in detail with examples.

[0042] In one implementation of the above-mentioned step 101, the recognition of the speech recognition text to obtain error information of the speech recognition text includes: recognizing the speech recognition text through an error learning module to obtain error information of the speech recognition text; the error learning module is used to compare the original text and the error correction text in the historical samples, and generate and store various error information.

[0043] In the embodiment of the present application, the function of the error learning module is to identify and record error information in the speech recognition text output by the speech recognition system, wherein the identified and recorded error information can be used for subsequent text error correction model learning.

[0044] Specifically, the corpus for learning by the error learning module is composed of supervised speech recognition text and weakly supervised speech recognition text. Wherein, the supervised speech recognition text is manually annotated, and the error position in the original text and the information after error correction are annotated to the original text through labels, and the weakly supervised speech recognition text includes the uncorrected speech recognition text (i.e., original text) and the text formed by the automatic error correction of the error correction system (i.e., error correction text), and does not include error position labels. Wherein the error correction system can be the text error correction model in this application or the error correction system currently used to correct the speech recognition text.

[0045] Furthermore, the error learning module may also be composed of a speech recognition evaluation unit and an error information memory unit, including:

[0046] The speech recognition evaluation unit can be used to organize unstructured text into structured data that can be learned by the model, and automatically compare the differences between the original text and the corrected text (i.e., the corrected text), and mark the error location in the original text and the corrected content.

[0047] The error information memory unit can be used to store the error information detected by the speech recognition evaluation unit; further, when training the text error correction model, the error information memory unit can also pass the error information to the text error correction model for training of the text error correction model.

[0048] The following two aspects are explanations of supervised speech recognition text and weakly supervised speech recognition text in combination with the error learning module.

[0049] 1. Supervised speech recognition text

[0050] Supervised speech recognition text can be used for supervised training of text error correction models.

[0051] For example, (w 1 ,w 2 ,…,w n ) represents a piece of speech recognition text, w i For the i-th word in the speech recognition text, and using (c 0 ,c 1 ,…,c n ,c n+1 ) indicates the text encoding, c 0 is the text start code, c n+1 The error position label is the code for the beginning and end of the error text in the text, for example (c i ,c i+3 ) indicates w i to w i+3 This is incorrect content.

[0052] For example, for a speech segment "Natural language processing refers to the processing of linguistic problems through computers", a speech recognition system recognizes it as a speech recognition text "Natural language processing essentially processes linguistic problems through computers". This speech recognition text contains 7 words, so the speech recognition text can be expressed as (w 1 ,w 2 ,…,w 7 ), and use (c 0 ,c 1 ,…,c 7 ,c 8) represents the text encoding of the speech recognition text. Then in the speech recognition text, the speech recognition evaluation unit in the error learning module can determine that the error position label of the speech recognition text is (c 2 ,c 2 ), the error position label (c 2 ,c 2 ) indicates w 2 This wording is wrong. "Substance" should be "refers to".

[0053] After that, the error information memory unit can store the original text, the text error position and the corrected text to form an error message. The above error information will be used as the text error feature of the text error correction model to train the text error correction model.

[0054] 2. Weakly supervised speech recognition text

[0055] The weakly supervised speech recognition text can come from the text error correction model in this application or the text data generated by the error correction system currently used to correct speech recognition text. After the data has accumulated to a certain extent, it can be used as an enhanced data set to train the text error correction model in this application.

[0056] For a weakly supervised speech recognition text, the speech recognition evaluation unit can automatically mark the error position by comparing the original text with the corrected text; after that, the error information memory unit can store the original text, the text error position and the corrected text, and form an error message. The above error information will be used as the text error feature of the text error correction model for training the text error correction model.

[0057] Based on the above error learning module, for a speech recognition text output by the speech recognition system, the error information of the speech recognition text can be determined through the error learning module, wherein the error information includes the error position and the text where the error occurs at the error position. The error position is the error correction position, and the text where the error occurs at the error position is the error correction information.

[0058] In one implementation of the above-mentioned step 101, the recognition of the speech recognition text to obtain the text semantics of the speech recognition text includes: recognizing the speech recognition text through a text semantics acquisition module to obtain the text semantics of the speech recognition text; the text semantics acquisition module is obtained by learning word vectors for original texts in historical samples and performing semantic learning based on the word vectors.

[0059] In the embodiment of the present application, the function of the text semantics acquisition module is to preprocess the speech recognition text, convert the text into word vectors through pre-training, and calculate the text semantics.

[0060] Specifically, the text semantics acquisition module can be composed of a text preprocessing unit, a word vector training unit, and a text semantics calculation unit, including:

[0061] The text preprocessing unit can be used to preprocess the input speech recognition text. The preprocessing content may include word segmentation of the speech recognition text of the Chinese sequence, and removing modal particles, stop words, etc. from each segmentation, so as to realize the conversion of the speech recognition text into a word set for the training of word vectors.

[0062] The word vector training unit can learn words by adopting a pre-trained model based on BERT (Bidirectional Encoder Representations from Transformers) to convert words into word vectors of set dimensions.

[0063] The text semantics calculation unit can further extract semantic features from word vectors based on word vector training to obtain deeper text semantics; the obtained text semantics can be used as input to the text error correction model for training.

[0064] Among them, word embedding refers to converting text into a vector form that can be understood by computers through training, which is used as the input of the next unit. Text semantic computing refers to converting the sparse word embedding matrix into a dense matrix that can convey the deep text semantics.

[0065] For example, the processing process of the speech recognition text "Natural language processing refers to the processing of linguistic problems through computers" can be: the word vector representation can convert the words "natural language processing", "refers to", "through", "computer", "to", "processing", and "linguistic problems" in the text into a vector of a fixed dimension, such as setting it to 100 dimensions, so that the speech recognition text can be converted into a word vector matrix of size 100*7, and the text semantic calculation can represent compressing the word vector matrix of size 100*7 into a text semantic matrix of size 20*5.

[0066] Specifically, in the embodiment of the present application, by adopting the BERT pre-training model, the speech recognition text can be converted into a word vector matrix, and the text semantics can be extracted through matrix dimensionality reduction, and finally a text semantic matrix can be obtained.

[0067] like Figure 2 , which is a schematic diagram of a BERT pre-training model provided in an embodiment of the present application. Figure 2In the figure, Trm is the core structure of the BERT pre-training model, Transformer. Transformer uses an encoder-decoder structure to convert the input matrix into an output matrix of the required size. The dimension of the output matrix is ​​controlled by the model parameters. In the word vector training stage, the input is a word sequence, and the output is a word vector of a set dimension. Each word is represented by a specific vector. In the text semantics calculation stage, the BERT pre-training model converts the word vector matrix into a text semantic matrix of a specific dimension, and represents the deep text semantics in a dense matrix form.

[0068] In one implementation of the above step 101, the speech recognition text is recognized to obtain the domain information of the speech recognition text, including: recognizing the speech recognition text through a domain information acquisition module to obtain the domain information of the speech recognition text; the domain information acquisition module performs domain learning on the word vectors of the original text in the historical samples, and sets the domain weight for the domain word vector of the original text based on the domain.

[0069] In the embodiment of the present application, the function of the domain information acquisition module is to obtain the domain features of the input text based on the domain dictionary, thereby determining the domain type, and finally adding domain weights to the relevant words, wherein the added domain weights can be used for subsequent text error correction model learning.

[0070] Specifically, the domain information acquisition module may be composed of a domain feature calculation unit, a domain determination unit, and a domain weight assignment unit, including:

[0071] The domain feature calculation unit can be used to learn the domain features of text through deep neural networks and obtain the tendency of text for specific fields.

[0072] The domain determination unit can be used to determine the domain type of the text according to the domain dictionary and the domain feature calculation results to obtain the domain type of the text.

[0073] The domain weight assignment unit can be used to increase the weight of words in a specific domain according to the domain type of the text, wherein the updated word weights can be used for training the text error correction model so that the text error correction model can make different error correction judgments for specific domains.

[0074] Among them, for the domain determination unit, on the one hand, the text word vector can be used as the input for domain feature calculation, such as in this application, the domain features of the word vector matrix of the text can be extracted through the bidirectional long short-term memory network (BiLSTM), thereby obtaining the domain features in the form of a set dimension matrix. On the other hand, for different fields, the domain determination unit can obtain the final domain type of the text based on the domain dictionary and the calculated text domain features.

[0075] like Figure 3 As shown, it is a schematic diagram of a BiLSTM model structure provided in an embodiment of the present application. Figure 3 In the model, the input x t is the word vector, output o t The domain features of the text are compared with the domain dictionary features to determine the domain type to which the text belongs. Given that the BiLSTM model has bidirectional learning capabilities and is based on the domain dictionary, the model trained in this way can determine the domain type of the speech recognition text.

[0076] For example, if the word vector matrix of the speech recognition text "Natural language processing refers to the processing of linguistic problems by computers" is input into the BiLSTM model, the network can learn the matrix from two directions to obtain domain features. For example, according to the domain dictionary, the domain to which the speech recognition text belongs is "artificial intelligence".

[0077] For the domain weight assignment unit, according to the determined domain category and domain dictionary, the words in the text sequence are compared, and the weight of the specific domain words in the text error correction model training process is increased, so that the text error correction model focuses on the domain words in the original text during the training process, thereby ensuring that the text generated in the actual text error correction process will meet the domain characteristics. In the above example, the "artificial intelligence" field contains several related words in the domain dictionary, such as "deep learning", so in the text error correction model training process, the model focuses on the "artificial intelligence" field words and re-weights these words so that the model does not ignore the "artificial intelligence" field words in the error location determination and error correction process.

[0078] In the general domain, the word weight is determined by model training. In the specific domain, during the text error correction model training process, the specific domain words are weighted according to their importance in the text, and the weight is greater than the general domain weight. For example, the word w in a specific domain i The weight W during training wi ′ is shown in the following formula:

[0079]

[0080] Among them, in the above formula is the word weight in the general field, C i For the word w i The number of times it appears in the text, is the total number of words in the text, N d is the total number of words in the input text of the text error correction model, N di is the word w in the input text i The number of times. Weight parameter Therefore, words in specific fields have greater weights than those in general fields, and the size of the weight is determined by the importance of the word itself in the field.

[0081] For example, in the aforementioned example, on the one hand, for the original text in the historical sample and the error correction text corresponding to the original text, the error information of the original text can be obtained through the error learning module; on the other hand, for the original text in the historical sample, the text semantics of the original text can be obtained through the text semantics acquisition module; on the other hand, for the original text in the historical sample, the domain information of the original text can be obtained through the domain information acquisition module; in this way, for any original text in the historical sample, the three aspects of error information, text semantics and domain information of the original text can be obtained. Then, the three aspects of information can be used as model input and input into the initial text error correction model. At the same time, the error correction position in the error information is used as the label value of the position sub-model in the training initial text error correction model, and the error text at the error correction position in the error information (i.e., the error correction information) is used as the label value of the error correction sub-model in the training initial text error correction model, thereby training the initial text error correction model. Assuming that after multiple trainings, the loss value of a text error correction model meets the set requirements, the text error correction model can be used to correct the speech recognition text output by the speech recognition system, thereby outputting the correct speech recognition text to the user.

[0082] Among them, the position sub-model in the text error correction model that meets the training requirements can be used to determine the location where the error occurs in the speech recognition text output by the speech recognition system, and the error correction sub-model in the text error correction model that meets the training requirements can be used to correct the erroneous text at the location where the error occurs in the speech recognition text to form a correct speech recognition text for output.

[0083] Specifically, the present application also includes a text error correction model training module. The text error correction model training module can be used to generate model parameters whose loss values ​​meet the loss threshold through step-by-step iterative training, and the model corresponding to the model parameters is the text error correction model.

[0084] The text error correction model training module can be composed of a model parameter adjustment unit, an error position marking unit and an error correction unit, including:

[0085] The model parameter adjustment unit can be used to obtain the error information, text semantics and domain weight corresponding to each speech recognition text, and use the three as input information, and adjust the parameters of the text error correction model according to the input information.

[0086] The error position labeling unit can be used to train the error position based on a deep neural network, using the error position in the error information of the speech recognition text as label information. The position sub-model formed after training can determine and label the position where the error occurs in the speech recognition text.

[0087] The error correction unit can be used to correct the erroneous text at the position where the error occurs in the speech recognition text.

[0088] like Figure 4 , which is a schematic diagram of a training network of a position sub-model provided in an embodiment of the present application. The present application uses a deep neural network to learn speech recognition text and trains the network for the wrong positions marked in the speech recognition text. Figure 4 ,The position sub-model is a network model built based on the encoder-decoder, where the input of the model is the text semantic feature matrix and the output is the error position tag.

[0089] In some implementations of the present application, the error correction sub-model includes a first sub-model based on a generation mode, a second sub-model based on a determination mode, and an evaluation sub-model.

[0090] Continuing with the above example, in the training process of the text error correction model, for a speech recognition text, the location where the error occurred in the speech recognition text can be obtained through the position sub-model in the text error correction model; then the problem of how to correct the erroneous text at the location where the error occurred can be achieved through the following method.

[0091] In the present application, the error correction unit includes an error correction subunit based on a generation mode, an error correction subunit based on a determination mode, and an evaluation unit. The error correction subunit based on a generation mode and the error correction subunit based on a determination mode can be used to correct the erroneous text at the wrong position in the speech recognition text, respectively, so as to obtain an error correction result respectively; for the two error correction results, the evaluation unit can evaluate the two, wherein the evaluation unit can judge the matching degree between the generated error correction content and the original text context through the semantic similarity between words and the semantic coherence at the sentence level, and select and generate the final error correction result.

[0092] It is noted that the error correction sub-unit of the generation model is the first sub-model based on the generation mode, the error correction sub-unit based on the determination mode is the second sub-model based on the determination mode, and the evaluation unit is the evaluation sub-model.

[0093] Specifically, the following describes the usage of the error correction subunit based on the generation mode and the error correction subunit based on the determination mode respectively:

[0094] (1) Error correction subunit based on generation mode

[0095] like Figure 5 , which is a schematic diagram of an error correction subunit based on a generation mode provided in an embodiment of the present application. Figure 5 In the generative mode, the error correction subunit is based on the sequence-to-sequence generative model (Seq2Seq), where x i The encoder input sequence contains text semantic features and error location information, y i It is the error correction text output by the decoder, that is, the error correction text that replaces the error text at the error position.

[0096] For example, the original text "Natural language processing is essentially the processing of linguistic problems through computers" has the following error position information: 2 ,c 2 ), then through model learning, the error correction text "refers to" is directly generated at the error position, thereby generating the correction text of the original text "Natural language processing refers to the processing of linguistic problems through computers".

[0097] (2) Error correction subunit based on judgment mode

[0098] The error correction subunit based on the judgment mode in this application adopts the traditional N-gram model. First, traverse and obtain the bigrams and trigrams in the input sequence; then, use the Bi-gram and Tri-gram models to calculate the error rate of the wrong position phrase; finally, calculate the confusion of all semantically similar phrases based on the text and domain dictionary, and select the word with the largest confusion as the correct word after error correction. i The perplexity calculation is shown in the following formula:

[0099]

[0100] Where S = w 1 w 2 ... i ... l For the word w i The sentence where p(w 1 w 2 ... i ... l ) is the probability of sentence S.

[0101] For example, the original text "Natural language processing is essentially the processing of linguistic problems through computers" has the following error position information: 2 ,c 2), the N-gram model takes characters as units, obtains two-character words and three-character words in the text, calculates the perplexity of the words in the domain dictionary in turn, and obtains the word with the highest perplexity as "refers to", and generates the corrected text of the original text "Natural language processing refers to the processing of linguistic problems through computers".

[0102] Finally, the correction texts (ie, error correction texts) generated by the two error correction sub-units are simultaneously transmitted to the next unit, ie, the evaluation unit.

[0103] For a speech recognition text in a historical sample, an error correction result can be generated by the error correction subunit based on the generation mode, and an error correction result can be generated by the error correction subunit based on the determination mode. The two error correction results can be further decided by the evaluation unit in the error correction unit. The function of the evaluation unit is described below:

[0104] When the evaluation unit makes a selection for the correction results generated by the two correction sub-units respectively, its evaluation indicators include the semantic similarity between the generated correction words and other words in the text sentence and the semantic coherence of the sentence. The semantic similarity between words is the semantic distance between words, which is represented by the Euclidean distance of word vectors in this application. The semantic coherence of a sentence refers to the distance between the semantics of sentences in the text. The Euclidean distance is also used in this application to calculate the distance between the semantics of sentences. Finally, the above two indicators can be added together to obtain the final score Score of the correction result. The score calculation method is shown in the following formula:

[0105]

[0106] In the above formula, w i is the error correction word generated by the model, w j w i Other words in the sentence, l is the number of words in the sentence. i is the sentence in the text, and n is the number of sentences in the text.

[0107] In the present application, when the Euclidean distance is used for calculation, the smaller the score of the correction result is, the more the correction word conforms to the semantics of the original text, and the stronger the semantic coherence of the text after correction is. Therefore, the two correction results are scored by the evaluation unit, and the evaluation unit can select the one with a smaller score as the output of the final text correction model.

[0108] In the above example, two error correction sub-units generate two error correction results. For example, the error correction sub-unit based on the determination mode outputs "natural language processing enables linguistic problems to be processed through computers", and the error correction sub-unit based on the generation mode outputs "natural language processing refers to processing linguistic problems through computers". After using the above evaluation method to calculate the scores of the two error correction results, it is assumed that the score of "natural language processing refers to processing linguistic problems through computers" output by the error correction sub-unit based on the generation mode is low. Therefore, in the embodiment of the present application, the error correction result of the error correction sub-unit based on the generation mode can be determined as the final model output.

[0109] Furthermore, the error correction unit includes, in addition to the error correction subunit based on the generation mode, the error correction subunit based on the determination mode and the evaluation unit, a training decision unit and an optimal model acquisition unit, the contents of which are as follows:

[0110] The training decision unit can be used to judge the training effect of the text error correction model and decide whether to continue training. When the training loss of the model is less than the set loss threshold, the model stops training and determines the optimal model parameters; otherwise, the model continues to be trained.

[0111] The optimal model acquisition unit can be used to obtain the parameters of the optimal model obtained after training, save and output it. That is, the optimal model acquisition unit can put the obtained optimal model into use to identify the correctness of the speech recognition text output by the speech recognition system, and correct the speech recognition text identified as erroneous, thereby generating a correct speech recognition text for the erroneous speech recognition text.

[0112] Specifically, whether the text correction model needs to continue training can be determined by the training decision unit. When the loss of the text correction model training reaches or is less than a certain set value, the training of the text correction model can be stopped. The loss of the text correction model includes the loss generated by the position sub-model. 1 and the loss loss generated by the error correction sub-model 2 The calculation method of the two parts is shown in the following formula.

[0113]

[0114]

[0115] Loss = loss 1 +loss 2

[0116] Among them, {x 1 ,x 2 ,…,x N} indicates the model error position output, {y 1 ,y2 ,…,y N} represents the model error correction output, γ≥0. loss 2 Parameters in The weights of positive and negative samples can be balanced so that the model focuses on difficult samples, that is, errors in the text that are difficult to correct.

[0117] Based on the training process of the text error correction model described above, in this application, the speech recognition text output by the speech recognition system can be detected for correctness based on the text error correction model whose loss value meets the loss threshold, that is, the optimal text error correction model.

[0118] In certain embodiments of the present application, the error correction sub-model in the text error correction model is used to correct the erroneous text at the error correction position to obtain the corrected speech recognition text, including: correcting the erroneous text at the error correction position through the first sub-model to obtain a first error correction result; correcting the erroneous text at the error correction position through the second sub-model to obtain a second error correction result; and evaluating the first error correction result and the second error correction result through the evaluation sub-model, and using the error correction result that meets the evaluation result as the corrected speech recognition text.

[0119] In certain implementations of the present application, the first error correction result and the second error correction result are evaluated, and the error correction result that meets the evaluation result is used as the speech recognition text after error correction, including: determining a first score based on the word semantic distance between the first error correction result and each word vector in the speech recognition text and the sentence semantic distance between the first error correction result and the speech recognition text; determining a second score based on the word semantic distance between the second error correction result and each word vector in the speech recognition text and the sentence semantic distance between the second error correction result and the speech recognition text; and using the error correction result with a lower score between the first score and the second score as the speech recognition text after error correction.

[0120] In an implementation of the above step 101, the speech recognition system is any one of various speech recognition systems.

[0121] Since the present application can learn and remember the errors of the speech recognition system, the present application has a mechanism to work in coordination with a variety of speech recognition systems. Figure 6 As shown, it is a schematic diagram of a collaborative working mechanism provided by an embodiment of the present application. Figure 6In the text correction device, ASRn represents a certain speech recognition system. For example, when the text correction device of the present application works with ASR1, the text correction device learns the corpus of ASR1, obtains the error information of the text output by the speech recognition system, learns and memorizes, and trains the text correction model. The text correction model obtained by training can perform text correction according to the characteristics of ASR1. When the correction device needs to adapt to other speech recognition systems, the above steps are repeated and will not be repeated.

[0122] Based on the same concept, the present application embodiment provides a text error correction device, such as Figure 7 , which is a schematic diagram of a text error correction device provided in an embodiment of the present application, the device includes an acquisition unit 701 and an error correction unit 702;

[0123] The acquisition unit 701 is used to recognize the speech recognition text based on the speech recognition text output by the speech recognition system, and obtain error information of the speech recognition text, text semantics of the speech recognition text and domain information of the speech recognition text;

[0124] The error correction unit 702 is used to obtain the error correction position of the speech recognition text based on the error information, the text semantics and the domain information through the position sub-model in the text error correction model; correct the erroneous text at the error correction position through the error correction sub-model in the text error correction model to obtain the corrected speech recognition text; the text error correction model is obtained by training the first loss value of the position sub-model and the second loss value of the error correction sub-model.

[0125] Furthermore, for the device, the acquisition unit 701 is specifically used to: recognize the speech recognition text through the error learning module to obtain error information of the speech recognition text; the error learning module is used to compare the original text and the error correction text in the historical sample, and generate and store various error information.

[0126] Furthermore, for the device, the acquisition unit 701 is specifically used to: recognize the speech recognition text through a text semantics acquisition module to obtain the text semantics of the speech recognition text; the text semantics acquisition module is obtained by learning word vectors for the original text in historical samples and performing semantic learning based on the word vectors.

[0127] Furthermore, for the device, the acquisition unit 701 is specifically used to: recognize the speech recognition text through the domain information acquisition module to obtain the domain information of the speech recognition text; the domain information acquisition module performs domain learning on the word vectors of the original text in the historical samples, and sets the domain weight for the domain word vector of the original text based on the domain.

[0128] Furthermore, the device also includes a text error correction model training unit 703; the text error correction model training unit 703 is used to: for the original text and the corrected text in the historical sample, obtain the error information of the original text through the error learning module; the error information includes the error correction position and the error correction information; for the original text in the historical sample, obtain the text semantics of the original text through the text semantics acquisition module; for the original text in the historical sample, obtain the domain information of the original text through the domain information acquisition module; use the error information of the original text, the text semantics of the original text and the domain information of the original text as input values ​​of the text error correction model, use the error correction position of the error information as the label value of the position sub-model, and use the error correction information of the error information as the label value of the error correction sub-model to train the text error correction model.

[0129] Further, for the device, the error correction sub-model includes a first sub-model based on a generation mode, a second sub-model based on a determination mode, and an evaluation sub-model; the text error correction model training unit 703 is specifically used to: correct the erroneous text at the error correction position through the first sub-model to obtain a first error correction result; correct the erroneous text at the error correction position through the second sub-model to obtain a second error correction result; evaluate the first error correction result and the second error correction result through the evaluation sub-model, and use the error correction result that meets the evaluation result as the corrected speech recognition text.

[0130] Further, for the device, the text correction model training unit 703 is specifically used to: determine a first score based on the word semantic distance between the first error correction result and each word vector in the speech recognition text and the sentence semantic distance between the first error correction result and the speech recognition text; determine a second score based on the word semantic distance between the second error correction result and each word vector in the speech recognition text and the sentence semantic distance between the second error correction result and the speech recognition text; and use the error correction result with a lower score between the first score and the second score as the corrected speech recognition text.

[0131] The embodiment of the present application also provides a computing device, which may be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), a memory, an input / output device, etc. The input device may include a keyboard, a mouse, a touch screen, etc. The output device may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), etc.

[0132] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In an embodiment of the present application, the memory may be used to store program instructions of the text error correction method;

[0133] The processor is used to call the program instructions stored in the memory and execute the text error correction method according to the obtained program.

[0134] like Figure 8 FIG. 1 is a schematic diagram of a computing device provided in an embodiment of the present application, wherein the computing device includes:

[0135] Processor 801, memory 802, transceiver 803, bus interface 804; wherein the processor 801, memory 802 and transceiver 803 are connected via bus 805;

[0136] The processor 801 is used to read the program in the memory 802 and execute the above-mentioned text error correction method;

[0137] The processor 801 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP. It may also be a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0138] The memory 802 is used to store one or more executable programs and can store data used by the processor 801 when performing operations.

[0139] Specifically, the program may include program code, and the program code includes computer operation instructions. The memory 802 may include volatile memory, such as random-access memory (RAM); the memory 802 may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 802 may also include a combination of the above-mentioned types of memory.

[0140] The memory 802 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:

[0141] Operation instructions: include various operation instructions, used to implement various operations.

[0142] Operating system: includes various system programs used to implement various basic businesses and handle hardware-based tasks.

[0143] The bus 805 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0144] The bus interface 804 may be a wired communication access port, a wireless bus interface or a combination thereof, wherein the wired bus interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface or a combination thereof. The wireless bus interface may be a WLAN interface.

[0145] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a text error correction method.

[0146] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0151] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A text error correction method, characterized in that: include: Based on the speech recognition text output by the speech recognition system, the speech recognition text is recognized to obtain the text semantics of the speech recognition text, and the speech recognition text is recognized through an error learning module to obtain error information of the speech recognition text; the error learning module is used to compare the original text and the error correction text in the historical sample, generate and store various error information, and the speech recognition text is recognized through a domain information acquisition module to obtain domain information of the speech recognition text; The domain information acquisition module performs domain learning on the word vectors of the original text in the historical samples, and sets domain weights for the domain word vectors of the original text based on the domain; Based on the error information, the text semantics and the domain information, the error correction position of the speech recognition text is obtained through the position sub-model in the text error correction model; the erroneous text at the error correction position is corrected through the error correction sub-model in the text error correction model to obtain the corrected speech recognition text; the text error correction model is obtained by training the first loss value of the position sub-model and the second loss value of the error correction sub-model.

2. The method according to claim 1, characterized in that The step of recognizing the speech recognition text to obtain text semantics of the speech recognition text includes: The speech recognition text is recognized by a text semantics acquisition module to obtain text semantics of the speech recognition text; the text semantics acquisition module is obtained by learning word vectors of original texts in historical samples and performing semantic learning based on the word vectors.

3. The method according to any one of claim 2, characterized in that The text error correction model is trained in the following manner, including: For the original text and the error-corrected text in the historical sample, the error information of the original text is obtained through the error learning module; the error information includes the error correction position and the error correction information; For the original text in the historical sample, the text semantics of the original text is obtained by the text semantics acquisition module; For the original text in the historical sample, the domain information of the original text is obtained by the domain information acquisition module; The error information of the original text, the text semantics of the original text and the domain information of the original text are used as input values ​​of the text error correction model, the correction position of the error information is used as the label value of the position sub-model, and the correction information of the error information is used as the label value of the error correction sub-model to train the text error correction model.

4. The method according to any one of claims 1 to 2, characterized in that: The error correction sub-model includes a first sub-model based on a generation mode, a second sub-model based on a determination mode, and an evaluation sub-model; The step of correcting the erroneous text at the error correction position by using the error correction submodel in the text error correction model to obtain the corrected speech recognition text includes: Correcting the erroneous text at the error correction position by using the first sub-model to obtain a first error correction result; Correcting the erroneous text at the error correction position by using the second sub-model to obtain a second error correction result; The first error correction result and the second error correction result are evaluated by the evaluation sub-model, and the error correction result that meets the evaluation result is used as the speech recognition text after error correction.

5. The method according to claim 4, characterized in that The step of evaluating the first error correction result and the second error correction result and using the error correction result that meets the evaluation result as the speech recognition text after error correction includes: Determining a first score based on a word semantic distance between the first error correction result and each word vector in the speech recognition text and a sentence semantic distance between the first error correction result and the speech recognition text; Determine a second score based on the word semantic distance between the second error correction result and each word vector in the speech recognition text and the sentence semantic distance between the second error correction result and the speech recognition text; The error correction result with a lower score between the first score and the second score is used as the speech recognition text after error correction.

6. A text error correction device, characterized in that: include: An acquisition unit is used to recognize the speech recognition text based on the speech recognition text output by the speech recognition system, obtain the text semantics of the speech recognition text, and recognize the speech recognition text through an error learning module to obtain error information of the speech recognition text; the error learning module is used to compare the original text and the error correction text in the historical sample, generate and store each error information, and recognize the speech recognition text through a domain information acquisition module to obtain domain information of the speech recognition text; The domain information acquisition module performs domain learning on the word vectors of the original text in the historical samples, and sets domain weights for the domain word vectors of the original text based on the domain; The error correction unit is used to obtain the error correction position of the speech recognition text based on the error information, the text semantics and the domain information through the position sub-model in the text error correction model; correct the erroneous text at the error correction position through the error correction sub-model in the text error correction model to obtain the corrected speech recognition text; the text error correction model is obtained by training the first loss value of the position sub-model and the second loss value of the error correction sub-model.

7. A computer device, characterized in that: include: Memory for storing computer programs; A processor is used to call the computer program stored in the memory, and execute the method according to any one of claims 1 to 5 according to the obtained program.

8. A computer-readable storage medium, characterized in that: The storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

Images