Speech recognition result error correction method and device, electronic equipment and storage medium

Abstract

The invention relates to the medical field and the voice text processing field, and discloses an error correction method for a voice recognition result, which comprises the following steps: matching a text to be corrected to obtain a first word group and a second word group; performing pinyin matching on the second phrase by using a pinyin fuzzy matching module to obtain a third phrase and a fourth phrase; generating a first candidate text based on the first and third phrases, calculating a vector of the first candidate text to generate a first word vector, generating a second candidate text based on the first and fourth phrases, and calculating a vector of the second candidate text to generate a second word vector; and correcting the first candidate text and the second candidate text based on a correction module, the first word vector and the second word vector to obtain a first correction result value and a second correction result value, and selecting the candidate text with the maximum correction result value as a target text. According to the method, optimization of global features between words and phrases of text information by an error correction model is improved in digital medical application, online medical popularization and use are realized, the efficiency is improved, and the cost is saved.

Description

Technical Field

[0001] The present invention relates to the medical field and the field of speech and text processing, and particularly to an error correction method, device, electronic device and storage medium for speech recognition results. Background Art

[0002] In the field of digital medical health, the automatic speech recognition model (ASR) is widely used to execute downstream tasks such as speech search, speech translation, and online shopping. The automatic speech recognition model is a technology that uses computer and digital signal processing technologies to convert the speech information of users into text information.

[0003] Since the speech information input into the automatic speech recognition model is easily affected by the user's spoken language level and speech environment, the output text information is prone to incorrect characters, resulting in a low recognition accuracy of the automatic speech recognition model.

[0004] For example, in the medical field, when doctors or patients use the automatic speech recognition model on mobile devices to perform speech recognition and retrieve medical data, due to the differences in the spoken languages of users, incorrect characters may appear in the recognized sentences, resulting in the need to manually modify the text again. For example, the common medical term "fatty liver zhī fanggān" is recognized as "only fatty liver zhǐ fanggān", causing low retrieval efficiency and unnecessary trouble.

[0005] In the prior art, an error correction model is usually used to optimize the output text information of the automatic speech recognition model, and further improve the word recognition rate of the automatic speech recognition model, and certain results have been achieved.

[0006] However, the mainstream error correction model is mainly a character model. When optimizing the output of the automatic speech recognition model, it cannot process the global features between words and phrases in the text information, resulting in the need to still improve the accuracy of word recognition. Summary of the Invention

[0007] In view of the above, it is necessary to provide an error correction method for speech recognition results, the purpose of which is to obtain the global features between words and phrases in the text information, improve the recognition accuracy of the error correction model, and make the speech information of doctors or patients be more accurately converted into text by using the error correction method for speech recognition results.

[0008] The error correction method for speech recognition results provided by the present invention includes: Using a preset error correction model to match the character-phrase sequence of the text to be corrected, obtaining at least one first phrase that matches a phrase in the phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, where the second phrase includes a first word and a second word; Using the pinyin fuzzy matching module of the error correction model, the second phrase is matched by pinyin to obtain at least one third phrase containing the first word, and at least one fourth phrase containing the second word; Generate a first candidate text based on the first word group and the third word group, calculate vectors of all words in the first candidate text to generate a first word vector, and generate a second candidate text based on the first word group and the fourth word group, calculate vectors of all words in the second candidate text to generate a second word vector; The first candidate text is corrected based on the correction module of the error correction model and the first word vector to obtain a first correction result value, and the second candidate text is corrected based on the correction module and the second word vector to obtain a second correction result value, the first correction result value is compared with the second correction result value, and the candidate text with the largest correction result value is selected as the target text of the text to be corrected.

[0009] Optionally, before matching the character-phrase sequence of the text to be corrected by using a preset error correction model, the method further includes: A character sequence of the text to be corrected is obtained, and the character sequence is expanded into a character-phrase sequence.

[0010] Optionally, the method of using a preset error correction model to match a character-phrase sequence of a text to be corrected to obtain at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word, comprises: Obtaining each phrase feature in the character-phrase sequence; Using the vocabulary enhancement module of the error correction model, calculating whether there is a phrase with the same features as the phrase in the phrase library; If there is an identical phrase, the corresponding phrase in the text to be corrected is used as the first phrase; If there is no identical phrase, the corresponding phrase in the text to be corrected is used as the second phrase.

[0011] Optionally, the pinyin fuzzy matching module of the error correction model is used to perform pinyin matching on the second phrase to obtain at least one third phrase containing the first word, and to obtain at least one fourth phrase containing the second word, including: shielding pronunciation information of the second phrase; Using the pinyin fuzzy matching module, matching the first word of the second phrase with a similar-sounding word to obtain the third phrase; and performing sound-similar word matching on the second word of the second phrase to obtain the fourth phrase.

[0012] Optionally, performing sound-similar word matching on the first word of the second phrase to obtain the third phrase includes: Masking the second word at the position of the text to be corrected; Using the pinyin fuzzy matching module, performing phonetic matching on the first word and all the phrases in the phrase library to obtain at least one first phonetic word that is similar in pronunciation to the second word; The third phrase is generated based on the first word and the first sound-similar word.

[0013] Optionally, performing sound-similar word matching on the second word of the second phrase to obtain the fourth phrase includes: Masking the first word at the position of the text to be corrected; Using the pinyin fuzzy matching module, performing phonetic matching on the second word and all the phrases in the phrase library to obtain at least one second phonetic word that is similar in pronunciation to the first word; The fourth phrase is generated based on the second word and the second sound-similar word.

[0014] Optionally, calculating the vectors of all words in the first candidate text to generate a first word vector includes: Utilizing the dictionary adapter of the error correction model, calculating first global information of the first phonetic word of the third phrase and each word of the first candidate text; Using the first global information as a coefficient of a character-phrase attention mechanism algorithm of the dictionary adapter; The first candidate text is weightedly processed by using the character-phrase attention mechanism algorithm to obtain the first word vector.

[0015] In order to solve the above problem, the present invention also provides a speech recognition result error correction device, the device comprising: A phrase matching module, used to match the character-phrase sequence of the text to be corrected using a preset error correction model, to obtain at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word; A fuzzy matching module, used to perform pinyin matching on the second word group using the pinyin fuzzy matching module of the error correction model to obtain at least one third word group including the first word, and to obtain at least one fourth word group including the second word; A word vector generation module, used to generate a first candidate text based on the first word group and the third word group, calculate the vectors of all words in the first candidate text to generate a first word vector, and generate a second candidate text based on the first word group and the fourth word group, calculate the vectors of all words in the second candidate text to generate a second word vector; A correction module is used to correct the first candidate text based on the correction module of the error correction model and the first word vector to obtain a first correction result value, and to correct the second candidate text based on the correction module and the second word vector to obtain a second correction result value, compare the first correction result value with the second correction result value, and select the candidate text with the largest correction result value as the target text of the text to be corrected.

[0016] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a speech recognition result error correction program that can be executed by the at least one processor, and the speech recognition result error correction program is executed by the at least one processor so that the at least one processor can execute the above-mentioned speech recognition result error correction method.

[0017] In order to solve the above problems, the present invention also provides a computer-readable storage medium, on which a speech recognition result error correction program is stored. The speech recognition result error correction program can be executed by one or more processors to implement the above speech recognition result error correction method.

[0018] Compared with the prior art, the present invention utilizes the pinyin fuzzy matching module of the error correction model to perform pinyin matching with the second phrase that does not match the phrase in the phrase library, adopts an error amplification strategy for the erroneous characters of the second phrase, and adopts a reduction strategy for the correct characters of the second phrase, which can better assist the error detection and correction of the error correction model.

[0019] The first word vector is generated by calculating the vectors of all words in the first candidate text, and the second word vector is generated by calculating the vectors of all words in the second candidate text; the global features between words and phrases between the text to be corrected and the matched phrases (implicit word-level syntactic and semantic knowledge) are captured, and a correction module is used to correct the candidate text. The correction module combines word-level information and syntactic structure to correct errors, thereby effectively improving the accuracy of error correction.

[0020] By using the error correction method of speech recognition results, the voice information of doctors or patients can be more accurately converted into text, and more and more useful medical data can be searched in the medical database to help doctors or patients understand the condition and treatment plan more accurately, thereby improving the accuracy and efficiency of medical services. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of a method for correcting speech recognition results provided by an embodiment of the present invention; Figure 2 A schematic diagram of a module of a device for correcting speech recognition results provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of an electronic device for implementing a method for correcting speech recognition results provided by an embodiment of the present invention; The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0023] It should be noted that the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0024] In the field of digital healthcare, the automatic speech recognition model, with the support of the existing error correction model optimization, is unable to handle the optimization of global features between words and phrases in text information, resulting in the accuracy of word recognition still needing to be improved.

[0025] The present invention provides a method for correcting speech recognition results, which can support the digital medical health field or the financial technology field and speech search and speech translation functions. The error correction model of the present invention includes a vocabulary enhancement module, a pinyin fuzzy matching module, a dictionary adapter, and a correction module.

[0026] The vocabulary enhancement module is used to accurately segment the text to be corrected to obtain matching phrases and non-matching phrases; By utilizing the pinyin fuzzy matching module of the error correction model, pinyin matching is performed on the second phrase that does not match the phrases in the phrase library. The erroneous characters of the second phrase are magnified by an error amplification strategy, while the correct characters of the second phrase are reduced by an error reduction strategy, which can better assist the error detection and correction of the error correction model.

[0027] The first word vector is generated by calculating the vectors of all words in the first candidate text through the dictionary adapter, and the second word vector is generated by calculating the vectors of all words in the second candidate text; the global features between words and phrases between the text to be corrected and the matched phrases (implicit word-level syntactic and semantic knowledge) are captured; A correction module is used to correct candidate texts. The correction module combines word-level information and syntactic structure to correct errors, effectively improving the accuracy of error correction.

[0028] Through the vocabulary enhancement module, pinyin fuzzy matching module, dictionary adapter, and correction module, these four modules are closely linked to each other to solve the technical problem that the error correction model in the existing technology cannot process the global features between words and phrases of text information when optimizing text information.

[0029] The error correction method of the present invention can be applied to scenarios such as online shopping, electronic financial transactions, medical diagnosis, etc. The error correction method of the present invention can convert the user's voice information into more accurate text, and then search for the shopping information, financial data, or medical information that the user wants, etc., which can improve the user's convenience and avoid the problem of erroneous words appearing in the converted text and the need for the user to manually correct the erroneous words.

[0030] Reference Figure 1 FIG. 1 is a flow chart of a method for correcting speech recognition results provided by an embodiment of the present invention. The method is executed by an electronic device.

[0031] In this embodiment, the method for correcting the error of the speech recognition result includes: S1, using a preset error correction model to match a character-phrase sequence of a text to be corrected, to obtain at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word; In this embodiment, the error correction model includes a vocabulary enhancement module, a pinyin fuzzy matching module, a dictionary adapter, and a correction module.

[0032] The character-phrase sequence refers to pairing each character (word) of the text information with a specified phrase; a character refers to the character corresponding to each word of the text, and a phrase refers to two words that can form a phrase, and the phrase is included in the commonly used dictionary.

[0033] When a user performs a voice operation on a voice terminal (for example, voice-to-text conversion), the automatic speech recognition model receives the user's voice and converts it into text to be corrected, obtains a character sequence of the text to be corrected, and uses the vocabulary enhancement module of the correction model to expand the character sequence into a character-phrase sequence, and matches a first phrase that can match a phrase in the phrase library, and a second phrase that cannot match a phrase in the phrase library.

[0034] In one embodiment, before matching the character-phrase sequence of the text to be corrected using the preset error correction model, the method further includes: A character sequence of the text to be corrected is obtained, and the character sequence is expanded into a character-phrase sequence.

[0035] In one embodiment, before obtaining the character sequence of the text to be corrected, the method further includes: The text output by the automatic speech recognition model is obtained as the text to be corrected.

[0036] When the automatic speech recognition model receives user speech, it is easily affected by the user's spoken language level and the current speech environment (for example, environmental noise and the performance of the speech receiving device). The text output by the automatic speech recognition model is prone to contain individual erroneous words, and the output text is used as the text to be corrected.

[0037] Get the character sequence of the text to be corrected, use the vocabulary enhancement module (Licion EnhancedBERT, LEBERT for short) of the error correction model to match the character sequence with all the phrases in the phrase library, and expand the text to be corrected into a character-phrase sequence.

[0038] The vocabulary enhancement module is a vocabulary-enhanced BERT for Chinese sequence tagging, which directly integrates external vocabulary knowledge into the BERT layer through the dictionary adaptation layer.

[0039] Compared with BERT, LBERT has two main differences. First, in the case of converting Chinese sentences into character-phrase sequences, LBERT takes both Chinese character features and dictionary features as input. Second, a dictionary adapter is attached between the transformer layers, so that the dictionary knowledge can be effectively integrated into BERT.

[0040] In one embodiment, the method of using a preset error correction model to match the character-phrase sequence of the text to be error corrected to obtain at least one first phrase that matches a phrase in the phrase library of the error correction model and at least one second phrase that does not match a phrase in the phrase library, where the second phrase includes a first word and a second word, includes: Obtain the respective phrase features in the character-phrase sequence; Use the vocabulary enhancement module of the error correction model to calculate whether there is a phrase in the phrase library that is the same as the phrase feature; If there is the same phrase, use the corresponding phrase in the text to be error corrected as the first phrase; If there is no same phrase, use the corresponding phrase in the text to be error corrected as the second phrase.

[0041] Using the vocabulary enhancement module of the error correction model to match all phrases in a pre-determined phrase library with the character-phrase sequence is equivalent to inputting the character-level features and word-level features of the text to be error corrected into the vocabulary enhancement module. At the first layer of the vocabulary enhancement module, the word-level features are fused to enhance the error correction ability of the vocabulary enhancement module for the text to be error corrected.

[0042] The phrase library pre-stores a preset number (for example, the preset number is 50,000 pairs of phrases. These 50,000 pairs of phrases are the vocabulary features of common sentences that conform to the voice scenario. A vocabulary list with a dictionary and matching index can also be generated from these 50,000 pairs of phrases, and the dictionary and vocabulary list are stored in the phrase library). By searching whether there are phrases with the same features between the character-phrase sequence and the phrase library, at least one first phrase with the same features as the phrase library and at least one second phrase without the same features as the phrase library are obtained.

[0043] All examples of the present invention use truncated common medical sentences. For example, medical patient A said a sentence - "false physical examination" in an automatic speech recognition model. Then, "false physical examination" is used as the text to be error corrected, and no limitation is imposed on the text to be error corrected.

[0044] If medical patient A uses "false physical examination" to retrieve from the database of a medical institution, the retrieved results will surely be of no help to medical patient A in understanding relevant physical examination matters and making an appointment for a physical examination.

[0045] In the present invention, the "physical" and "examination" in the character-phrase sequence of the text to be error corrected can be directly matched to the correct word "physical examination" in the phrase library of the medical institution, and "physical examination" is used as the first phrase; However, "false" and "examination" do not match the correct word in the phrase library, and "false examination" is used as the second phrase.

[0046] In other embodiments, if the text to be corrected does not match the second phrase with the same features in the phrase library, but only the first phrase, it means that the text to be corrected output by the automatic speech recognition model does not contain erroneous words, and the method of the present invention does not need to perform subsequent correction.

[0047] In other embodiments, if the text to be corrected and the phrase library only match a second phrase with different features, but not the first phrase, it means that the user's voice is non-standard voice, and none of the phrases in the text to be corrected represented by it has lexical features of common sentences that conform to the voice scenario, and there is no need for subsequent correction using the method of the present invention.

[0048] That is to say, the method of the present invention is aimed at the lexical features of common sentences in the text to be corrected that have part of the content that conforms to the speech scene, for example, the text to be corrected that has correct phrases and incorrect phrases.

[0049] In the prior art, text information (Chinese sentences) is usually represented as a character sequence, which only contains word-level features, and the existing error correction model is difficult to use word-level (phrase) features for the following reasons: 1) The existing error correction model is based on the character model, which lacks phrase information. This is because the wrong character detection interferes with the word segmentation results, and the direct use of the word-based model for error correction detection leads to miscorrection on the wrong word segmentation results. Therefore, although the character model avoids the error caused by word segmentation and further affects the error correction effect, it cannot effectively utilize the semantic information of the phrase.

[0050] 2) Existing error correction models have difficulty correcting phrase errors: Chinese words usually consist of one to four characters, which are much shorter than English words. If there is a phonetic error in the semantic core word of a certain text, it will affect the accuracy of the semantics of the entire sentence. The system directly based on words is too complex and difficult to optimize globally. The mainstream error correction model is mainly based on the character model, which is difficult to effectively correct word-level errors.

[0051] In step S1, the character sequence is expanded into a character-phrase sequence. In order to make full use of the role of the phrase library of the error correction model, the vocabulary enhancement module is used to apply the correlation between the words and phrases of the text to be corrected to the error correction, so as to improve the accuracy of word segmentation, thereby solving the technical problem that the semantic information of the phrase cannot be effectively utilized when segmenting words in the prior art.

[0052] S2, using the pinyin fuzzy matching module of the error correction model to perform pinyin matching on the second phrase to obtain at least one third phrase containing the first word, and to obtain at least one fourth phrase containing the second word; In this embodiment, the pinyin fuzzy matching module includes backward pinyin fuzzy matching and forward pinyin fuzzy matching, which are composed of these two steps.

[0053] The Pinyin fuzzy matching module refers to masking the pronunciation information of a phrase. With the two words in the phrase as the restricted original characters in turn, and based on the Pinyin of all phrases in the phrase library, compare the second phrase with the Pinyin of all these phrases (referring to the case where the pronunciation information of all phrases is masked), and calculate the possibility of their similar Pinyin belonging to the same entity.

[0054] In one embodiment, the Pinyin fuzzy matching module using the error correction model performs Pinyin matching on the second phrase to obtain at least one third phrase containing the first word and at least one fourth phrase containing the second word, including: Mask the pronunciation information of the second phrase; Use the Pinyin fuzzy matching module to perform homophone matching on the first word of the second phrase to obtain the third phrase; And perform homophone matching on the second word of the second phrase to obtain the fourth phrase.

[0055] Mask or ignore the pronunciation information of the second phrase that has no identical phrase in the phrase library. The Pinyin fuzzy matching module performs homophone matching on the first word of the second phrase to obtain at least one third phrase with a similar Pinyin to the second phrase and containing the first word.

[0056] For example, in the medical field, for the second phrase "canjia" in the text to be corrected "canjia tijian", mask or ignore the pronunciation information such as the front and back nasal sounds, flat and curled tongues, and tones when patient A pronounces "canjia". Use backward Pinyin fuzzy matching and forward Pinyin fuzzy matching to perform homophone matching on "can" for the second phrase, and obtain "canjia" as the third phrase.

[0057] That is to say, when patient A pronounces the third phrase "canjia" and the second phrase "canjia" with the pronunciation information masked, they are still relatively close during pronunciation, which can truly restore how patient A uses the automatic speech recognition model to perform speech recognition and retrieve medical data in the medical scenario, so as to find the key points to solve the problem and make improvements; For the correct characters in the second phrase, if the wrong word information is incorporated, it is likely to cause overcorrection. The present invention adopts an error reduction strategy to prevent overcorrection caused by the influence of the word-level (phrase) information of the wrong word. Use the Pinyin fuzzy matching module to match the correct word similar to its homophone, ignoring the differences in front and back nasal sounds, flat and curled tongues, and tones.

[0058] For the correct character "can" in the second phrase, through two-way (backward Pinyin fuzzy matching and forward Pinyin fuzzy matching) Pinyin fuzzy matching, ensure that the character is in the middle of the word and match the correct word "canjia" for it.

[0059] In the second phrase "can jia", one or both words may be misspelled. Therefore, "jia" can be selected as the restricted original character, and forward and backward pinyin fuzzy matching can be used to perform homophone matching on "jia" for the second phrase, obtaining "chan jia" as the fourth phrase. That is to say, when the pronunciation information of the fourth phrase "chan jia" and the second phrase "can jia" is masked, their pronunciations are still relatively similar when pronounced.

[0060] For the misspelled characters in the second phrase, an error amplification strategy is adopted. During the matching process of the pinyin fuzzy matching module, the original character is restricted to be in the homophone word, and an incorrect word is matched for the misspelled character. For example, the misspelled character "jia" is matched to "chan jia". Error scaling is achieved to better assist in error detection and correction.

[0061] In one embodiment, the pronunciation information includes information on the front and back nasal sounds, flat and rolled tongues, and tones of the pronunciation of the second phrase.

[0062] In one embodiment, the homophone matching of the first word of the second phrase to obtain the third phrase includes: Mask the position of the second word in the text to be corrected; Use the pinyin fuzzy matching module to perform homophone matching on the first word and all phrases in the phrase library, obtaining at least one first homophone word with a pronunciation similar to that of the second word; Generate the third phrase based on the first word and the first homophone word.

[0063] In one embodiment, the homophone matching of the second word of the second phrase to obtain the fourth phrase includes: Mask the position of the first word in the text to be corrected; Use the pinyin fuzzy matching module to perform homophone matching on the second word and all phrases in the phrase library, obtaining at least one second homophone word with a pronunciation similar to that of the first word; Generate the fourth phrase based on the second word and the second homophone word.

[0064] Masking is a bitwise AND operation on the characters of the text to be corrected with a string of binary codes to mask the current input bit.

[0065] Homophone words refer to words that have very similar pronunciations and spellings but different meanings.

[0066] The error correction models in the prior art usually use homophonic confusion sets to judge the incorrect characters in a phrase. Since homophonic confusion sets are often constructed based on artificial rules, it is difficult to cover all real error situations. Therefore, the advantage of the pinyin fuzzy matching module of the present invention over the homophonic confusion set matching lies in being more comprehensive and controllable.

[0067] In step S2, for the correct characters in the second phrase, if the incorrect word information is incorporated, it is likely to cause overcorrection. The present invention adopts an error reduction strategy to prevent overcorrection caused by the influence of the word-level (phrase) information of the incorrect word. The pinyin fuzzy matching module is used to match the correct word that sounds similar to it, ignoring the differences in front and back nasals, flat and rolled tongues, and tones.

[0068] For the incorrect characters in the second phrase, an error amplification strategy is adopted. During the matching process of the pinyin fuzzy matching module, it is restricted that the original character is located in the homophonic word, and an incorrect word is matched for the incorrect character. For example, the incorrect character "假" is matched to "产假". Error scaling is achieved to better assist in error detection and correction.

[0069] S3. Generate a first candidate text based on the first phrase and the third phrase, calculate the vectors of all words in the first candidate text to generate a first word vector, and generate a second candidate text based on the first phrase and the fourth phrase, calculate the vectors of all words in the second candidate text to generate a second word vector; In this embodiment, in order to obtain the context information of all the matched words and the original characters of the text to be error-corrected, and this context information contains implicit word-level syntactic and semantic knowledge, the dictionary adapter of the error correction model is used to dynamically extract the context information of each original character and the matched word.

[0070] In one embodiment, the calculating the vectors of all words in the first candidate text to generate a first word vector includes: Using the dictionary adapter of the error correction model, calculate the first global information of the first homophonic word of the third phrase and each word in the first candidate text; Take the first global information as the coefficient of the character-phrase attention mechanism algorithm of the dictionary adapter; Use the character-phrase attention mechanism algorithm to perform weighted processing on the first candidate text to obtain the first word vector.

[0071] In one embodiment, the calculating the vectors of all words in the second candidate text to generate a second word vector includes: Using the dictionary adapter to calculate the second global information of the second homophonic word of the fourth phrase and each word in the second candidate text; Take the second global information as the coefficient of the character-phrase attention mechanism algorithm; The second candidate text is weighted using the character-phrase attention mechanism algorithm to obtain the second word vector.

[0072] The dictionary adapter is constructed based on the character-phrase attention mechanism algorithm, enabling the context information of lexical features to be better captured by the error correction model, thereby improving the text error correction ability of the error correction model.

[0073] The dictionary adapter is designed to dynamically extract the most relevant matching words for each character using a character-to-word bilinear attention mechanism. The dictionary adapter is applied between adjacent transformers in LEBERT so that the dictionary function and LEBERT representations can fully interact through the multi-layer encoder within LEBERT. During training, LEBERT and the dictionary adapter are fine-tuned to make full use of word information, which is very different from the built-in adapter of LEBERT (the built-in adapter fixes the LEBERT parameters).

[0074] The global information refers to the context information of the matched words and the original characters of the text to be error-corrected.

[0075] A first candidate text is generated based on the first phrase and the third phrase. For example, the first candidate text generated by the first phrase "physical examination" and the third phrase "participate in" is - "participate in a physical examination".

[0076] And a second candidate text is generated based on the first phrase and the fourth phrase. For example, the second candidate text generated by the first phrase "physical examination" and the fourth phrase "maternity leave" is - "maternity leave physical examination".

[0077] The context information between the characters corresponding to the words matched in the first candidate text and the original characters of the text to be error-corrected is calculated to obtain the first word vector. That is to say, the first word vector contains the context information (global features) of each word in the first candidate text.

[0078] For example, when capturing the first word vector of the first candidate text - "participate in a physical examination" through the dictionary adapter, first calculate the context information between "can" and the original characters "ti", "jian" to obtain the first candidate word vector; Then calculate the context information between "jia" and the original characters "ti", "jian" to obtain the second candidate word vector; The first candidate word vector and the second candidate word vector are added together to obtain the first word vector.

[0079] For example, the first candidate text "Participate in a physical examination" and the second candidate text "Maternity leave physical examination" are both common sentences in medical data. The context information of each word in the first candidate text "Participate in a physical examination" and the second candidate text "Maternity leave physical examination" can only be calculated through the character-phrase attention mechanism algorithm verified under a large amount of medical data. It is very effective to correct common sentences in medical data through the context information of words, which can improve the efficiency of error correction.

[0080] The second word vector is obtained by referring to the method for obtaining the first word vector mentioned above.

[0081] In one embodiment, the character-phrase attention mechanism algorithm includes formulas 1 to 3, including: The first formula is: in, is the first global information, is the hidden vector of the i-th character of the first candidate text, is the weight matrix of bilinear attention, for The transpose of , whose size is m-by- , m is the total number of assigned words.

[0082] The first formula is used to obtain the relevance of each phonetic word of the third phrase and / or the fourth phrase in the context information (global information) of the text information.

[0083] The second formula is: in, is the weighted sum of all words in the first candidate text, j is the jth matching word of the i-th character in the phrase library, is the correlation between the ith character and the jth matching word, is the word vector of the j-th matching word.

[0084] The third phrase and all the words in the first phrase are weighted together using the second formula to obtain a first weighted value.

[0085] The third formula is: in, A new hidden vector that integrates word information into the first candidate text, namely, the first word vector.

[0086] Referring to the above character-phrase attention mechanism algorithm, the formula algorithm for calculating the first candidate text to obtain the first word vector can be used to obtain the second word vector.

[0087] In step S3, a dictionary adapter constructed based on a character-phrase attention mechanism algorithm is used to obtain contextual information of lexical features of the first and second candidate texts, so that the text to be corrected can be corrected more accurately.

[0088] S4. Correct the first candidate text based on the correction module of the error correction model and the first word vector to obtain a first correction result value, and correct the second candidate text based on the correction module and the second word vector to obtain a second correction result value, compare the first correction result value with the second correction result value, and select the candidate text with the largest correction result value as the target text of the text to be corrected.

[0089] In this embodiment, the correction module is trained based on word-level information and syntactic structure of a preset number of text samples. These text samples can be digital medical samples in a medical institution database, and these samples are all medical information obtained from historical patients.

[0090] Using the correction module to correct the word-level information and syntactic structure of the first candidate text and the second candidate text respectively, to obtain a first correction result value of the first candidate text and a second correction result value of the second candidate text; Compare the numerical values ​​of the first correction result value and the second correction result value, select the candidate text of the correction result value with the largest numerical value as the target text of the text to be corrected, and display the target text on the user's voice terminal for the user to perform downstream tasks, such as downstream tasks including voice search, voice translation, etc.

[0091] In scenarios such as medical diagnosis, the user's voice information can be converted into more accurate text through the error correction method of the present invention, and then the user can search for the consultation information or medical information that the user wants. This can improve the user's convenience and avoid the problem of erroneous words appearing in the converted text, which requires the user to manually correct the erroneous words.

[0092] For example, if medical patient A says a sentence "Participate in a fake physical examination" in the automatic speech recognition model, then "Participate in a fake physical examination" is used as the text to be corrected. The present invention corrects "Participate in a fake physical examination" before the automatic speech recognition model outputs the text, and automatically generates the correct text "Participate in a physical examination".

[0093] Then, medical patient A directly uses the correct text "attend physical examination" to search from the database of the medical institution to obtain medical information related to "attend physical examination". Medical patient A can choose to make an appointment for physical examination or get precautions before the physical examination.

[0094] The correction module includes a single-layer feedforward network with softmax normalization and a two-layer feedforward network with layer normalization.

[0095] The correction module corrects the first candidate text and, and the first correction result value finally outputted is the weighted sum of the generation distribution and the copy distribution of the first candidate text, wherein the weight is the copy probability learned by the correction module.

[0096] copy distribution ∈ Computed by a single-layer feed-forward network normalized by softmax; copy distribution is a one-hot vector; the copy probability ω is calculated through a two-layer feed-forward network with layer normalization.

[0097] The final output of the correction module, the first correction result value (p), is calculated by the following second formula set, which includes the fourth to seventh formulas: is the generation distribution of the first candidate text, used to generate the correct character corresponding to the wrong character, and ∈ are the parameters of the generating distribution, The R in represents the vector space, and 768 represents the dimension of the word vector. is the size of the vocabulary of the phrase library.

[0098] The fifth formula is: is the copy distribution, which is used to determine whether the character of the first candidate text is a correct character. The correct character is directly copied, and the wrong character is generated through the generation distribution. is the character at the Kth position, X is the i-th character of the input correction module, c is the character, The index in the vocabulary is denoted by idx( ).

[0099] The sixth formula is: Given a text sample (X, Y), the correction loss L of the token of the i-th word is defined as: The seventh formula is: (5) in, is the cross entropy loss, X is the character sequence of the text sample, Y is the correction sequence of X, and p is the first correction result value.

[0100] The second correction result value may be obtained by referring to the formula algorithm for obtaining the first correction result value by correcting the first candidate text by the correction module.

[0101] In step S4, the word-level information and syntactic structure of the first and second candidate texts are corrected by the correction module to generate corresponding probability distribution correction result values, and the candidate text with the maximum correction result value is used as the text with a lower probability of erroneous words, thereby improving the error correction accuracy of the error correction model.

[0102] In steps S1-S4, the pinyin fuzzy matching module of the error correction model is used to adopt an error magnification strategy for the erroneous characters of the second phrase and a reduction strategy for the correct characters of the second phrase, which can better assist the error detection and correction of the error correction model.

[0103] The dictionary adapter is used to capture the implicit word-level syntactic and semantic knowledge between the text to be corrected and the matched phrases.

[0104] A correction module is used to correct candidate texts. The correction module combines word-level information and syntactic structure to correct errors, effectively improving the accuracy of error correction.

[0105] like Figure 2 , which is a module diagram of a device for correcting speech recognition results provided by an embodiment of the present invention.

[0106] The speech recognition result error correction device 100 of the present invention can be installed in an electronic device. According to the functions implemented, the speech recognition result error correction device 100 can include a phrase matching module 110, a fuzzy matching module 120, a word vector generation module 130 and a correction module 140. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0107] In this embodiment, the functions of each module / unit are as follows: The phrase matching module 110 is used to match the character-phrase sequence of the text to be corrected using a preset error correction model to obtain at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word.

[0108] The fuzzy matching module 120 is used to use the pinyin fuzzy matching module of the error correction model to perform pinyin matching on the second phrase to obtain at least one third phrase containing the first word, and to obtain at least one fourth phrase containing the second word.

[0109] The word vector generation module 130 is used to generate a first candidate text based on the first word group and the third word group, calculate the vectors of all words in the first candidate text to generate a first word vector, and generate a second candidate text based on the first word group and the fourth word group, and calculate the vectors of all words in the second candidate text to generate a second word vector.

[0110] The correction module 140 is used to correct the first candidate text based on the correction module of the error correction model and the first word vector to obtain a first correction result value, and to correct the second candidate text based on the correction module and the second word vector to obtain a second correction result value, compare the first correction result value with the second correction result value, and select the candidate text with the largest correction result value as the target text of the text to be corrected.

[0111] In one embodiment, before matching the character-phrase sequence of the text to be corrected using the preset error correction model, the method further includes: A character sequence of the text to be corrected is obtained, and the character sequence is expanded into a character-phrase sequence.

[0112] In one embodiment, the character-phrase sequence of the text to be corrected is matched using a preset error correction model to obtain at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word, including: Obtaining each phrase feature in the character-phrase sequence; Using the vocabulary enhancement module of the error correction model, calculating whether there is a phrase with the same features as the phrase in the phrase library; If there is an identical phrase, the corresponding phrase in the text to be corrected is used as the first phrase; If there is no identical phrase, the corresponding phrase in the text to be corrected is used as the second phrase.

[0113] In one embodiment, the pinyin fuzzy matching module of the error correction model is used to perform pinyin matching on the second phrase to obtain at least one third phrase containing the first word, and to obtain at least one fourth phrase containing the second word, including: shielding pronunciation information of the second phrase; Using the pinyin fuzzy matching module, matching the first word of the second phrase with a similar-sounding word to obtain the third phrase; and performing sound-similar word matching on the second word of the second phrase to obtain the fourth phrase.

[0114] In one embodiment, performing sound-similar word matching on the first word of the second word group to obtain the third word group includes: Masking the second word at the position of the text to be corrected; Using the pinyin fuzzy matching module, performing phonetic matching on the first word and all the phrases in the phrase library to obtain at least one first phonetic word that is similar in pronunciation to the second word; The third phrase is generated based on the first word and the first sound-similar word.

[0115] In one embodiment, performing sound-similar word matching on the second word of the second phrase to obtain the fourth phrase includes: Masking the first word at the position of the text to be corrected; Using the pinyin fuzzy matching module, performing phonetic matching on the second word and all the phrases in the phrase library to obtain at least one second phonetic word that is similar in pronunciation to the first word; The fourth phrase is generated based on the second word and the second sound-similar word.

[0116] In one embodiment, calculating the vectors of all words in the first candidate text to generate a first word vector includes: Utilizing the dictionary adapter of the error correction model, calculating first global information of the first phonetic word of the third phrase and each word of the first candidate text; Using the first global information as a coefficient of a character-phrase attention mechanism algorithm of the dictionary adapter; The first candidate text is weightedly processed by using the character-phrase attention mechanism algorithm to obtain the first word vector.

[0117] like Figure 3 , which is a schematic diagram of the structure of an electronic device for implementing a method for correcting speech recognition results provided by an embodiment of the present invention.

[0118] In this embodiment, the electronic device 1 includes, but is not limited to, a memory 11, a processor 12, and a network interface 13, which can be interconnected and communicated via a system bus. The memory 11 stores an error correction program 10 for speech recognition results, and the error correction program 10 for speech recognition results can be executed by the processor 12. Figure 3 Only the electronic device 1 having components 11-13 and the error correction program 10 of the speech recognition result is shown, and it can be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0119] Among them, the memory 11 includes a memory and at least one type of readable storage medium. The memory provides a cache for the operation of the electronic device 1; the readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 1; in other embodiments, the non-volatile storage medium may also be an external storage device of the electronic device 1, such as a plug-in hard disk equipped on the electronic device 1, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. In this embodiment, the readable storage medium of the memory 11 is generally used to store an operating system and various application software installed on the electronic device 1, such as the code of the error correction program 10 storing the speech recognition result in an embodiment of the present invention. In addition, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.

[0120] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 12 is generally used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication with other devices. In this embodiment, the processor 12 is used to run the program code stored in the memory 11 or process data, such as running the error correction program 10 of the speech recognition result.

[0121] The network interface 13 may include a wireless network interface or a wired network interface, and the network interface 13 is used to establish a communication connection between the electronic device 1 and a terminal (not shown in the figure).

[0122] Optionally, the electronic device 1 may further include a user interface, which may include a display (Display), an input unit such as a keyboard (Keyboard), and the optional user interface may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0123] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0124] The error correction program 10 of the speech recognition result stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 12, it can achieve: Using a preset error correction model to match a character-phrase sequence of a text to be corrected, obtaining at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word; Using the pinyin fuzzy matching module of the error correction model, the second phrase is matched by pinyin to obtain at least one third phrase containing the first word, and at least one fourth phrase containing the second word; Generate a first candidate text based on the first word group and the third word group, calculate vectors of all words in the first candidate text to generate a first word vector, and generate a second candidate text based on the first word group and the fourth word group, calculate vectors of all words in the second candidate text to generate a second word vector; The first candidate text is corrected based on the correction module of the error correction model and the first word vector to obtain a first correction result value, and the second candidate text is corrected based on the correction module and the second word vector to obtain a second correction result value, the first correction result value is compared with the second correction result value, and the candidate text with the largest correction result value is selected as the target text of the text to be corrected.

[0125] Specifically, the specific implementation method of the processor 12 for the above-mentioned speech recognition result correction program 10 can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0126] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or non-volatile. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0127] The computer-readable storage medium stores a speech recognition result error correction program 10, which can be executed by one or more processors. The specific implementation method of the computer-readable storage medium of the present invention is basically the same as the various embodiments of the speech recognition result error correction method mentioned above, and will not be repeated here.

[0128] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0129] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0131] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0132] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0133] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for correcting speech recognition results, characterized in that: The method comprises: Using a preset error correction model to match a character-phrase sequence of a text to be corrected, obtaining at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word; Using the pinyin fuzzy matching module of the error correction model, the second phrase is matched by pinyin to obtain at least one third phrase containing the first word, and at least one fourth phrase containing the second word; Generate a first candidate text based on the first word group and the third word group, calculate vectors of all words in the first candidate text to generate a first word vector, and generate a second candidate text based on the first word group and the fourth word group, calculate vectors of all words in the second candidate text to generate a second word vector; The first candidate text is corrected based on the correction module of the error correction model and the first word vector to obtain a first correction result value, and the second candidate text is corrected based on the correction module and the second word vector to obtain a second correction result value, the first correction result value is compared with the second correction result value, and the candidate text with the largest correction result value is selected as the target text of the text to be corrected.

2. The method for correcting speech recognition results as claimed in claim 1, characterized in that: Before matching the character-phrase sequence of the text to be corrected by using the preset error correction model, the method further includes: A character sequence of the text to be corrected is obtained, and the character sequence is expanded into a character-phrase sequence.

3. The method for correcting speech recognition results as claimed in claim 1, characterized in that: The method uses a preset error correction model to match a character-phrase sequence of a text to be corrected, and obtains at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word, including: Obtaining each phrase feature in the character-phrase sequence; Using the vocabulary enhancement module of the error correction model, calculating whether there is a phrase with the same features as the phrase in the phrase library; If there is an identical phrase, the corresponding phrase in the text to be corrected is used as the first phrase; If there is no identical phrase, the corresponding phrase in the text to be corrected is used as the second phrase.

4. The method for correcting speech recognition results as claimed in claim 1, characterized in that: The pinyin fuzzy matching module of the error correction model is used to perform pinyin matching on the second phrase to obtain at least one third phrase containing the first word, and to obtain at least one fourth phrase containing the second word, including: shielding pronunciation information of the second phrase; Using the pinyin fuzzy matching module, matching the first word of the second phrase with a similar-sounding word to obtain the third phrase; and performing sound-similar word matching on the second word of the second phrase to obtain the fourth phrase.

5. The method for correcting speech recognition results as claimed in claim 4, characterized in that: The performing sound-similar word matching on the first word of the second phrase to obtain the third phrase comprises: Masking the second word at the position of the text to be corrected; Using the pinyin fuzzy matching module, performing phonetic matching on the first word and all the phrases in the phrase library to obtain at least one first phonetic word that is similar in pronunciation to the second word; The third phrase is generated based on the first word and the first sound-similar word.

6. The method for correcting speech recognition results as claimed in claim 4, characterized in that: The performing sound-similar word matching on the second word of the second phrase to obtain the fourth phrase comprises: Masking the first word at the position of the text to be corrected; Using the pinyin fuzzy matching module, performing phonetic matching on the second word and all the phrases in the phrase library to obtain at least one second phonetic word that is similar in pronunciation to the first word; The fourth phrase is generated based on the second word and the second sound-similar word.

7. The method for correcting speech recognition results as claimed in claim 1, characterized in that: The step of calculating the vectors of all words in the first candidate text to generate a first word vector includes: Utilizing the dictionary adapter of the error correction model, calculating first global information of the first phonetic word of the third phrase and each word of the first candidate text; Using the first global information as a coefficient of a character-phrase attention mechanism algorithm of the dictionary adapter; The first candidate text is weightedly processed by using the character-phrase attention mechanism algorithm to obtain the first word vector.

8. A device for correcting speech recognition results, characterized in that: The device comprises: A phrase matching module, used to match the character-phrase sequence of the text to be corrected using a preset error correction model, to obtain at least one first phrase that matches a phrase in a phrase library of the error correction model, and at least one second phrase that does not match a phrase in the phrase library, wherein the second phrase includes a first word and a second word; A fuzzy matching module, used to perform pinyin matching on the second word group using the pinyin fuzzy matching module of the error correction model to obtain at least one third word group including the first word, and to obtain at least one fourth word group including the second word; A word vector generation module, used to generate a first candidate text based on the first word group and the third word group, calculate the vectors of all words in the first candidate text to generate a first word vector, and generate a second candidate text based on the first word group and the fourth word group, calculate the vectors of all words in the second candidate text to generate a second word vector; A correction module is used to correct the first candidate text based on the correction module of the error correction model and the first word vector to obtain a first correction result value, and to correct the second candidate text based on the correction module and the second word vector to obtain a second correction result value, compare the first correction result value with the second correction result value, and select the candidate text with the largest correction result value as the target text of the text to be corrected.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores an error correction program for speech recognition results that can be executed by the at least one processor, and the error correction program for speech recognition results is executed by the at least one processor so that the at least one processor can execute the error correction method for speech recognition results as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a speech recognition result error correction program, and the speech recognition result error correction program can be executed by one or more processors to implement the speech recognition result error correction method as described in any one of claims 1 to 7.

Images