Text correction method, apparatus, device, and medium

Abstract

The application relates to the technical field of artificial intelligence, and discloses a text correction method and device, equipment and a medium, wherein the method comprises the following steps: obtaining initial text input by a user based on a preset ASR speech recognition technology or input method technology, performing pinyin conversion on the initial text to obtain an initial pinyin sequence; and performing pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary tree in a preset pinyin dictionary tree configuration to obtain corrected text, wherein each pinyin dictionary tree in the pinyin dictionary tree configuration comprises a wrong word pinyin dictionary tree, a multiple-word pinyin dictionary tree, a few-word pinyin dictionary tree and a disordered pinyin dictionary tree. Thus, the correction of homophonic words, few words, multiple words and disordered text is realized, and the accuracy and comprehensiveness of the correction are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a text correction method, apparatus, device, and medium. Background Technology

[0002] Text correction, a challenging task in Natural Language Processing (NLP), aims to improve the accuracy of text, ensuring the correct execution and output of downstream NLP tasks. With wide-ranging applications, addressing various text error types across different scenarios presents a significant challenge. As the most fundamental and upstream module in NLP, the performance of the text correction module directly impacts the effectiveness and output of all downstream tasks, highlighting its crucial importance. User text input methods include input via input methods and speech-to-text conversion. Errors in the resulting text primarily include homophones, missing characters, extra characters, and disordered text. Current academic text correction methods mainly focus on semantic errors, resulting in low accuracy and insufficient comprehensiveness. Similarly, industrial text correction methods primarily address obfuscation errors, also leading to low accuracy and insufficient comprehensiveness. Summary of the Invention

[0003] The main objective of this application is to provide a text correction method, apparatus, device, and medium, aiming to solve the technical problems of low accuracy and insufficient comprehensiveness in existing text correction methods for text input by input methods and voice input converted into text.

[0004] To achieve the aforementioned objectives, this application proposes a text correction method, the method comprising:

[0005] The initial text input by the user is obtained based on preset ASR speech recognition technology or input method technology;

[0006] Based on the various pinyin dictionary trees in the preset pinyin dictionary tree configuration, the initial text is converted into pinyin and corrected based on pinyin to obtain the corrected text. The various pinyin dictionary trees in the pinyin dictionary tree configuration include: a pinyin dictionary tree for misspelled characters, a pinyin dictionary tree for characters with many characters, a pinyin dictionary tree for characters with few characters, and a pinyin dictionary tree for characters in random order.

[0007] Further, the pinyin dictionary configuration sequentially includes: the misspelled pinyin dictionary, the multi-character pinyin dictionary, the short-character pinyin dictionary, and the disordered pinyin dictionary. The step of performing pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration to obtain the corrected text includes:

[0008] The initial text is converted to pinyin to obtain an initial pinyin sequence;

[0009] Based on the misspelled pinyin dictionary tree and the initial pinyin sequence, the initial text is corrected to obtain the first text and the first pinyin sequence;

[0010] Based on the multi-character pinyin dictionary and the first pinyin sequence, the first text is corrected to obtain the second text and the second pinyin sequence;

[0011] Based on the missing-character pinyin dictionary and the second pinyin sequence, the second text is corrected to obtain the third text and the third pinyin sequence;

[0012] Based on the disordered pinyin dictionary and the third pinyin sequence, the third text is corrected to obtain the corrected text.

[0013] Further, the step of correcting the initial text based on the misspelled pinyin dictionary and the initial pinyin sequence to obtain the first text and the first pinyin sequence includes:

[0014] The initial pinyin sequence is input into the misspelled pinyin dictionary tree for entity node matching to obtain the first result;

[0015] If the first result is a failure, then the initial text is used as the first text, and the initial pinyin sequence is used as the first pinyin sequence;

[0016] If the first result is successful, then each entity node matched from the misspelled pinyin dictionary tree will be used as the node set to be evaluated. The initial text will be corrected according to the preset N-gram statistical language model and the node set to be evaluated to obtain the first text, and the pinyin sequence corresponding to the first text will be used as the first pinyin sequence.

[0017] Further, the step of correcting the initial text according to a preset N-gram statistical language model and the set of nodes to be evaluated to obtain the first text includes:

[0018] Take any entity node in the set of nodes to be evaluated as a candidate node;

[0019] Determine whether the substring corresponding to the candidate node in the initial text exists in the entity word set of the candidate node. If it exists, repeat the step of taking any entity node in the node set to be evaluated as a candidate node until the nodes in the node set to be evaluated are obtained. Otherwise, determine the substring as the problem string.

[0020] Based on the preset string extraction method and the question string, a string is extracted from the initial text to obtain the string to be evaluated;

[0021] The N-gram statistical language model is used to score the string to be evaluated, and the first misspelling score is obtained.

[0022] Based on each entity word in the entity word set corresponding to the string to be evaluated, the string to be evaluated is replaced to obtain a candidate string;

[0023] The N-gram statistical language model is used to score each candidate string to obtain a second misspelling score;

[0024] Determine whether there exists a second misspelled word score that is greater than the product of the first misspelled word score and a preset score coefficient;

[0025] If it exists, the entity word corresponding to the maximum value among the second misspelling scores that are greater than the score product value will replace the initial text;

[0026] Repeat the step of selecting any entity node in the set of nodes to be evaluated as a candidate node until all nodes in the set of nodes to be evaluated are obtained.

[0027] The initial text is used as the first text.

[0028] Further, the step of extracting strings from the initial text according to a preset string extraction method and the question string to obtain the string to be evaluated includes:

[0029] If the question string is at the beginning of the initial text, then the preset beginning string will be used as the forward extracted character substring;

[0030] If the question string is not at the beginning of the initial text, then extract an additional first number of characters as the forward extracted character substring;

[0031] If the question string is located at the end of the initial text, then the preset ending string will be used as the substring to be extracted later.

[0032] If the question string is not located at the end of the initial text, then extract a second number of characters forward as the backward extracted character substring;

[0033] The forward-extracted character substring, the question string, and the backward-extracted character substring are concatenated sequentially to obtain the string to be evaluated.

[0034] Furthermore, the pinyin sequence of the pinyin dictionary in the pinyin dictionary configuration needs to have tones, retroflex consonants, and nasal consonants removed.

[0035] Furthermore, the misspelling dictionary is used to correct characters that are similar in both pronunciation and form;

[0036] The multi-character pinyin dictionary tree is used to correct errors of having an extra character. It is a dictionary tree generated based on preset wildcard pinyin symbols and preset placeholders for incorrect characters.

[0037] The missing-character pinyin dictionary tree is used to correct errors where one character is missing.

[0038] The disordered pinyin dictionary is used to correct entity words whose character length is greater than or equal to a preset character length. Each preset word to be processed is segmented, and a disordered error text set is generated from the segmented data. A preset N-gram statistical language model is used to score each disordered error text in the disordered error text set. Scores greater than a preset score threshold are identified from the scores obtained. The disordered error texts corresponding to the identified scores are used as target disordered error text sets. The disordered pinyin dictionary is generated based on the target disordered error text sets corresponding to each word to be processed.

[0039] This application also proposes a text correction device, the device comprising:

[0040] The text acquisition module is used to acquire the initial text input by the user based on preset ASR speech recognition technology or input method technology, and to convert the initial text into pinyin to obtain an initial pinyin sequence;

[0041] The text correction module is used to perform pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration, so as to obtain the corrected text. The pinyin dictionary in the preset pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for multiple characters, a pinyin dictionary for missing characters, and a pinyin dictionary for disordered characters.

[0042] This application also proposes a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0043] This application also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0044] The text correction method, apparatus, device, and medium of this application, wherein the method performs pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in a preset pinyin dictionary configuration, to obtain the corrected text. The pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for characters with many characters, a pinyin dictionary for characters with few characters, and a pinyin dictionary for text in a disordered order, thereby achieving error correction for text with similar-sounding characters, characters with few characters, characters with many characters, and text in a disordered order, improving the accuracy and comprehensiveness of error correction. Errors in speech-to-text conversion caused by user accents and / or potential environmental background noise are mainly due to inaccurate pinyin pronunciation recognition. To address this problem, the pinyin dictionary configuration of this application uses only pinyin dictionary trees to specifically correct errors caused by inaccurate pinyin pronunciation recognition, further improving the accuracy and comprehensiveness of error correction. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating a text correction method according to an embodiment of this application;

[0046] Figure 2 This is a schematic block diagram of the structure of a text correction device according to an embodiment of this application;

[0047] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application.

[0048] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] The execution subject of this method is a device capable of implementing text correction methods, including but not limited to terminals and servers. Terminals include desktop terminals and mobile terminals; desktop terminals include but are not limited to desktop computers, industrial control computers, and in-vehicle computers; mobile terminals include but are not limited to mobile phones, tablets, laptops, smartwatches, and other wearable devices. Servers include high-performance computers and high-performance computer clusters.

[0051] Reference Figure 1 This application provides a text correction method, the method comprising:

[0052] S1: Obtain the initial text input by the user based on preset ASR speech recognition technology or input method technology;

[0053] Specifically, the initial text can be obtained from a database, from user input, or from a third-party application.

[0054] The initial text is the text that needs to be corrected.

[0055] ASR speech recognition technology, also known as automatic speech recognition technology, is used to convert speech into text.

[0056] Input method technology refers to the technology of text input using any one of the following input methods: Pinyin input method, Wubi input method, or handwriting input method.

[0057] S2: Based on each pinyin dictionary in the preset pinyin dictionary configuration, the initial text is converted into pinyin and text is corrected based on pinyin to obtain the corrected text. The pinyin dictionary in the preset pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for multiple characters, a pinyin dictionary for fewer characters, and a pinyin dictionary for disordered characters.

[0058] The various pinyin dictionary trees in the pinyin dictionary configuration include: the pinyin dictionary tree for misspelled characters, the pinyin dictionary tree for characters with many characters, the pinyin dictionary tree for characters with few characters, and the pinyin dictionary tree for characters in random order.

[0059] A trie (or misspelled pinyin dictionary) is a tree that uses pinyin for different characters, including those with fewer characters, those with fewer characters, and those with disordered pinyin. Each edge in a trie corresponds to the pinyin of a single character. Nodes in a trie are divided into blank nodes and entity nodes. Blank nodes contain no data, while each entity node stores at least one entity word. Nodes are connected by edges (each edge corresponding to a single pinyin character). By dividing nodes into entity nodes and blank nodes, appropriate entity nodes can be set according to the error correction goals of the trie.

[0060] It is understood that the specific implementation of performing pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration is as follows: the first pinyin dictionary in the configuration is used to correct the initial text based on the pinyin of the initial text to achieve the first correction; the second pinyin dictionary in the configuration is used to correct the initial text based on the pinyin of the initial text after the first correction to achieve the second correction; the third pinyin dictionary in the configuration is used to correct the initial text based on the pinyin of the initial text after the second correction to achieve the third correction; and the fourth pinyin dictionary in the configuration is used to correct the initial text based on the pinyin of the initial text after the third correction to achieve the fourth correction.

[0061] Optionally, the initial text can be corrected using a dictionary tree for incorrectly spelled characters, a dictionary tree for characters with more characters, a dictionary tree for characters with fewer characters, and a dictionary tree for characters in a random order. It is understood that other orders can also be used for text correction, such as using a dictionary tree for incorrectly spelled characters, a dictionary tree for characters with fewer characters, a dictionary tree for characters with more characters, and a dictionary tree for characters in a random order; this is not limited to these orders.

[0062] The step of sequentially using a misspelled pinyin dictionary, a multi-character pinyin dictionary, a short-character pinyin dictionary, and a disordered pinyin dictionary to correct errors in the initial text includes: converting the initial text into pinyin to obtain an initial pinyin sequence; correcting the initial text based on the misspelled pinyin dictionary and the initial pinyin sequence to obtain a first text and a first pinyin sequence; correcting the first text based on the multi-character pinyin dictionary and the first pinyin sequence to obtain a second text and a second pinyin sequence; correcting the second text based on the short-character pinyin dictionary and the second pinyin sequence to obtain a third text and a third pinyin sequence; and correcting the third text based on the disordered pinyin dictionary and the third pinyin sequence to obtain the corrected text.

[0063] It is understandable that similar-looking characters are generally similar-sounding characters, so the misspelling pinyin dictionary tree in this application can also correct characters that are similar in both appearance and sound.

[0064] The existing pinyin dictionary nodes store keywords, and the edges of the pinyin dictionary correspond to the pinyin of characters. During error correction, as long as the pinyin of each edge from the root node to the matched node is equal to the pinyin of the text to be corrected, the keyword stored in the matched node is used as a candidate keyword. The whole process is to correct misspelled words. The number of characters of the keyword stored in the matched node is the same as the number of edges corresponding to the entire path from the root node to the matched node and the corresponding pinyin order. The pinyin dictionary tree in this application corrects errors for misspelled characters, the pinyin dictionary tree for multiple characters, the pinyin dictionary tree for fewer characters, and the pinyin dictionary tree for disordered text. The number of characters in the text stored in the entity nodes of the multi-character pinyin dictionary tree and the pinyin dictionary tree for fewer characters is different from the number of edges corresponding to the entire path from the root node to the matched node. Similarly, the order of pinyin in the entity nodes of the disordered pinyin dictionary tree is different from the order of pinyin in the entire path from the root node to the matched node. Therefore, the pinyin dictionary tree in this application differs from existing pinyin dictionary trees in its design concept. Furthermore, existing technologies do not propose correcting errors for multiple characters, fewer characters, or disordered text based on pinyin dictionary trees, and do not provide corresponding technical inspiration. Furthermore, the error correction targets of each pinyin dictionary tree in the misspelled pinyin dictionary tree, multi-character pinyin dictionary tree, short-character pinyin dictionary tree, and disordered pinyin dictionary tree are different. Therefore, the misspelled pinyin dictionary tree, multi-character pinyin dictionary tree, short-character pinyin dictionary tree, and disordered pinyin dictionary tree are not simply a stack of multiple pinyin dictionary trees. Compared with the existing pinyin dictionary trees, which can only correct pinyin errors, the text error correction method of this application can correct the problems of misspelled characters, extra characters, missing characters, and text disorder in the initial text based on pinyin, thereby improving the error correction effect. Moreover, this application is more suitable for application scenarios that correct errors in text based on preset ASR speech recognition technology or input method technology.

[0065] This embodiment performs pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration, resulting in corrected text. The pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for characters with many characters, a pinyin dictionary for characters with few characters, and a pinyin dictionary for text in random order. This achieves error correction for text with similar-sounding characters, characters with few characters, characters with many characters, and text in random order, improving the accuracy and comprehensiveness of error correction. Errors in speech-to-text conversion due to user accents and / or potential environmental background noise are mainly caused by inaccurate pinyin pronunciation recognition. To address this issue, the pinyin dictionary configuration in this application uses only pinyin dictionary trees to specifically correct errors caused by inaccurate pinyin pronunciation recognition, further improving the accuracy and comprehensiveness of error correction.

[0066] In one embodiment, the above-mentioned pinyin dictionary configuration sequentially includes: the misspelled pinyin dictionary, the multi-character pinyin dictionary, the short-character pinyin dictionary, and the disordered pinyin dictionary. The step of performing pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration to obtain the corrected text includes:

[0067] S21: Perform pinyin conversion on the initial text to obtain an initial pinyin sequence;

[0068] Specifically, the initial text is input into a preset pinyin module to convert the text into pinyin, thereby obtaining the initial pinyin sequence.

[0069] The Pinyin module is used to convert text into Pinyin. Pinyin modules can be selected from existing technologies, which will not be elaborated upon here.

[0070] S22: Correct the initial text according to the misspelled pinyin dictionary tree and the initial pinyin sequence to obtain the first text and the first pinyin sequence;

[0071] Specifically, the initial pinyin sequence is input into the misspelled pinyin dictionary for entity node matching. If no entity node is matched, it means that the initial text is error-free relative to the misspelled pinyin dictionary, and the initial text is directly used as the first text, and the initial pinyin sequence is used as the first pinyin sequence. If an entity node is matched, it means that the initial text has an error relative to the misspelled pinyin dictionary, and the best replacement word is selected from each entity word in the matched entity node. Then, the initial text is replaced with the best replacement word, and finally, the initial text is used as the first text, and the pinyin sequence corresponding to the first text is used as the first pinyin sequence.

[0072] S23: Based on the multi-character pinyin dictionary and the first pinyin sequence, the first text is corrected to obtain the second text and the second pinyin sequence;

[0073] Specifically, the first pinyin sequence is input into the multi-character pinyin dictionary for entity node matching. If no node is matched, it means that the first text has no errors relative to the multi-character pinyin dictionary, and the first text is directly used as the second text, and the first pinyin sequence is used as the second pinyin sequence. If an entity node is matched, it means that the first text has errors relative to the multi-character pinyin dictionary, and the best replacement word is selected from each entity word in the matched entity node. Then, the best replacement word is used to replace the first text, and finally, the first text is used as the second text, and the pinyin sequence corresponding to the second text is used as the second pinyin sequence.

[0074] S24: Based on the missing-character pinyin dictionary and the second pinyin sequence, the second text is corrected to obtain the third text and the third pinyin sequence;

[0075] Specifically, the second pinyin sequence is input into the short-character pinyin dictionary for entity node matching. If no entity node is matched, it means that the second text has no errors relative to the short-character pinyin dictionary, and the second text is directly used as the third text, and the second pinyin sequence is used as the third pinyin sequence. If an entity node is matched, it means that the second text has errors relative to the short-character pinyin dictionary, and the best replacement word is selected from each entity word in the matched entity node. Then, the best replacement word is used to replace the second text, and finally, the second text is used as the third text, and the pinyin sequence corresponding to the third text is used as the third pinyin sequence.

[0076] S25: Correct the third text according to the disordered pinyin dictionary and the third pinyin sequence to obtain the corrected text.

[0077] Specifically, the third pinyin sequence is input into the disordered pinyin dictionary for entity node matching. If no entity node is matched, it means that the third text has no errors relative to the disordered pinyin dictionary, and the third text is directly used as the corrected text. If an entity node is matched, it means that the third text has errors relative to the disordered pinyin dictionary, the best replacement word is selected from each entity word in the matched entity node, and then the third text is replaced with the best replacement word. Finally, the third text is used as the corrected text.

[0078] This embodiment implements text correction by sequentially using a misspelled pinyin dictionary tree, a multi-character pinyin dictionary tree, a short-character pinyin dictionary tree, and a disordered pinyin dictionary tree. This achieves correction of homophones, missing characters, multi-characters, and disordered text, improving the accuracy and comprehensiveness of correction. Errors in speech-to-text conversion caused by user accents and / or potential environmental background noise are mainly due to inaccurate pinyin pronunciation recognition. To address this issue, this application uses a pinyin dictionary tree configuration throughout, specifically correcting errors caused by inaccurate pinyin pronunciation recognition, further improving the accuracy and comprehensiveness of correction. Since errors in speech-to-text conversion after inputting text and voice input decrease in the order of misspelled characters, multi-characters, missing characters / images, and disordered text, sequentially using a misspelled pinyin dictionary tree, a multi-character pinyin dictionary tree, a short-character pinyin dictionary tree, and a disordered pinyin dictionary tree further enhances the accuracy and comprehensiveness of correction.

[0079] In one embodiment, the step of correcting the initial text based on the misspelled pinyin dictionary and the initial pinyin sequence to obtain the first text and the first pinyin sequence includes:

[0080] S221: Input the initial pinyin sequence into the misspelled pinyin dictionary tree for entity node matching to obtain a first result;

[0081] Specifically, based on a greedy strategy, the initial pinyin sequence is input into the misspelled pinyin dictionary to search for nodes. If there are pinyin sequences corresponding to each edge between an entity node and the root node that are the same as the initial pinyin sequence, the first result is determined to be successful; otherwise, the first result is determined to be unsuccessful.

[0082] S222: If the first result is a failure, then the initial text is used as the first text, and the initial pinyin sequence is used as the first pinyin sequence;

[0083] Specifically, if the first result is a failure, it means that the initial text does not contain any errors relative to the misspelled pinyin dictionary. Therefore, the initial text is used as the first text, and the initial pinyin sequence is used as the first pinyin sequence.

[0084] S223: If the first result is successful, then each entity node matched from the misspelled pinyin dictionary tree will be used as the set of nodes to be evaluated. The initial text will be corrected according to the preset N-gram statistical language model and the set of nodes to be evaluated to obtain the first text. The pinyin sequence corresponding to the first text will be used as the first pinyin sequence.

[0085] Specifically, if the first result is successful, it means that the initial text contains errors relative to the misspelled pinyin dictionary. Therefore, each entity node matched from the misspelled pinyin dictionary based on the initial pinyin sequence is used as the set of nodes to be evaluated. This achieves node search based on a greedy strategy.

[0086] According to the N-gram statistical language model, the best replacement word is selected from each entity word in the set of nodes to be evaluated, and then the initial text is replaced by the best replacement word. Finally, the initial text is used as the first text, and the pinyin sequence corresponding to the first text is used as the first pinyin sequence.

[0087] It is understood that the N-gram statistical language model in this application can be a 2-gram statistical language model or a 3-gram statistical language model.

[0088] N-gram is an algorithm based on statistical language models. Its basic idea is to process the text content into a sliding window of size N bytes, forming a sequence of byte segments of length N. Each byte segment is called a gram. The frequency of all grams is statistically analyzed and filtered according to a pre-defined threshold to form a list of key grams, which is the vector feature space of the text. Each gram in the list represents a feature vector dimension. This model is based on the assumption that the occurrence of the Nth word is only related to the preceding N-1 words and not to any other words; the probability of the entire sentence is the product of the probabilities of each word. These probabilities can be obtained by directly counting the number of times N words appear simultaneously in the corpus. Commonly used algorithms are the bigram (Bi-gram) and the trigram (Tri-gram).

[0089] This embodiment implements error correction of the initial text based on the misspelled pinyin dictionary tree, thereby avoiding the presence of misspelled characters in the first text.

[0090] In one embodiment, the step of correcting the initial text based on a preset N-gram statistical language model and the set of nodes to be evaluated to obtain the first text includes:

[0091] S2231: Select any entity node in the set of nodes to be evaluated as a candidate node;

[0092] Specifically, any entity node in the set of nodes to be evaluated is selected as a candidate node, thus providing a basis for using one entity node from the set of nodes to be evaluated for text correction each time.

[0093] S2232: Determine whether the substring corresponding to the candidate node in the initial text exists in the entity word set of the candidate node. If it exists, repeat the step of taking any entity node in the node set to be evaluated as a candidate node until the nodes in the node set to be evaluated are obtained. Otherwise, determine the substring as the problem string.

[0094] Specifically, it is determined whether the substring corresponding to the candidate node in the initial text exists in the entity word set of the candidate node; if it exists, it means that the initial text does not have an error relative to the candidate node, and the step of taking any entity node in the node set to be evaluated as a candidate node is repeated, that is, steps S2231 to S2232 are repeated. If the acquisition of the node set to be evaluated is completed at this time, the error correction of the initial text ends and step S22310 begins; if it does not exist, it means that the initial text has an error relative to the candidate node, therefore, the substring is determined as the problem string.

[0095] S2233: Based on the preset string extraction method and the question string, extract the string from the initial text to obtain the string to be evaluated;

[0096] Specifically, a string is extracted from the initial text, and the extracted string is used as the string to be evaluated. When extracting the string, a first number of characters are extracted forward and a second number of characters are extracted backward.

[0097] S2234: Using the N-gram statistical language model, score the string to be evaluated to obtain the first misspelling score;

[0098] Specifically, the string to be evaluated is input into the N-gram statistical language model for scoring, and the data obtained from the scoring is used as the first misspelling score.

[0099] S2235: Replace the string to be evaluated with each entity word in the entity word set corresponding to the string to be evaluated to obtain a candidate string;

[0100] Specifically, the string to be evaluated is replaced with each entity word in the entity word set corresponding to the string to be evaluated, and each resulting string is used as a candidate string. In other words, a candidate string is obtained by replacing the string to be evaluated with an entity word in the entity word set corresponding to the string to be evaluated.

[0101] S2236: Using the N-gram statistical language model, score each candidate string to obtain a second misspelling score;

[0102] Specifically, each candidate string is input into the N-gram statistical language model for scoring, and the data obtained from the scoring is used as the second misspelling score.

[0103] S2237: Determine whether there exists a second misspelled word score that is greater than the product of the first misspelled word score and a preset score coefficient;

[0104] Specifically, the score for the first misspelled word is multiplied by a preset scoring coefficient to obtain a score product value; then it is determined whether there is a second misspelled word score greater than the score product value.

[0105] Optionally, the scoring coefficient is set to 10, thereby determining whether the score of the second misspelled word is greater than the score of the first misspelled word by an order of magnitude, thus improving the accuracy of text correction.

[0106] S2238: If it exists, then replace the initial text with the entity word corresponding to the maximum value among the second misspelling scores that are greater than the product of the scores;

[0107] Specifically, if there exists a second misspelling score greater than the score product value, it means that the second misspelling score is an order of magnitude greater than the first misspelling score. Therefore, the entity word corresponding to the maximum value among all the second misspelling scores that are greater than the score product value is used to replace the initial text.

[0108] S2239: Repeat the step of taking any entity node in the set of nodes to be evaluated as a candidate node until the acquisition of nodes in the set of nodes to be evaluated is completed.

[0109] Specifically, the step of selecting any entity node in the set of nodes to be evaluated as a candidate node is repeated, that is, steps S2231 to S2239 are repeated until the nodes in the set of nodes to be evaluated are obtained. At this point, the repeated execution of steps S2231 to S2239 is stopped, and step S22310 is started. When the nodes in the set of nodes to be evaluated are obtained, the error correction of the initial text based on the misspelled pinyin dictionary tree has been completed.

[0110] S22310: Use the initial text as the first text.

[0111] Specifically, the initial text after correction based on the misspelled pinyin dictionary is taken as the first text, and the pinyin sequence corresponding to the first text is taken as the first pinyin sequence.

[0112] It is understandable that the implementation principle of step S23 is the same as that of steps S2231 to S22310, the implementation principle of step S24 is the same as that of steps S2231 to S22310, and the implementation principle of step S25 is the same as that of steps S2231 to S22310.

[0113] This embodiment uses an N-gram statistical language model to select the best replacement word from each entity word in the set of nodes to be evaluated, then uses the best replacement word to replace the initial text, and finally uses the initial text as the first text, thereby improving the accuracy of the replacement.

[0114] In one embodiment, the step of extracting a string from the initial text according to a preset string extraction method and the question string to obtain the string to be evaluated includes:

[0115] S22331: If the question string is at the beginning of the initial text, then the preset beginning string is used as the forward extracted character substring;

[0116] Specifically, if the question string is at the beginning of the initial text, it means that no more characters can be extracted from it. Therefore, the preset starting string is used as the substring to be extracted from it.

[0117] S22332: If the question string is not at the beginning of the initial text, then extract an additional first number of characters as the forward extracted character substring;

[0118] S22333: If the question string is located at the end of the initial text, then the preset ending string is used as the character substring to be extracted later;

[0119] Specifically, if the problem string is located at the end of the initial text, it means that no more characters can be extracted from it. Therefore, the preset ending string is used as the substring to be extracted from it.

[0120] S22334: If the question string is not located at the end of the initial text, then extract a second number of characters forward as the backward extracted character substring;

[0121] S22335: Concatenate the forward-extracted character substring, the question string, and the backward-extracted character substring sequentially to obtain the string to be evaluated.

[0122] Specifically, the forward-extracted character substring, the question string, and the backward-extracted character substring are concatenated sequentially to obtain a string that includes the question string itself and its context.

[0123] Optionally, both the first and second quantities are 2. If the question string is at the beginning of the initial text, the first number of characters are not extracted further; instead, the preset starting character is used as the additional extracted characters. If the question string is at the end of the initial text, the second number of characters are not extracted further; instead, the preset ending character is used as the additional extracted characters. For example, if the initial text is "desktop terminals include, but are not limited to, desktop computers, industrial control computers, and vehicle-mounted computers," and the question string is "desktop terminals include," with the preset starting character being #, then the string to be evaluated is "#desktop terminals include but not." Another example: if the initial text is "desktop terminals include, but are not limited to, desktop computers, industrial control computers, and vehicle-mounted computers," and the question string is "machine, vehicle-mounted computer," with the preset starting character being #, then the string to be evaluated is "industrial control computers, vehicle-mounted computers #."

[0124] This embodiment extracts an additional first number of characters forward as the forward extracted character substring and an additional second number of characters forward as the backward extracted character substring. This approach considers the context when selecting the best replacement word from the entity words in the set of nodes to be evaluated, thereby improving the accuracy of determining the best replacement word and enhancing the accuracy of text correction.

[0125] In one embodiment, the pinyin sequence of the pinyin dictionary in the above-mentioned input pinyin dictionary configuration needs to remove tones, retroflex consonants, and nasal consonants.

[0126] Because incorrect characters often have the same pronunciation in ASR speech recognition, a constructed pinyin dictionary is used to correct characters that are similar in sound and form. Considering the issue of users with widely distributed geographical locations and varying Mandarin pronunciation, the pinyin sequences input into the pinyin dictionary configuration need to have tones, retroflex consonants, and nasal consonants removed, thereby improving the recall of erroneous text and enhancing the text correction capability.

[0127] In one embodiment, the aforementioned misspelling dictionary is used to correct characters that are similar in sound and form;

[0128] The multi-character pinyin dictionary tree is used to correct errors of having an extra character. It is a dictionary tree generated based on preset wildcard pinyin symbols and preset placeholders for incorrect characters.

[0129] The missing-character pinyin dictionary tree is used to correct errors where one character is missing.

[0130] The disordered pinyin dictionary is used to correct entity words whose character length is greater than or equal to a preset character length. Each preset word to be processed is segmented, and a disordered error text set is generated from the segmented data. A preset N-gram statistical language model is used to score each disordered error text in the disordered error text set. Scores greater than a preset score threshold are identified from the scores obtained. The disordered error texts corresponding to the identified scores are used as target disordered error text sets. The disordered pinyin dictionary is generated based on the target disordered error text sets corresponding to each word to be processed.

[0131] The misspelled pinyin dictionary, multi-character pinyin dictionary, short-character pinyin dictionary, and disordered pinyin dictionary are all pinyin dictionary trees constructed based on a preset vocabulary list. The preset vocabulary list can be a domain vocabulary list, or it can be a set of words that are intended to be error-free.

[0132] Because ASR speech recognition and pinyin conversion (pinyin input method) have some built-in text repair capabilities, a multi-character pinyin dictionary is constructed to correct errors of adding one extra character. For each word to be processed in the vocabulary, a placeholder for the incorrect character is inserted at one position of the word to be processed, resulting in a multi-character erroneous version of the text. A multi-character pinyin dictionary is generated based on the various multi-character erroneous versions of the text corresponding to the vocabulary. The pinyin corresponding to the placeholder for the incorrect character is a wildcard pinyin symbol. The entity nodes of the multi-character pinyin dictionary store the words to be processed, and the edges corresponding to the entity nodes of the multi-character pinyin dictionary represent the pinyin sequence corresponding to the multi-character erroneous version of the text. For example, if the word to be processed is v... i =c1c2c3, then v i The corresponding set of multi-word error versions d(v) i = {hc1c2c3,c1hc2c3,c1c2hc3,c1c2c3h}, where h is a placeholder for incorrect characters. It is understood that, in another embodiment of this application, the multi-character pinyin dictionary is used to correct errors in one or more characters.

[0133] For each word to be processed in the vocabulary, a character at one position of the word is deleted, resulting in a text with missing characters. A trie (prefix tree) is then generated based on these missing-character versions of the text. The entity nodes of this trie store the words to be processed, and the edges corresponding to these entity nodes represent the pinyin sequences of the missing-character versions of the text. For example, if the word to be processed is "v",... i =c1c2c3, then v i The corresponding set of multi-word error versions s(v i = {c2c3, c1c3, c1c2}. It is understood that, in another embodiment of this application, the missing-character pinyin dictionary is used to correct errors involving missing one or more characters.

[0134] Since disordered text is generally formed by arranging the word components in a longer domain term as units, the preset character length is set to 5.

[0135] The entity nodes of the disordered pinyin dictionary tree store the words to be processed. For example, if the word to be processed is vi = c1c2c3c4c5, and vi is segmented into c1c2 / c3 / c4c5, then all disordered erroneous texts of vi are all permutations based on the above segmentation s(vi) = {c1c2c4c5c3,c3c1c2c4c5,c4c5c1c2c3,c4c5c3c1c2,c3c4c5c1c2}.

[0136] Reference Figure 2 This application also proposes a text correction device, the device comprising:

[0137] The text acquisition module 100 is used to acquire the initial text input by the user based on a preset ASR speech recognition technology or input method technology, and to convert the initial text into pinyin to obtain an initial pinyin sequence.

[0138] The text correction module 200 is used to perform pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration, so as to obtain the corrected text. The pinyin dictionary in the pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for multiple characters, a pinyin dictionary for missing characters, and a pinyin dictionary for disordered characters.

[0139] This embodiment performs pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration, resulting in corrected text. The pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for characters with many characters, a pinyin dictionary for characters with few characters, and a pinyin dictionary for text in random order. This achieves error correction for text with similar-sounding characters, characters with few characters, characters with many characters, and text in random order, improving the accuracy and comprehensiveness of error correction. Errors in speech-to-text conversion due to user accents and / or potential environmental background noise are mainly caused by inaccurate pinyin pronunciation recognition. To address this issue, the pinyin dictionary configuration in this application uses only pinyin dictionary trees to specifically correct errors caused by inaccurate pinyin pronunciation recognition, further improving the accuracy and comprehensiveness of error correction.

[0140] Reference Figure 3 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3As shown, this computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data such as text correction methods. The network interface is used for communication with external terminals via a network connection. When the processor executes the computer program, it implements a text correction method. The text correction method includes: acquiring initial text input by the user based on preset ASR speech recognition technology or input method technology; converting the initial text into pinyin to obtain an initial pinyin sequence; and performing pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration to obtain corrected text. The pinyin dictionary in the pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for multiple characters, a pinyin dictionary for characters with fewer characters, and a pinyin dictionary for disordered characters.

[0141] This embodiment performs pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration, resulting in corrected text. The pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for characters with many characters, a pinyin dictionary for characters with few characters, and a pinyin dictionary for text in random order. This achieves error correction for text with similar-sounding characters, characters with few characters, characters with many characters, and text in random order, improving the accuracy and comprehensiveness of error correction. Errors in speech-to-text conversion due to user accents and / or potential environmental background noise are mainly caused by inaccurate pinyin pronunciation recognition. To address this issue, the pinyin dictionary configuration in this application uses only pinyin dictionary trees to specifically correct errors caused by inaccurate pinyin pronunciation recognition, further improving the accuracy and comprehensiveness of error correction.

[0142] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements a text correction method, including the steps of: obtaining initial text input by a user based on a preset ASR speech recognition technology or input method technology; performing pinyin conversion on the initial text to obtain an initial pinyin sequence; and performing pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in a preset pinyin dictionary configuration to obtain corrected text. The pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for multiple characters, a pinyin dictionary for fewer characters, and a pinyin dictionary for disordered characters.

[0143] The text correction method described above performs pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in a preset pinyin dictionary configuration, resulting in corrected text. The pinyin dictionary configuration includes: a pinyin dictionary for misspelled characters, a pinyin dictionary for characters with many characters, a pinyin dictionary for characters with few characters, and a pinyin dictionary for text in a disordered order. This achieves correction of text with similar pronunciations, characters with few characters, characters with many characters, and text in a disordered order, improving the accuracy and comprehensiveness of the correction. Errors in speech-to-text conversion caused by user accents and / or potential environmental background noise are mainly due to inaccurate pinyin pronunciation recognition. To address this issue, the pinyin dictionary configuration in this application uses only pinyin dictionary trees to specifically correct errors caused by inaccurate pinyin pronunciation recognition, further improving the accuracy and comprehensiveness of the correction.

[0144] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0146] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A text error correction method, characterized in that, The method includes: The initial text input by the user is obtained based on the preset ASR speech recognition technology or input method technology. The initial text is then converted into pinyin to obtain an initial pinyin sequence. The initial pinyin sequence needs to remove tones, retroflex consonants and nasal consonants. According to the pinyin dictionary tree configuration, the initial text is converted into pinyin and text is corrected based on pinyin to obtain the corrected text. The pinyin dictionary tree configuration includes: a pinyin dictionary tree for misspelled characters, a pinyin dictionary tree for multiple characters, a pinyin dictionary tree for missing characters, and a pinyin dictionary tree for disordered characters. The disordered pinyin dictionary tree is generated as follows: each preset word to be processed is segmented, and a disordered error text set is generated from the segmented data. A preset N-gram statistical language model is used to score each disordered error text in the disordered error text set. From the scores obtained, the scores greater than a preset score threshold are found. The disordered error texts corresponding to the found scores are used as the target disordered error text set. The disordered pinyin dictionary tree is generated based on the target disordered error text sets corresponding to each word to be processed. The steps of performing pinyin conversion and pinyin-based text correction on the initial text include: sequentially performing correction through the misspelled pinyin dictionary, the multi-character pinyin dictionary, the missing-character pinyin dictionary, and the disordered pinyin dictionary; updating the pinyin sequence corresponding to the current text after each correction is completed; and performing the next correction based on the updated pinyin sequence.

2. The text correction method of claim 1, wherein, The step of performing pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration to obtain the corrected text includes: The initial text is converted to pinyin to obtain an initial pinyin sequence; Based on the misspelled pinyin dictionary tree and the initial pinyin sequence, the initial text is corrected to obtain the first text and the first pinyin sequence; Based on the multi-character pinyin dictionary and the first pinyin sequence, the first text is corrected to obtain the second text and the second pinyin sequence; Based on the missing-character pinyin dictionary and the second pinyin sequence, the second text is corrected to obtain the third text and the third pinyin sequence; Based on the disordered pinyin dictionary and the third pinyin sequence, the third text is corrected to obtain the corrected text.

3. The text correction method of claim 2, wherein, The step of correcting the initial text based on the misspelled pinyin dictionary and the initial pinyin sequence to obtain the first text and the first pinyin sequence includes: The initial pinyin sequence is input into the misspelled pinyin dictionary tree for entity node matching to obtain the first result; If the first result is a failure, then the initial text is used as the first text, and the initial pinyin sequence is used as the first pinyin sequence; If the first result is successful, then each entity node matched from the misspelled pinyin dictionary tree will be used as the node set to be evaluated. The initial text will be corrected according to the preset N-gram statistical language model and the node set to be evaluated to obtain the first text, and the pinyin sequence corresponding to the first text will be used as the first pinyin sequence.

4. The text correction method of claim 3, wherein, The step of correcting the initial text according to a preset N-gram statistical language model and the set of nodes to be evaluated to obtain the first text includes: Take any entity node in the set of nodes to be evaluated as a candidate node; Determine whether the substring corresponding to the candidate node in the initial text exists in the entity word set of the candidate node. If it exists, repeat the step of taking any entity node in the node set to be evaluated as a candidate node until the nodes in the node set to be evaluated are obtained. Otherwise, determine the substring as the problem string. Based on the preset string extraction method and the question string, a string is extracted from the initial text to obtain the string to be evaluated; The N-gram statistical language model is used to score the string to be evaluated, and the first misspelling score is obtained. Based on each entity word in the entity word set corresponding to the string to be evaluated, the string to be evaluated is replaced to obtain a candidate string; The N-gram statistical language model is used to score each candidate string to obtain a second misspelling score; Determine whether there exists a second misspelled word score that is greater than the product of the first misspelled word score and a preset score coefficient; If it exists, the entity word corresponding to the maximum value among the second misspelling scores that are greater than the score product value will replace the initial text; Repeat the step of selecting any entity node in the set of nodes to be evaluated as a candidate node until all nodes in the set of nodes to be evaluated are obtained. The initial text is used as the first text.

5. The text correction method of claim 4, wherein, The step of extracting a string from the initial text according to a preset string extraction method and the question string to obtain the string to be evaluated includes: If the question string is at the beginning of the initial text, then the preset beginning string will be used as the forward extracted character substring; If the question string is not at the beginning of the initial text, then extract an additional first number of characters as the forward extracted character substring; If the question string is located at the end of the initial text, then the preset ending string will be used as the substring to be extracted later. If the question string is not located at the end of the initial text, then extract a second number of characters forward as the backward extracted character substring; The forward-extracted character substring, the question string, and the backward-extracted character substring are concatenated sequentially to obtain the string to be evaluated.

6. The text correction method of claim 1, wherein, The pinyin sequence of the pinyin dictionary in the input pinyin dictionary configuration needs to have tones, retroflex consonants, and nasal consonants removed.

7. The text correction method of claim 1, wherein, The misspelling dictionary tree is used to correct characters that are similar in sound and form; The multi-character pinyin dictionary tree is used to correct errors of having an extra character. It is a dictionary tree generated based on preset wildcard pinyin symbols and preset placeholders for incorrect characters. The missing-character pinyin dictionary tree is used to correct errors where one character is missing. The disordered pinyin dictionary is used to correct entity words whose character length is greater than or equal to a preset character length.

8. A text correction device, characterized in that, The device includes: The text acquisition module is used to acquire the initial text input by the user based on preset ASR speech recognition technology or input method technology, and to convert the initial text into pinyin to obtain an initial pinyin sequence; The text correction module is used to perform pinyin conversion and pinyin-based text correction on the initial text according to each pinyin dictionary in the preset pinyin dictionary configuration, to obtain the corrected text. The pinyin dictionary configuration includes: a misspelled pinyin dictionary, a multi-character pinyin dictionary, a short-character pinyin dictionary, and a disordered pinyin dictionary. The disordered pinyin dictionary is generated as follows: each preset word to be processed is segmented; the segmented data is used to generate a disordered error text set; a preset N-gram statistical language model is used to score each disordered error text in the disordered error text set; scores greater than a preset score threshold are found from the scores; the disordered error texts corresponding to the found scores are used as target disordered error text sets; and the disordered pinyin dictionary is generated based on the target disordered error text sets corresponding to each word to be processed. The steps of performing pinyin conversion and pinyin-based text correction on the initial text include: sequentially performing correction through the misspelled pinyin dictionary, the multi-character pinyin dictionary, the missing-character pinyin dictionary, and the disordered pinyin dictionary; updating the pinyin sequence corresponding to the current text after each correction is completed; and performing the next correction based on the updated pinyin sequence. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images