Error correction method, device, equipment and readable storage medium

Abstract

The present invention relates to the field of artificial intelligence technology, and discloses an error correction method, comprising the following steps: obtaining text data to be corrected, performing word segmentation processing on the text data to obtain characters to be converted, converting the characters to be converted into a pinyin sequence, matching the correct characters corresponding to the pinyin sequence from an error correction dictionary using a preset multi-mode matching algorithm according to the pinyin sequence, and replacing the corresponding characters to be converted in the text data with the correct characters. The present invention also discloses an error correction device, equipment, and computer-readable storage medium, so as to realize rapid error correction of erroneous characters in text, and in subsequent use, only new text needs to be added to the error correction dictionary, so the update cost is low and maintenance is convenient.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an error correction method, device, equipment and readable storage medium. Background Art

[0002] With the continuous development of the information age, we are receiving information every day and outputting information every day. No matter in which output mode, such as voice input, input method input, etc., there are inevitably erroneous characters. In the existing error correction technology, there is a method of obtaining an error correction model by collecting erroneous characters for model training, and making a mapping dictionary for one-to-one correspondence between erroneous characters and correct characters. However, the above methods are difficult to collect all the erroneous characters that appear in daily life due to the variety of typos. At the same time, in subsequent use, due to the lack of previously collected erroneous characters, the update cost in subsequent use is too high and difficult to maintain. Summary of the invention

[0003] The main purpose of the present invention is to provide an error correction method, device, equipment and readable storage medium, aiming to solve the technical problem of how to effectively perform character error correction.

[0004] To achieve the above object, the present invention provides an error correction method, which comprises the following steps:

[0005] Acquiring text data to be corrected, wherein the text data includes text information input by a user and a preset text template;

[0006] Performing word segmentation processing on the text data to obtain characters to be converted, wherein the characters to be converted are one of a single Chinese character, a phrase, and a product name;

[0007] Convert the characters to be converted into a pinyin sequence;

[0008] According to the pinyin sequence, a preset multi-mode matching algorithm is used to match the correct characters corresponding to the pinyin sequence from an error correction dictionary, wherein the error correction dictionary is a dictionary of correct characters corresponding to characters that often appear in errors in the chat corpus of the chat robot in advance;

[0009] The correct character replaces the corresponding character to be converted in the text data.

[0010] Optionally, the error correction dictionary is obtained in the following manner:

[0011] Collecting chat corpus in the chat robot;

[0012] Extracting characters that are erroneous and corrected in the chat corpus;

[0013] Convert the characters into pinyin features, and establish a mapping relationship between the pinyin features and the characters;

[0014] According to the matching principle of the multi-mode matching algorithm, a search tree is constructed for the pinyin features corresponding to all characters extracted from the chat corpus to obtain the error correction dictionary.

[0015] Optionally, the step of constructing a search tree for the pinyin features corresponding to all characters extracted from the chat corpus according to the matching principle of the multi-mode matching algorithm to obtain the error correction dictionary includes:

[0016] Path-based pinyin features corresponding to all characters to obtain path features;

[0017] Extracting the same part and the different part of the pinyin in the path feature, wherein the same part means that at least two consecutive syllables are the same;

[0018] Create a main trunk based on the same section;

[0019] On the basis of the main trunk, a sub-trunk is created, wherein the sub-trunk is created based on different parts to obtain an error correction dictionary in the shape of the search tree;

[0020] Wherein, a plurality of nodes are arranged on the main trunk and the sub-trunk, and each node corresponds to a letter or a syllable in the pinyin feature.

[0021] Optionally, the step of performing word segmentation processing on the text data to obtain characters to be converted includes:

[0022] Set a variable-length sliding window with an initial length of one;

[0023] The text data is iteratively scanned based on the sliding window to obtain characters to be converted, wherein the characters to be converted are characters selected by the sliding window.

[0024] Optionally, the step of matching the correct characters corresponding to the pinyin sequence from an error correction dictionary using a preset multi-mode matching algorithm according to the pinyin sequence includes:

[0025] Using the multi-mode matching algorithm, querying the error correction dictionary for all characters that contain the pinyin sequence;

[0026] Calculate the length of each character, and select the character with the largest length from all characters to obtain the first error correction character;

[0027] adjusting the sliding window according to the length of the first error correction character, and reacquiring the character to be converted from the text data;

[0028] comparing the similarity between the first error correction character and the reacquired character to be converted;

[0029] If the similarity is greater than a preset value, the first misconnected character is selected as the correct character;

[0030] If the similarity is less than a preset value, the character with the smallest length is selected from all the characters as the correct character.

[0031] Optionally, if the node is a syllable in the pinyin feature, the step of using the multi-mode matching algorithm to query the error correction dictionary for all characters containing the pinyin sequence includes:

[0032] According to the syllable division rule in Pinyin, the Pinyin sequence is subjected to syllable division processing to obtain a syllable sequence, wherein the division processing is to sequentially combine the initial consonants and the finals in the Pinyin sequence into a syllable and then perform segmentation processing;

[0033] According to the syllable sequence, the nodes on the main trunk and the sub-trunk in the error correction dictionary are matched in sequence to obtain corresponding pinyin features;

[0034] All characters of the pinyin sequence are determined according to the pinyin features and the corresponding relationship between the pinyin features and the characters.

[0035] Optionally, the step of matching the syllable sequence with the nodes on the main trunk and the sub-trunk in the error correction dictionary in sequence to obtain the corresponding pinyin features includes:

[0036] Inputting the syllables in the syllable sequence into the model corresponding to the multi-modal matching algorithm in sequence;

[0037] By using the model, the first syllables in the syllable sequence are sequentially matched with the first nodes on all the main trunks in the error correction dictionary;

[0038] If the match fails, the second syllable in the syllable sequence is read and matched with all the main trunks in the error correction dictionary;

[0039] If the match is successful, then matching the second syllable in the syllable sequence with the next node of the main trunk along the successfully matched main trunk, and determining whether the sub-trunk exists in the next node;

[0040] If it exists, then read the third syllable in the syllable sequence and match it with the first child node on the sub-trunk in the next node;

[0041] If the match is successful, the next syllable in the syllable sequence is matched along the sub-trunk that is successfully matched;

[0042] If the matching fails, the jump table preset in the first node is read, and the matching process of the syllable sequence is jumped to continue matching with other main trunks or sub-trunks.

[0043] In addition, to achieve the above object, the present invention further provides an error correction device, the error correction device comprising:

[0044] An acquisition module, used for acquiring text data to be corrected, wherein the text data includes text information input by a user and a preset text template;

[0045] A word segmentation module, used for performing word segmentation processing on the text data to obtain characters to be converted, wherein the characters to be converted are one of a single Chinese character, a phrase and a product name;

[0046] A conversion module, used for converting the characters to be converted into a pinyin sequence;

[0047] A matching module, for matching the correct characters corresponding to the pinyin sequence from an error correction dictionary using a preset multi-mode matching algorithm according to the pinyin sequence, wherein the error correction dictionary is a dictionary of correct characters corresponding to characters that often appear in errors in the chat corpus of the chat robot, which is previously observed;

[0048] The replacement module is used to replace the characters to be converted in the text data with the correct characters.

[0049] Optionally, the error correction device further includes a dictionary construction module, the dictionary construction module includes a collection unit, an extraction unit, a pinyin unit, and a construction unit, wherein:

[0050] The collecting unit is used to collect the chat corpus in the chat robot;

[0051] The extraction unit is used to extract characters in the chat corpus that are erroneous and corrected;

[0052] The pinyin unit is used to convert the characters into pinyin features and establish a mapping relationship between the pinyin features and the characters;

[0053] The construction unit is used to construct a search tree for the pinyin features corresponding to all characters extracted from the chat corpus according to the matching principle of the multi-mode matching algorithm to obtain the error correction dictionary.

[0054] Optionally, the construction unit is used to path-ize the pinyin features corresponding to all characters to obtain path features; extract the same part and the different part of the pinyin in the path features, wherein the same means that there are at least two consecutive syllables that are the same; create a main trunk based on the same part; based on the main trunk, create a sub-trunk, wherein the sub-trunk is created based on the different parts to obtain the error correction dictionary in the shape of the search tree; wherein the main trunk and the sub-trunk are both provided with a number of nodes, each node corresponding to a letter or a syllable in the pinyin feature.

[0055] Optionally, the word segmentation module includes a setting unit and a scanning unit, wherein:

[0056] The setting unit is used to set a sliding window of variable length with an initial length of one;

[0057] The scanning unit is used to perform iterative scanning processing on the text data based on the sliding window to obtain characters to be converted, wherein the characters to be converted are characters selected by the sliding window.

[0058] Optionally, the matching module includes a query unit, a length calculation unit, an adjustment unit and a comparison unit, wherein:

[0059] The query unit is used to query all characters in the error correction dictionary that contain the pinyin sequence using the multi-mode matching algorithm;

[0060] The length calculation unit is used to calculate the length of each character, and select the character with the largest length from all characters to obtain the first error correction character;

[0061] The adjusting unit is used to adjust the sliding window according to the length of the first error correction character, and reacquire the character to be converted from the text data;

[0062] The comparison unit is used to compare the similarity between the first error correction character and the re-acquired character to be converted;

[0063] If the similarity is greater than a preset value, the first misconnected character is selected as the correct character;

[0064] If the similarity is less than a preset value, the character with the smallest length is selected from all the characters as the correct character.

[0065] Optionally, if the node is a syllable in the pinyin feature, the query unit is used to perform syllable division processing on the pinyin sequence according to the syllable division rule in pinyin to obtain a syllable sequence, and the division processing is to sequentially combine the initial consonants and finals in the pinyin sequence into a syllable and then perform segmentation processing; according to the syllable sequence, match it with the nodes on the main trunk and sub-trunk in the error correction dictionary in turn to obtain the corresponding pinyin feature; determine the correct character based on the pinyin feature and the correspondence between the pinyin feature and the character.

[0066] Optionally, the query unit is used to input the syllables in the syllable sequence into the model corresponding to the multi-mode matching algorithm in sequence; through the model, the first syllable in the syllable sequence is sorted and matched with the first nodes on all the main trunks in the error correction dictionary in sequence; if the match fails, the second syllable in the syllable sequence is read and continued to match with all the main trunks in the error correction dictionary; if the match is successful, the second syllable in the syllable sequence is matched with the next node of the main trunk along the main trunk with successful matching, and it is determined whether the sub-trunk exists in the next node; if so, the third syllable in the syllable sequence is read and matched with the first sub-node on the sub-trunk in the next node; if the match is successful, the next syllable in the syllable sequence is matched along the sub-trunk with successful matching; if the match fails, the jump table pre-set in the first node is read, and the matching process of the syllable sequence is jumped to continue matching with other main trunks or sub-trunks.

[0067] Furthermore, to achieve the above-mentioned purpose, the present invention also provides an error correction device, which includes a memory, a processor, and an error correction program stored in the memory and executable on the processor, and the error correction program, when executed by the processor, implements the steps of the error correction method as described in any one of the above items.

[0068] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, on which an error correction program is stored, and when the error correction program is executed by a processor, the steps of the error correction method as described in any one of the above items are implemented.

[0069] The embodiment of the present invention proposes a method of observing character errors in the chat corpus of the chat robot in advance, summarizing the error trends of users, constructing an error correction dictionary based on the correct characters corresponding to the observed erroneous characters and the pinyin sequences corresponding to the correct characters, inputting the text data to be corrected into the error correction dictionary for matching based on a preset multi-mode matching algorithm, performing replacement correction processing on the successfully matched characters, and completing the correction of the erroneous characters in the text data to be corrected. The error correction dictionary constructed by this method can simply and quickly realize the correction of text data with erroneous characters through a preset multi-mode matching algorithm, with a low error rate. At the same time, when a new erroneous character appears, it is only necessary to establish a mapping relationship between the new correct character and the corresponding pinyin sequence in the error correction dictionary, which is convenient for maintenance and can reduce labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of the structure of the operating environment of the error correction device involved in the embodiment of the present invention;

[0071] Figure 2 A schematic diagram of a flow chart of a first embodiment of an error correction method provided by the present invention;

[0072] Figure 3 A schematic diagram of constructing a search tree provided by an embodiment of the present invention;

[0073] Figure 4 A schematic diagram of a process flow of a pinyin sequence matching process provided by an embodiment of the present invention;

[0074] Figure 5 It is a flow chart of the second embodiment of the error correction method of the present invention;

[0075] Figure 6 Another schematic diagram of constructing a search tree provided by an embodiment of the present invention;

[0076] Figure 7 A schematic diagram of the functional modules of the error correction device provided by the present invention.

[0077] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0078] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0079] The present invention provides an error correction device.

[0080] Reference Figure 1 , Figure 1 It is a schematic diagram of the structure of the operating environment of the error correction device involved in the embodiment of the present invention.

[0081] like Figure 1 As shown, the error correction device includes: a processor 101, such as a CPU, a communication bus 102, a user interface 103, a network interface 104, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 103 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the network interface 104 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 105 can be a high-speed RAM memory, or it can be a stable memory (non-volatile memory), such as a disk memory. The memory 105 can also be a storage device independent of the aforementioned processor 101.

[0082] Those skilled in the art will understand that Figure 1 The hardware structure of the error correction device shown in the figure does not constitute a limitation on the error correction device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0083] like Figure 1 As shown, the memory 105 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module and an error correction program. The operating system is a program that manages and controls error correction devices and software resources, and supports the operation of the error correction program and other software and / or programs.

[0084] exist Figure 1 In the hardware structure of the error correction device shown, the network interface 104 is mainly used to access the network; the user interface 103 is mainly used to detect confirmation instructions, etc., and the processor 101 can be used to call the error correction program stored in the memory 105 and execute the operations of the various embodiments of the following error correction method.

[0085] Based on the above-mentioned hardware structure of the error correction device, various embodiments of the error correction method of the present invention are proposed.

[0086] Reference Figure 2 , Figure 2 The flowchart of an embodiment of the error correction method of the present invention is shown in FIG. In this embodiment, the error correction method includes the following steps:

[0087] Step S210, obtaining text data to be corrected, wherein the text data includes text information input by a user and a preset text template;

[0088] In this step, the text information to be corrected includes the text information input by the user through speech recognition and the input content input by the user into the input method system through the input method. The input method system can convert the input content into word candidates of the input text according to the current input method of the user for the user to select. The user can input text information through various input means, such as a keyboard, a touchpad, a mouse, etc. At the same time, the user can also select any input method to input text information, such as pinyin, Wubi, strokes, handwriting, etc., which are not limited in the present invention. The text template to be corrected can be collected from data sources such as books, news, web pages, etc., or can be pre-stored in a preset database, and is not the existing text data input by the user in real time. For example, the text template is obtained by performing character recognition on the text in a book, by performing speech recognition on the broadcast news, and by performing a text scraping operation on the web page.

[0089] Step S220: Perform word segmentation on the text data to obtain characters to be converted, where the characters to be converted are one of a single Chinese character, a phrase, and a product name.

[0090] In this step, the characters to be converted are the characters that need to be converted into pinyin sequences later. There may be two situations in the characters to be converted. For example, there may be incorrect characters or there may be no incorrect characters. However, regardless of whether there are incorrect characters or not, the characters to be converted will be converted into pinyin in the subsequent steps.

[0091] In this step, since various meaningless character strings will appear after the text data is divided. For example, for the text data "I have caught a cold these days. Can I buy a peace charm?", there will be a division situation such as "le ke" and other meaningless characters. For these meaningless characters, since these characters are not the target characters we need to correct, these characters need to be filtered out. For example, character strings containing prepositions are filtered out. In this embodiment, there are also various filtering methods, which are not limited in this embodiment. It should be noted that for the symbols, numeric characters, and alphabetic characters in the text data, such as the question mark "?", the comma ",", the numeric character "500", and the letter "a", for the symbols in the text characters, obviously they are not the objects we need to correct. For the numeric characters and alphabetic characters, since it is not known whether there are errors in the numeric characters and alphabetic characters in the text data. For example, for the text data "I earned 500 yuan today.", it is obviously difficult to judge whether "500" is a correct number. Therefore, the symbols, numeric characters, and alphabetic characters in the text data need to be filtered out before word segmentation.

[0092] Step S230: Convert the characters to be converted into pinyin sequences.

[0093] In this step, the characters to be converted can be converted into a pinyin sequence according to the ASCII code of the Chinese characters. Because in the computer system, Chinese characters are represented by ASCII codes, it is only necessary to use the correspondence between each pinyin and each ASCII code that is already in the computer system or established by the user to realize the conversion of the characters to be converted into a pinyin sequence. If the characters to be converted contain polyphones, multiple pinyins of the polyphones can be listed and the correct pinyin selected by the user can be received. Alternatively, the characters to be converted can be converted into a pinyin sequence according to the Unicode value of the Chinese characters. The specific steps are as follows:

[0094] (1) Establish a pinyin-number comparison table, number all pinyins and add the numbers corresponding to all pinyins to the pinyin-number comparison table. The pinyins of all Chinese characters do not exceed 512, and the pinyins can be numbered with two bytes. Each pinyin corresponds to a number.

[0095] (2) Establish a Unicode value-pinyin number comparison table, and add the pinyin number corresponding to the Chinese character to the Unicode value-pinyin number comparison table according to the Unicode value of the Chinese character.

[0096] (3) Read the Chinese characters to be converted one by one in the characters to be converted, determine the Unicode value of the Chinese characters to be converted, obtain the number of the pinyin corresponding to the Chinese characters to be converted from the Unicode value-pinyin number comparison table according to the Unicode value of the Chinese characters to be converted, and obtain the pinyin corresponding to the Chinese characters to be converted from the pinyin-number comparison table according to the number of the pinyin corresponding to the Chinese characters to be converted, thereby converting each Chinese character to be corrected into pinyin.

[0097] If the character to be converted contains a polyphone, the numbers of the multiple pinyins corresponding to the polyphone can be added to the Unicode value-pinyin number comparison table according to the Unicode value of the polyphone in the above step (2), the Unicode value of the polyphone is determined in the above step (3), the numbers of the multiple pinyins corresponding to the polyphone are obtained from the Unicode value-pinyin number comparison table according to the Unicode value of the polyphone, and the multiple pinyins corresponding to the polyphone are obtained from the pinyin-number comparison table according to the numbers of the multiple pinyins corresponding to the polyphone. The correct pinyin selected by the user from the multiple pinyins can be received, and the pinyin selected by the user is used as the correct pinyin of the polyphone in the character to be converted.

[0098] Step S240, according to the pinyin sequence, using a preset multi-mode matching algorithm to match the correct characters corresponding to the pinyin sequence from an error correction dictionary, wherein the error correction dictionary is a dictionary of correct characters corresponding to characters that often appear in errors in the chat corpus of the chat robot in advance;

[0099] In this step, the preset multi-mode matching algorithm is constructed based on the pinyin corresponding to the correct characters collected in advance. Whether the character to be converted is correct or not, when the pinyin sequence of the character to be converted matches the pinyin sequence of the correct character in the error correction dictionary, the successfully matched pinyin sequence is converted into the correct character, so that the correct character corresponding to the pinyin sequence can be matched from the error correction dictionary according to the preset multi-mode matching algorithm.

[0100] In real life, character error correction is divided into two cases. One is to correct words. Whether in daily use or in the dictionary, there is no word form of the character to be converted. For example, the character to be converted "wú néng wéi lì" does not exist as such a string in daily use or in the dictionary, which is obviously an incorrect character and should be corrected to "wú néng wéi lì". In this case, the existing word dictionary needs to be used. The other is to correct the position and context of the word in the sentence. For example, in the company's insurance products, there are product names such as "Fubao Insurance" and "Ping An Fu". In daily use, users may easily input them as "Fubao Baby" and "Ping An Symbol" through the input method or the voice recognition system. Since the characters to be converted in this case do not need to be corrected in other sentences, in normal applications, we need to classify the fields where the error correction dictionary needs to be applied. For example, for certain company products, a special product dictionary for company product names can be constructed, and for incorrect characters in daily communication, a common error dictionary for daily communication can be constructed.

[0101] In this step, there may be a situation where the pinyin sequences of two or more correct characters in the error correction dictionary are the same. The pinyin sequences in this situation need to be marked. When the pinyin sequence of the character to be converted is successfully paired with the pinyin sequence in this situation, the two or more correct characters corresponding to the pinyin sequence are sent to the business personnel for manual error correction selection.

[0102] Step S250: Replace the corresponding character to be converted in the text data with the correct character.

[0103] In this step, replacing the character to be converted in the text data mainly involves finding the position of the character to be converted in the text data, deleting the character to be converted, and adding the corrected correct character in the original position. For example, for the text data "I've had a cold these days. Can I apply for Ping An Symbol?", the position of the character to be converted is found to be after the character "bao" and before the character "ma". The character to be converted "Ping An Symbol" is deleted, and the correct character is placed after the character "bao" and before the character "ma" to achieve the effect of replacing the character to be converted with the correct character, thereby completing the error correction of the text data.

[0104] Further, in this embodiment, for the error correction dictionary, it is specifically constructed from the chat corpus of users. The specific construction process is as follows:

[0105] Collect the chat corpus in the chatbot;

[0106] Extract the characters that appear wrongly and are corrected in the chat corpus;

[0107] Convert the characters into pinyin features and establish the mapping relationship between the pinyin features and the characters;

[0108] According to the matching principle of the multi-mode matching algorithm, construct a search tree for the pinyin features corresponding to the corrected characters extracted from the chat corpus to obtain the error correction dictionary.

[0109] In this step, for the extraction of the characters that appear wrongly and are corrected in the chat corpus, mainly collect the chat corpus when observing the chat of the chatbot, extract the wrong characters in the chat corpus, and convert them into correct characters, so as to achieve the extraction of the collection of correct characters. When the correct characters obtained after correction are converted into pinyin features, for the pinyin features, there are two situations, one is the syllable, and the other is the pinyin character. The error correction dictionary can match according to the different situations of the pinyin features according to actual needs. In this embodiment, the way of constructing the search tree is based on the syllable as the pinyin feature. For example, for the pinyin sequence "fubaobao" of the character "福保保", if its pinyin feature is the syllable, the pinyin feature is "fu bao bao". Taking the company's product name as the error correction object as an example, assume that the set of characters that appear wrongly and are corrected extracted from the chat corpus is {平安福, 平安福保, 平安驾驶优保险, 驾驶易保保, 康寿安易驾保, 康寿安保保}, then the converted pinyin sequences are {pinganfu, pinganfubao, pinganjiashiyoubaoxian, jiashiyibaobao, kangshouanyijiabao, kangshouanbaobao}. When the pinyin feature that constitutes the error correction dictionary is the syllable, establish a search tree according to the set of characters that appear wrongly and have been corrected, such as Figure 3 .

[0110] Further, in the embodiment of the present invention, for the construction of the search tree for the pinyin features corresponding to all the characters extracted from the chat corpus according to the matching principle of the multi-mode matching algorithm to obtain the error correction dictionary, its specific implementation can be:

[0111] Pathify the pinyin features corresponding to all characters to obtain path features;

[0112] Extracting the same part and the different part of the pinyin in the path feature, wherein the same part means that at least two consecutive pinyin features are the same;

[0113] Create a main trunk based on the same section;

[0114] On the basis of the main trunk, a sub-trunk is created, wherein the sub-trunk is created based on different parts to obtain an error correction dictionary in the shape of the search tree;

[0115] Wherein, a plurality of nodes are arranged on the main trunk and the sub-trunk, and each node corresponds to a letter or a syllable in the pinyin feature.

[0116] In this embodiment, different path features can be obtained by pathing the pinyin features. For example, for the pinyin sequence "pinganfubao", the corresponding path is Figure 3 The leftmost path in the tree; the same part of the path features is used as the main trunk. For example, for the pinyin sequences "pinganfubao" and "pinganyjiashiyoubaoxian", the same part is "pingan", corresponding to Figure 3 The unbranched part of the left path of , constructs sub-trunks with different parts of the path features. For example, for the pinyin sequences "pinganfubao" and "pinganjiashiyoubaoxian", the different parts are "fubao" and "jiashiyoubaoxian", corresponding to the different branches of the left path. There are several nodes on the main trunk and sub-trunk, such as Figure 3 The circular part of each node contains a pinyin feature. It should be noted that for the starting node of the search tree, we set it to not contain any pinyin features, that is, its value is null.

[0117] Furthermore, for step S220, the specific implementation steps may be:

[0118] Set a variable-length sliding window with an initial length of one;

[0119] The text data is iteratively scanned based on the sliding window to obtain characters to be converted, wherein the characters to be converted are characters selected by the sliding window.

[0120] In this step, the text data is segmented, mainly by means of a preset sliding window, a sliding scan is performed on the text data and the characters contained in the sliding window are extracted. The preset sliding window has a preset size of 1 character, and a step length of 1 character is used to slide the text data from left to right. Each time the sliding window is slid, the characters within the range of the sliding window are obtained to obtain the characters to be converted.

[0121] In the present embodiment, it should be noted that, since the sliding window is a variable-length sliding window, as the window is iteratively scanned, the length of the window will continue to increase. When the length of the window is greater than the length of the erroneous and corrected characters that constitute the error correction dictionary, the scan ends. For example, in the present embodiment, the application scenario is the company's product name, the constructed error correction dictionary is a company-specific name dictionary, and the correct characters are the company's product names. When the length of the scanned window is greater than the longest product name in the company's product names, it is obvious that the characters to be converted scanned by the window are longer than the longest product name characters in the company's product names. At this time, the error correction dictionary has corrected the erroneous characters in the text data, and further correction will waste time and cost.

[0122] Furthermore, for step S240, the specific implementation steps may be:

[0123] Using the multi-mode matching algorithm, querying the error correction dictionary for characters corresponding to all pinyin sequences beginning with the pinyin sequence;

[0124] Calculating the length of the characters, and selecting the character with the largest length from the characters to obtain a first error correction character;

[0125] adjusting the sliding window according to the length of the first error correction character, and reacquiring the character to be converted from the text data;

[0126] comparing the similarity between the first error correction character and the reacquired character to be converted;

[0127] If the similarity is greater than a preset value, the first error correction character is selected as the correct character;

[0128] If the similarity is less than a preset value, the character with the smallest length is selected from all the characters as the correct character.

[0129] In this embodiment, the query of the characters corresponding to all pinyin sequences starting with the pinyin sequence in the error correction dictionary can search for correct characters of different lengths for the same starting character in the error correction dictionary. For example, in company products, the company product names starting with "Ping An" include "Ping An Fu" and "Ping An Fu Bao Bao". Assuming the text data is "I have a cold these few days. Can I buy Ping An Fu Bao Bao insurance?" If the short correction is taken, the "Ping An Fu" and "Bao Bao" in the text data may be corrected to "Ping An Fu" and "Bao Bao". If the long correction is taken, the long character "Ping An Fu Bao Bao" is directly corrected to "Ping An Fu Bao Bao". Therefore, when multiple candidate characters to be converted with the same starting character are matched, it is necessary to calculate the lengths of all candidate characters to be converted, and select the character with the longest length as the first error correction character.

[0130] Furthermore, if the node is a syllable in the pinyin feature, the specific implementation steps of using the multi-mode matching algorithm to query the error correction dictionary for all characters containing the pinyin sequence include:

[0131] According to the syllable division rule in Pinyin, the Pinyin sequence is subjected to syllable division processing to obtain a syllable sequence, wherein the division processing is to sequentially combine the initial consonants and the finals in the Pinyin sequence into a syllable and then perform segmentation processing;

[0132] According to the syllable sequence, the nodes on the main trunk and the sub-trunk in the error correction dictionary are matched in sequence to obtain corresponding pinyin features;

[0133] According to the pinyin features and the corresponding relationship between the pinyin features and the characters, all characters containing the pinyin sequence are determined.

[0134] Furthermore, the syllable sequence is matched with the nodes on the main trunk and the sub-trunk in the error correction dictionary in turn to obtain the corresponding pinyin features, which can be specifically implemented as follows:

[0135] Inputting the syllables in the syllable sequence into the model corresponding to the multi-modal matching algorithm in sequence;

[0136] By using the model, the first syllables in the syllable sequence are sequentially matched with the first nodes on all the main trunks in the error correction dictionary;

[0137] If the match fails, the second syllable in the syllable sequence is read and matched with all the main trunks in the error correction dictionary;

[0138] If the match is successful, then matching the second syllable in the syllable sequence with the next node of the main trunk along the successfully matched main trunk, and determining whether the sub-trunk exists in the next node;

[0139] If it exists, then read the third syllable in the syllable sequence and match it with the first child node on the sub-trunk in the next node;

[0140] If the match is successful, the next syllable in the syllable sequence is matched along the sub-trunk that has been matched successfully;

[0141] If the matching fails, the jump table preset in the first node is read, and the matching process of the syllable sequence is jumped to continue matching with other main trunks or sub-trunks.

[0142] Assuming that the pinyin sequence corresponding to the character to be converted is "pinganjiashiyibaobao", the matching process is as follows: Figure 4 As shown in the dotted line, the first syllable input is matched with "ping" and the first node of all main trunks, and the matched main trunk is the left main trunk, and the matching main trunk is matched downward. After the syllable "an" is matched, there are two sub-trunks. At this time, the input syllable is matched with the first node on the two sub-trunks, and the matched sub-trunk is the right sub-trunk. When the downward matching fails, the jump table is accessed. The jump table contains the node position where the current syllable needs to jump to another node for downward matching after the syllable fails to match the node. For example, the syllable "shi" in the pinyin series is successfully matched and continues to match downward. If it fails, the next syllable "yi" does not match the next node "you", then the syllable "shi" is jumped to the middle main tree The "shi" node on the stem continues to match downward. It should be noted that in the path, the node where the last character of each pinyin sequence is located is set as the terminal node. For example, for the pinyin sequence "fubaobao", when the pinyin feature is a pinyin character, the terminal node is the node where "o" is located, and when the pinyin feature is a syllable, the terminal node is the node where "bao" is located; when the terminal node is encountered during the matching process, it means that the correct character with the terminal node as the path exists in the pinyin sequence. For example, in the above matching process, the terminal node "bao" in "jiashiyibaobao" is matched, which means that the character "jiashiyibaobao" exists in the pinyin sequence "pinganjiashiyibaobao".

[0143] The present invention can construct a search tree for different pinyin features. For the construction of a search tree with pinyin characters in this embodiment, the specific implementation process is as follows: Figure 5 As shown:

[0144] Step S310, converting the pinyin characters corresponding to the correct characters into paths to obtain pinyin paths;

[0145] In this embodiment, taking the company's product name as an example, assuming that the correct characters are {Fubaobao, Pinganfu, Pinganyijiabao, Kangshoubao, Kanganfu}, the corresponding pinyin characters are {fubaobao, pinganfu, pinganyijiabao, kangshoubao, kanganfu}.

[0146] Step S320, extracting the same part and the different part of the pinyin in the pinyin path, wherein the same part is that at least two consecutive pinyin paths are the same;

[0147] In this embodiment, for the same part and the different part in the pinyin path, for example, the same part of the pinyin paths "pinganfu" and "pinganyijiabao" is "pingan", and the different parts are "fu" and "yijiabao".

[0148] Step S330, creating a main trunk based on the same part;

[0149] Step S340, based on the main trunk, creating a sub-trunk, wherein the sub-trunk is created based on different parts to obtain an error correction dictionary in the shape of the search tree;

[0150] Wherein, a plurality of nodes are arranged on the main trunk and the sub-trunk, and each node corresponds to a letter or a syllable in the pinyin feature.

[0151] In this embodiment, the same part is used as the main trunk of the search tree, e.g. Figure 5 The same part of the Chinese pinyin paths "pinganfu" and "pinganyijiabao" is "pingan" as the middle main trunk, and the different parts are sub-trunks, such as Figure 5 The bifurcation in the middle section.

[0152] This embodiment can observe the character errors in the chat corpus of the chat robot in advance, summarize the user's error trend, build an error correction dictionary based on the correct characters corresponding to the observed erroneous characters and the pinyin sequence corresponding to the correct characters, input the text data to be corrected into the error correction dictionary for matching based on a preset multi-mode matching algorithm, replace the successfully matched characters for error correction, and complete the error correction of the erroneous characters in the text data to be corrected. The error correction dictionary constructed by this method can simply and quickly realize the error correction of text data with erroneous characters through a preset multi-mode matching algorithm, with a low error rate. At the same time, when a new erroneous character appears, it is only necessary to establish a mapping relationship between the new correct character and the corresponding pinyin sequence in the error correction dictionary, which is convenient for maintenance and can reduce labor costs.

[0153] In order to solve the above problems, the embodiment of the present invention further provides an error correction device, referring to Figure 6 , Figure 6 A schematic diagram of the functional modules of the error correction device provided by an embodiment of the present invention. In this embodiment, the device includes:

[0154] An acquisition module 41 is used to acquire text data to be corrected, wherein the text data includes text information input by a user and a preset text template;

[0155] A word segmentation module 42, used for performing word segmentation processing on the text data to obtain characters to be converted, wherein the characters to be converted are one of a single Chinese character, a phrase and a product name;

[0156] A conversion module 43, used for converting the characters to be converted into a pinyin sequence;

[0157] A matching module 44 is used to match the correct characters corresponding to the pinyin sequence from an error correction dictionary using a preset multi-mode matching algorithm according to the pinyin sequence, wherein the error correction dictionary is a dictionary of correct characters corresponding to characters that often appear in errors in the chat corpus of the chat robot in advance;

[0158] The replacement module 45 is used to replace the corresponding to-be-converted character in the text data with the correct character.

[0159] The present invention also provides a computer-readable storage medium.

[0160] In this embodiment, the computer-readable storage medium stores an error correction program, and when the error correction program is executed by the processor, the steps of the error correction method described in any of the above embodiments are implemented. The method implemented when the error correction program is executed by the processor can refer to the various embodiments of the error correction method of the present invention, so it will not be described in detail.

[0161] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM) and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0162] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

Claims

1. An error correction method, characterized in that: The error correction method comprises the following steps: Acquiring text data to be corrected, wherein the text data includes text information input by a user and a preset text template; Performing word segmentation processing on the text data to obtain characters to be converted, wherein the characters to be converted are one of a single Chinese character, a phrase, and a product name; Convert the characters to be converted into a pinyin sequence; According to the pinyin sequence, a preset multi-mode matching algorithm is used to match the correct characters corresponding to the pinyin sequence from an error correction dictionary, wherein the error correction dictionary is a dictionary of correct characters corresponding to characters that often appear in errors in the chat corpus of the chat robot in advance; Replacing the corresponding to-be-converted character in the text data with the correct character; The error correction dictionary is constructed as follows: Collecting chat corpus in the chat robot; Extracting characters that are erroneous and corrected in the chat corpus; Convert the characters into pinyin features, and establish a mapping relationship between the pinyin features and the characters; According to the matching principle of the multi-mode matching algorithm, a search tree is constructed for the pinyin features corresponding to all characters extracted from the chat corpus to obtain the error correction dictionary; The step of constructing a search tree for the pinyin features corresponding to all characters extracted from the chat corpus according to the matching principle of the multi-mode matching algorithm to obtain the error correction dictionary includes: Path-based pinyin features corresponding to all characters to obtain path features; Extracting the same part and the different part of the pinyin in the path feature, wherein the same part means that at least two consecutive syllables are the same; Create a main trunk based on the same section; On the basis of the main trunk, a sub-trunk is created, wherein the sub-trunk is created based on different parts to obtain an error correction dictionary in the shape of the search tree; Wherein, a plurality of nodes are arranged on the main trunk and the sub-trunk, and each node corresponds to a letter or a syllable in the pinyin feature.

2. The error correction method according to claim 1, characterized in that: The step of performing word segmentation processing on the text data to obtain characters to be converted comprises: Set a variable-length sliding window with an initial length of one; The text data is iteratively scanned based on the sliding window to obtain characters to be converted, wherein the characters to be converted are characters selected by the sliding window.

3. The error correction method according to claim 2, characterized in that: The step of matching the correct characters corresponding to the pinyin sequence from the error correction dictionary using a preset multi-mode matching algorithm according to the pinyin sequence comprises: Using the multi-mode matching algorithm, querying the error correction dictionary for all characters that contain the pinyin sequence; Calculate the length of each character, and select the character with the largest length from all characters to obtain the first error correction character; adjusting the sliding window according to the length of the first error correction character, and reacquiring the character to be converted from the text data; comparing the similarity between the first error correction character and the reacquired character to be converted; If the similarity is greater than a preset value, the first error correction character is selected as the correct character; If the similarity is less than a preset value, the character with the smallest length is selected from all the characters as the correct character.

4. The error correction method according to claim 3, characterized in that: If the node is a syllable in a pinyin feature, the step of using the multi-mode matching algorithm to query the error correction dictionary for all characters containing the pinyin sequence includes: According to the syllable division rule in Pinyin, the Pinyin sequence is subjected to syllable division processing to obtain a syllable sequence, wherein the division processing is to sequentially combine the initial consonants and the finals in the Pinyin sequence into a syllable and then perform segmentation processing; According to the syllable sequence, the nodes on the main trunk and the sub-trunk in the error correction dictionary are matched in sequence to obtain corresponding pinyin features; According to the pinyin features and the corresponding relationship between the pinyin features and the characters, all characters containing the pinyin sequence are determined.

5. The error correction method according to claim 4, characterized in that: The step of matching the syllable sequence with the nodes on the main trunk and the sub-trunk in the error correction dictionary in turn to obtain the corresponding pinyin features comprises: Inputting the syllables in the syllable sequence into the model corresponding to the multi-modal matching algorithm in sequence; By using the model, the first syllables in the syllable sequence are sequentially matched with the first nodes on all the main trunks in the error correction dictionary; If the match fails, the second syllable in the syllable sequence is read and matched with all the main trunks in the error correction dictionary; If the match is successful, then matching the second syllable in the syllable sequence with the next node of the main trunk along the successfully matched main trunk, and determining whether the sub-trunk exists in the next node; If it exists, then read the third syllable in the syllable sequence and match it with the first child node on the sub-trunk in the next node; If the match is successful, the next syllable in the syllable sequence is matched along the sub-trunk that is successfully matched; If the matching fails, the jump table preset in the first node is read, and the matching process of the syllable sequence is jumped to continue matching with other main trunks or sub-trunks.

6. An error correction device, characterized in that: The error correction device comprises: An acquisition module, used for acquiring text data to be corrected, wherein the text data includes text information input by a user and a preset text template; A word segmentation module, used for performing word segmentation processing on the text data to obtain characters to be converted, wherein the characters to be converted are one of a single Chinese character, a phrase and a product name; A conversion module, used for converting the characters to be converted into a pinyin sequence; A matching module, for matching the correct characters corresponding to the pinyin sequence from an error correction dictionary using a preset multi-mode matching algorithm according to the pinyin sequence, wherein the error correction dictionary is a dictionary of correct characters corresponding to characters that often appear in errors in the chat corpus of the chat robot, which is previously observed; A replacement module, used for replacing the corresponding to-be-converted character in the text data with the correct character; The error correction device further includes a dictionary construction module, which includes a collection unit, an extraction unit, a pinyin unit, and a construction unit, wherein: The collecting unit is used to collect the chat corpus in the chat robot; The extraction unit is used to extract characters in the chat corpus that are erroneous and corrected; The pinyin unit is used to convert the characters into pinyin features and establish a mapping relationship between the pinyin features and the characters; The construction unit is used to construct a search tree for the pinyin features corresponding to all characters extracted from the chat corpus according to the matching principle of the multi-mode matching algorithm to obtain the error correction dictionary; The construction unit is used to path-ize the pinyin features corresponding to all characters to obtain path features; extract the same part and the different part of the pinyin in the path features, wherein the same means that there are at least two consecutive syllables that are the same; create a main trunk based on the same part; based on the main trunk, create a sub-trunk, wherein the sub-trunk is created based on the different parts to obtain the error correction dictionary in the shape of the search tree; wherein the main trunk and the sub-trunk are both provided with a number of nodes, each node corresponding to a letter or a syllable in the pinyin feature.

7. The error correction device according to claim 6, characterized in that: The word segmentation module includes a setting unit and a scanning unit, wherein: The setting unit is used to set a sliding window of variable length with an initial length of one; The scanning unit is used to perform iterative scanning processing on the text data based on the sliding window to obtain characters to be converted, wherein the characters to be converted are characters selected by the sliding window.

8. The error correction device according to claim 7, characterized in that: The matching module includes a query unit, a length calculation unit, an adjustment unit and a comparison unit, wherein: The query unit is used to query all characters in the error correction dictionary that contain the pinyin sequence using the multi-mode matching algorithm; The length calculation unit is used to calculate the length of each character, and select the character with the largest length from all characters to obtain the first error correction character; The adjusting unit is used to adjust the sliding window according to the length of the first error correction character, and reacquire the character to be converted from the text data; The comparison unit is used to compare the similarity between the first error correction character and the re-acquired character to be converted; If the similarity is greater than a preset value, the first error correction character is selected as the correct character; If the similarity is less than a preset value, the character with the smallest length is selected from all the characters as the correct character.

9. An error correction device, characterized in that: The error correction device comprises a memory, a processor, and an error correction program stored in the memory and executable on the processor. When the error correction program is executed by the processor, the steps of the error correction method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an error correction program, and when the error correction program is executed by the processor, the steps of the error correction method according to any one of claims 1 to 5 are implemented.

Images