A text error correction method and system that takes into account both text length and similarity.

Abstract

The present invention discloses a text error correction method and system that takes into account both text length and similarity. By searching in a relevant database, the long word closest to the unit to be corrected is obtained. If the length of the long word is in a preset length interval, a sliding window is created, and a text block is intercepted with the window, and the edit distance between the text block and the long word is obtained to obtain the error type, so as to correct the text. If the length of the long word is greater than the preset length interval, it is determined whether the similarity exceeds the preset threshold, and then the error type is obtained through a text comparison algorithm, and error correction is performed. There are no special requirements for hardware, and it has very good system compatibility, high error correction efficiency, and greatly shortens the proofreading time. New words can be added in real time, quickly, and conveniently to meet the error correction of new words. By setting databases in different fields, long word error correction in different fields can be met, and it does not rely on deep learning technology.

Description

Technical Field

[0001] The present invention relates to the field of search technology, and in particular to a text error correction method and system that take into account both text length and similarity. Background Art

[0002] Text error correction technology usually includes two parts: detection and error correction. It is one of the important topics of natural language processing under the branch of deep learning. Text error correction relies on deep learning models to detect errors and uses complex calculations to recall correct words and phrases to correct errors. This error correction method requires large amounts of calculations, a relatively complex model, and low overall error correction efficiency, making it difficult to apply this technology to domestic machines. In addition, deep learning models generally have domain limitations. In different fields, the performance of the models varies greatly, requiring human and machine resources to continuously train to meet the needs of different fields.

[0003] As society continues to develop and progress, new words, slogans, and inherent expressions are constantly being created. Current deep learning models often rely on a larger, universal pre-trained model, and make appropriate fine-tuning on this basis. However, as new words continue to emerge, the previous model is not compatible with new words, and it is impossible to obtain sufficient training samples to optimize the model in a short period of time. Therefore, a technology that can be separated from deep learning is needed to provide the function of correcting newly generated words.

[0004] Therefore, how to provide a technology that is independent of deep learning to correct text errors is a technical problem that needs to be urgently solved by technicians in this field. Summary of the invention

[0005] The present invention provides a text error correction method that takes into account both text length and similarity, and is used to solve the technical problems in the prior art that text error correction relies on deep learning, has low overall error correction efficiency, and is time-consuming and labor-intensive. The method includes:

[0006] Obtain the text to be corrected, and divide it into units of text to be corrected using punctuation marks;

[0007] Searching in a database to obtain a number of long words closest to the text unit to be corrected;

[0008] Based on the length of the long word, a determination method is selected to obtain the error type of the text unit to be corrected, and the text unit to be corrected is corrected according to the error type.

[0009] Preferably, based on the length of the long word, a determination method is selected to obtain the error type of the text unit to be corrected, specifically:

[0010] When the length of the long word is within a preset length range, length determination is selected;

[0011] The length determination is specifically as follows:

[0012] Three sliding windows are set, and the lengths of the three sliding windows are:

[0013] The length of the long word minus one, the length of the long word, and the length of the long word plus one;

[0014] The three sliding windows correspond to the error types representing missing words, wrong words and multiple words respectively;

[0015] According to the increasing order of the sliding window lengths, the three sliding windows are used in sequence to intercept the text units to be corrected, to obtain text blocks of corresponding lengths;

[0016] The edit distances between the text block and the long word are calculated in sequence, and it is determined whether the edit distance is 1. If so, the error type corresponding to the sliding window is obtained.

[0017] Preferably, based on the length of the long word, a determination method is selected to obtain the error type of the text unit to be corrected, specifically:

[0018] When the length of the long word is greater than a preset length interval, similarity determination is selected;

[0019] The similarity determination is specifically as follows:

[0020] Comparing the length of the text unit to be corrected with the length of the long word to obtain the longest length of the two;

[0021] Calculating the similarity between the text unit to be corrected and the long word,

[0022] Similarity = (longest length - edit distance) / longest length,

[0023] The edit distance is the edit distance between the text unit to be corrected and the long word;

[0024] It is determined whether the similarity exceeds a preset threshold. If so, the difference between the text unit to be corrected and the long word is found through a text comparison method to obtain three error types: missing characters, wrong characters and extra characters.

[0025] Preferably, the difference between the text unit to be corrected and the long word is found by a text comparison method, and three error types of missing characters, wrong characters and extra characters are obtained, specifically:

[0026] Construct a matrix with the long words as rows and the text units to be corrected as columns;

[0027] The outer loop traverses the rows of the matrix, and the inner loop traverses the columns of the matrix;

[0028] The inner loop and the outer loop determine whether the current position is the same as the character at the index position. If so, the current position is recorded as (i now , j now ), and calculate i dis =i now -i before , j dis =j now -j before ;

[0029] If i dis =j dis ≠1, it means that this is a typo error type;

[0030] If i dis >j dis , it means that this is a missing word error type;

[0031] If i dis <j dis , it means that this is a multi-word error type;

[0032] Among them, i dis is the difference of the text unit to be corrected, i now is the current value of the text unit to be corrected, i before is the past value of the text unit to be corrected, j dis is the difference of the long word, j now is the present value of the long word, j before is the past value of the long word.

[0033] Preferably, if the preceding sliding window satisfies the judgment condition, the subsequent sliding window interception and calculation actions are canceled.

[0034] Accordingly, the present invention also provides a text error correction system that takes into account both text length and similarity, including:

[0035] An acquisition module is used to acquire the text to be corrected and divide it into units of text to be corrected using punctuation marks;

[0036] A search module, used for searching in a database for a number of long words closest to the text unit to be corrected;

[0037] The error correction module is used to select a judgment method based on the length of the long word to obtain the error type of the text unit to be corrected, and correct the text unit to be corrected according to the error type.

[0038] Preferably, the error correction module is further specifically used for:

[0039] When the length of the long word is within a preset length range, length determination is selected;

[0040] The length determination is specifically as follows:

[0041] Three sliding windows are set, and the lengths of the three sliding windows are:

[0042] The length of the long word minus one, the length of the long word, and the length of the long word plus one;

[0043] The three sliding windows correspond to the error types representing missing words, wrong words and multiple words respectively;

[0044] According to the increasing order of the sliding window lengths, the three sliding windows are used in sequence to intercept the text units to be corrected, to obtain text blocks of corresponding lengths;

[0045] The edit distances between the text block and the long word are calculated in sequence, and it is determined whether the edit distance is 1. If so, the error type corresponding to the sliding window is obtained.

[0046] Preferably, the error correction module is further specifically used for:

[0047] When the length of the long word is greater than a preset length interval, similarity determination is selected;

[0048] The similarity determination is specifically as follows:

[0049] Comparing the length of the text unit to be corrected with the length of the long word to obtain the longest length of the two;

[0050] Calculating the similarity between the text unit to be corrected and the long word,

[0051] Similarity = (longest length - edit distance) / longest length,

[0052] The edit distance is the edit distance between the text unit to be corrected and the long word;

[0053] It is determined whether the similarity exceeds a preset threshold. If so, the difference between the text unit to be corrected and the long word is found through a text comparison method to obtain three error types: missing characters, wrong characters and extra characters.

[0054] Preferably, the error correction module is further specifically used for:

[0055] Construct a matrix with the long words as rows and the text units to be corrected as columns;

[0056] The outer loop traverses the rows of the matrix, and the inner loop traverses the columns of the matrix;

[0057] The inner loop and the outer loop determine whether the current position is the same as the character at the index position. If so, the current position is recorded as (i now , j now ), and calculate i dis =i now -i before , j dis =j now -j before ;

[0058] If i dis =j dis ≠1, it means that this is a typo error type;

[0059] If i dis >j dis , it means that this is a missing word error type;

[0060] If i dis <j dis , it means that this is a multi-word error type;

[0061] Among them, i dis is the difference of the text unit to be corrected, i now is the current value of the text unit to be corrected, i before is the past value of the text unit to be corrected, j dis is the difference of the long word, j now is the present value of the long word, j before is the past value of the long word.

[0062] Preferably, the error correction module is further used for:

[0063] If the preceding sliding window satisfies the judgment condition, the subsequent sliding window interception and calculation actions are canceled.

[0064] Therefore, the present invention adopts the above method and system, which has the following beneficial effects:

[0065] By searching in the relevant database, the long word closest to the unit to be corrected is obtained. If the length of the long word is in the preset length range, a sliding window is created, and the text block is intercepted with the window. The edit distance between the text block and the long word is obtained to obtain the error type, and the text is corrected. If the length of the long word is greater than the preset length range, it is determined whether the similarity exceeds the preset threshold, and then the text comparison algorithm is used to obtain the error type and perform correction. There are no special requirements for hardware, and it has very good system compatibility, high error correction efficiency, and greatly shortens the proofreading time. New words can be added in real time, quickly, and conveniently to meet the correction of new words. By setting up databases in different fields, long word correction in different fields can be met, and it does not rely on deep learning technology.

[0066] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flow chart of a text error correction method that takes into account both text length and similarity in an embodiment of the present invention;

[0068] Figure 2 It is a structural schematic diagram of a text error correction system that takes into account both text length and similarity in another embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0070] Text error correction generally relies on deep learning models to detect errors and uses complex calculations to recall correct words and phrases to correct them. Deep learning (DL) is a new research direction in the field of machine learning (ML). It is introduced into machine learning to make it closer to its original goal - artificial intelligence (AI).

[0071] Deep learning is the process of learning the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in interpreting data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technology, and other related fields.

[0072] However, deep learning in natural language processing requires large amounts of computation, has a relatively complex model, and has a relatively poor overall error correction effect, making it difficult to apply this technology to domestic machines. Deep learning models often rely on a large, universal pre-trained model, but with the popularization of the Internet, there are more and more new words, slogans, and Internet terms. Previous models are not compatible with new words, and it is impossible to obtain sufficient training samples to optimize the model in a short period of time.

[0073] Based on the above background, the present invention proposes a text error correction method that takes into account both text length and similarity. Figure 1 As shown, the steps are as follows:

[0074] S101, obtaining a text to be corrected, and dividing it by punctuation marks to obtain text units to be corrected.

[0075] The text to be corrected is obtained, and is divided by punctuation marks, including commas, semicolons, colons, periods, etc., to obtain text units to be corrected.

[0076] S102, searching in a database to obtain a plurality of long words that are closest to the text unit to be corrected.

[0077] According to text databases in different fields, a number of long words that are closest to the text unit to be corrected are searched, wherein the closest means satisfying a preset difference range.

[0078] S103, based on the length of the long word, selecting a determination method to obtain the error type of the text unit to be corrected, and correcting the text unit to be corrected according to the error type.

[0079] When the length of the long word is within a preset length range, length determination is selected. The preset length range is set according to the specific text situation and can be changed according to the situation.

[0080] The length determination is specifically as follows:

[0081] Three sliding windows are set, and the lengths of the three sliding windows are:

[0082] The length of the long word minus one, the length of the long word, and the length of the long word plus one;

[0083] The three sliding windows correspond to the error types representing missing words, wrong words and multiple words respectively.

[0084] The wrong word is a long word and has the same length as the unit of text to be corrected, but the characters are different. The corresponding sliding window is the length of the long word.

[0085] Missing characters refer to the text unit to be corrected lacking some characters at a certain position (except the beginning and end of the word) compared to the long word. The corresponding length is the sliding window of the length of the long word minus one.

[0086] The multi-word text unit to be corrected has some more words at a certain position (except the beginning and end of the word) compared to the long word. The corresponding length is the sliding window of the length of the long word plus one.

[0087] According to the increasing order of the sliding window lengths, the three sliding windows are used in turn to intercept the text units to be corrected to obtain text blocks of corresponding lengths.

[0088] The text unit to be corrected is intercepted using three sliding windows of lengths equal to the length of the long word minus one, the length of the long word, and the length of the long word plus one, respectively, to obtain three text blocks of corresponding lengths.

[0089] The minimum edit distance between the text block and the long word is calculated in sequence, and it is determined whether the minimum edit distance is 1, if so, the error type corresponding to the sliding window is obtained. If not, no operation is performed.

[0090] Calculate the edit distances of the three text blocks and the long word in order, and determine whether the edit distance is 1. If so, get the error type corresponding to the sliding window. Note that when the previous sliding window with a smaller length meets the condition, the subsequent windows will no longer be calculated.

[0091] When the length of the long word is greater than a preset length interval, similarity determination is selected;

[0092] The similarity determination is specifically as follows:

[0093] Compare the length of the text unit to be corrected with the length of the long word to obtain the longest length of the two.

[0094] The lengths of the long word and the text unit to be corrected are compared to obtain the longer length, namely the longest length.

[0095] Calculating the similarity between the text unit to be corrected and the long word,

[0096] Similarity = (longest length - edit distance) / longest length,

[0097] The edit distance is the edit distance between the text unit to be corrected and the long word.

[0098] It is determined whether the similarity exceeds a preset threshold value. If so, the difference between the text unit to be corrected and the long word is found through a text comparison method to obtain three error types: missing characters, wrong characters and extra characters. If not, no operation is performed.

[0099] The text comparison method is as follows:

[0100] Construct a matrix with the long words as rows and the text units to be corrected as columns;

[0101] The outer loop traverses the rows of the matrix, and the inner loop traverses the columns of the matrix;

[0102] The inner loop and the outer loop determine whether the current position is the same as the character at the index position. If so, the current position is recorded as (i now , j now ), and calculate i dis =i now -i before , j dis =j now -j before ;

[0103] If i dis =j dis ≠1, it means that this is a typo error type;

[0104] If i dis >j dis , it means that this is a missing word error type;

[0105] If i dis <j dis , it means that this is a multi-word error type;

[0106] Among them, i dis is the difference of the text unit to be corrected, i now is the current value of the text unit to be corrected, i before is the past value of the text unit to be corrected, j dis is the difference of the long word, j now is the present value of the long word, j before is the past value of the long word.

[0107] By applying the above technical scheme, the present invention provides a text correction method that takes into account both text length and similarity. The method includes: obtaining a text to be corrected, and dividing it with punctuation marks to obtain a text unit to be corrected; searching in a database to obtain several long words closest to the text unit to be corrected; based on the length of the long word, selecting a judgment method to obtain the error type of the text unit to be corrected, and correcting the text unit to be corrected according to the error type. By searching in a related database, the long word closest to the unit to be corrected is obtained. If the length of the long word is in a preset length interval, a sliding window is created, a text block is intercepted with the window, and the edit distance between the text block and the long word is obtained to obtain the error type, so as to correct the text. If the length of the long word is greater than the preset length interval, it is determined whether the similarity exceeds the preset threshold, and then the error type is obtained through a text comparison algorithm, and correction is performed. There are no special requirements for hardware, and it has very good system compatibility, high error correction efficiency, and greatly shortens the proofreading time. New words can be added in real time, quickly and conveniently to meet the needs of new word correction. By setting up databases in different fields, long word correction can be met in different fields without relying on deep learning technology.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present invention can be implemented by hardware, or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present invention.

[0109] In order to further explain the technical idea of ​​the present invention, the technical solution of the present invention is now described in combination with specific application scenarios.

[0110] Traversal refers to visiting each node in a tree (or graph) in sequence along a certain search route. The operation performed when visiting a node depends on the specific application problem. The specific access operation may be to check the value of a node, update the value of a node, etc. Different traversal methods have different order of visiting nodes. Traversal is one of the most important operations on a binary tree and is the basis for other operations on a binary tree. Of course, the concept of traversal is also applicable to multi-element collections, such as arrays.

[0111] First, the text to be corrected is obtained, and the text to be corrected is divided by punctuation marks such as commas, semicolons, colons, periods, etc. to obtain text units to be corrected, and each text unit to be corrected is traversed, and a search engine is used to search in databases in different fields to obtain the n long words closest to the text unit to be corrected.

[0112] According to the length of the long word, it is divided into two situations and corrected separately. When the length of the long word is between 4-6, three sliding windows are set, and their lengths are: the length of the long word minus one, the length of the long word, and the length of the long word plus one. Use these three windows to determine whether the error type is a wrong word, a missing word, or an extra word. Missing words refer to the text unit to be corrected that lacks some words in a certain position (except the beginning and end of the word) compared to the long word. The corresponding length is a sliding window of the length of the long word minus one. Wrong words refer to the length of the long word and the text unit to be corrected that are the same, but the words are different. The corresponding length is a sliding window of the length of the long word. Extra words refer to the text unit to be corrected that has some extra words in a certain position (except the beginning and end of the word) compared to the long word. The corresponding length is a sliding window of the length of the long word plus one.

[0113] Use three sliding windows to intercept the text units to be corrected and obtain text blocks of corresponding lengths. Use the minimum edit distance algorithm to calculate the minimum edit distance between the text block and the long word in each sliding window, and determine whether the minimum edit distance is 1. If yes, obtain the error type of the corresponding sliding window. If not, do not perform any operation.

[0114] When the error type of the text unit to be corrected is a missing word, that is, the length of the text unit to be corrected is less than the length of the long word, at this time, only one text block is obtained by using all the sliding windows. When classifying the error type, the missing word error type will be classified as the wrong word type. In order to avoid this situation, the sliding window with a smaller length is placed in front. When one window meets the conditions, the calculation of the subsequent window can be cancelled. Based on the error type, the unit to be corrected is corrected.

[0115] When the length of the long word is greater than 6, first compare the length of the long word with the length of the text unit to be corrected to obtain the longest length of the two texts, and calculate the similarity between the text unit to be corrected and the long word.

[0116] Similarity = (longest length - edit distance) / longest length.

[0117] Determine whether the similarity exceeds the set threshold. If yes, use the text comparison algorithm to find the difference between the text unit to be corrected and the long word. If no, no operation is performed. There are three types of errors through comparison, namely missing words, extra words and wrong words.

[0118] Text comparison algorithm: First, use the length of the two texts to construct a matrix, with long words as rows and text units to be corrected as columns. Initialize all the contents in the matrix to "F", traverse the contents of the two texts (text units to be corrected and long words), and if the characters at the same index position are the same, replace "F" with "T" at the corresponding position on the matrix. Set the initial position of the matrix index_ = (0,0), the outer loop traverses the rows of the matrix, and the inner loop traverses the columns of the matrix. In the inner loop, determine whether the current position is "T". If so, record the current position and change index_ to (i now ,j now ), and calculate i dis =i now -i before , j dis =j now -j before If not, no operation is performed. dis is the difference of the text unit to be corrected, i now is the current value of the text unit to be corrected, i before is the past value of the text unit to be corrected, j dis is the difference of the long word, j now is the present value of the long word, j before is the past value of the long word.

[0119] If i dis Equal to j dis and is not equal to 1, it means that this is a typo error type; if i dis >j dis , it means that this is a missing word error type; if i dis <j dis , it means that this is a multi-word error type. After obtaining the error type, the index positions of the two texts are compared to obtain the error text in the text unit to be corrected and the correct text of the long word, and then the error is corrected.

[0120] It should be noted that long words are usually surrounded by other words when they appear in the text, so it is necessary to filter out the multi-word errors before and after the words. Finally, the filtered results are used to correct the erroneous text.

[0121] Accordingly, the present invention also provides a text error correction system that takes into account both text length and similarity, such as Figure 2 As shown, the system includes:

[0122] S201, an acquisition module, used to acquire a text to be corrected, and segment it with punctuation marks to obtain text units to be corrected;

[0123] S202, a search module, used to search in a database for a number of long words closest to the text unit to be corrected;

[0124] S203, a determination and correction module, used to select a determination method based on the length of the long word, obtain the error type of the text unit to be corrected, and correct the text unit to be corrected according to the error type.

[0125] The error correction module is further specifically used for:

[0126] When the length of the long word is within a preset length range, length determination is selected;

[0127] The length determination is specifically as follows:

[0128] Three sliding windows are set, and the lengths of the three sliding windows are:

[0129] The length of the long word minus one, the length of the long word, and the length of the long word plus one;

[0130] The three sliding windows correspond to the error types representing missing words, wrong words and multiple words respectively;

[0131] According to the increasing order of the sliding window lengths, the three sliding windows are used in sequence to intercept the text units to be corrected, to obtain text blocks of corresponding lengths;

[0132] The minimum edit distances between the text block and the long word are calculated in sequence, and it is determined whether the minimum edit distance is 1. If so, the error type corresponding to the sliding window is obtained.

[0133] The error correction module is further specifically used for:

[0134] When the length of the long word is greater than a preset length interval, similarity determination is selected;

[0135] The similarity determination is specifically as follows:

[0136] Comparing the length of the text unit to be corrected with the length of the long word to obtain the longest length of the two;

[0137] Calculating the similarity between the text unit to be corrected and the long word,

[0138] Similarity = (longest length - edit distance) / longest length,

[0139] The edit distance is the edit distance between the text unit to be corrected and the long word;

[0140] It is determined whether the similarity exceeds a preset threshold. If so, the difference between the text unit to be corrected and the long word is found through a text comparison method to obtain three error types: missing characters, wrong characters and extra characters.

[0141] The error correction module is further specifically used for:

[0142] Construct a matrix with the long words as rows and the text units to be corrected as columns;

[0143] The outer loop traverses the rows of the matrix, and the inner loop traverses the columns of the matrix;

[0144] The inner loop and the outer loop determine whether the current position is the same as the character at the index position. If so, the current position is recorded as (i now , j now ), and calculate i dis =i now -i before , j dis =j now -j before ;

[0145] If i dis =j dis ≠1, it means that this is a typo error type;

[0146] If i dis >j dis , it means that this is a missing word error type;

[0147] If i dis <j dis , it means that this is a multi-word error type;

[0148] Among them, i dis is the difference of the text unit to be corrected, i now is the current value of the text unit to be corrected, i before is the past value of the text unit to be corrected, j dis is the difference of the long word, j now is the present value of the long word, j before is the past value of the long word.

[0149] The error correction module is further used for:

[0150] If the preceding sliding window satisfies the judgment condition, the subsequent sliding window interception and calculation actions are canceled.

[0151] Those skilled in the art will appreciate that the modules in the system in the implementation scenario can be distributed in the device of the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules of the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A text error correction method that takes into account both text length and similarity, characterized in that: include: Obtain the text to be corrected, and divide it into units of text to be corrected using punctuation marks; Searching in a database to obtain a number of long words closest to the text unit to be corrected; Based on the length of the long word, a determination method is selected to obtain the error type of the text unit to be corrected, and the text unit to be corrected is corrected according to the error type; When the length of the long word is within a preset length range, length determination is selected; The length determination is specifically as follows: Three sliding windows are set, and the lengths of the three sliding windows are: The length of the long word minus one, the length of the long word, and the length of the long word plus one; The three sliding windows correspond to the error types representing missing words, wrong words and multiple words respectively; According to the increasing order of the sliding window lengths, the three sliding windows are used in sequence to intercept the text units to be corrected, to obtain text blocks of corresponding lengths; The edit distances between the text block and the long word are calculated in sequence, and it is determined whether the edit distance is 1. If so, the error type corresponding to the sliding window is obtained.

2. The method according to claim 1, characterized in that Based on the length of the long word, a determination method is selected to obtain the error type of the text unit to be corrected, specifically: When the length of the long word is greater than a preset length interval, similarity determination is selected; The similarity determination is specifically as follows: Comparing the length of the text unit to be corrected with the length of the long word to obtain the longest length of the two; Calculating the similarity between the text unit to be corrected and the long word, Similarity = (longest length - edit distance) / longest length, The edit distance is the edit distance between the text unit to be corrected and the long word; It is determined whether the similarity exceeds a preset threshold. If so, the difference between the text unit to be corrected and the long word is found through a text comparison method to obtain three error types: missing characters, wrong characters and extra characters.

3. The method according to claim 2, characterized in that The text comparison method is used to find the difference between the text unit to be corrected and the long word, and three error types of missing words, wrong words and extra words are obtained, specifically: Construct a matrix with the long words as rows and the text units to be corrected as columns; The outer loop traverses the rows of the matrix, and the inner loop traverses the columns of the matrix; The inner loop and the outer loop determine whether the current position is the same as the character at the index position. If so, the current position is recorded as (i now , j now ), and calculate i dis =i now -i before , j dis =j now -j before ; If i dis =j dis ≠1, it means that this is a typo error type; If i dis >j dis , it means that this is a missing word error type; If i dis <j dis , it means that this is a multi-word error type; Among them, i dis is the difference of the text unit to be corrected, i now is the current value of the text unit to be corrected, i before is the past value of the text unit to be corrected, j dis is the difference of the long word, j now is the present value of the long word, j before is the past value of the long word.

4. The method according to claim 1, characterized in that If the preceding sliding window satisfies the judgment condition, the subsequent sliding window interception and calculation actions are canceled.

5. A text error correction system that takes into account both text length and similarity, characterized in that: include: An acquisition module is used to acquire the text to be corrected and divide it into units of text to be corrected using punctuation marks; A search module, used for searching in a database for a number of long words closest to the text unit to be corrected; A determination and correction module, configured to select a determination method based on the length of the long word, obtain an error type of the text unit to be corrected, and perform error correction on the text unit to be corrected according to the error type; When the length of the long word is within a preset length range, length determination is selected; The length determination is specifically as follows: Three sliding windows are set, and the lengths of the three sliding windows are: The length of the long word minus one, the length of the long word, and the length of the long word plus one; The three sliding windows correspond to the error types representing missing words, wrong words and multiple words respectively; According to the increasing order of the sliding window lengths, the three sliding windows are used in sequence to intercept the text units to be corrected, to obtain text blocks of corresponding lengths; The edit distances between the text block and the long word are calculated in sequence, and it is determined whether the edit distance is 1. If so, the error type corresponding to the sliding window is obtained.

6. The system according to claim 5, characterized in that The error correction module is further specifically used for: When the length of the long word is greater than a preset length interval, similarity determination is selected; The similarity determination is specifically as follows: Comparing the length of the text unit to be corrected with the length of the long word to obtain the longest length of the two; Calculating the similarity between the text unit to be corrected and the long word, Similarity = (longest length - edit distance) / longest length, The edit distance is the edit distance between the text unit to be corrected and the long word; It is determined whether the similarity exceeds a preset threshold. If so, the difference between the text unit to be corrected and the long word is found through a text comparison method to obtain three error types: missing characters, wrong characters and extra characters.

7. The system according to claim 6, characterized in that The error correction module is further specifically used for: Construct a matrix with the long words as rows and the text units to be corrected as columns; The outer loop traverses the rows of the matrix, and the inner loop traverses the columns of the matrix; The inner loop and the outer loop determine whether the current position is the same as the character at the index position. If so, the current position is recorded as (i now , j now ), and calculate i dis =i now -i before , j dis =j now -j before ; If i dis =j dis ≠1, it means that this is a typo error type; If i dis >j dis , it means that this is a missing word error type; If i dis <j dis , it means that this is a multi-word error type; Among them, i dis is the difference of the text unit to be corrected, i now is the current value of the text unit to be corrected, i before is the past value of the text unit to be corrected, j dis is the difference of the long word, j now is the present value of the long word, j before is the past value of the long word.

8. The system according to claim 5, characterized in that The error correction module is further used for: If the preceding sliding window satisfies the judgment condition, the subsequent sliding window interception and calculation actions are canceled.

Images