Text error correction method and device, electronic device and storage medium

Abstract

The embodiments of the present application provide a text error correction method and device, an electronic device, and a storage medium, the method comprising: obtaining text data of a target field and determining a first text to be corrected according to a preset rule corresponding to the target field; processing the first text to be corrected using a target model, wherein the target model is obtained by training an initial model with a set of data pairs, and the data pairs are vocabulary data pairs including words to be corrected and correct words; and in the case where the words to be corrected are included in the first text to be corrected, outputting an error correction result through the target model, wherein the error correction result includes the first text to be corrected and the correct words corresponding to the words to be corrected. The above scheme solves the technical problem in the prior art that only clear syntax and grammar rules are corrected, resulting in low error correction efficiency.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a text error correction method and device, electronic equipment and storage medium. Background Art

[0002] At present, the text data error correction scheme is a rule-based method, which refers to the use of a series of clear syntactic and grammatical rules to judge spelling errors, which also requires text analysis, chunking and phrase detection. Once the segmentation result cannot guarantee all language criteria, it is considered an error.

[0003] Although the rule-based method is easy to implement and has a high error detection accuracy, its disadvantage is that the accuracy is also limited by these established rules and cannot cover all grammatical errors, so it has poor transferability. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a text error correction method and device, an electronic device and a storage medium to solve the technical problem that the prior art only corrects errors based on clear syntax and grammar rules, resulting in low error correction efficiency. The specific technical solution is as follows:

[0005] In a first aspect of the implementation of the present application, a text correction method is first provided, comprising: acquiring text data of a target field and determining a first text to be corrected according to a preset rule corresponding to the target field; processing the first text to be corrected using a target model, wherein the target model is obtained by training an initial model with a set of data pairs, and the data pairs are vocabulary data pairs including vocabulary to be corrected and correct vocabulary; and in a case where the vocabulary to be corrected is included in the first text to be corrected, outputting a correction result through the target model, wherein the correction result includes the first text to be corrected and the correct vocabulary corresponding to the vocabulary to be corrected.

[0006] Optionally, the vocabulary to be corrected is determined by a confusion set, and the correct vocabulary is determined by a data set; before using the target model to process the first text to be corrected, the method also includes: collecting multiple keywords in the target field, and using the multiple keywords to construct the data set; determining homophones associated with each word in the multiple keywords to obtain a homophone set; wherein the homophones are words with exactly the same initial consonants and finals; determining pictographic characters associated with each word in the multiple keywords based on four-corner coding to obtain a pictographic character set; and constructing the confusion set using the homophone set and the pictographic character set.

[0007] Optionally, constructing the confusion set using the homophone set and the pictographic character set includes: when the text length of a first keyword in the data set meets a first preset length standard, randomly selecting a character in the first keyword and replacing the homophone with the homophone or replacing the pictographic character with the pictographic character set to obtain the vocabulary to be corrected; when the text length of a second keyword in the data set meets a second preset length standard, randomly selecting two characters in the second keyword and replacing the homophone with the homophone or replacing the pictographic character with the pictographic character set to obtain the vocabulary to be corrected; and constructing the confusion set using the vocabulary to be corrected.

[0008] Optionally, before using the target model to process the first text to be corrected, the method also includes: constructing an error detection network of the target model based on a bidirectional long short-term memory Bi LSTM model; constructing an error detection network of an initial model and an error correction model of the initial model based on a converter-based bidirectional coding representation BERT model; setting a maximum sequence length for model training according to the target domain; and training the initial model according to a data pair set to obtain the target model.

[0009] Optionally, the use of the target model to process the first text to be corrected includes: obtaining an input vector corresponding to the first text, wherein the input vector is jointly composed of a word vector, a position vector and a segment vector of the first text; performing error probability calculation on each character in the input vector based on the error detection network to obtain a calculation result; multiplying the difference between 1 and the calculation result by the input vector as the input of the error correction network of the target model; outputting multiple correction probabilities corresponding to the character to be corrected based on the error correction network, and selecting the character with the highest probability among the multiple correction probabilities from the candidate character list as the correction character of the character to be corrected in the first text.

[0010] Optionally, when the first text to be corrected includes the vocabulary to be corrected, outputting the correction result through the target model includes: when there is at least one of the correction characters, outputting the correction result.

[0011] Optionally, the error correction result includes: a tuple consisting of the first text sequence, an erroneous text character, a correct text character, and a start index of a position of the erroneous text character in the first text sequence.

[0012] In a second aspect of the implementation of the present application, there is also provided a text correction device, comprising: an acquisition module, for acquiring text data of a target field and determining a first text to be corrected according to a preset rule corresponding to the target field; a processing module, for processing the first text to be corrected using a target model, wherein the target model is obtained by training an initial model with a set of data pairs, and the data pairs are vocabulary data pairs including vocabulary to be corrected and correct vocabulary; and a correction module, for outputting a correction result through the target model when the vocabulary to be corrected is included in the first text to be corrected, wherein the correction result includes the first text to be corrected and the correct vocabulary corresponding to the vocabulary to be corrected.

[0013] Optionally, the vocabulary to be corrected is determined by a confusion set, and the correct vocabulary is determined by a data set; the device also includes: a first construction module, used to collect multiple keywords in the target field before processing the first text to be corrected using the target model, and use the multiple keywords to construct the data set; a first determination module, used to determine homophones associated with each word in the multiple keywords to obtain a homophone set; wherein the homophones are words with exactly the same initial consonants and finals; a second determination module, used to determine pictographic characters associated with each word in the multiple keywords based on four-corner coding to obtain a pictographic character set; and a second construction module, used to construct the confusion set using the homophone set and the pictographic character set.

[0014] Optionally, the second construction module includes: a first replacement unit, which is used to randomly select a word in the first keyword and replace the homophone with the homophone set or replace the pictographic characters with the pictographic characters set to obtain the vocabulary to be corrected when the text length of the first keyword in the data set meets the first preset length standard; a second replacement unit, which is used to randomly select two words in the second keyword and replace the homophone with the homophone set or replace the pictographic characters with the pictographic characters set to obtain the vocabulary to be corrected when the text length of the second keyword in the data set meets the second preset length standard; and a construction unit, which is used to construct the confusion set using the vocabulary to be corrected.

[0015] Optionally, the device also includes: a third construction module, used to construct an error detection network of the target model according to a bidirectional long short-term memory BiLSTM model before using the target model to process the first text to be corrected; a fourth construction module, used to construct an error detection network of the initial model and an error correction model of the initial model according to a converter-based bidirectional encoding representation BERT model; a setting module, used to set the maximum sequence length of model training according to the target domain; a training module, used to train the initial model according to a data pair set to obtain the target model.

[0016] Optionally, the processing module includes: an acquisition unit, used to acquire an input vector corresponding to the first text, wherein the input vector is jointly composed of a word vector, a position vector and a segment vector of the first text; a calculation unit, used to calculate the error probability of each character in the input vector based on the error detection network to obtain a calculation result; an input unit, used to multiply the difference between 1 and the calculation result by the input vector as an input to the error correction network of the target model; and a processing unit, used to output multiple correction probabilities corresponding to the character to be corrected based on the error correction network, and select the character with the highest probability among the multiple correction probabilities from the candidate character list as the correction character of the character to be corrected in the first text.

[0017] Optionally, the error correction module includes: an output unit, configured to output the error correction result when there is at least one correction character.

[0018] Optionally, the error correction result includes: a tuple consisting of the first text sequence, an erroneous text character, a correct text character, and a start index of a position of the erroneous text character in the first text sequence.

[0019] In another aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes any of the random testing methods for host operations described above.

[0020] In another aspect of the implementation of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned random testing methods for host operations.

[0021] In an embodiment of the present application, text data of the target field can be obtained and the first text to be corrected can be determined based on preset rules corresponding to the target field, and then the first text to be corrected can be processed using the target model. When the first text to be corrected includes words to be corrected, the correction result can be output through the target model, that is, the correction of the first text to be corrected can be achieved through the target model, which can cover all grammatical errors, thereby solving the problem of low correction efficiency in the prior art that only clear syntax and grammar rules are corrected. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0023] Figure 1 This is a flow chart of the error correction method of the text in the embodiment of the present application;

[0024] Figure 2 This is a schematic diagram of the structure of the error detection network implemented by Bi LSTM in an embodiment of the present application;

[0025] Figure 3 This is a schematic diagram of the mask vector construction process in an embodiment of the present application;

[0026] Figure 4 This is a schematic diagram of the model structure of the error correction network implemented by BERT in an embodiment of the present application;

[0027] Figure 5 This is a flow chart of a model error correction method for searching query words in an embodiment of the present application;

[0028] Figure 6 It is a structural schematic diagram of the error correction device in the text of the embodiment of the present application;

[0029] Figure 7 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0031] First, the relevant terms involved in the embodiments of the present application are explained;

[0032] Four-corner coding: This method divides each character into four corners, assigns a number to each corner, and then arranges all the characters in the order of the four numbers. It divides the strokes of Chinese characters into ten categories: head, horizontal, vertical, dot, cross, insert, square, corner, eight, and small, represented by numbers 0 to 9 respectively. The strokes of the four corners of each character are numbered in the order of their positions: upper left, upper right, lower left, and lower right.

[0033] Confusion set: Confusion set is one of the important resources in text error correction. It is used to store text collections that are similar in appearance or pronunciation and are easy to cause confusion and text errors.

[0034] Seq2seq: Seq2Seq is a variant of a recurrent neural network, consisting of an encoder and a decoder. Seq2Seq is an important model in natural language processing and can be used for machine translation, dialogue systems, automatic summarization, etc.

[0035] BiLSTM (Bi-directional Long Short-Term Memory) is composed of forward LSTM and backward LSTM. It controls the transmission state through a gating mechanism and is often used to model contextual information. It can well capture the information of each position in the text sequence.

[0036] BERT (Bidirectional Encoder Representation from Transformers) is a pre-trained model that uses the Transformer Encoder block for connection and is a typical bidirectional encoding model.

[0037] The present application embodiment provides a text error correction method, such as Figure 1 As shown, the steps of the method include:

[0038] Step 102, obtaining text data of a target domain and determining a first text to be corrected according to a preset rule corresponding to the target domain;

[0039] It should be noted that the length of the first text to be corrected in the embodiment of the present application is greater than 1 and less than the maximum sequence length. The maximum sequence length is related to the target domain and / or model capability. The maximum sequence lengths corresponding to different domains may be different, and the maximum sequence lengths corresponding to different model capabilities may also be different.

[0040] Step 104, using a target model to process the first text to be corrected, wherein the target model is obtained by training an initial model with a set of data pairs, where the data pairs are vocabulary data pairs including vocabulary to be corrected and correct vocabulary; and

[0041] Step 106: When the first text to be corrected includes the vocabulary to be corrected, output a correction result through the target model, wherein the correction result includes the first text to be corrected and the correct vocabulary corresponding to the vocabulary to be corrected.

[0042] Through the above steps 102 to 106, the text data of the target field can be obtained and the first text to be corrected can be determined according to the preset rules corresponding to the target field, and then the first text to be corrected can be processed by the target model. When the first text to be corrected includes the vocabulary to be corrected, the correction result can be output through the target model, that is, the correction of the first text to be corrected can be achieved through the target model, which can cover all grammatical errors, thereby solving the problem of low correction efficiency in the prior art that only clear syntax and grammar rules are corrected.

[0043] In an optional implementation manner of the embodiment of the present application, the words to be corrected involved in the embodiment of the present application are determined by a confusion set, and the correct words are determined by a data set; based on this, before the first text to be corrected is processed using the target model involved in step 102, the method of the embodiment of the present application may also include:

[0044] Step 11, collecting multiple keywords in the target field and constructing a data set using the multiple keywords;

[0045] It should be noted that in the embodiment of the present application, all Chinese characters can be sorted out first as a basic data set, and then the multiple keywords can be determined based on different fields. Since all Chinese characters are used as the basic data set, the multiple keywords can cover the entire field as much as possible, such as commonly used keywords, and also uncommon keywords can be collected in the keywords.

[0046] Step 12, determining homophones associated with each of the multiple keywords to obtain a set of homophones; wherein a homophone is a word with the same initial consonant and final vowel;

[0047] Step 13, determining similar characters associated with each character in the plurality of keywords based on the four-corner codes, and obtaining a set of similar characters; and

[0048] Step 14, constructing a confusion set using the homophone set and the similar-looking character set.

[0049] Based on the above steps 11 to 14, it can be seen that the confusion set is mainly constructed from two aspects: homophonic characters and similar-looking characters. Among them, homophonic characters refer to Chinese characters in modern Chinese that have exactly the same initials and finals, and the tones do not necessarily need to be exactly the same. For example, the group of characters "文问紊蚊闻纹吻瘟温稳" are homophonic characters, and their pronunciation is all "wen". Similar-looking characters refer to Chinese characters that are similar in form but different in meaning, and have the same radical or main body of the character. For example, the group of characters "青清晴睛蜻猜情倩" are similar-looking characters, and the similar text main body is "青". The above are just examples, and corresponding settings can be made according to the actual situation in specific application scenarios. Compared with the method of error correction based on grammar rules in the prior art, using homophonic characters and similar-looking characters to determine whether the text of the present application needs error correction is more direct and has a higher effect, and can avoid the limitations brought by grammar rules.

[0050] Further, for the method of constructing a confusion set by using the homophonic character set and the similar-looking character set involved in the above step 14, in the embodiments of the present application, it may include:

[0051] Step 21, when the text length of the first keyword in the dataset meets the first preset length standard, randomly select a character in the first keyword and use the homophonic character set to replace it with a homophonic character or use the similar-looking character set to replace it with a similar-looking character to obtain a word to be error-corrected;

[0052] Step 22, when the text length of the second keyword in the dataset meets the second preset length standard, randomly select two characters in the second keyword and use the homophonic character set to replace them with homophonic characters or use the similar-looking character set to replace them with similar-looking characters to obtain a word to be error-corrected; and

[0053] Step 23, use the words to be error-corrected to construct a confusion set.

[0054] For the above steps 21 to 23, in a specific example, if the text length of the current first keyword is 2 or 3 (the first preset length standard), then among the three positions of the word, only one position is randomly selected each time to extract a homophonic character or a similar-looking character for replacement; if the text length of the current word is greater than 3 (the second preset length standard), then two positions in the word are randomly selected to extract homophonic characters or similar-looking characters for replacement. It should be noted that the above first preset length standard and second preset length standard are just examples, and the specific values can be set according to the actual situation. It can be seen that the construction of the confusion set in the embodiments of the present application does not limit grammar rules, but directly replaces the text, so as to avoid the limitations brought by grammar rules.

[0055] In yet another alternative of the embodiments of the present application, before processing the first text to be corrected using the target model, the method steps of the embodiments of the present application may further include:

[0056] Step 31, constructing an error detection network of the target model according to a bidirectional long short-term memory (BiLSTM) model;

[0057] Step 32, constructing an error detection network and an error correction model of the initial model according to a Bidirectional Encoder Representations from Transformers (BERT) model;

[0058] Step 33, setting the maximum sequence length for model training according to the target domain;

[0059] Step 34, training the initial model according to the data pair set to obtain the target model.

[0060] For the above steps 31 to 34, in a specific example, as Figure 2 shown, the input of the error detection network of the initial model is the input in the BERT model format, that is, the input vector is jointly composed of the word vector, position vector, and segment vector of the current text string. If the data object of the present application is a Chinese short text search term, that is, the text has a short length, so in the embodiments of the present application, the maximum sequence length for model training is set to 16, the unit of the word vector is a single character, and the word vector is obtained through the ID word table of each character of the BERT Chinese pre-training model. For example, the ID corresponding to "菲" in the BERT word table is 5839, the partial value of the segment vector within the text is 1, the position vector is randomly initialized, and if the text length is less than the maximum sequence length, the word vectors, segment vectors, and position vectors of the redundant parts are filled with 0.

[0061] Based on this, as Figure 3 shown, in the error detection network in the embodiments of the present application, the process of constructing the mask vector can calculate the error probability for each character of the text sequence. The error probability is a value between 0 and 1, and the masked part is obtained by multiplying the input vector by (1 - the error probability). If the error probability of the current character is very high, that is, the error probability is close to 1, the larger the amplitude of the input vector is masked. If the error probability of the current character is 0, the value of the masked vector is the value of the input vector.

[0062] In a specific example, as Figure 4As shown in the figure, the error correction network is implemented by BERT, the input vector is the mask vector sequence calculated by the error detection network, and the output is the character sequence. BERT consists of a 12-layer encoder structure, each layer of the encoder contains a multi-headed self-attention, takes the mask vector sequence as input, and performs encoding operation through the 12-layer encoder, and then performs a full connection operation, and finally uses SoftMax as the activation function to output the correction probability of the character, and then selects the character with the highest probability from the candidate character list as the correct character after the error correction at that position.

[0063] Based on the above steps 31 to 34, the method of processing the first text to be corrected using the target model involved in step 104 of the present application may further include:

[0064] Step 41, obtaining an input vector corresponding to the first text, wherein the input vector is jointly formed by a word vector, a position vector and a segment vector of the first text;

[0065] Step 42, calculating the error probability of each character in the input vector based on the error detection network to obtain a calculation result;

[0066] Step 43, multiplying the difference between 1 and the calculated result by the input vector as the input of the error correction network of the target model;

[0067] Step 44: outputting multiple correction probabilities corresponding to the character to be corrected based on the error correction network, and selecting a corresponding character with the highest probability among the multiple correction probabilities from the candidate character list as a correction character for the character to be corrected in the first text.

[0068] For the above steps 41 to 43, in a specific example, combined with Figures 2 to 4 It can be seen that according to the error probability calculation, the corresponding multiple correction probabilities of the character to be corrected are determined, and the corresponding character with the largest probability among the multiple correction probabilities is selected from the candidate character list as the correction character of the character to be corrected in the first text. In other words, corresponding error correction processing can be performed on each character in the first text, instead of performing corresponding error correction processing based on the grammatical rules in the first text, and the processing efficiency is higher and more accurate.

[0069] In an optional implementation manner of the embodiment of the present application, in the case where the first text to be corrected includes the vocabulary to be corrected involved in step 106, the method of outputting the correction result through the target model may further include:

[0070] Step 51, when there is at least one correction character, output the error correction result; where the error correction result includes: a tuple composed of the first text sequence, the incorrect text character, the correct text character, and the start index of the position of the incorrect text character in the first text sequence.

[0071] It can be seen that the first text is a search entity word to be corrected, and the output result is a tuple composed of the input text sequence to be error corrected, the incorrect text character, the correct text character, and the start index of the position of the incorrect text character in the text sequence to be error corrected. Multiple combinations of misspelled words form an array. For example: "The first text to be error corrected is 'dress group', and the obtained error correction result is 'dress group, error: [(group, skirt, 3)]'. The above is only an example and does not constitute a limitation to this application.

[0072] The following uses the specific implementation manners of the embodiments of this application to illustrate this application. Taking the e-commerce field as an example, based on this, a model error correction method for search query words in the e-commerce field is provided, as Figure 5 shown, the steps of this method include:

[0073] Step 501, set model hyperparameters;

[0074] Among them, the hyperparameters involved in this specific implementation manner include: the data path of the homophone confusion set, the data path of the similar-looking character confusion set, the training and test data set paths, the model path, Batch size, Epoch, the maximum text sequence length, the BERT pre-trained model, the number of units of LSTM, Dropout, Epoch, learning rate, optimizer, regularization term, whether to use GPU, etc.

[0075] Step 502, organize the entire set of Chinese characters as the basic data set;

[0076] Step 503, perform Chinese character to pinyin conversion based on pypinyin (a Chinese character pinyin conversion tool, Python-Pinyin), and aggregate the same pinyins to obtain homophone combinations, thereby obtaining the homophone confusion set;

[0077] Step 504, perform Chinese character to four-corner code conversion based on char_featurizer, and then judge and aggregate to obtain similar-looking character combinations under certain rules, thereby obtaining the similar-looking character confusion set;

[0078] Among them, char_featurizer is a Chinese character feature extraction tool that can extract information such as the pronunciation (including initials, finals, tones), glyphs (radicals, components), and four-corner symbols of Chinese characters.

[0079] In addition, in the embodiments of the present application, the Chinese character confusion set is mainly constructed from two aspects: homophonic characters and similar-looking characters. Among them, homophonic characters refer to Chinese characters in modern Chinese that have exactly the same initials and finals, and the tones do not necessarily have to be exactly the same. For example, the group of characters "文问紊蚊闻纹吻瘟温稳" are homophonic characters, and their pronunciation is all "wen". Similar-looking characters refer to Chinese characters that are similar in form but different in meaning, and have the same radical or main body of the character. For example, the group of characters "青清晴睛蜻猜情倩" are similar-looking characters, and the similar text main body is "青".

[0080] Among them, the construction of homophonic characters is carried out using the pypinyin library in python. The pypinyin library can obtain the pinyin corresponding to the input Chinese character according to the input text. In this application, the tone-free method can be adopted. Write a script to traverse all Chinese character files found on the Internet, convert Chinese characters to pinyin through pypinyin, then aggregate homophonic characters, and write them into a file to generate a homophonic character set file. For example, in the above example, the group of characters "文问紊蚊闻纹吻瘟温稳" are homophonic characters, and their pronunciation is all "wen", and they are regarded as homophonic characters.

[0081] Among them, the construction of similar-looking characters is carried out by parsing the four-corner codes of Chinese characters using the char_featurizer library. The char_featurizer obtains the four-corner codes of Chinese characters according to the input text respectively. If there are more than three pairs of Chinese character groups with the same relative positions in a group of four-corner codes, they are considered similar-looking characters. Aggregate the similar-looking characters, and then write them into a file to generate a homophonic character set file. For example, the four-corner codes of the above example "青清晴睛蜻猜情倩" are (50227), (30127), (65027), (65027), (55127), (45227), (95027), (25227) respectively. There are two groups of codes with the same relative positions, namely 527 and 127, so this group is considered a group of similar-looking characters.

[0082] Step 505, search for and sort out e-commerce search query words of a preset number of categories, and construct a data set using the homophonic character set and the similar-looking character set based on the method of text replacement with no more than two misspelled characters.

[0083] If the application field of the embodiment of the present application is the vertical field of e-commerce search, the data object is the e-commerce search item vocabulary. The search object in the e-commerce search field is the e-commerce short text data. In the implementation process of this application, the keywords of 10 preset categories such as beauty, clothing, electrical appliances, home furnishings, stationery, maternal and child, accessories, medical and health care, accessories, luggage, etc. are sorted and collected. The relevant vocabulary includes product names, brand names, abbreviations, common words, adjectives of related categories, etc. For example, taking "Little Brown Bottle Eye Cream" as an example, the relevant content may include Little Brown Bottle Eye Cream, Little Brown Bottle, etc. The abbreviation and common words of "Repair Essence" are "Red Waist", and the description of clothing adjectives such as "Little Chanel Style" and "Boyfriend Style" in the clothing category. 10 kinds of product category search terms, among which, 1) Clothing category: sweatshirts, jeans, Little Chanel style, jackets, sweaters, suits, Martin boots, beautiful leg socks, dresses, high heels, one-shoulder, sweaters, stockings. 2) Electrical appliances: electric fans, televisions, refrigerators, computers, notebooks, keyboards, mice, electric ovens, air conditioners, rice cookers, wall breakers, electric baking pans, soybean milk makers, electric pressure cookers. 3) Furniture: desks, dining tables, chairs, beds, bookcases, bedside tables, coffee tables, chopping boards, kitchen knives, frying pans, chopsticks, spoons, water ladles, shelves, table lamps, sofas, pillows, carpets, tatami mats, cushions. 4) Stationery: notebooks, books, pencils, pens, pencil cases, pencil cases, colored pens, grid books, brushes, sheep hair brushes, wolf hair brushes, ink, rice paper, copybooks, folders, sticky notes, signature pens, memo pads, refills, and grid books. 5) Maternal and infant products: diapers, milk powder, baby bottles, baby powder, bibs, breast pumps, baby food machines, pull-up pants, safety seats, breast pads, baby walkers, crawling mats, diapers, cribs, soothing teethers, bibs, scooters, rocking chairs, safety fences, twisting cars. 6) Food products: duck necks, fat beef, spicy hot pot, tiger skin chicken feet, egg yolk crisps, snail noodles, hot pot base, osmanthus cake, luncheon meat, sardines, steamed buns. 7) Medical and health products: bird's nests, cordyceps, blood pressure monitors, blood glucose meters, deer antlers, thermometers, snow clam, ventilators, dendrobium, oxygen generators, probiotics, collagen, enzymes, L-carnitine, vitamins, wolfberries. 8) Accessories: earrings, watches, ear wires, bracelets, necklaces, hats, sunglasses, rings, ear studs, pinky rings. 9) Luggage and bags: tote bags, dumpling bags, bucket bags, backpacks, messenger bags, wallets, chest bags, suitcases, trolley cases, handbags, liner bags, cloud bags. 10) Digital products: SLR cameras, camcorders, speakers, webcams, voice recorders, cameras, scanners, printers, monitors, consoles, tripods, PDAs, data cables, Bluetooth headsets, CDs, game consoles, adapters.

[0084] It can be seen that the construction of the training data set and the test data set in the embodiment of the present application is by aggregating the above 11 categories of e-commerce product vocabulary data, and then traversing the Chinese characters in each vocabulary, and using the constructed confusion set of homophones and similar characters for replacement processing. If the text length of the current vocabulary is 1, it is discarded; if the text length of the current vocabulary is 2 or 3, then only one position is selected at a time from the three positions of the word to randomly extract homophones or similar characters for replacement; if the text length of the current vocabulary is greater than three, then two positions are selected in the word to randomly extract homophones or similar characters for replacement.

[0085] Based on this, through the construction of the wrong character confusion set and the construction of the data set, the vocabulary data pairs of the words to be corrected and the correct words similar to the format of the SIGHAN data set are generated as the experimental data of the search terms in the e-commerce field of this application. The training set and the test set are divided according to the ratio of 9:1. The data format is shown in Table 1, where the #src column is the text to be corrected and the target column is the target correct text.

[0086] #src Target Electric pancake stall Electric baking pan Tea tea

[0087] Table 1

[0088] Step 206, using a BiLSTM+BERT-based model structure to implement the error detection network and the error correction network respectively, and then perform joint training, using the accuracy and recall of the error detection and error correction parts as verification indicators, and respectively calculating the loss function, and weighting to obtain the final loss function;

[0089] Among them, the transformation of the BERT model structure uses the BiLSTM+BERT model as the baseline model. Since BERT has the relevant characteristics of masked language modeling, but cannot detect whether erroneous text appears in each position, this application uses BERT in the error text correction part, and the BiLSTM model is used for typo detection in the error detection part, which can better predict the probability of erroneous text appearing in each position, and then use the probability of erroneous text appearing in the current position to implement the masking operation of the position, that is, the masking of the current position of the original BERT is complete masking, and the probability value is 1. When the probability is no longer 1, the masking situation of each position can be input into the error correction network. Through joint training, the model can perform more accurate error detection and error correction.

[0090] The error detection network is implemented by the BiLSTM model. The BiLSTM is composed of a forward LSTM and a backward LSTM. The bidirectional LSTM structure can better capture bidirectional semantic dependencies. The transmission state is controlled by the gating mechanism, which is often used to model context information and can capture the information of each position in the text sequence. Further error detection networks such as Figure 2 shown.

[0091] The input of the error detection network is in the format of the input of the BERT model, that is, the word vector, position vector, and segment vector of the current text string jointly constitute the input vector. Since the data object in this application is a Chinese short text search term, which has the characteristic of short text length, the maximum sequence length for model training is set to 16 in this application. The unit of the word vector is a single character, and the word vector is obtained through the ID vocabulary of each character of the BERT Chinese pre-trained model. For example, the ID corresponding to "菲" in the BERT vocabulary is 5839. The partial value of the segment vector within the text is 1, and the position vector is randomly initialized. If the text length is less than the maximum sequence length, the word vectors, segment vectors, and position vectors of the redundant parts are filled with 0.

[0092] It can be seen that the error detection network uses the hidden state output of the gating unit of each layer of BiLSTM as the input feature vector of the next layer, uses the sigmoid function as the activation function, and calculates the conditional probability as the output of the network. In addition, in the embodiments of this application, an error detection network implemented by a BiLSTM model can calculate the error probability for each character of the text sequence. The error probability is a value between 0 and 1. The masked part is obtained by multiplying the input vector by (1 - the error probability). If the error probability of the current character is very high, that is, the error probability is closer to 1, the larger the amplitude of the input vector is masked. If the error probability of the current character is 0, the value of the masked vector is the value of the input vector. Figure 3 This is the process of constructing the masked vector.

[0093] In this application, the error correction network is implemented by BERT. The input vector is the masked vector sequence calculated by the error detection network, and the output is a character sequence. BERT consists of 12 layers of encoder structures. Each layer of encoder contains a multi-head self-attention. Using the masked vector sequence as the input, it undergoes encoding operations through 12 layers of encoders, then a fully connected operation, and finally uses SoftMax as the activation function to output the correction probability of the character. Then, the character with the highest probability is selected from the candidate character list as the correct character after error correction at this position.

[0094] As Figure 4 shown, it is the model of the error correction network implemented by BERT. Based on this, training, testing, and verification are carried out. In addition, the model structure in the embodiments of this application is jointly trained using BiLSTM and the BERT Chinese pre-trained model, which are respectively responsible for the detection of error texts and the correction of misaligned texts. The training data consists of pairs of text sequences to be error-corrected and correct text sequences. The optimization objectives of the model are distinguished according to the error detection network and the error correction network respectively, and the overall learning objective is the weighted sum of two learning objectives.

[0095] In a specific example, the inventors of the present application adopted the TensorFlow framework in model training. For the BERT pre-trained model, the Chinese pre-trained model chinese_L-12_H-768_A-12 was selected. The maximum sequence length was set to 16, the batch_size of the training data was set to 128, the learning rate was 0.0003, the number of LSTM hidden units was set to 128, the optimizer was adam, and the total training data was about 2 million short text search terms in the e-commerce field. The model converged at 30 epochs. The model was evaluated using precision and recall as evaluation metrics for the error detection network and the error correction network respectively. The precision of the error detection network could reach 78.8%, and the recall was 59.7%. The precision of the error correction network was 74.2%, and the recall was 48.9%.

[0096] Based on this, if the input data is a search entity word to be corrected, the output result is a tuple composed of the text sequence to be error-corrected, the error text characters, the correct text characters, and the start index of the position of the error text characters in the text sequence to be error-corrected. Multiple combinations of misspelled words form an array. For example: "The input text to be error-corrected is 'dress group', and the error correction result is 'dress group, error: [(group, skirt, 3)]'."

[0097] In summary, through the construction scheme of the text error correction data set in the embodiments of the present application, the problem of insufficient text error correction data sets in the current vertical field can be solved, and the difficulty of applying deep learning in the field of text error correction can be reduced. Moreover, the text error correction scheme of BiLSTM + BERT strengthens the error detection ability and has higher error correction accuracy compared with a single sequence model.

[0098] Corresponding to the above Figure 1 , the embodiments of the present application further provide an error correction device for text, as Figure 6 described, including:

[0099] An acquisition module 62, configured to acquire text data in the target field and determine the first text to be error-corrected according to a preset rule corresponding to the target field;

[0100] A processing module 64, configured to process the first text to be error-corrected by using a target model, where the target model is obtained by training an initial model with a data pair set, and the data pair is a vocabulary data pair including a word to be error-corrected and a correct word; and

[0101] An error correction module 66, configured to output an error correction result through the target model when the first text to be error-corrected includes a word to be error-corrected, where the error correction result includes the first text to be error-corrected and the correct word corresponding to the word to be error-corrected.

[0102] Through the device in the embodiment of the present application, text data of the target field can be obtained and the first text to be corrected can be determined according to the preset rules corresponding to the target field, and then the first text to be corrected can be processed by using the target model. When the first text to be corrected includes the vocabulary to be corrected, the correction result can be output through the target model, that is, the correction of the first text to be corrected can be achieved through the target model, which can cover all grammatical errors, thereby solving the problem of low correction efficiency in the prior art that only clear syntax and grammar rules are corrected.

[0103] Optionally, the vocabulary to be corrected in the embodiment of the present application is determined by a confusion set, and the correct vocabulary is determined by a data set; based on this, the device in the embodiment of the present application also includes: a first construction module, which is used to collect multiple keywords in the target field and construct a data set using multiple keywords before using the target model to process the first text to be corrected; a first determination module, which is used to determine the homophones associated with each word in the multiple keywords to obtain a homophone set; wherein the homophones are words with exactly the same initial consonants and finals; a second determination module, which is used to determine the morphologically similar words associated with each word in the multiple keywords based on four-corner coding to obtain a morphologically similar word set; and a second construction module, which is used to construct a confusion set using the homophone set and the morphologically similar word set.

[0104] Optionally, the second construction module in the embodiment of the present application includes: a first replacement unit, which is used to randomly select a word in the first keyword and replace the homophone with a homophone set or replace the pictographic characters with a pictographic character set to obtain a vocabulary to be corrected when the text length of the first keyword in the data set meets the first preset length standard; a second replacement unit, which is used to randomly select two words in the second keyword and replace the homophone with a homophone set or replace the pictographic characters with a pictographic character set to obtain a vocabulary to be corrected when the text length of the second keyword in the data set meets the second preset length standard; and a construction unit, which is used to construct a confusion set using the vocabulary to be corrected.

[0105] Optionally, the device in the embodiment of the present application may also include: a third construction module, used to construct an error detection network of the target model according to the bidirectional long short-term memory BiLSTM model before using the target model to process the first text to be corrected; a fourth construction module, used to construct an error detection network of the initial model and an error correction model of the initial model according to the converter-based bidirectional encoding representation BERT model; a setting module, used to set the maximum sequence length of model training according to the target domain; a training module, used to train the initial model according to the data pair set to obtain the target model.

[0106] Optionally, the processing module 64 in the embodiment of the present application may further include: an acquisition unit, used to acquire an input vector corresponding to the first text, wherein the input vector is jointly composed of a word vector, a position vector and a segment vector of the first text; a calculation unit, used to calculate the error probability of each character in the input vector based on an error detection network to obtain a calculation result; an input unit, used to multiply the difference between 1 and the calculation result by the input vector as an input to the error correction network of the target model; a processing unit, used to output multiple correction probabilities corresponding to the character to be corrected based on the error correction network, and select the character with the highest probability among the multiple correction probabilities from the candidate character list as the correction character of the character to be corrected in the first text.

[0107] Optionally, the error correction module 66 in the embodiment of the present application may further include: an output unit, configured to output an error correction result when there is at least one correction character.

[0108] Optionally, the error correction result in the embodiment of the present application includes: a tuple consisting of a first text sequence, an erroneous text character, a correct text character, and a start index of the position of the erroneous text character in the first text sequence.

[0109] The present application also provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704;

[0110] Memory 703, used for storing computer programs;

[0111] Processor 701 is used to execute the program stored in memory 703 to implement Figure 1 The method steps in the method have similar functions and will not be repeated here.

[0112] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0113] The communication interface is used for communication between the above terminal and other devices.

[0114] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0115] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0116] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the text error correction method described in any of the above embodiments.

[0117] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the text error correction method described in any one of the above embodiments.

[0118] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0119] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0120] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0121] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A text error correction method, characterized in that: include: Acquire text data of a target domain and determine a first text to be corrected according to a preset rule corresponding to the target domain; Processing the first text to be corrected using a target model, wherein the target model is obtained by training an initial model with a set of data pairs, wherein the data pairs are vocabulary data pairs including vocabulary to be corrected and correct vocabulary; and In a case where the first text to be corrected includes the vocabulary to be corrected, outputting a correction result through the target model, wherein the correction result includes the first text to be corrected and a correct vocabulary corresponding to the vocabulary to be corrected; Before using the target model to process the first text to be corrected, the method further includes: constructing an error detection network of the target model according to a bidirectional long short-term memory BiLSTM model; constructing an error detection network of an initial model and an error correction model of the initial model according to a bidirectional encoding representation BERT model based on a converter; setting a maximum sequence length for model training according to the target domain; and training the initial model according to a data pair set to obtain the target model; Among them, the use of the target model to process the first text to be corrected includes: obtaining an input vector corresponding to the first text, wherein the input vector is jointly composed of a word vector, a position vector and a segment vector of the first text; calculating the error probability of each character in the input vector based on the error detection network to obtain a calculation result; multiplying the difference between 1 and the calculation result by the input vector as the input of the error correction network of the target model; outputting multiple correction probabilities corresponding to the character to be corrected based on the error correction network, and selecting the character with the highest probability among the multiple correction probabilities from the candidate character list as the correction character of the character to be corrected in the first text.

2. The method according to claim 1, characterized in that The to-be-corrected vocabulary is determined by a confusion set, and the correct vocabulary is determined by a data set; before the target model is used to process the first text to be corrected, the method further includes: Collecting a plurality of keywords in the target field, and constructing the data set using the plurality of keywords; Determine homophones associated with each of the multiple keywords to obtain a set of homophones; wherein the homophones are words with identical initial consonants and finals; Determine the similar characters associated with each character in the plurality of keywords based on the four-corner codes to obtain a set of similar characters; and The confusion set is constructed by using the homophone set and the similar-looking character set.

3. The method according to claim 2, characterized in that The step of constructing the confusion set by using the homophone set and the similar-looking character set includes: When the text length of the first keyword in the data set meets the first preset length standard, randomly select a word in the first keyword and replace the homophone with the homophone set or replace the similar word with the similar word set to obtain the vocabulary to be corrected; When the text length of the second keyword in the data set meets the second preset length standard, randomly selecting two characters in the second keyword and replacing the homophones with the homophones set or replacing the similar characters with the similar characters set to obtain the vocabulary to be corrected; and The confusion set is constructed using the words to be corrected.

4. The method according to claim 1, characterized in that: In a case where the first text to be corrected includes the vocabulary to be corrected, outputting the correction result through the target model includes: When there is at least one of the correction characters, the error correction result is output.

5. The method according to claim 4, characterized in that The error correction result includes: a tuple consisting of the first text sequence, an erroneous text character, a correct text character, and a start index of a position of the erroneous text character in the first text sequence.

6. A text error correction device, characterized in that: include: An acquisition module, used to acquire text data of a target field and determine a first text to be corrected according to a preset rule corresponding to the target field; a processing module, configured to process the first text to be corrected using a target model, wherein the target model is obtained by training an initial model with a set of data pairs, wherein the data pairs are vocabulary data pairs including vocabulary to be corrected and correct vocabulary; and an error correction module, configured to output an error correction result through the target model when the first text to be corrected includes the word to be corrected, wherein the error correction result includes the first text to be corrected and a correct word corresponding to the word to be corrected; A third construction module is used to construct an error detection network of the target model according to a bidirectional long short-term memory BiLSTM model before using the target model to process the first text to be corrected; A fourth building module is used to build an error detection network of the initial model and an error correction model of the initial model according to the converter-based bidirectional encoding representation BERT model; The settings module is used to set the maximum sequence length for model training according to the target domain; A training module is used to train the initial model based on the data pair set to obtain the target model; Among them, the processing module includes: an acquisition unit, used to acquire an input vector corresponding to the first text, wherein the input vector is jointly composed of a word vector, a position vector and a segment vector of the first text; a calculation unit, used to calculate the error probability of each character in the input vector based on an error detection network to obtain a calculation result; an input unit, used to multiply the difference between 1 and the calculation result by the input vector as the input of the error correction network of the target model; a processing unit, used to output multiple correction probabilities corresponding to the character to be corrected based on the error correction network, and select the character with the highest probability among the multiple correction probabilities from the candidate character list as the correction character of the character to be corrected in the first text.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-5 when executing a program stored in a memory.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images