A training method for single-word typo recognition model and its training set generation method

Abstract

The present invention provides a method for generating a training set for a single-word typo recognition model, comprising: S1, obtaining a corpus, performing word segmentation processing and part-of-speech tagging on each sentence in the corpus, and obtaining a plurality of original word segmentation sequences, each of which includes a plurality of word segmentation items and the part of speech of each word segmentation item; S2, constructing labels for each word segmentation item based on the original word segmentation sequence, and obtaining a label sequence, wherein an initial label value indicates that the text corresponding to each word segmentation item is a correct text; S3, replacing a word segmentation item in the corresponding original word segmentation sequence with an incorrect word similar to the corresponding word, and obtaining a word segmentation sequence with incorrect words, and changing the label value corresponding to the word segmentation item whose word is replaced in the corresponding label sequence to indicate that the corresponding word is an incorrect text; S4, repeating step S3, and constructing a training set including a plurality of samples, each of which includes a word segmentation sequence and a corresponding label sequence, and the word segmentation sequence is an original word segmentation sequence or a word segmentation sequence with incorrect words.

Description

Technical Field

[0001] The present invention relates to the field of automatic recognition of Chinese character typos, and more specifically, to a method for training a single-character typo recognition model and a method for generating a training set therefor. Background Art

[0002] With the rapid development of Internet and mobile Internet technologies, the pace of people's life and work has been continuously accelerating. When typing Chinese characters in daily work and study, typos often occur from time to time. In fact, the recognition and proofreading of Chinese character typos consume a large amount of manpower and financial resources in all walks of life. The recognition of Chinese character typos can be divided into word-level typo recognition and character-level typo recognition.

[0003] Good results have been achieved in the research of word-level typo recognition. However, single characters are often polysemous, have a higher usage frequency than words, have complex context, and it is difficult to collect error training samples. For example, in the sentence "The new governments of the two countries should work together", the single character "界" is a typo corresponding to the character "届". Character-level typo recognition of Chinese characters is very difficult, and the existing models have unsatisfactory recognition effects on single characters. To obtain a model with good recognition effect on single characters, it is necessary to consider the complex usage of single characters in different contexts, and there are various single characters similar to the corresponding single character. At the same time, it is necessary to overcome the difficulties in collecting and constructing training samples.

[0004] Therefore, there is an urgent need for a method for generating a training set, so that the model trained by this training set can not only improve the recognition accuracy of single characters, but also improve its generalization ability. Summary of the Invention

[0005] Therefore, the object of the present invention is to overcome the above-mentioned defects of the prior art, and provide a method for training a single-character typo recognition model and a method for generating a training set therefor.

[0006] The object of the present invention is achieved by the following technical solutions:

[0007] In a first aspect of the present invention, a method for generating a training set for a single-word typo recognition model is provided, the method comprising: S1, obtaining a corpus, performing word segmentation processing and part-of-speech tagging on each sentence in the corpus, and obtaining a plurality of original word segmentation sequences, wherein each original word segmentation sequence comprises a plurality of word segmentation items and the part of speech of each word segmentation item; S2, constructing labels for each word segmentation item based on the original word segmentation sequence, and obtaining a label sequence, wherein an initial label value indicates that the text corresponding to each word segmentation item is correct text; S3, replacing a word segmentation item in the corresponding original word segmentation sequence with an incorrect word similar to the corresponding word, and obtaining a word segmentation sequence with incorrect words, and changing the label value corresponding to the word segmentation item whose word is replaced in the corresponding label sequence to indicate that the corresponding word is incorrect text; S4, repeating step S3, and constructing a training set comprising a plurality of samples, wherein each sample comprises a word segmentation sequence and a corresponding label sequence, and the word segmentation sequence is an original word segmentation sequence or a word segmentation sequence with incorrect words.

[0008] In some embodiments of the present invention, the replacing of the segmentation items of the original segmentation sequence that are single words with erroneous single words similar to the corresponding single words to obtain the segmentation sequence with erroneous characters includes: for each part of speech, obtaining multiple original segmentation sequences including segmentation items of the corresponding part of speech for the single words; according to the part of speech of the segmentation item, replacing the segmentation item of the corresponding part of speech for the single words in each original segmentation sequence with one of the multiple erroneous single words similar to the corresponding single word to obtain multiple segmentation sequences with erroneous characters.

[0009] In some embodiments of the present invention, the replacing of a word segmentation item with an incorrect word similar to the corresponding word is to refer to a plurality of word set confusion pairs and perform the replacement according to the part of speech of the segmentation item, wherein the generation method of the plurality of word set confusion pairs includes: for each part of speech, constructing a word confusion pair set based on a word of the corresponding part of speech and a plurality of incorrect words similar to the corresponding word, wherein the word confusion pair set includes a plurality of word confusion pairs constructed by the corresponding word and an incorrect word similar to the word; constructing a word set confusion pair set of the corresponding part of speech based on a plurality of word confusion pairs of a plurality of words of the same part of speech, and obtaining a plurality of word set confusion pairs of different parts of speech.

[0010] In some embodiments of the present invention, the multiple erroneous words similar to the corresponding words include: multiple erroneous words similar to the corresponding words in shape, multiple erroneous words similar to the corresponding words in pronunciation, multiple erroneous words similar to the corresponding words in meaning, or any different combination thereof.

[0011] In some embodiments of the present invention, the changing of the label value corresponding to the segmentation item in which a single word is replaced in the corresponding label sequence to indicate that the corresponding single word is an erroneous text includes: according to a predetermined label window size, changing the label value of the segmentation item in which a single word is replaced in the corresponding label sequence to indicate that the corresponding single word is an erroneous text, and changing the label values ​​corresponding to multiple segmentation items before and after the segmentation item to indicate that the text of the multiple segmentation items before and after the segmentation item is an erroneous text.

[0012] In some embodiments of the present invention, the corpus is a Chinese corpus, a Japanese corpus, or a Korean corpus.

[0013] In some embodiments of the present invention, the method for generating a training set for a single-word typo recognition model also includes: abstracting the segmentation items of corresponding parts of speech in the original segmentation sequence and the segmentation sequence with typos, including replacing the text of all segmentation items whose parts of speech are person names, place names, time and organization names with unified person name symbols, place name symbols, time symbols and organization name symbols respectively.

[0014] In a second aspect of the present invention, a method for training a single-word typo recognition model is provided, comprising iterating the single-word typo recognition model multiple times in the following manner: obtaining a training set generated using the method described in the first aspect of the present invention; using the training set to train the single-word typo recognition model to identify whether the single word corresponding to the single-word segmentation item in the segmentation sequence of the sample is a typo, and obtaining a prediction result sequence; calculating a total loss value based on the prediction result sequence and label sequence of the corresponding sample; and updating the parameters of the single-word typo recognition model according to the total loss value.

[0015] In a third aspect of the present invention, a single-word typo recognition system is provided, comprising: a word segmentation module, configured to perform word segmentation processing on an input sentence to obtain a word segmentation sequence to be recognized; a single-word typo recognition model obtained according to the training method of the second aspect of the present invention, configured to process the word segmentation sequence to be recognized and output a prediction result sequence, wherein the prediction result sequence indicates whether the single word corresponding to the word segmentation item in the word segmentation sequence is a typo.

[0016] In a fourth aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the method described in the first and second aspects of the present invention by executing the executable instructions.

[0017] Compared with the prior art, the advantages of the present invention are:

[0018] 1. In the method for generating a training set of the present invention, the segmentation items of a single word in the original segmentation sequence are replaced with an incorrect single word similar to the corresponding single word, and an original segmentation sequence includes multiple segmentation items of a single word, and there are multiple incorrect single words similar to the corresponding single word, that is, an original segmentation sequence can obtain multiple different segmentation sequences with incorrect words, and a label indicating whether the text of the segmentation item is an incorrect text is constructed for each segmentation item in the original segmentation sequence and the segmentation sequence with incorrect words. Each segmentation sequence with incorrect words in the training set includes an incorrect single word. The training set obtained in this way is used to train the model, which can greatly improve the model's recognition accuracy for incorrect single words.

[0019] 2. The present invention generates multiple corresponding word segmentation sequences with wrong characters for each part of speech. The generated multiple word segmentation sequences with wrong characters are rich and diverse, which solves the problem that the frequency of occurrence of words with some parts of speech is low, resulting in uneven categories of wrong words in the generated word segmentation sequences with wrong characters, making the model's recognition effect on words with some parts of speech poor.

[0020] 3. When generating corresponding multiple word segmentation sequences with typos for individual words of each part of speech, the present invention refers to a plurality of confusion pairs of individual words with different parts of speech and replaces the individual words of the corresponding part of speech, thereby solving the problem of difficulty in constructing samples of word segmentation sequences with typos and low efficiency. Among the confusion pairs of individual words with multiple different parts of speech, the confusion pairs of individual words of each part of speech include a variety of confusion pairs of individual words that are easily confused with the pronunciation, glyph structure, and meaning of the corresponding individual words, enriching the samples of word segmentation sequences with typos and improving the generalization performance of the model.

[0021] 4. The present invention not only changes the label of the segmentation item with the wrong word in the segmentation sequence to indicate that the text of its segmentation item is wrong, but also changes the labels of several segmentation items before and after the segmentation item with the wrong word to indicate that the text of its segmentation item is wrong, according to the predetermined label window size. This has an important effect on improving the recall rate of single-word typo recognition. In addition, the segmentation items of some parts of speech with very low occurrence frequency in the training set are abstracted to solve the problem of poor generalization performance of the model caused by sparse data in some training sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0023] Figure 1 2. A structural diagram of a single-word typo recognition model according to an embodiment of the present invention;

[0024] Figure 2 A flow chart of a method for generating a training set for a single-word typo recognition model according to an embodiment of the present invention;

[0025] Figure 3Schematic diagram of a word segmentation sequence with typos and its label sequence according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0027] As mentioned in the background technology section, since words are often polysemous, the frequency of use of words is higher than that of words, the context of words is complex, and it is difficult to collect training samples for word errors. In addition, word errors related to parts of speech are very common in texts. This makes it more difficult to identify typos at the Chinese character level, and the effect is very unsatisfactory. In order to obtain a model with good single-word recognition effect, it is necessary to consider the complex usage of words in different contexts, and there are many different words similar to the corresponding words. At the same time, it is necessary to overcome the difficulties in collecting and constructing training samples.

[0028] Therefore, in general, the present invention relates to improvements in two aspects: 1) how to generate a sample set for a single-word typo recognition model; and 2) the structure of a single-word typo recognition model and its training method. Each of these will be described in detail below.

[0029] The present invention obtains a plurality of original segmentation sequences including a plurality of segmentation items by performing segmentation processing on each sentence in a corpus; constructs a label for each segmentation item in the original segmentation sequence to obtain a label sequence, wherein the initial label value of each label indicates that the text corresponding to each segmentation item is a correct text, and the original segmentation sequence and the corresponding label sequence are used as samples; a segmentation item that is a single word in the corresponding original segmentation sequence is replaced by an incorrect single word similar to the corresponding single word, and each segmentation sequence with incorrect characters obtained includes an incorrect single word, and the label value of the segmentation item that is an incorrect single word in the corresponding label sequence is changed to indicate that it is an incorrect text, thereby obtaining a sample including the segmentation sequence with incorrect characters and the corresponding label sequence; the above multiple samples constitute a rich training set, and a single-word typo recognition model is trained by the training set, effectively improving the recognition accuracy of the model for single-word typos.

[0030] Furthermore, an original word segmentation sequence includes multiple word segmentation items, and a word segmentation item in the corresponding original word segmentation sequence is replaced with an incorrect word similar to the corresponding word, and the incorrect words similar to the corresponding word include multiple words, such as multiple words with similar pronunciation, similar glyphs, and similar meanings. That is, an original word segmentation sequence can obtain multiple word segmentation sequences with incorrect words, so as to effectively improve the generalization ability of the model.

[0031] According to one embodiment of the present invention, a method for generating a training set for a single-word typo recognition model is provided. Figure 2 The method includes steps S1, S2, S3 and S4. In order to better understand the present invention, each step will be described in detail below in conjunction with specific embodiments.

[0032] Step S1: Obtain a corpus, perform word segmentation processing and part-of-speech tagging on each sentence in the corpus, and obtain multiple original word segmentation sequences, wherein each original word segmentation sequence includes multiple word segmentation items and the part of speech of each word segmentation item.

[0033] According to one embodiment of the present invention, the corpus can be a Chinese corpus, a Japanese corpus or a Korean corpus. The present invention uses the Chinese corpus to generate a training set, and the sentences in the Chinese corpus are recorded as S, and the M sentences are recorded as {S1, S2, ..., S M}, use word segmentation tools (such as word segmentation tool ICTCLAS (Institute of Computing Technology, Chinese Lexical Analysis System)) to perform word segmentation and part-of-speech tagging on each sentence in the Chinese corpus, forming multiple word segmentation sequences recorded as {T1, T2, ..., T M The word segmentation tool gives each word in the word segmentation sequence a part-of-speech tag, such as the word segmentation sequence T i =w i1 / pos(w i1 )w i2 / pos(w i2 )…w ij / pos(w ij ), and (1≤i≤M), (1≤j≤k), where w ij / pos(w ij ), called T i The jth participle item, pos(w ij ) is the word w ij For example, after the sentence S1 "a pair of exquisite bracelets" is segmented and tagged with parts of speech, a segmentation sequence "a / m a pair / v exquisite / a a / u bracelet / n" is obtained, which contains 5 segmentation items, whose parts of speech are m (indicating numerals), v (indicating verbs), a (indicating adjectives), u (indicating auxiliary words), and n (indicating nouns).

[0034] Step S2: construct labels for each segmentation item based on the original segmentation sequence to obtain a label sequence, wherein the initial label value indicates that the text corresponding to each segmentation item is the correct text.

[0035] According to one embodiment of the present invention, each segmentation item in the original segmentation sequence is assigned a corresponding label. Since the text of each segmentation item in the original segmentation sequence is correct, the assigned initial label value indicates that the text corresponding to each segmentation item is correct. At the same time, the corresponding label sequence Tag i =CC…C(1≤i≤M), the label C can be specified as 0, indicating that the initial label value is 0, and the text of each segmentation item is correct.

[0036] Step S3: Replace a single-word segmentation item in the corresponding original segmentation sequence with an incorrect single word similar to the corresponding single word to obtain a segmentation sequence with an incorrect word, and change the label value corresponding to the segmentation item whose single word is replaced in the corresponding label sequence to indicate that the corresponding single word is an incorrect word.

[0037] According to one embodiment of the present invention, a segmentation item that is a single word in an original segmentation sequence is replaced with an incorrect single word that is similar to the corresponding single word to obtain a segmentation sequence with incorrect characters, including: for each part of speech, obtaining multiple original segmentation sequences including segmentation items with the corresponding part of speech of the single word; according to the part of speech of the segmentation item, replacing the segmentation item with the corresponding part of speech of the single word in each original segmentation sequence with one of the multiple incorrect single words that are similar to the corresponding single word to obtain multiple segmentation sequences with incorrect characters.

[0038] According to one embodiment of the present invention, for example, with respect to the particle part of speech, a predetermined number of original segmentation sequences are obtained, and each original segmentation sequence includes a segmentation item of the particle part of speech that is a single word. According to the particle part of speech of the segmentation item, the segmentation item of the particle part of speech that is a single word in each original segmentation sequence obtained above is replaced with one of multiple incorrect single words similar to the corresponding single word, and the replaced segmentation item is recorded as w′ ij , that is, the word segmentation sequence T with typos i =w i1 / pos(w i1 )w i2 / pos(w i2 )…w′ ij / pos(w ij )…. Similarly, the above operation is performed for various parts of speech, such as adverbs, quantifiers, and conjunctions. The resulting multiple word segmentation sequences with typos are rich and diverse, and each part of speech generates a word segmentation sequence with typos. This solves the problem that the frequency of individual words in some parts of speech is low, resulting in poor recognition performance for these words. The following is Example 1 for particle parts of speech and Example 2 for adverbs.

[0039] Among them, Example 1: Original word segmentation sequence T i: A delicate bracelet, including the particle term "de / u" with the particle part-of-speech of a single character. Replace the particle term with a single character similar to the corresponding single character among multiple wrong single characters, such as replacing it with the wrong single character "de" or "di", etc., to obtain a segmented sequence with wrong characters: A delicate bracelet.

[0040] Example 2: The original segmented sequence T i : A delicate bracelet, including the adverb term "fu / v" with the adverb part-of-speech of a single character. Replace the adverb term with a single character similar to the corresponding single character among multiple wrong single characters, such as replacing it with the wrong single characters "fu", "fu", or "fu", etc., to obtain a segmented sequence with wrong characters: A delicate bracelet.

[0041] According to an embodiment of the present invention, when replacing the segmented term with the corresponding part-of-speech of a single character in each original segmented sequence with a wrong single character, the segmented term with the corresponding part-of-speech of a single character in the original segmented sequence can be randomly replaced with a single character similar to the corresponding single character among multiple wrong single characters; or specified to be replaced with a single character similar to the corresponding single character among multiple wrong single characters; or in accordance with the arrangement order of multiple wrong single characters similar to the corresponding single character, select a wrong single character in turn to replace the segmented term with the corresponding part-of-speech of a single character, to obtain multiple different samples to avoid sample repetition. Among them, multiple wrong single characters similar to the corresponding single character include: multiple wrong single characters with similar glyphs to the corresponding single character, multiple wrong single characters with similar pronunciations to the corresponding single character, one of multiple wrong single characters with similar meanings to the corresponding single character or any different combination thereof.

[0042] According to an embodiment of the present invention, change the label value corresponding to the segmented term where the single character is replaced in the corresponding label sequence to indicate that the corresponding character is a wrong character, including: according to the predetermined label window size, change the label value of the segmented term where the single character is replaced in the corresponding label sequence to indicate that the corresponding single character is a wrong character, and change the label values corresponding to multiple segmented terms before and after the segmented term to indicate that the characters of multiple segmented terms before and after the segmented term are wrong characters. Preferably, the predetermined label window size (WinSize = 2d + 1) can be 3, 5, 7, 9, see Figure 3 , for the segmented sequence with wrong characters T i = w i1 / pos(w i1 )…w ij-d / pos(w ij-d )…w′ ij / pos(w ij )…w ij+d / pos(w ij+d )…w ik / pos(w ik ), change the word item w′ in the corresponding label sequence where the word is replaced ij / pos(w ij ) to indicate that the single word of the segmentation item is an incorrect text, and the tag values ​​corresponding to the d segmentation items before and after the segmentation item are changed to indicate that the text of the d segmentation items before and after the segmentation item is an incorrect text. The changed tag value is 1, represented by the symbol E, and the tag sequence Tag with the incorrect word segmentation sequence is obtained. i =C…E…E…E…C. For example, when the predetermined tag window size is 3, d is set to 1, that is, the segmentation item w′ is changed ij / pos(w ij ) to indicate that the word in the participle is an error word, and to change w ij-1 / pos(w ij-1 ) and w ij+1 / pos(w ij+1 ) The label value corresponding to the two participle items is to indicate that the text of the participle item before and after the participle item is an incorrect text.

[0043] According to another embodiment of the present invention, replacing a word segmentation item with an incorrect word similar to the corresponding word can refer to multiple word set confusion pairs and be replaced according to the part of speech of the segmentation item, so as to replace the corresponding word segmentation item more simply and effectively.

[0044] The method for generating multiple word confusion pairs includes: for each part of speech, constructing a word confusion pair set based on a word of the corresponding part of speech and multiple incorrect words similar to the corresponding word. The word confusion pair set includes multiple word confusion pairs constructed by the corresponding word and an incorrect word similar to the word; constructing a word confusion pair set of the corresponding part of speech based on multiple word confusion pairs of multiple words of the same part of speech, thereby obtaining multiple word confusion pair sets with different parts of speech.

[0045] According to one embodiment of the present invention, for example, for particle parts of speech, a single-word confusion pair set of particle parts of speech is constructed. First, it is necessary to collect the single words of particle parts of speech that appear frequently in the Chinese corpus, and obtain a single-word set consisting of single words of particle parts of speech. The collection result is shown in Table 1 below. The single-word set consisting of single words of particle parts of speech includes 10 single-word particles that appear frequently.

[0046] Table 1: Frequency distribution of each word in the word set composed of words of the particle part of speech

[0047] A word set consisting of words of the particle part of speech Frequency of 23853039 Got it 3997818 wait 1496229 Wear 661947 land 512835 Pass 392214 have to 363028 Of 352169 Place 322354 even 63945

[0048] Secondly, by the above method, a set of single characters for each word class is constructed, and multiple sets of single characters with different word classes are obtained, as shown in Table 2 below, including sets of single characters composed of multiple frequently occurring single characters with different word classes.

[0049] Table 2: Sets of single characters and the number of single characters in each set

[0050]

[0051]

[0052] Finally, a set of single character confusion pairs with multiple different word classes is constructed based on the sets of single characters composed of single characters with multiple different word classes. According to an embodiment of the present invention, for example, based on each single character of the particle word class and multiple incorrect single characters similar to the corresponding single character, a set of single character confusion pairs for the corresponding single character is constructed. The set of single characters composed of single characters of the particle word class includes 10 single characters, so 10 sets of single character confusion pairs can be constructed. Denote any single character in the set of single characters of the particle word class as w ij , and the incorrect single character similar to the corresponding single character is denoted as w' ij , which can be one of multiple incorrect single characters similar in glyph to the corresponding single character, multiple incorrect single characters similar in pronunciation to the corresponding single character, multiple incorrect single characters similar in meaning to the corresponding single character, or a combination thereof. A single character confusion pair <w ij , w'> is formed. For example, the incorrect single characters w' similar to the single character "的" ij include "地", "得", "昀", etc., and the set of single character confusion pairs for the single character "的" is obtained as {<的, 地>, <的, 得>, …, <的, 昀>}. For the single character "了", the incorrect single characters similar to it include "子", "孑", "孓", etc., and the set of single character confusion pairs for the single character "了" is obtained as {<了, 子>, <了, 孑>, …, <了, 孓>}. The set of single character confusion pairs for all single characters of the particle word class forms the set of single character confusion pairs for the set of single characters of the particle word class, denoted as Cset 助词 . 助词 .

[0053] According to an embodiment of the present invention, replacing the tokenized term that is a single character with an incorrect single character similar to the corresponding single character can refer to multiple sets of single character confusion pairs and be replaced according to the word class of the tokenized term. For example, according to the particle word class of the tokenized term, referring to the set of single character confusion pairs for the single character "的" in the set of single character confusion pairs Cset 助词 of the particle word class, in the original tokenized sequence T i : 一 / m副 / v精致 / a的 / u手镯 / n, the particle "的" which is a single character is replaced with "地", "得", or "昀", and a tokenized sequence with incorrect characters is obtained, such as: 一 / m副 / v精致 / a得 / u手镯 / n.

[0054] Step S4: Repeat step S3 above to construct a training set including multiple samples, where each sample includes a word segmentation sequence and a corresponding label sequence. The word segmentation sequence is the original word segmentation sequence or a word segmentation sequence with typos. That is, the corresponding sample form is: the original word segmentation sequence T i =w i1 / pos(w i1 )w i2 / pos(w i2 )…w ij / pos(w ij ) and the corresponding tag sequence Tag i =CC…C, word segmentation sequence T with typos i =w i1 / pos(w i1 )w i2 / pos(w i2 )…w′ ij / pos(w ij ) and the corresponding tag sequence Tag i =CC…E…E.

[0055] Furthermore, since the frequency of occurrence of the segmentation items of some parts of speech in the corpus is very low, and the corresponding texts under the same part of speech are different, the sparse data in the single-word training set affects the recognition accuracy of the trained model for single-word typos. According to one embodiment of the present invention, the method for generating a training set for a single-word typo recognition model provided by the present invention also includes step S5: abstract processing of the segmentation items of corresponding parts of speech in the original segmentation sequence and the segmentation sequence with typos, including replacing the text text of all segmentation items whose parts of speech are person names, place names, time and organization names with unified person name symbols, place name symbols, time symbols and organization name symbols. The technical solution of this embodiment can at least achieve the following beneficial technical effects: the present invention abstracts the segmentation items of some parts of speech in the training set, overcomes the problem of sparse data in the training set, and improves the recognition accuracy of the model.

[0056] According to one embodiment of the present invention, the word segmentation sequence T i =w i1 / pos(w i1 )w i2 / pos(w i2 )…w ij / pos(w ij ), if pos(w ij )=nr(name) and w ij In Tag i The corresponding mark in is "C", then T i Middle w ijReplaced with the unified "<person name>" symbol.

[0057] If pos(w ij )=ns(name of place), and w ij In Tag i The corresponding mark in is "C", then T i Middle w ij Replaced with a unified “<place name type>” symbol.

[0058] If pos(w ij ) = t (i.e. time) and w ij In Tag i The corresponding mark in is "C", then T i Middle w ij Replaced with the unified "<time class>" symbol.

[0059] If pos(w ij )=nt(name of the organization) and w ij In Tag i The corresponding mark in is "C", then T i Middle w ij Replaced with the unified “<Organization Name Category>” symbol.

[0060] According to one embodiment of the present invention, the part of speech marked on each segmentation item in the segmentation sequence can also be removed to simplify the training set. The part of speech marked on each segmentation item in the original segmentation sequence is removed to obtain the training sentence corresponding to the sample: TT i =w i1 / w i2 / …w ij-d / …w ij / and the corresponding tag sequence: Tag i =CC…C…C, and remove the part of speech marked on each segmentation item in the segmentation sequence with typos to obtain the training sentence corresponding to the sample: TT i =w i1 / w i2 / …w ij-d / …w′ ij / and the corresponding tag sequence: Tag i =CC…E…E, where one of the label values ​​of C and E is 0 and the other is 1.

[0061] The training set obtained based on the above method includes multiple word segmentation sequences with typos corresponding to various parts of speech. Separate training or mixed training can be adopted. When separate training is adopted, the training process is to train for a certain part of speech, and obtain a single-word typo recognition model of the corresponding part of speech to separately identify whether the single word of the corresponding part of speech is wrong. For example, multiple word segmentation sequences with typos in which the single word of the part of speech segmentation item is wrong are used to train the single-word typo recognition model, and a single-word typo recognition model of the part of speech is obtained to improve the recognition accuracy of the single word of a certain part of speech; or a mixed training method can be adopted, that is, multiple word segmentation sequences with typos corresponding to all parts of speech in the training set are used to train the single-word typo recognition model, and the obtained model can detect whether the single words of all parts of speech are wrong.

[0062] According to one embodiment of the present invention, the single-word typo recognition model of the present invention adopts a bidirectional sequence tagging model (Bi-directional Long Short-Term Memory-Conditional Random Fields, BILSTM-CRFModels) to identify whether a single word in a word segmentation item in a word segmentation sequence corresponding to an input sentence is a typo. Figure 1 The input of the bidirectional sequence labeling model is the word segmentation sequence after the sentence is segmented, and the output is the prediction result corresponding to each word item in the word segmentation sequence. The structure of the model from input to output includes the word vector layer, forward LSTM layer, backward LSTM layer, splicing layer and CRF layer.

[0063] According to one embodiment of the present invention, a method for training a single-word typo recognition model is provided by adopting a hybrid training method, comprising iterating the single-word typo recognition model multiple times according to the following steps:

[0064] Step a1: Obtain the training set generated using the above method.

[0065] Step a2: Use the training set to train a single-word typo recognition model to identify whether the single word corresponding to the single-word segmentation item in the sample segmentation sequence is a typo, and obtain a prediction result sequence.

[0066] refer to Figure 1 First, the word vector layer of the single-word typo recognition model is used to extract the features of each word segmentation item in the word segmentation sequence, including converting each word segmentation item into the corresponding one-hot vector: vec′. d represents the size of the dictionary, and each vec′ is mapped into a low-dimensional dense word vector through the word embedding matrix D represents the word vector dimension after each word is embedded, such as vec ijis the word vector of the j-th segmented term of the i-th sample. Secondly, the forward LSTM layer reads vec from left to right ij and then outputs the forward hidden state of the j-th segmented term of the i-th sample The backward LSTM layer reads vec from right to left ij and then outputs the backward hidden state of the j-th segmented term of the i-th sample Finally, the concatenation layer concatenates the forward hidden state of the j-th segmented term of the i-th sample and the backward hidden state into a vector to obtain the output matrix Z of the i-th sample i =(z i1 ,…,z ij …,z ik ). The CRF layer takes this matrix Z i =(z i1 ,…,z ij …,z ik ) as the input of the CRF layer and obtains the predicted result sequence y i =(y i1 ,…,y ij ,…,y ik ) of the i-th sample. Among them, the predicted result y ij is the predicted result E indicating that the text of the j-th segmented term of the i-th sample is an incorrect text or the predicted result C indicating that the text of the corresponding segmented term is a correct text.

[0067] For example Figure 1 in, the training sentence TT i =a / delicate / bracelet corresponding to the i-th sample is input into the model. Among them, the training sentence of the sample has five segmented terms, and the second segmented term "delicate" is an incorrect single character. When the label window size is taken as 3, the label sequence of the i-th sample is (E, E, E, C, C), where C represents that the text of the corresponding segmented term is correct and E represents that the text of the corresponding segmented term is incorrect; the word vectors corresponding to the five segmented terms are obtained through the word vector layer as vec i1 , vec i2 , vec i3 , vec i4 and vec i5 . The forward hidden states corresponding to the five segmented terms are obtained through the forward LSTM layer as and . The backward hidden states corresponding to the five segmented terms are obtained through the backward LSTM layer as and . The output matrix Z of the i-th sample is output through the concatenation layer i =(z i1 , zi2 , z i3 , z i4 , z i5 ), finally, the prediction result sequence of the i-th sample is obtained through the CRF layer as y i =(y i1 ,y i2 ,y i3 ,y i4 ,y i5 ), such as y i =(E, E, E, C, C).

[0068] Step a3: Calculate the total loss value based on the prediction result sequence and label sequence of the corresponding sample.

[0069] According to one embodiment of the present invention, the total loss is calculated using a log-likelihood function as follows:

[0070]

[0071] Among them, n represents the number of samples, p(y i |Z i ) represents the probability distribution of the prediction result sequence of the i-th sample, y i is the prediction result sequence of the i-th sample, Z i is the output matrix of the i-th sample in the splicing layer, k represents the number of segmentation items of the i-th sample, T is the label transfer matrix, Represents the j-th prediction result y in the prediction result sequence ij Transfer to the j+1th prediction result y i(j+1) The probability of Represents the training sentence TT i The jth participle item w in ij Labeled as predicted result y ij The score, represents the label sequence of the i-th sample, Indicates TT i The sequence of all prediction results for this round.

[0072] Step a4: Update the parameters of the single-word typo recognition model based on the total loss value until the preset number of iterations is reached, then stop updating to obtain a trained single-word typo recognition model.

[0073] Based on the above-trained single-word typo recognition model, according to one embodiment of the present invention, a single-word typo recognition system is provided, including: a word segmentation module, which is used to perform word segmentation processing on an input sentence to obtain a word segmentation sequence to be recognized; a single-word typo recognition model, which is trained using the above-mentioned training method, is used to process the word segmentation sequence to be recognized and output a prediction result sequence, wherein the prediction result sequence indicates whether the single word corresponding to the word item of the single word in the word segmentation sequence is a typo.

[0074] The training process will be described below, and the recognition effect of the single-word typo recognition model of the present invention will be verified through a test set after the training is completed.

[0075] In the experiment, after embedding the word segmentation sequence into the model's word embedding layer, the word embedding dimension for each word segmentation item was 300. Furthermore, the number of hidden layer units was set to 200, the learning rate was 0.001, the batch size was 512, and the model training was stopped after 50 iterations. The number of samples for both positive samples (i.e., the original word segmentation sequence and the corresponding label sequence) and negative samples (i.e., the word segmentation sequence with typos and the corresponding label sequence) in the corpus containing part-of-speech errors was 15,000. The entire single-word typo recognition model was built using TensorFlow, and the graphics card used in the experiment was a TITAN RTX. Furthermore, when training the single-word typo recognition model, the label window size (WinSize) indicating that the text in the label sequence in the training set was incorrect was set to 3, 5, 7, and 9, respectively, resulting in four different single-word typo recognition models.

[0076] After training, we randomly collected 2,000 negative samples from the particle part of speech as a test set. We then used this test set to test four different single-word typo recognition models. The results are shown in Table 3.

[0077] Table 3: Test results of the corresponding single-word typo recognition model for test sets with different label window sizes

[0078]

[0079]

[0080] From the test results, it can be found that the method of the present invention achieves good results in terms of precision, recall rate and average F1 value (harmonic mean); the single-word typo recognition model trained with a training set in which the label window size of the text indicating the participle item is the wrong text is set to 5 has the best effect, that is, the precision rate, recall rate and average F1 value (harmonic mean) are the largest.

[0081] It should be noted that although the above describes the various steps in a specific order, it does not mean that the steps must be performed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0082] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0083] Computer-readable storage media can be a tangible device that holds and stores the instructions used by an instruction execution device. Computer-readable storage media can, for example, include, but are not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, a punch card or a raised structure in a groove on which instructions are stored, for example, and any suitable combination thereof.

[0084] While various embodiments of the present invention have been described above, the above descriptions are intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating a training set for a single-word typo recognition model, characterized in that: The method comprises: S1. Obtain a corpus, perform word segmentation and part-of-speech tagging on each sentence in the corpus, and obtain multiple original word segmentation sequences, wherein each original word segmentation sequence includes multiple word segments and the part of speech of each word segmentation item; S2. Construct labels for each segmentation item based on the original segmentation sequence to obtain a label sequence, where the initial label value indicates that the text corresponding to each segmentation item is the correct text; S3. Replace a single-word segmentation item in the corresponding original segmentation sequence with an incorrect single word similar to the corresponding single word to obtain a segmentation sequence with an incorrect word, and change the label value corresponding to the segmentation item whose single word is replaced in the corresponding label sequence to indicate that the corresponding single word is an incorrect word, wherein the replacing of the single-word segmentation item in the corresponding original segmentation sequence with an incorrect single word similar to the corresponding single word is referring to a plurality of confused pairs of single word sets, and performing the replacement according to the part of speech of the segmentation item, wherein the generating method of the plurality of confused pairs of single word sets includes: For each part of speech, a word confusion pair set is constructed based on a word of the corresponding part of speech and a plurality of incorrect words similar to the corresponding word, wherein the word confusion pair set includes a plurality of word confusion pairs constructed by the corresponding word and an incorrect word similar to the word; Based on multiple single-word confusion pair sets of multiple single-words with the same part of speech, a single-word set confusion pair set of corresponding part of speech is constructed to obtain multiple single-word set confusion pair sets with different parts of speech; S4. Repeat step S3 to construct a training set including multiple samples, wherein each sample includes a word segmentation sequence and a corresponding label sequence, and the word segmentation sequence is an original word segmentation sequence or a word segmentation sequence with typos.

2. The method according to claim 1, characterized in that The step of replacing a single-word segmentation item in the corresponding original segmentation sequence with an incorrect single word similar to the corresponding single word to obtain a segmentation sequence with an incorrect word includes: For each part of speech, obtaining a plurality of original segmentation sequences including segmentation items of the corresponding part of speech of the single word; According to the part of speech of the segmentation item, the segmentation item of the corresponding part of speech of each word in each original segmentation sequence is replaced with one of multiple incorrect words similar to the corresponding word, thereby obtaining multiple segmentation sequences with incorrect words.

3. The method according to claim 2, characterized in that The multiple erroneous words similar to the corresponding words include: One of a plurality of incorrect characters that are similar in shape to the corresponding character, a plurality of incorrect characters that are similar in pronunciation to the corresponding character, a plurality of incorrect characters that are similar in meaning to the corresponding character, or any different combination thereof.

4. The method according to claim 1, wherein The step of changing the label value corresponding to the segmentation item whose single word is replaced in the corresponding label sequence to indicate that the corresponding single word is an erroneous word includes: According to a predetermined label window size, the label value of the segmentation item in the corresponding label sequence whose single word is replaced is changed to indicate that the corresponding single word is an incorrect text, and the label values ​​corresponding to the multiple segmentation items before and after the segmentation item are changed to indicate that the text of the multiple segmentation items before and after the segmentation item is an incorrect text.

5. The method according to claim 1, wherein The corpus is a Chinese corpus, a Japanese corpus or a Korean corpus.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: Abstract processing is performed on the corresponding part-of-speech participle items in the original participle sequence and the participle sequence with typos, including replacing the text of all participle items whose parts of speech are person names, place names, time and organization names with unified person name symbols, place name symbols, time symbols and organization name symbols respectively.

7. A training method for a single-word typo recognition model, characterized in that: This involves multiple iterations of the single-word typo recognition model in the following manner: Obtaining a training set generated using the method of any one of claims 1 to 6; Using the training set to train a single-word typo recognition model to identify whether a single word corresponding to a single-word segmentation item in a sample's segmentation sequence is a typo, and obtain a prediction result sequence; Calculate the total loss value based on the prediction result sequence and label sequence of the corresponding sample; Update the parameters of the single-word typo recognition model based on the total loss value.

8. A system for identifying single-word typos, characterized in that: include: The word segmentation module is used to perform word segmentation on the input sentence to obtain the word segmentation sequence to be recognized; The single-word typo recognition model obtained by the method according to claim 7 is used to process the word segmentation sequence to be recognized and output a prediction result sequence, wherein the prediction result sequence indicates whether the single word corresponding to the word segmentation item in the word segmentation sequence is a typo.

9. An electronic device, characterized in that: include: one or more processors; as well as a memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1 to 7 by executing the executable instructions.

Images