A Chinese text spelling checking method, system, device and storage medium

Abstract

The application discloses a Chinese text spelling checking method, system, device and storage medium, and belongs to the field of natural language processing. The method comprises the following steps: performing feature extraction on semantic information, pinyin information and character structure information of an input text by using a corresponding pre-training encoder; after obtaining the three kinds of modal information, performing feature fusion by using a neural network to obtain feature information of the input text in multiple modes; and finally, performing error detection and correction on each word segmentation of the input text based on the feature information. The application comprehensively uses various information of Chinese characters in terms of semantics, pronunciation and character structure, and can improve the performance of the model in spelling checking of Chinese text.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and particularly to a Chinese text spelling check method, system, device and storage medium. Background Art

[0002] Natural language understanding is a branch of natural language processing technology, and tasks under this branch are usually used to evaluate indicators such as model understanding ability and semantic richness. Chinese text spelling check is a task under natural language understanding, and its purpose is to correct misspelled characters in the text. For example, "I hope to possibly attend your party, but his health is really not good!" The model needs to change "生" to "身". Existing Chinese spelling check datasets include: SIGHAN13, 14, 15, Wang271K, and OCR datasets. These data all include correct-incorrect sentence pairs, and the evaluation methods for the test set are usually based on the character or sentence level, and the precision (Precision), recall (Recall), F1 value, and accuracy (Accuracy) are calculated through the original text, correct text, and predicted text. As can be seen from the above example, Chinese character errors usually occur between homophonic or similar-shaped characters, so it is necessary to introduce information of pinyin and font modalities. Current Chinese text spelling check solutions more or less involve multi-modal information, but there are the following defects: (1) It is necessary to design complex rules, such as feature information determined by stroke order for font types. (2) The semantic information between pinyin or font types is not fully considered, and the pinyin or font feature information of adjacent characters is not modeled, and each pinyin or font is processed in isolation. (3) It is difficult to construct an end-to-end training scheme; due to the reason of defect (1), some solutions need to perform cumbersome post-processing to obtain the finally output text. Summary of the Invention

[0003] To at least partly solve one of the technical problems existing in the prior art, the purpose of the present invention is to provide a Chinese text spelling check method, system, device and storage medium.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A Chinese text spelling check method includes the following steps:

[0006] S1. Obtain the original text and the target text, and obtain the text token sequence, pinyin string sequence, and font picture feature vector sequence from the original text;

[0007] S2. Pre-train the pinyin encoder BERT and the font encoder BERT respectively according to the pinyin string sequence and the font feature vector sequence;

[0008] S3. Input the text segmentation sequence, pinyin string sequence, and character image feature vector sequence into the pre-trained text encoder BERT, pinyin encoder BERT, and character encoder BERT respectively to obtain the semantic, phonetic, and character feature vectors of the original text, denoted as H. T H P H S ;

[0009] S4. Transfer the feature vector H T H P H S Feature fusion is performed to obtain a feature hidden vector H that integrates semantic, phonetic, and character shape features;

[0010] S5. Input the feature hidden vector H into the preset error detection BERT and error correction BERT to obtain the error probability of each character in the original text, as well as the probability distribution of the character that should be modified to another character.

[0011] Further, step S1 includes:

[0012] The original text is represented as X = {x1, x2, ..., x...} m The target correct text is represented as Y = {y1, y2, ..., y}. m}, where m is the length of the original text;

[0013] Inputting X into the word segmenter yields the discrete encoded sequence T = {t} of the text modality. CLS ,t1,t2,...,t m , t SEP}, t i , i∈{1,…,m} represents the discrete code in the vocabulary corresponding to each character in the sentence, CLS at the beginning of the sentence is used as the sentence vector to represent the word, and SEP at the end of the sentence is used as the end of the sentence text;

[0014] Inputting X into pypinyin yields a sequence of pinyin strings. Then, a word segmenter for the pinyin modality is used to obtain a discrete encoding sequence P = {p}. CLs p1, p2, ..., p m p SEP}, where p i , i∈{1,…,m} is t i The discrete encoding of the corresponding pinyin in the pinyin dictionary;

[0015] Input X into the preset character structure feature extraction module to obtain the sequence S = {s} CLS s1, s2, ..., s m s SEP}, where s i , i∈{1,…,m} is ti The corresponding character feature vector.

[0016] Furthermore, the character structure feature extraction module operates as follows:

[0017] The Chinese characters at each position are converted into three fixed 32*32 font images, including Small Seal Script, Simplified Black, and Traditional Black.

[0018] The RGB format images of the three fonts are stacked in the third dimension and used as the input tensor. The tensor dimension of the image corresponding to each character position is [32, 32, 3].

[0019] The input tensor is fed into a 6-layer convolutional neural network to obtain the character feature vector corresponding to each position, thus forming a sequence S.

[0020] Further, step S2 includes:

[0021] 15% of the positions in the pinyin string sequence are processed according to the following rules: 80% are replaced with discrete codes of similar pinyin, 10% are randomly replaced with discrete codes of another pinyin, and 10% remain unchanged;

[0022] The following rules apply to 15% of the character feature vector sequence: 80% are replaced with feature vectors of similar characters, 10% are randomly replaced with feature vectors of another character, and 10% remain unchanged.

[0023] The modified sequence is used as input and the original sequence is used as output. The encoder BERT outputs the probability distribution of the modified position and compares the probability distribution with the true value of the original position to obtain the loss value and perform backpropagation. Finally, the BERT phonetic encoder and the BERT character encoder learn the relationship between the phonetic and character features of characters respectively.

[0024] Further, step S3 includes:

[0025] The discrete encoded sequence T of the text modality is input into the pre-trained text encoder BERT to obtain the feature vector.

[0026] The discrete encoded sequence P of the Pinyin modality is input into the pre-trained Pinyin encoder BERT to obtain the feature vector.

[0027] The sequence S of image modalities is input into the pre-trained character encoder BERT to obtain feature vectors.

[0028] Further, step S4 includes:

[0029] eigenvectors Feature vector and eigenvectors The last dimension of each positional feature vector is concatenated with a 1, then pairwise outer products are performed, and finally flattened to obtain the hidden feature vector of the sentence after incorporating semantics, pinyin, and character type modality: H = {h GLS h1, h2, ..., h m h SEP},in The flatten operation is used to transform a multidimensional tensor into a 1-dimensional vector.

[0030] Further, step S5 includes:

[0031] The hidden feature vector H is input into the error detection BERT to obtain the hidden feature vector of the last layer. Will Passing through dimension [n] dim The probability of an error at that location is obtained by using a linear layer [2] and Softmax. Where det represents error detection, and l represents the number of layers in the error detection BERT model. n dim The dimension representing the hidden vector;

[0032] The feature hidden vector H is input into the error correction BERT to obtain the feature hidden vector of the last layer. Where cor represents the error correction module and l represents the number of layers in the error correction BERT model;

[0033] vector and Add them together, passing through a dimension of [n] dim The linear layer of |V| is then passed through a Softmax layer to obtain the probability distribution of that position with respect to all Chinese characters in the vocabulary. Cor stands for error correction. The dimension is |V|, where V is the vocabulary size. The Chinese character with the highest probability is the final error correction result R = {r1, r2, ..., r}. m}

[0034] Based on the target text Y and the original text X, two types of labels are obtained: a label indicating whether an error occurred at each position. det ={l det_1 , l det_2 , ..., l det_m}, where l det_i Let i ∈ {0, 1}, i ∈ {1, ..., m}, and the character Label that should appear at each position. cor ={l cor_1, l cor_2 , ..., l cor_m}, where l cor_i ∈{0,…,|V|}, i∈{1,…,m}, where V is a vocabulary.

[0035] via Label det Label cor With Prob det The cross-entropy loss values ​​are calculated using R and R respectively. The feature fusion module, error detection and correction module, and two linear layers are optimized using gradient descent.

[0036] Another technical solution adopted in this invention is:

[0037] A Chinese text spell checking system, comprising:

[0038] The modality conversion module is used to obtain the original text and extract the text segmentation sequence, pinyin string sequence, and character image feature vector sequence from the original text.

[0039] The encoder module is used to pre-train the Pinyin encoder BERT and the character encoder BERT respectively based on the Pinyin string sequence and the character feature vector sequence.

[0040] The feature extraction module is used to input the text segmentation sequence, the pinyin string sequence, and the character image feature vector sequence into the pre-trained text encoder BERT, pinyin encoder BERT, and character encoder BERT, respectively, to obtain the semantic, phonetic, and character feature vectors of the original text, denoted as H. T H P H S ;

[0041] The feature fusion module is used to fuse the feature vector H T H P H S Feature fusion is performed to obtain a feature hidden vector H that integrates semantic, phonetic, and character shape features;

[0042] The error detection and correction module is used to input the feature hidden vector H into the preset error detection BERT and error correction BERT to obtain the error probability of each character in the original text, as well as the probability distribution of the character that should be modified to another character.

[0043] Another technical solution adopted in this invention is:

[0044] A Chinese text spell checking device, comprising:

[0045] At least one processor;

[0046] At least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor performs the method as described above.

[0048] Another technical solution adopted in this invention is:

[0049] A computer-readable storage medium storing a processor-executable program, which, when executed by a processor, performs the method described above.

[0050] The beneficial effects of this invention are: this invention integrates various information about Chinese characters in terms of semantics, pronunciation and character structure, which can effectively improve the performance of the model in Chinese text spell checking. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of a Chinese text spelling check method based on multimodal technology in an embodiment of the present invention;

[0053] Figure 2 This is a flowchart illustrating the steps of a Chinese text spelling check method based on multimodal technology in an embodiment of the present invention. Detailed Implementation

[0054] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0055] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0056] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0057] Furthermore, in the description of this invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0058] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0059] To address the shortcomings of existing technologies, this embodiment provides a Chinese text spelling check method based on multimodal technology. This method, based on multimodal feature fusion, fully integrates semantic, pinyin, and character structure features into the hidden vector, and then processes them through error detection and correction modules for downstream tasks. It has broad practical value. Figure 1 and Figure 2 As shown, the method includes the following steps:

[0060] S1. Obtain the original text and target text, and extract the text segmentation sequence, pinyin string sequence and character image feature vector sequence from the original text.

[0061] All Chinese characters in the original text are converted into pinyin string sequences using pypinyin, and into font images using font files of Xiaozhuan, Simplified Heiti, and Traditional Heiti. Feature vectors are then extracted using a grid-based convolutional neural network. Through these operations, text segmentation sequences, pinyin string sequences, and font image feature vector sequences can be obtained.

[0062] As an optional implementation, the original text is represented as X = {x1, x2, ..., x...} m The target correct text is represented as Y = {y1, y2, ..., y}. m}, where m is the length of the original text; inputting X into the word segmenter yields the discrete encoded sequence T = {t} of the text modality. CLS ,t1,t2,...,t m , t SEP}, t i Let i ∈ {1, ..., m} represent the discrete code in the vocabulary corresponding to each character in the sentence. The CLS at the beginning of the sentence represents the word in the sentence vector, and the SEP at the end of the sentence represents the end of the sentence text. Inputting X into pypinyin yields a sequence of pinyin strings, which is then processed by a word segmenter for the pinyin modality to obtain the discrete code sequence P = {p...}. CLS p1, p2, ..., p m p SEP}, where p i , i∈{1,…,m} is t i The corresponding discrete encoding of the pinyin in the pinyin dictionary; input X into Figure 1 The character structure feature extraction module on the right can obtain S = {s} CLS s1, s2, ..., s m s SEP}, where s i , i∈{1,…,m} is t i The corresponding character feature vectors. The character structure feature extraction module first converts the Chinese character at each position into three fixed 32*32 font images, including Small Seal Script, Simplified Black, and Traditional Black. The RGB format images of the three fonts are stacked in the third dimension (channel dimension) as the input tensor, and the tensor dimension corresponding to each character position is [32, 32, 3]. This tensor is input into a 6-layer convolutional neural network to obtain the character feature vector corresponding to each position, thus forming S.

[0063] S2. Pre-train the Pinyin encoder BERT and the character encoder BERT according to the Pinyin string sequence and the character feature vector sequence respectively.

[0064] In some embodiments, 15% of the positions in the discrete encoding sequence of the Pinyin string are processed according to the following rules: 80% are replaced with discrete encodings of similar Pinyin, 10% are randomly replaced with discrete encodings of another Pinyin, and 10% remain unchanged. Similarly, 15% of the positions in the character feature vector sequence are processed according to the following rules: 80% are replaced with feature vectors of similar characters, 10% are randomly replaced with feature vectors of another character, and 10% remain unchanged. Using the modified sequence as input and the original sequence as output, the BERT encoder outputs the probability distribution of the modified positions. The loss value is obtained by comparing the probability distribution with the true value of the original position and then backpropagated. Ultimately, the Pinyin and character encoders BERT learn the relationships between Pinyin and character features, respectively.

[0065] S3. Input the text segmentation sequence, pinyin string sequence, and character image feature vector sequence into the pre-trained text encoder BERT, pinyin encoder BERT, and character encoder BERT respectively to obtain the semantic, phonetic, and character feature vectors of the original text, denoted as H. T H P H S .

[0066] Specifically, step S3 includes:

[0067] The discrete encoded sequence T of the text modality is input into the pre-trained text encoder BERT to obtain the feature vector.

[0068] The discrete encoded sequence P of the Pinyin modality is input into the pre-trained Pinyin encoder BERT to obtain the feature vector.

[0069] The sequence S of image modalities is input into the pre-trained character encoder BERT to obtain feature vectors.

[0070] S4. Transfer the feature vector H T H P H S Feature fusion is performed to obtain a hidden feature vector H that integrates semantic, phonetic, and character shape features.

[0071] As an optional implementation method, firstly... The last dimension of each positional feature vector is concatenated with a 1, then pairwise outer products are performed, and finally flattened to obtain the hidden feature vector H = {h} of the sentence, which integrates semantics, pinyin, and character type modality. CLS h1, h2, ..., h m h SEP},in The flatten operation is used to transform a multidimensional tensor into a 1-dimensional vector.

[0072] S5. Input the feature hidden vector H into the preset error detection BERT and error correction BERT to obtain the error probability of each character in the original text, as well as the probability distribution of the character that should be modified to another character.

[0073] H is input into the error detection BERT and the error correction BERT respectively, and the error probability of each character in the input sentence and the probability distribution of the character that should be changed to another character are obtained respectively. Based on the output probability and probability distribution, the loss value can be calculated by the loss function, thereby optimizing the network parameters.

[0074] Specifically, H is input into the error detection BERT to obtain the feature hidden vector of the last layer. Will Passing through dimension [n] dim The probability of an error at that location can be obtained from the linear layer [2] and Softmax. Where det represents error detection, and 1 represents the number of layers in the error detection BERT model. n dim H represents the dimension of the hidden vector; inputting H into the error-correcting BERT yields the feature hidden vector of the last layer. Where cor represents the error correction module, and 1 represents the number of layers in the error-correcting BERT model; and Add them together, passing through a dimension of [n] dim A linear layer of |V| followed by a softmax layer yields the probability distribution of that position for all Chinese characters in the vocabulary. Cor stands for error correction. The dimension is |V|, where V is the vocabulary size. The Chinese character with the highest probability is the final error correction result R = {r1, r2, ..., r}. m Based on Y and X, two types of labels can be obtained: a label indicating whether an error occurred at each position. det ={l det_1 , l det_2 , ..., l det_m}, where l det_i Let i ∈ {0, 1}, i ∈ {1, ..., m}, and the character Label that should appear at each position. cor ={l cor_1 , l cor_2 , ..., l cor_m}, where l cor_i ∈{0, ...,|V|}, i∈{1, ...,m}, where V is a vocabulary. (Using Labels...) det Labelcor With Prob det R and R can be used to calculate the cross-entropy loss value, and the feature fusion module, error detection and correction module, and the two linear layers mentioned above can be optimized by gradient descent.

[0075] The neural network model used in this embodiment mainly consists of an encoder module, a feature fusion module, and an error detection and correction module. In this step, it is only necessary to add the error detection loss and the error correction loss together, and continuously optimize the feature fusion module, the error detection and correction module, and the linear layer during the training process to reduce the loss.

[0076] Based on the validation on the SIGHAN dataset, the results obtained by this example using the sentence-level evaluation method are shown in Table 1 below:

[0077] Table 1

[0078]

[0079]

[0080] The calculation methods for the above indicators are as follows:

[0081] Error detection section:

[0082]

[0083]

[0084]

[0085]

[0086] Error correction section:

[0087]

[0088]

[0089]

[0090]

[0091] In summary, compared with the prior art, the present invention has at least the following advantages and beneficial effects:

[0092] (1) The Chinese text spelling check method based on multimodal technology designed in this invention does not require the design of complex and cumbersome modification rules, and the entire system can realize the end-to-end training process.

[0093] (2) The Chinese text spelling check method based on multimodal technology designed in this invention is based on modular design, which can easily add or delete modules, and can easily add more modal information, and has strong scalability.

[0094] (3) The Chinese text spelling check method based on multimodal technology designed in this invention fully considers the semantic information of the pinyin and character shape between each character.

[0095] This embodiment also provides a Chinese text spelling check system based on multimodal technology, including:

[0096] The modality conversion module is used to convert Chinese characters into pinyin strings and character feature vectors;

[0097] The encoder module is used to process the features of each modality separately and obtain the hidden feature vectors of the input sentence in terms of semantics, pinyin, and character shape.

[0098] The feature fusion module is used to perform tensor fusion of the feature hidden vectors of the three modalities and output a multimodal feature hidden vector.

[0099] The error detection and correction module is used to process the multimodal feature hidden vectors for downstream tasks, and output the feature hidden vector of the last layer and the error probability at each position.

[0100] The generation module weights and maps the last layer of hidden feature vectors output by the error detection and correction module, and then outputs the probability distribution of each position with respect to the vocabulary through the Softmax layer.

[0101] The network optimization module obtains the error detection loss and error correction loss based on the probability distribution output by the generation module, weights the two losses, and optimizes the feature fusion, error detection and correction modules, and linear layers using gradient descent.

[0102] This embodiment of a Chinese text spelling check system based on multimodal technology can execute a Chinese text spelling check method based on multimodal technology provided in the method embodiment of the present invention. It can execute any combination of implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0103] This embodiment also provides a Chinese text spelling checker based on multimodal technology, including:

[0104] At least one processor;

[0105] At least one memory for storing at least one program;

[0106] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 2 The method shown.

[0107] This embodiment of the Chinese text spelling checker based on multimodal technology can execute the Chinese text spelling checker method based on multimodal technology provided in the method embodiment of the present invention. It can execute any combination of the implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0108] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform... Figure 2 The method shown.

[0109] This embodiment also provides a storage medium storing instructions or programs that can execute the Chinese text spelling check method based on multimodal technology provided in the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the method has the corresponding functions and beneficial effects.

[0110] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0111] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0112] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0114] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0115] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0116] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0117] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0118] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A Chinese text spelling checking method, characterized in that, The method comprises the following steps: S1, obtaining an original text, and acquiring a text segmentation sequence, a pinyin string sequence, and a font picture feature vector sequence in the original text; S2, pre-training a pinyin encoder BERT and a font encoder BERT according to the pinyin string sequence and the font feature vector sequence; S3, input the text word segmentation sequence, the pinyin string sequence and the font picture feature vector sequence into the pre-trained text encoder BERT, pinyin encoder BERT and font encoder BERT respectively, and obtain the feature vectors of the original text in semantics, pronunciation and font respectively, denoted as ; S4, the feature vector feature fusion is performed to obtain a feature hidden vector that integrates semantic, pronunciation and font features ; S5、the feature hidden vector Input into the preset error detection BERT and error correction BERT to obtain the error probability of each word in the original text and the probability distribution of the word that should be modified. The step S2 comprises: 80% of the positions in the pinyin string sequence are replaced into similar pinyin discrete codes, 10% are randomly replaced into discrete codes of another pinyin, and 10% remain unchanged; 80% of the positions in the font feature vector sequence are replaced into feature vectors of similar characters, 10% are randomly replaced into feature vectors of another character, and 10% remain unchanged; The modified sequence is taken as input, the original sequence is taken as output, the encoder BERT outputs a probability distribution of the modified position, and a loss value is obtained by comparing the probability distribution with the true value of the original position, and then back propagation is performed, so that the pinyin encoder BERT and the font encoder BERT learn the relationship between the pinyin and the font feature of the characters respectively; The step S5 comprises: Feature hidden vector The input is fed into the error detection BERT to obtain the feature hidden vector of the last layer. ,Will After the dimension is The linear layer and Softmax are used to obtain the probability of an error at that location. Where det represents error detection. This represents the number of layers in the error-detection BERT model. , The dimension representing the hidden vector; the feature hidden vector input to the error correction BERT to obtain the feature hidden vector of the last layer wherein cor represents an error correction module, represents the number of layers of the error correction BERT model; The vector is added, passed through a linear layer of dimension , and passed through a Softmax layer to obtain the probability distribution of all Chinese characters in the vocabulary for this position , where cor represents the correction, dimension , V is the size of the vocabulary, and the Chinese character with the highest probability in is the final correction result .​ 2. The Chinese text spelling checking method of claim 1, wherein, The step S1 comprises: The original text is represented as where m is the length of the original text; Will Input to the word segmenter, get the discrete encoding sequence of the text modal , Each word in the sentence corresponds to a discrete encoding in the word table, and the CLS at the beginning of the sentence represents the word vector. The SEP at the end of the sentence represents the end of the sentence text. will be described in detail below with reference to the accompanying drawings. The pinyin string sequence obtained by inputting to pypinyin is then input into the pinyin modality tokenizer to obtain the discrete encoding sequence of the pinyin modality wherein is the discrete encoding of the corresponding pinyin in the pinyin vocabulary. Will be input to the preset font structure feature extraction module, get the sequence , is The corresponding font feature vector.

3. The Chinese text spelling checking method of claim 2, wherein, The working mode of the font structure feature extraction module is as follows: The Chinese character at each position is converted into three font pictures; The RGB format pictures of the three font pictures are stacked in the third dimension as an input tensor; The input tensor is input into a 6-layer convolutional neural network to obtain a font feature vector corresponding to each position, thereby forming a sequence .

4. The Chinese text spelling checking method of claim 2, wherein, The step S3 comprises: input the discrete encoding sequence T of the text modality into the pre-trained text encoder BERT to obtain a feature vector ; The discrete coding sequence P of the pinyin mode is input into the pre-trained pinyin encoder BERT to obtain a feature vector ; inputting a sequence S of picture modalities into a pre-trained font encoder BERT to obtain a feature vector .

5. The Chinese text spelling checking method of claim 4, wherein, The step S4 comprises: concatenate the feature vectors , the feature vectors , and the feature vectors The last 1 dimension of each position feature vector is concatenated with 1, then pairwise outer product is performed, and finally flattened to obtain the feature hidden vector of the sentence after fusing the semantic, pinyin, and font modalities wherein , is a flattening operation.

6. A Chinese text spelling checking system for implementing the method of any one of claims 1-5, characterized in that, It comprises: A modal conversion module is configured to acquire a text segmentation sequence, a pinyin string sequence, and a font picture feature vector sequence in an original text according to the original text; An encoder module is configured to pre-train a pinyin encoder BERT and a font encoder BERT according to the pinyin string sequence and the font feature vector sequence; The feature extraction module is configured to input the text word segmentation sequence, the pinyin string sequence, and the font picture feature vector sequence into pre-trained text encoder BERT, pinyin encoder BERT, and font encoder BERT respectively, and obtain feature vectors of the original text in semantics, pronunciation, and font respectively, and the feature vectors are represented as ; a feature fusion module configured to fuse the feature vectors perform feature fusion to obtain a feature hidden vector fused with semantic, pronunciation and font features ; An error detection and correction module is configured to input the feature hidden vector to a preset error detection BERT and error correction BERT to obtain an error probability of each word in the original text and a probability distribution of the word that should be modified.

7. A Chinese text spelling checking apparatus, characterized by comprising: It comprises: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-5.

8. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The processor-executable program, when executed by the processor, is configured to perform the method of any one of claims 1-5.

Images