Training method of high-performance wrongly written character recognition model

Abstract

The invention discloses a training method for a high-performance wrongly written character recognition model, and relates to the field of model training, and the method comprises the steps: obtaining character and word associated information, and constructing a character and word knowledge graph; extracting word multi-dimensional feature vectors, and performing sample expansion on the primary sample set to obtain a word and sentence sample set; constructing a wrongly written character recognition model, wherein the wrongly written character recognition model comprises a primary recognition main model and a primary recognition sub-model; completing iterative training of the primary identification main model to obtain a wrongly written character identification main model; outputting a wrongly written character recognition result set through the wrongly written character recognition main model; and training the primary recognition sub-model by using the wrongly written character recognition result set and the word and sentence sample set to obtain a wrongly written character recognition sub-model. The wrongly written character recognition precision and recognition efficiency of the wrongly written character recognition model can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of model training, and more particularly to a training method for a high-performance misspelling recognition model. Background Technology

[0002] With the development of artificial intelligence and natural language processing technologies, automatic misspelling recognition plays an increasingly important role in various application scenarios such as text editing, information retrieval, and intelligent input methods. Some pre-trained language models with strong semantic understanding and generation capabilities have been widely used in misspelling detection and correction tasks.

[0003] However, existing pre-trained language models still have some limitations in the task of misspelling recognition. On the one hand, these models typically learn language patterns through repeated training on massive amounts of data, causing them to prioritize outputting words and phrases that appear more frequently in the dataset during inference. This results in insufficient accuracy in recognizing misspelled words and phrases that appear less frequently. On the other hand, existing pre-trained language models usually have a large number of parameters and high computational complexity, resulting in slow inference speeds in real-world applications and thus insufficient efficiency in misspelling detection and correction tasks. Summary of the Invention

[0004] This application provides a training method for a high-performance misspelling recognition model, which is used to improve the misspelling recognition accuracy and efficiency of the misspelling recognition model.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions: Firstly, a training method for a high-performance misspelling recognition model is provided, the method comprising: Obtain word association information for all target words in the target word set, and construct a word knowledge graph based on the word association information; Based on the word knowledge graph, the multi-dimensional feature vectors of all target words are extracted, and the pre-acquired primary sample set is expanded according to the multi-dimensional feature vectors of words to obtain the word and sentence sample set; A misspelling recognition model is constructed based on a pre-trained language model. The misspelling recognition model includes a primary recognition main model and a primary recognition sub-model. The initial recognition model was iteratively trained based on the word and sentence sample set to obtain the misspelling recognition model. The negative sample set of words in the word and sentence sample set is input into the main model for misspelling recognition to identify misspellings, and the misspelling recognition result set is output through the main model for misspelling recognition. The initial recognition sub-model is trained iteratively using the misspelling recognition result set and the word and sentence sample set to obtain the misspelling recognition sub-model.

[0006] Optionally, obtaining word association information for all target words in the target word set and constructing a word knowledge graph based on the word association information includes the following steps: Obtain multidimensional association information of all target words in the target word set; The multidimensional association information is standardized to obtain word association information, which includes word pronunciation information, word shape information, word semantic information, and word domain information. The multidimensional similarity of all target words is calculated based on word association information. The multidimensional similarity includes pronunciation similarity, character shape similarity, semantic similarity, and domain similarity. All target words are used as graph nodes. Connection edges are constructed between all graph nodes based on multidimensional similarity, and connection edge weights are assigned to all connection edges based on multidimensional similarity to obtain a word knowledge graph.

[0007] Optionally, extracting multi-dimensional feature vectors of all target words based on the word knowledge graph includes the following steps: For any graph node, select the neighboring graph nodes from the word knowledge graph; Based on the word knowledge graph, multidimensional similarity and associated edge weights between neighboring graph nodes are extracted, and word misuse labels are generated for graph nodes and neighboring graph nodes by combining multidimensional similarity and associated edge weights. By combining multidimensional similarity and word association information, word difference features between neighboring graph nodes and graph nodes are determined; After vectorizing the word misuse labels, word difference features, and associated edge weights, the vectors are concatenated to obtain the word multidimensional feature vector.

[0008] Optionally, the primary sample set includes a primary positive sample set and a primary negative sample set. The step of expanding the pre-acquired primary sample set according to the multi-dimensional feature vectors of words to obtain the word and sentence sample set includes the following steps: Based on the multi-dimensional feature vectors of words, a primary positive sample set of multiple types of words is selected; Similar word sets of various types are selected from the word knowledge graph based on the multi-dimensional feature vectors of words; By using a set of similar words to replace words in the primary positive sample set, multiple primary perturbation samples are obtained; For any primary perturbation sample, it is beneficial for the pre-trained language model to perform contextual semantic consistency verification on the primary perturbation sample; If the contextual semantic consistency verification of the primary perturbation sample passes, the primary perturbation sample will be added to the primary negative sample set. Repeat the above steps until the number of samples in the primary negative sample set reaches the preset standard sample number, and then output the word and phrase sample set.

[0009] Optionally, the primary recognition main model includes a first word segmentation embedding module, a first graph enhancement module, a first encoding module, and a first classification output module. The first classification output module includes a first error label prediction unit and a first correct word regression unit. The primary recognition sub-model includes a second word segmentation embedding module, a second graph enhancement module, a second encoding module, and a second classification output module. The second classification output module includes a second error label prediction unit and a second correct word regression unit.

[0010] Optionally, the iterative training of the primary recognition model based on the word and sentence sample set to obtain the misspelling recognition model includes the following steps: Initialize the primary model parameters of the primary recognition main model; The word and phrase sample set is input into the primary recognition main model after the primary model parameters have been initialized in batches for iterative training, and the weight parameters of the primary model are adjusted according to the iterative training results. When the primary model's main indicator reaches the preset indicator threshold, the iterative training stops, and the misspelling recognition main model is obtained.

[0011] Optionally, inputting the negative sample set of words from the word and phrase sample set into the main model for misspelling recognition to perform misspelling recognition, and outputting the misspelling recognition result set through the main model includes the following steps: The negative sample set of words is input into the main model for misspelling recognition in batches. The negative sample set of words is segmented by the first word segmentation and embedding module to obtain the negative sample word set. The negative sample word set is input into the first graph enhancement module. The first graph enhancement module extracts the associated word features and associated attribute features of the negative sample word set, and then fuses the associated word features and associated attribute features into a composite feature embedding vector. The composite feature embedding vector is input into the first encoding module, and the word composite feature vector of the composite feature embedding vector is extracted by the first encoding module. The first classification output module completes the prediction of erroneous words in the compound feature vector of words and characters, and outputs the erroneous word information and correct word and sentence sequence in the negative sample set of words and characters according to the prediction results; By integrating the erroneous word information and correct word sequence from all batches of the negative word sample set, a set of misspelling recognition results is obtained.

[0012] Optionally, the initial recognition sub-model can be iteratively trained using the misspelling recognition result set and the word and sentence sample set to obtain the misspelling recognition sub-model, including the following steps: The result set of misspelled words and the sample set of words and sentences are mixed and integrated into a mixed sample set of words and sentences. Extract the main model parameters of the main model for identifying erroneous characters, and initialize the parameters of the primary recognition sub-model based on the main model parameters; Input the mixed word sample set into the primary recognition sub-model that has completed parameter initialization for iterative training of the model until the sub-model index of the primary recognition sub-model reaches the preset first index threshold, and obtain the basic recognition sub-model. Anchor point training set is selected from the word-mixed sample set based on the sub-model indicators; Extract the anchor semantic features and anchor error features from the anchor training set, and combine the anchor semantic features and anchor error features to select the anchor positive sample set and anchor negative sample set from the word-mixed sample set; By integrating the anchor training set, the anchor positive sample set, and the anchor negative sample set, an enhanced sample set is obtained. By using an enhanced sample set to perform model comparison enhancement training on the basic recognition sub-model, a misspelling recognition sub-model is obtained.

[0013] Secondly, this application provides a machine-readable storage medium storing instructions for causing a machine to execute a training method for a high-performance misspelling recognition model as described in the first aspect.

[0014] Thirdly, this application provides a training device for a high-performance misspelling recognition model, comprising: The memory is configured to store instructions; and The processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the training method for the high-performance misspelling recognition model according to the first aspect.

[0015] The above technical solution transforms unstructured word association information into structured information by constructing a word knowledge graph, providing a data foundation for subsequent steps. The relationships within the word knowledge graph can serve as external knowledge supplementation, guiding the model to determine error types. Next, multi-dimensional feature vectors of target words are extracted from the word knowledge graph, and the primary sample set is expanded based on these vectors. This addresses the limitations of the primary sample set, such as its limited scope and incomplete error coverage, especially for less common error samples. Expanding the primary sample set increases the number of samples, thereby improving the model's accuracy in recognizing less frequently occurring misspellings. Two models are constructed and trained: a primary misspelling recognition model and a secondary misspelling recognition model. The primary model captures complex contextual semantics, providing a foundation for high-precision misspelling recognition, while the secondary model is adapted for lightweight scenarios and exhibits higher efficiency in misspelling recognition. Furthermore, during model training, using the spell recognition inference results of the main spell recognition model as the training set for the spell recognition sub-model can effectively reduce the training complexity of the spell recognition sub-model. In summary, the spell recognition model jointly constructed based on the main spell recognition model and the spell recognition sub-model can effectively balance the accuracy and efficiency of task execution results.

[0016] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0017] Figure 1 A flowchart illustrating the training method of a high-performance misspelling recognition model provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a method for constructing a word knowledge graph, as provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0019] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0020] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0021] Figure 1 The illustration schematically shows a flowchart of a training method for a high-performance misspelling recognition model according to an embodiment of this application. Figure 1 As shown in the embodiments of this application, a training method for a high-performance misspelling recognition model is provided, which may include the following steps: S101. Obtain word association information for all target words in the target word set, and construct a word knowledge graph based on the word association information; S102. Extract the multi-dimensional feature vectors of all target words based on the word knowledge graph, and expand the pre-acquired primary sample set according to the multi-dimensional feature vectors of words to obtain the word and sentence sample set. S103. Construct a misspelling recognition model based on a pre-trained language model. The misspelling recognition model includes a primary recognition main model and a primary recognition sub-model. S104. Based on the word and sentence sample set, complete the iterative training of the primary recognition model to obtain the misspelling recognition model; S105. Input the negative sample set of words and phrases from the word and phrase sample set into the main model for misspelling recognition to perform misspelling recognition, and output the misspelling recognition result set through the main model for misspelling recognition; S106. Use the misspelling recognition result set and the word and sentence sample set to iteratively train the primary recognition sub-model to obtain the misspelling recognition sub-model.

[0022] In this embodiment, the target word set refers to the core words required to cover the application scenario, such as commonly used words in general scenarios. Word association information includes pronunciation information, character shape information, semantic information, and domain information. A word knowledge graph can display word association information in a structured manner, providing a core basis for subsequent feature extraction and sample expansion. The word multidimensional feature vector is obtained by concatenating word misuse label vectors, word difference feature vectors, and associated edge weight vectors. Then, based on the word multidimensional feature vectors, the pre-acquired primary sample set is expanded. This supplements rare errors missing in the primary sample set, solves the problem of the original data being too singular and incomplete in its coverage, provides sufficient and diverse samples for subsequent model training, and improves the model's misspelling recognition accuracy.

[0023] The misspelling detection model comprises a primary main model and a primary sub-model. The primary main model can be built using BERT-base-chinese as its core framework, including a first word segmentation and embedding module, a first graph enhancement module, a first encoding module, and a first classification output module. The first classification output module includes a first error label prediction unit and a first correct word regression unit. The primary sub-model's core framework can be DistilBERT-base-chinese, which includes a second word segmentation and embedding module, a second graph enhancement module, a second encoding module, and a second classification output module. The second classification output module includes a second error label prediction unit and a second correct word regression unit. The primary main model focuses on accurate error identification and correction, and is responsible for learning complex semantics, knowledge associations, and error rules. The primary sub-model emphasizes efficient inference, inheriting the capabilities of the main model and adapting to lightweight scenarios. The misspelling detection model built on this dual-model architecture can meet both high-precision recognition requirements and adapt to efficient scenarios such as edge devices and real-time response.

[0024] During training, a primary recognition model is first trained to obtain the misspelling recognition model. Then, the negative word sample set from the word and sentence sample set is input into the misspelling recognition model for misspelling recognition, resulting in a misspelling recognition result set. This result set contains the misspelling information and correct word / sentence sequence for each negative word sample in the negative word sample set. Subsequently, when training the primary recognition sub-model, a mixed word sample set (a combination of the misspelling recognition result set and the word and sentence sample set) is used to train the primary recognition sub-model. This allows the primary recognition sub-model to inherit the model accuracy of the primary misspelling recognition model, avoiding the primary recognition sub-model learning from scratch, and ensuring both the accuracy and efficiency of its inference.

[0025] In one embodiment, reference is made to Figure 2, obtaining the word association information of all target words in the target word set, and constructing a word knowledge graph based on the word association information includes the following steps: S201. Obtain the multi-dimensional association information of all target words in the target word set; S202. Normalize the multi-dimensional association information to obtain word association information, where the word association information includes word pronunciation information, word glyph information, word semantic information, and word domain information; S203. Complete the multi-dimensional similarity of all target words according to the word association information, where the multi-dimensional similarity includes pronunciation similarity, glyph similarity, semantic similarity, and domain similarity; S204. Use all target words as graph nodes, construct association edges between all graph nodes according to the multi-dimensional similarity, and assign association edge weights to all association edges according to the multi-dimensional similarity to obtain a word knowledge graph.

[0026] In this embodiment, the target word set refers to the core words required to cover the application scenario, such as the commonly used words in the general scenario, that is, the target words, such as "must", "necessary", "try", "wipe", etc., which can be obtained through a Chinese dictionary. The multi-dimensional association information refers to all the original data related to word association, such as pronunciation, glyph, semantics, domain, etc., without format regularization and screening for these target words. For example, the multi-dimensional association information of "amine" may include "pinyin an, tone the fourth tone, radical month, is a nitrogen-containing organic drug, commonly used in the medical field", etc. Normalizing the multi-dimensional association information means converting the multi-dimensional association information of the target words into a unified format and clearly dividing the multi-dimensional association information into word pronunciation information, word glyph information, word semantic information, and word domain information. For example, [amine, pronunciation: an, glyph: month, an, semantics: nitrogen-containing organic compound, domain: medicine]. Then, using the target words as graph nodes, connecting the graph nodes with overlapping word pronunciation information and word glyph information to form a word knowledge graph. For example, for "amine" and "ammonia", the pronunciation is both "an", so there is a pronunciation association, and thus an pronunciation association edge is constructed between the corresponding graph nodes of the two. For "ammonia" and "oxygen", both contain "gas" in the glyph, so a glyph association edge is constructed between the corresponding graph nodes of the two.

[0027] Specifically, calculate the multi-dimensional similarity between all graph nodes. If the pinyin of two words is exactly the same, the pronunciation similarity between them is set to 1. If they are completely different, such as "bei" and "mo", the pronunciation similarity is set to 0. If the initials are the same or the finals are the same, it is set to 0.5. For example, "ba" and "ben", "ban" and "pan". If the partial structures of two words are the same, use the regularization method based on glyph coding for similarity calculation. Commonly used regularization methods based on glyph coding include Wubi coding and Four-Corner Codes, etc. Taking the method of integrating Wubi coding and Four-Corner coding as an example, according to the Wubi coding rules and Four-Corner Codes (such as using the "Four-Corner Method for Looking up Chinese Characters"), for any two words, obtain the codes through the glyph coding library. For example, the Wubi coding of "哑" is "KUGG", and the Four-Corner Code is "60117"; the Wubi coding of "娅" is "VGOG", and the Four-Corner Code is "40417". Then, perform unified format processing on the obtained codes. The Wubi coding is complemented to 4 digits (filled with "X" if it is less than 4 digits to ensure the same length); the Four-Corner Code retains the first 4-digit core code (the complement code is used for auxiliary distinction, and the similarity calculation is mainly based on the core code). Then, calculate the Wubi coding similarity and Four-Corner Code similarity of the two words respectively, and then perform weighted summation on the two similarities to obtain the structure similarity. Among them, the weight of Wubi coding can be greater than the weight of Four-Corner Code. For example, the weight of Wubi coding is 0.6, and the weight of Four-Corner Code is 0.4. This is because Wubi coding is more in line with the logic of stroke combination, and Four-Corner Code focuses more on the overall structure. For semantic similarity, through large-scale corpus training, map the word and semantic information of each target word into a dense vector (such as Word2Vec, GloVe), and then calculate the Euclidean distance or cosine similarity between the two vectors to obtain their semantic similarity. For domain similarity, a topic model can be trained on a corpus containing multiple domains to identify different domain topics, and then the probability distribution of the two belonging to each topic can be output through the trained topic model. Then, use the similarity formula (such as the cosine similarity formula) to calculate the probability distribution similarity between the two to obtain their domain similarity. The cosine similarity formula is as follows:

[0028] where A and B are the probability distribution vectors of two target words respectively, and are the norms of the probability distribution vectors A and B respectively.

[0029] Next, for any similarity in the multidimensional similarity, if any similarity between two graph nodes is greater than or equal to the corresponding preset similarity threshold, an association edge is constructed between them. For example, for pronunciation similarity and structural similarity, the similarity threshold can be set to 0.5, and for semantic similarity, the similarity threshold can be set to 0.7. The multidimensional similarities between graph nodes with associated edges are normalized and then weighted to obtain the weight of the associated edge. The weight required for the weighted average can be determined based on the frequency of error causes. For example, if pronunciation and structural similarity are high, the frequency of typos is high, so their weights are also higher; conversely, if domain and semantic similarity are high, the frequency of typos is low, so their weights are also lower.

[0030] In one embodiment, extracting the multidimensional feature vectors of all target words based on a word knowledge graph includes the following steps: For any graph node, select the neighboring graph nodes from the word knowledge graph; Based on the word knowledge graph, multidimensional similarity and associated edge weights between neighboring graph nodes are extracted, and word misuse labels are generated for graph nodes and neighboring graph nodes by combining multidimensional similarity and associated edge weights. By combining multidimensional similarity and word association information, word difference features between neighboring graph nodes and graph nodes are determined; After vectorizing the word misuse labels, word difference features, and associated edge weights, the vectors are concatenated to obtain the word multidimensional feature vector.

[0031] In this embodiment, neighboring graph nodes of each graph node are screened from the word knowledge graph. The graph nodes having associated edges with the graph node can be used as its neighboring graph nodes. Then, the multi-dimensional similarity and the associated edge weight between the neighboring graph node and the graph node are extracted. Then, the highest similarity in the multi-dimensional similarity is screened out as the reason for the misuse of words between the neighboring graph node and the graph node. The multi-dimensional similarity includes pronunciation similarity, glyph similarity, semantic similarity, and domain similarity. Among the multi-dimensional similarities, whichever similarity is the highest, it is used as the reason for the misuse of words between the neighboring graph node and the graph node. For example, if the pronunciation similarity is the highest, the reason for the misuse of words between the two is that they are similar in pronunciation. Then, the associated edge weight is used as the misuse probability between the two. Then, the misuse probability and the reason for the misuse of words are vectorized and used as the word misuse label between the two graph nodes. For example, pronunciation similarity misuse: 0.7. The word misuse label obtained after vectorization using the One-Hot coding rule is [1, 0, 0, 0, 0.7]. Then, the word difference feature between the neighboring graph node and the graph node is determined by combining the multi-dimensional similarity and the word association information, converting the abstract similarity value into a concrete feature description, enabling the model to clarify the difference between the two, and providing a data basis for the subsequent model to output incorrect word information and correct sentence sequences. First, the difference points between the neighboring graph node and the graph node are determined according to the multi-dimensional similarity. For example, for "amine" and "ammonia", their pronunciation similarity is 1, and there is no difference point. However, their structure similarity is not 1, indicating that there is a difference point in their structures. Therefore, the word structure difference feature can be extracted according to their word association information, which is [structural difference: month, gas]. Then, the word misuse label, the word difference feature, and the associated edge weight are vectorized, and the word misuse label vector, the word difference feature vector, and the associated edge weight vector are obtained respectively. After the three vectors are concatenated, the word multi-dimensional feature vector is obtained. In addition, after the word misuse label vector and the word difference feature vector of each target word are extracted, they are concatenated and used as the associated edge attribute to be embedded into the word knowledge graph to further optimize and update the word knowledge graph.

[0032] In one of the embodiments, the primary sample set includes a primary positive sample set and a primary negative sample set. The steps of obtaining the sentence sample set by sample expansion of the pre-obtained primary sample set according to the word multi-dimensional feature vector are as follows: Screen out multi-type word sets in the primary positive sample set according to the word multi-dimensional feature vector; Screen out similar word sets of the multi-type word sets from the word knowledge graph according to the word multi-dimensional feature vector; Use the similar word sets to replace words in the primary positive sample set to obtain multiple primary perturbation samples; For any primary perturbation sample, use the pre-trained language model to verify the context semantic consistency of the primary perturbation sample; If the contextual semantic consistency verification of the primary perturbation sample passes, the primary perturbation sample will be added to the primary negative sample set. Repeat the above steps until the number of samples in the primary negative sample set reaches the preset standard sample number, and then output the word and phrase sample set.

[0033] In this embodiment, the initial positive sample set refers to the training sample set containing correct sentences and words, while the initial negative sample set refers to the training sample set containing incorrect sentences and words, labeled with the error location and reason. First, a multi-type word set is selected from the initial positive sample set based on the multi-dimensional feature vectors of the words. The multi-type word set refers to a set of target words containing different associated edge weights, different word misuse labels, and different word difference features. For example, it includes target words misused for pronunciation similarity, structural similarity, semantic similarity, and domain similarity. Furthermore, for target words with different word misuse labels, different misuse probabilities are included: target words with a misuse probability greater than or equal to 0.7, target words between 0.7 and 0.4, and target words less than or equal to 0.4. This ensures that the multi-type word set can cover most word misuse scenarios. Next, based on the multi-dimensional feature vectors of the words, similar words associated with all multi-type words in the multi-type word set are selected from the word knowledge graph and integrated into a similar word set. Next, the various types of words in the primary positive sample set are replaced one by one with similar words to obtain multiple primary perturbation samples. This is because the number of negative samples is relatively small compared to the number of positive samples. In order to ensure the balance between the number of positive and negative samples, the primary negative sample set needs to be expanded.

[0034] Next, it is beneficial to use a pre-trained language model to verify the context semantic consistency of the primary perturbation samples and filter out the primary perturbation samples with inconsistent semantics or no practical meaning. This is to ensure that the finally generated negative samples conform to the daily expression logic and avoid misleading the model training. For example, the sentence "他尝拭擦拭桌面" has inconsistent semantics and needs to be excluded. A pre-trained language model adapted to the Chinese scenario (such as BERT-base-chinese, RoBERTa-wwm-ext) can be selected, load the publicly available pre-trained weights, and no additional fine-tuning is required (only for judging semantic smoothness). Taking RoBERTa-wwm-ext as an example, RoBERTa-wwm-ext is a pre-trained language model optimized for Chinese. Based on the RoBERTa architecture and combined with the whole-word masking technology, after removing meaningless characters, unified encoding (can be converted to UTF-8 encoding), and short sentence filtering (directly removing samples with a length ≤ 2 characters) from the primary perturbation samples and splitting them, multiple short sentence perturbation samples are obtained. Then, special tokens are added to each short sentence perturbation sample in the format of [CLS] + short sentence perturbation sample + [SEP]. Next, all the short sentence perturbation samples with special tokens added are sequentially input into the RoBERTa-wwm-ext model with parameter configuration completed. The parameter configuration mainly includes batch size, maximum sequence length, padding strategy, etc. Then, model inference is performed through the RoBERTa-wwm-ext model to output the semantic smoothness probability (the value range is 0 - 1.0, and the higher the value, the smoother the semantics). For the primary perturbation samples with a semantic smoothness probability less than the preset probability threshold (such as 0.5), it is determined that the context semantic consistency verification fails and they need to be removed. For the primary perturbation samples with a semantic smoothness probability greater than or equal to the preset probability threshold (such as 0.5), it is determined that the context semantic consistency verification passes, add a negative sample label (such as label "1") to them, and add them to the primary negative sample set after marking the error reason and error location. Continuously repeat the above process of expanding the primary negative sample set until the number of samples in the primary negative sample set reaches the preset standard sample number (for example, forty thousand), and output it as the word and sentence sample set.

[0035] Through the above steps, high-quality negative samples can be generated, making the positive and negative samples in the finally formed word and sentence sample set balanced and the error types comprehensive, which can provide sufficient and high-quality training materials for the subsequent typo recognition model, thereby improving the recognition accuracy and generalization ability of the model for different types of errors.

[0036] In one embodiment, the primary recognition main model includes a first word segmentation embedding module, a first graph enhancement module, a first encoding module, and a first classification output module. The first classification output module includes a first error label prediction unit and a first correct word regression unit. The primary recognition sub-model includes a second word segmentation embedding module, a second graph enhancement module, a second encoding module, and a second classification output module. The second classification output module includes a second error label prediction unit and a second correct word regression unit.

[0037] In this embodiment, during the construction of the primary recognition main model, BERT-base-chinese can be used as its core framework. Then, a dual-task header layer is extended to BERT-base-chinese to obtain the primary recognition main model. The primary recognition main model includes a first word segmentation and embedding module, a first graph enhancement module, a first encoding module, and a first classification output module. The first classification output module includes a first error label prediction unit and a first correct word regression unit. The first word segmentation and embedding module can use the BERT Chinese word segmenter (WordPiece) to convert the input text into word symbols and generate corresponding word embeddings, position embeddings, and segment embeddings, with an output dimension of 768. The first graph enhancement module is used to extract the input association features and error rule features. Association features refer to words that are related to the input words, and error rule features refer to word misuse label vectors and word difference feature vectors. The extracted association features and error rule features are concatenated with the output of the first word segmentation and embedding module and then input into the first encoding module.

[0038] The first encoding module includes a multi-head self-attention mechanism, a feedforward neural network (FFN), layer normalization (LayerNorm), and residual connections. Each layer of the multi-head self-attention mechanism contains 12 attention heads, enabling parallel capture of semantic relationships (such as contextual dependencies and syntactic structures) at different locations in the text, supporting the contextual understanding required for error detection. Each layer of the feedforward neural network is equipped with two fully connected layers (with GELU activation function) to perform non-linear transformations on the features output by the attention mechanism, enhancing feature representation capabilities. Layer normalization and residual connections are used to stabilize the training process, alleviate the gradient vanishing problem, and ensure effective learning of deep networks. The first classification output module is constructed by removing the default classification output layer of BERT-base-chinese and adding two parallel task-specific header modules, which respectively handle error detection and correct sentence generation. The first error label prediction unit comprises a two-layer fully connected network (intermediate layer with 384 dimensions and ReLU activation function) and a softmax output layer. It is used to label error locations and classify error types when performing misspelling recognition tasks, outputting the error probability and error type label for each word, such as misuse due to similar pronunciation, similar structure, similar semantics, or similar domain. The first correct word regression unit comprises a three-layer fully connected network (intermediate layer with 768 dimensions and GELU activation function) and a linear mapping layer (mapping dimension consistent with the Chinese vocabulary dimension). It is used to regress and generate a text sequence that perfectly matches the correct sentence, based on the contextual features and word information of the erroneous sentence.

[0039] The core framework for building the primary recognition sub-model can be DistilBERT-base-chinese, a lightweight version of BERT-base-chinese, which is more streamlined than the primary recognition main model. It includes a second word segmentation embedding module, a second graph enhancement module, a second encoding module, and a second classification output module. The second classification output module includes a second error label prediction unit and a second correct word regression unit. The second word segmentation embedding module removes segment embedding compared to the first word segmentation embedding module. This is because the misspelling recognition scenario primarily uses single-sentence input, eliminating the need to distinguish between sentence paragraphs. Its output dimension is 768, consistent with the first word segmentation embedding module. The second graph enhancement module has the same function as the first graph enhancement module. The second encoding module has the same structure as the first encoding module, both including a multi-head self-attention mechanism, a feedforward neural network, layer normalization, and residual connections. However, the feedforward neural network in the second module has fewer layers than the first encoding module; for example, the feedforward neural network in the first encoding module has 12 layers, while the feedforward neural network in the second encoding module has 6 layers. The scale of the layer normalization parameters is also halved compared to the first encoding module. The second classification output module includes a second error label prediction unit and a second correct word regression unit. The second error label prediction unit has the same structure as the first error label prediction unit, and the model parameters of the first error label prediction unit can be reused during subsequent training. Only the output layer dimension of the second classification output module needs to be adjusted to match its output parameters, i.e., the error word information and correct word sequence output by the misspelling recognition sub-model. In summary, the model structure and function of the primary recognition main model and the primary recognition sub-model are not significantly different, but the primary recognition sub-model is more lightweight and can identify and infer error words more quickly.

[0040] In one embodiment, the iterative training of the primary recognition master model based on the word and sentence sample set to obtain the misspelling recognition master model includes the following steps: Initialize the primary model parameters of the primary recognition main model; The word and phrase sample set is input into the primary recognition main model after the primary model parameters have been initialized in batches for iterative training, and the weight parameters of the primary model are adjusted according to the iterative training results. When the primary model's main indicator reaches the preset indicator threshold, the iterative training stops, and the misspelling recognition main model is obtained.

[0041] In this embodiment, the primary model parameters of the primary recognition main model are first initialized. The main initialization steps include: reusing the word embeddings, position embeddings, segment embeddings, and Transformer encoder parameters from the BERT-base-chinese pre-trained model to save training time. The first graph enhancement module is initialized, and its mapping matrix can be initialized using Xavier uniform initialization. The parameters of the first incorrect label prediction unit and the first correct word regression unit are initialized, and their fully connected layer weights can be initialized using He normalization, with the bias term initialized to 0. Simultaneously, training parameters are configured. The optimizer can be set to AdamW, and the loss function can be set to joint loss, for example, joint loss function = 0.6 × classification loss (cross-entropy loss) + 0.4 × generation loss (mean squared error). The batch size (e.g., 32), initial learning rate (e.g., 2e-5), and maximum number of training iterations (e.g., 20) are also set. Next, the word and sentence sample set is input into the primary recognition main model in batches according to the set batch size for iterative training. First, forward propagation is performed, then the classification loss for each batch is calculated using the joint loss function, including error localization, error type prediction error, and the deviation between the generated sentence and the correct sentence. The total loss is then summed. The total loss is backpropagated to the parameters of each layer, and the model parameters (including the weights and biases of the Transformer encoder, the first graph augmentation module, and the first classification output module) are updated through the AdamW optimizer. The word and sentence sample set is then divided into a word and sentence training set and a word and sentence validation set (which can be divided in an 8:2 ratio). During training, training logs for each batch are stored, including loss values, learning rates, and parameter update magnitudes. This iterative training is repeated multiple times until the preset maximum number of training iterations is reached, or the primary recognition main model's main model metrics reach preset threshold values. At this point, iterative training stops, resulting in a completed misspelling recognition main model. The primary recognition main model reaching the preset threshold values ​​means that the misspelling accuracy, error type accuracy, and generated sentence BLEU score, among other main model metrics, reach their respective preset threshold values, indicating that the model training is complete. Among them, the error recognition accuracy is calculated as the number of correctly identified errors divided by the total number of validation samples, with a threshold of 95%. The error type accuracy is calculated as the number of samples correctly identified by error type divided by the number of samples with errors identified, with a corresponding threshold of 90%. The generated sentence BLEU score refers to the similarity between the generated correct sentence and the labeled correct sentence, which can be vectorized separately and calculated using pre-calculated similarity, with a corresponding threshold of 0.8. Through the above steps, a high-accuracy misspelling recognition master model can be trained.

[0042] In one embodiment, the steps of inputting the negative sample set of words from the word and phrase sample set into the main model for misspelling recognition, and outputting the misspelling recognition result set through the main model for misspelling recognition include the following: The negative sample set of words is input into the main model for misspelling recognition in batches. The negative sample set of words is segmented by the first word segmentation and embedding module to obtain the negative sample word set. The negative sample word set is input into the first graph enhancement module. The first graph enhancement module extracts the associated word features and associated attribute features of the negative sample word set, and then fuses the associated word features and associated attribute features into a composite feature embedding vector. The composite feature embedding vector is input into the first encoding module, and the word composite feature vector of the composite feature embedding vector is extracted by the first encoding module. The first classification output module completes the prediction of erroneous words in the compound feature vector of words and characters, and outputs the erroneous word information and correct word and sentence sequence in the negative sample set of words and characters according to the prediction results; By integrating the erroneous word information and correct word sequence from all batches of the negative word sample set, a set of misspelling recognition results is obtained.

[0043] In this embodiment, the negative sample set of words is input into the main model for misspelling recognition in batches. First, the first word segmentation and embedding module is started, and the BERT-base-chineseWordPiece Chinese word segmenter, consistent with the main model training phase, is called to perform word-level word segmentation on each negative sample word set. The negative samples in the complete negative sample set are split into the smallest semantic units, resulting in a sequence of smallest semantic units, i.e., the negative sample word set. Markers are forcibly added to the beginning and end of each smallest semantic unit sequence, such as adding [CLS] to the beginning and [SEP] to the end. For sequences that exceed the maximum input length of the misspelling recognition main model (e.g., 512 smallest semantic units), the core segment containing the erroneous character is truncated first. For smallest semantic unit sequences shorter than the maximum length, [PAD] (padding marker) is used to pad to the maximum length to ensure that the length of the smallest semantic unit sequence is uniform across all batches. Then, a corresponding attention mask vector is generated for each smallest semantic unit sequence.

[0044] Each smallest semantic unit in the negative sample word set is mapped to each graph node in the word knowledge graph, and the associated word features and associated attribute features corresponding to that node are obtained. For each graph node corresponding to the smallest semantic unit, the Word2Vec mean vector of the other graph nodes connected in the word knowledge graph is used as the associated word features of the smallest semantic unit. The associated attribute features refer to the core attribute information of the smallest semantic unit extracted from the word knowledge graph, including but not limited to error tendency attributes, i.e., whether it is a high-frequency error word of similar-looking characters or a high-frequency error word of similar-sounding characters, which can be represented by Boolean values ​​1 / 0, and grammatical attributes, such as part of speech (noun, verb, or adjective, which can be represented by One-Hot encoding), etc. All core attribute information is concatenated in a preset order and mapped to a 64-dimensional associated attribute feature vector through a 1-layer fully connected network to ensure the dimensionality uniformity of the associated attribute features. The associated word features and associated attribute features of each smallest semantic unit are extracted and directly concatenated in the order of associated word features first and associated attribute features last to obtain an initial fusion vector. Then, layer normalization is performed on the initial fusion vector to eliminate the differences in the numerical range of different feature dimensions. Finally, a composite feature embedding vector corresponding one-to-one with the negative sample word set is output to ensure that each smallest semantic unit in the negative sample word set corresponds to a composite feature embedding vector.

[0045] Next, the composite feature embedding vector is input into the first encoding module, where it is concatenated dimensionally with the word embeddings, position embeddings, and segment embeddings extracted by the first word segmentation embedding module to obtain the concatenated feature embedding vector. Then, the first encoding module's 12-layer Transformer encoder iteratively processes the feature embedding vector, transforming it into a feature vector containing deep semantic information, knowledge association information, and contextual dependency information. All feature vectors are then combined according to the original sequence (i.e., the sequence of smallest semantic units from the word segmentation stage) to form a composite word feature vector corresponding to each smallest semantic unit in the negative sample word set. This composite word feature vector retains both the independent features of each smallest semantic unit and the global association information between them.

[0046] The composite feature vector of words is input into the first classification output module. The first error label prediction unit adopts a hierarchical structure of a 2-layer fully connected network + Softmax activation function. The 2-layer fully connected network maps the high-dimensional composite feature vector of words to an (N+1)-dimensional output feature vector, where N is the total number of preset error types, including misuse due to similar pronunciation, misuse due to similar structure, misuse due to similar semantics, or misuse due to similar domain, etc. The additional dimension corresponds to the no-error category. After the output of the second fully connected network, the output feature vector is normalized by the Softmax activation function to obtain the corresponding (N+1)-dimensional probability distribution, where the probability value of each dimension ranges from [0,1], and the sum of the probabilities of all dimensions is 1. This probability distribution contains two core types of information: one is the probability of no-error category, used to determine whether the smallest semantic unit is an error word; the other is the probability corresponding to each error type, used to determine the specific type of error word. For example, the probability distribution of the smallest semantic unit might be [no error: 0.02, similar-looking character error: 0.95, similar-sounding character error: 0.03, ...], which intuitively reflects the error attributes of the smallest semantic unit. Then, based on the position of the erroneous word in the sequence of smallest semantic units, its error word information is output, including the error type and its corresponding error position.

[0047] By fusing the word composite feature vector of the misspelled word with its corresponding error position, a binary mask vector consistent with the length of the minimum semantic unit sequence can be constructed. The mask bit corresponding to the misspelled word is set to 1, and the mask bit corresponding to the correct word is set to 0. Multiply this mask vector element-wise with the word composite feature vector to obtain a composite enhanced feature vector. Through this method, the feature weight of the misspelled word can be enhanced while retaining the semantic features of the correct word, providing targeted support for accurate correction. Input the composite enhanced feature vector into the first correct word regression unit. The first correct word regression unit adopts a structure of 3-layer fully connected network + linear mapping. The first two fully connected networks perform deep non-linear transformation on the composite enhanced feature vector. The third fully connected network should set the composite enhanced feature vector after non-linear transformation to an M-dimensional composite enhanced feature vector consistent with the size of the Chinese word list. M is the total number of minimum semantic units in the BERT-base-chinese word list. Then, normalize the M-dimensional composite enhanced feature vector through the Softmax activation function to obtain the probability distribution of the candidate corrected words corresponding to each misspelled word. For each misspelled word, select the candidate corrected word with the highest probability in the probability distribution of the candidate corrected words to perform correction and replacement, and the correct words are not processed, forming a corrected word vector sequence. Then decode the word vector sequence to convert it into natural Chinese text to obtain the correct sentence sequence. During the decoding process, special characters such as [CLS], [SEP], [PAD], etc. are automatically filtered, and at the same time, invalid information such as redundant spaces and consecutive punctuation marks is cleaned to ensure the fluency and normality of the output text. For example, if the word vector sequence is [CLS], he, today, very, angry, [SEP], it is decoded as "He is very angry today".

[0048] Integrate the error word information and correct sentence sequence of all batches in the word negative sample set to obtain the misspelled word recognition result set, providing a data basis for the training of the subsequent primary recognition sub-model.

[0049] In one embodiment, using the misspelled word recognition result set and the sentence sample set to perform model iterative training on the primary recognition sub-model, the steps for obtaining the misspelled word recognition sub-model are as follows: Mix and integrate the misspelled word recognition result set and the sentence sample set into a word mixed sample set; Extract the main model parameters of the misspelled word recognition main model, and complete the parameter initialization of the primary recognition sub-model according to the main model parameters; Input the word mixed sample set into the primary recognition sub-model that has completed parameter initialization for model iterative training until the sub-model index of the primary recognition sub-model reaches the preset first index threshold to obtain the basic recognition sub-model; Screen out the anchor training set in the word mixed sample set according to the sub-model index; Extract the anchor semantic features and anchor error features from the anchor training set, and combine the anchor semantic features and anchor error features to select the anchor positive sample set and anchor negative sample set from the word-mixed sample set; By integrating the anchor training set, the anchor positive sample set, and the anchor negative sample set, an enhanced sample set is obtained. By using an enhanced sample set to perform model comparison enhancement training on the basic recognition sub-model, a misspelling recognition sub-model is obtained.

[0050] In this embodiment, the data formats of the typo recognition result set and the sentence and word sample set are aligned to ensure that fields such as incorrect sentences, correct sentences, incorrect positions, and error types are exactly matched. At the same time, duplicate samples in the typo recognition result set and the sentence and word sample set are removed, and only unique valid samples are retained. Then, the two are mixed to obtain a word and character mixed sample set, and the source of each sample in the word and character mixed sample set (typo recognition result set or sentence and word sample set) is recorded for subsequent training analysis. Then, the main model parameters of the main typo recognition model are extracted. The main model parameters include the key layer parameters of the Transformer encoder in the first encoding module, the layer normalization weights of the first graph enhancement module, and the fully connected layer weights and biases of the first classification output module, etc. The main model parameters of the main typo recognition model are directly reused to complete the parameter initialization of the primary recognition sub-model. At the same time, the training parameters of the primary recognition sub-model are set, including the batch size (such as 64) and the initial learning rate (such as 3e-5). The loss function can follow the joint loss function used by the main typo recognition model. The word and character mixed sample set is divided into a word and character mixed training set and a word and character mixed validation set according to a certain ratio (such as an 8:2 ratio). Then, the word and character mixed training set is input into the primary recognition sub-model that has completed parameter initialization, and the same iterative training steps as the main typo recognition model are executed until, during the validation process, the sub-model metrics of the primary recognition sub-model reach the preset first metric threshold. The sub-model metrics include accuracy and BLEU value, etc., such as the accuracy of error types ≥ 85% and the BLEU value ≥ 0.75, to obtain a basic recognition sub-model that has completed primary model training. Then, the basic recognition sub-model is enhanced trained. The samples recognized as incorrect in each word and character mixed validation set are recorded, and these samples are integrated into an anchor training set. The anchor semantic features and anchor error features of the anchor training set are extracted. They can be input into the basic recognition sub-model for feature extraction. Although the final output result of the basic recognition sub-model has deficiencies, it can still be used for feature extraction. The anchor semantic features are extracted by the Transformer encoder in the basic recognition sub-model and output by the second encoding module. The anchor error features are output by the second graph enhancement module, which includes error association features and error attribute features. The error association features refer to the形近 / 音近 / 语义关联字词 of the incorrect word, such as "份" associated with "愤", "分", "粉". The error attribute features refer to the error tendency of the incorrect word, such as high-frequency words with形近错误. Then, according to the anchor semantic features and anchor error features of the anchor training set, samples with similar semantics and the same error type as the anchor samples are screened out from the word and character mixed training set and integrated into an anchor positive sample set. It can be screened by calculating the cosine similarity of their semantic features (threshold ≥ 0.85) to ensure semantic consistency. At the same time, samples with similar semantics and different error types from the anchor samples are screened out from the word and character mixed training set and integrated into an anchor negative sample set.Next, an augmented sample set is constructed by combining one anchor sample with one positive anchor sample and one negative anchor sample. This augmented sample set is then divided into an augmented training set and an augmented validation set at a certain ratio (e.g., 9:1). The augmented training set is then input into the basic recognition sub-model in batches for comparative augmented training. Forward propagation is performed to calculate the total loss, and backpropagation is used to update the parameters until the preset maximum number of training iterations is reached, resulting in a completed misspelling recognition sub-model.

[0051] The misspelling recognition main model and misspelling recognition sub-model trained through the above steps can effectively improve the misspelling recognition accuracy and efficiency of the misspelling recognition model.

[0052] This application also provides a machine-readable storage medium storing instructions for causing a machine to execute a training method for a high-performance misspelling recognition model according to any one of the preceding claims.

[0053] This application also provides a training device for a high-performance misspelling recognition model, comprising: The memory is configured to store instructions; and A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement a training method for a high-performance misspelling recognition model according to any one of the preceding statements.

[0054] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0055] The memory can be an internal storage unit of a computer device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) provided on the computer device. Furthermore, the memory can be a combination of internal storage units and external storage devices of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0056] This application embodiment also provides a machine-readable storage medium storing instructions that cause a machine to execute the training method of the high-performance misspelling recognition model described above.

[0057] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0059] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0061] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0062] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0063] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0064] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0065] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A training method for a high-performance misspelling recognition model, characterized in that, The method includes the following steps: Obtain word association information for all target words in the target word set, and construct a word knowledge graph based on the word association information; Based on the word knowledge graph, the multi-dimensional feature vectors of all target words are extracted, and the pre-acquired primary sample set is expanded according to the multi-dimensional feature vectors of words to obtain the word and sentence sample set; A misspelling recognition model is constructed based on a pre-trained language model. The misspelling recognition model includes a primary recognition main model and a primary recognition sub-model. The initial recognition model was iteratively trained based on the word and sentence sample set to obtain the misspelling recognition model. The negative sample set of words in the word and sentence sample set is input into the main model for misspelling recognition to identify misspellings, and the misspelling recognition result set is output through the main model for misspelling recognition. The initial recognition sub-model is trained iteratively using the misspelling recognition result set and the word and sentence sample set to obtain the misspelling recognition sub-model.

2. The method according to claim 1, characterized in that, The process of obtaining word association information for all target words in the target word set and constructing a word knowledge graph based on the word association information includes the following steps: Obtain multidimensional association information of all target words in the target word set; The multidimensional association information is standardized to obtain word association information, which includes word pronunciation information, word shape information, word semantic information, and word domain information. The multidimensional similarity of all target words is calculated based on word association information. The multidimensional similarity includes pronunciation similarity, character shape similarity, semantic similarity, and domain similarity. All target words are used as graph nodes. Connection edges are constructed between all graph nodes based on multidimensional similarity, and connection edge weights are assigned to all connection edges based on multidimensional similarity to obtain a word knowledge graph.

3. The method according to claim 2, characterized in that, The process of extracting multidimensional feature vectors of all target words based on word knowledge graphs includes the following steps: For any graph node, select the neighboring graph nodes from the word knowledge graph; Based on the word knowledge graph, multidimensional similarity and associated edge weights between neighboring graph nodes are extracted, and word misuse labels are generated for graph nodes and neighboring graph nodes by combining multidimensional similarity and associated edge weights. By combining multidimensional similarity and word association information, word difference features between neighboring graph nodes and graph nodes are determined; After vectorizing the word misuse labels, word difference features, and associated edge weights, the vectors are concatenated to obtain the word multidimensional feature vector.

4. The method according to claim 2, characterized in that, The primary sample set includes a primary positive sample set and a primary negative sample set. The step of expanding the pre-obtained primary sample set according to the multi-dimensional feature vectors of words to obtain the word and sentence sample set includes the following steps: Based on the multi-dimensional feature vectors of words, a primary positive sample set of multiple types of words is selected; Similar word sets of various types are selected from the word knowledge graph based on the multi-dimensional feature vectors of words; By using a set of similar words to replace words in the primary positive sample set, multiple primary perturbation samples are obtained; For any primary perturbation sample, it is beneficial for the pre-trained language model to perform contextual semantic consistency verification on the primary perturbation sample; If the contextual semantic consistency verification of the primary perturbation sample passes, the primary perturbation sample will be added to the primary negative sample set. Repeat the above steps until the number of samples in the primary negative sample set reaches the preset standard sample number, and then output the word and phrase sample set.

5. The method according to claim 1, characterized in that, The primary recognition main model includes a first word segmentation embedding module, a first graph enhancement module, a first encoding module, and a first classification output module. The first classification output module includes a first error label prediction unit and a first correct word regression unit. The primary recognition sub-model includes a second word segmentation embedding module, a second graph enhancement module, a second encoding module, and a second classification output module. The second classification output module includes a second error label prediction unit and a second correct word regression unit.

6. The method according to claim 5, characterized in that, The iterative training of the primary recognition model based on the word and sentence sample set to obtain the misspelling recognition model includes the following steps: Initialize the primary model parameters of the primary recognition main model; The word and phrase sample set is input into the primary recognition main model after the primary model parameters have been initialized in batches for iterative training, and the weight parameters of the primary model are adjusted according to the iterative training results. When the primary model's main indicator reaches the preset indicator threshold, the iterative training stops, and the misspelling recognition main model is obtained.

7. The method according to claim 6, characterized in that, The process of inputting the negative sample set of words from the word and phrase sample set into the main model for misspelling recognition to identify misspellings, and then outputting the misspelling recognition result set through the main model for misspelling recognition, includes the following steps: The negative sample set of words is input into the main model for misspelling recognition in batches. The negative sample set of words is segmented by the first word segmentation and embedding module to obtain the negative sample word set. The negative sample word set is input into the first graph enhancement module. The first graph enhancement module extracts the associated word features and associated attribute features of the negative sample word set, and then fuses the associated word features and associated attribute features into a composite feature embedding vector. The composite feature embedding vector is input into the first encoding module, and the word composite feature vector of the composite feature embedding vector is extracted by the first encoding module. The first classification output module completes the prediction of erroneous words in the compound feature vector of words and characters, and outputs the erroneous word information and correct word and sentence sequence in the negative sample set of words and characters according to the prediction results; By integrating the erroneous word information and correct word sequence from all batches of the negative word sample set, a set of misspelling recognition results is obtained.

8. The method according to claim 1, characterized in that, The step of iteratively training the primary recognition sub-model using the misspelling recognition result set and the word and sentence sample set to obtain the misspelling recognition sub-model includes the following steps: The result set of misspelled words and the sample set of words and sentences are mixed and integrated into a mixed sample set of words and sentences. Extract the main model parameters of the main model for identifying erroneous characters, and initialize the parameters of the primary recognition sub-model based on the main model parameters; Input the mixed word sample set into the primary recognition sub-model that has completed parameter initialization for iterative training of the model until the sub-model index of the primary recognition sub-model reaches the preset first index threshold, and obtain the basic recognition sub-model. Anchor point training set is selected from the word-mixed sample set based on the sub-model indicators; Extract the anchor semantic features and anchor error features from the anchor training set, and combine the anchor semantic features and anchor error features to select the anchor positive sample set and anchor negative sample set from the word-mixed sample set; By integrating the anchor training set, the anchor positive sample set, and the anchor negative sample set, an enhanced sample set is obtained. By using an enhanced sample set to perform model comparison enhancement training on the basic recognition sub-model, a misspelling recognition sub-model is obtained.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to execute the training method of the high-performance misspelling recognition model according to any one of claims 1 to 8.

10. A training device for a high-performance misspelling recognition model, characterized in that, include: The memory is configured to store instructions; as well as A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement a training method for a high-performance misspelling recognition model according to any one of claims 1 to 8.

Images