Chinese character correction method and device

Abstract

The application discloses a Chinese correction method and device. The application comprises: determining a correction scene according to an initial Chinese text to be corrected; constructing a sentence classification data set according to a Chinese training text corresponding to the correction scene, and constructing a sentence classifier by using the sentence classification data set; constructing a correction training model based on the sentence classifier, generating multi-path correction data according to a dictionary corresponding to the correction scene and the Chinese training text, and performing adaptive dynamic mask correction training on the correction training model by using the multi-path correction data to construct a Chinese correction model; and sending the initial Chinese text into the trained Chinese correction model to obtain a target Chinese text after correction. The application improves the semantic recognition ability and generalization ability of Chinese correction, reduces the training cost, and improves the correction accuracy, stability and reliability.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a Chinese rectification method and device. Background Art

[0002] With the development of artificial intelligence technology, Chinese error correction has become increasingly important. Chinese error correction can significantly improve the accuracy of language understanding and help the system better grasp the text semantics. In human-computer interaction, by automatically correcting user input errors, the user experience can be improved and the system reliability can be enhanced. For data processing, Chinese error correction can optimize the quality of training data, increase data diversity, and thus improve the model training effect and generalization ability.

[0003] Although Chinese error correction provides strong support for the wide application of artificial intelligence, there are still the following problems: on the one hand, over-correction and mis-correction are likely to occur during error correction, resulting in inaccurate error correction results. For example, in the sentence "This is an exciting moment", when correcting the error, "moment" is changed to "interval". On the other hand, the training corpus is limited, resulting in low training efficiency of the Chinese error correction model and insufficient generalization ability of the model, making it difficult to adapt to texts in different fields and styles, which affects the stability and reliability of the error correction effect. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a Chinese rectification method and device, aiming to improve the accuracy, stability and reliability of Chinese error correction to solve the problems in the background art.

[0005] To achieve the above object, according to the first aspect of the embodiments of the present invention, a Chinese rectification method is provided, including:

[0006] Determine a rectification scenario according to the initial Chinese text to be rectified;

[0007] Construct a sentence classification data set according to the Chinese training text corresponding to the rectification scenario, and use the sentence classification data set to construct a sentence classifier, which is used to judge the relevant attributes of the position and semantics between the input sentences;

[0008] Construct a rectification training model based on the sentence classifier, generate multi-channel rectification data according to the dictionary corresponding to the rectification scenario and the Chinese training text, and perform adaptive dynamic mask rectification training on the rectification training model using the multi-channel rectification data to construct a Chinese rectification model;

[0009] Send the initial Chinese text into the trained Chinese rectification model to obtain the rectified target Chinese text.

[0010] Further, the method is based on a knowledge base, and the knowledge base includes scenario identifiers and scenario features corresponding to each scenario;

[0011] The correction scenario is determined based on the initial Chinese text to be corrected, including:

[0012] The TF-IDF algorithm was used to obtain the high-frequency word features of the initial Chinese text;

[0013] The high-frequency word features are matched with the scene features corresponding to each scene identifier to obtain the target scene identifier, which is used to determine the correction scene.

[0014] Furthermore, the method is based on a knowledge base, which includes multiple Chinese training texts corresponding to the correction scenario;

[0015] A sentence classification dataset is constructed based on the Chinese training text corresponding to the aforementioned correction scenario. A sentence classifier is then built using this dataset. The sentence classifier is used to determine the positional and semantic correlation attributes between sentences, including:

[0016] Multiple Chinese training texts are obtained, and the sentences in the Chinese training texts are identified in chronological order to obtain the sequence identifiers corresponding to each sentence;

[0017] Positive samples of the sentence classification dataset are constructed using consecutive sentences and their sequence identifiers from the same Chinese training text; negative samples of the sentence classification dataset are constructed using non-consecutive sentences and their sequence identifiers from different Chinese training texts; wherein each positive sample includes a first sentence and its sequence identifier, a second sentence and its sequence identifier, the first sentence and the second sentence being consecutive sentences; each negative sample includes a third sentence and its sequence identifier, a fourth sentence and its sequence identifier, the third sentence and the fourth sentence being non-consecutive sentences;

[0018] Calculate the relevance of each positive sample and each negative sample in the statement classification dataset, and construct the statement classifier based on the statistical results of the relevance of the positive and negative samples.

[0019] Further, the relevance of each positive sample and the relevance of each negative sample in the statement classification dataset are calculated, and the statement classifier is constructed based on the statistical results of the relevance of the positive and negative samples, including:

[0020] The Word2Vec model is used to map each statement and its sequence identifier in the positive and negative samples into multi-dimensional space vectors, resulting in positive sample multi-dimensional space vector pairs and negative sample multi-dimensional space vector pairs.

[0021] Calculate the correlation p between the two multidimensional space vectors in the i-th positive sample multidimensional space vector pair.i And calculate the mean correlation of positive samples. Where N1 is the number of the positive sample multidimensional space vector pairs;

[0022] Calculate the correlation n between the two multidimensional space vectors in the j-th negative sample multidimensional space vector pair. j And calculate the mean correlation of negative samples. Where N2 is the number of the negative sample multidimensional space vector pairs;

[0023] The decision threshold K of the statement classifier is obtained based on the mean relevance of the positive samples and the mean relevance of the negative samples, where,

[0024] Construct the statement classifier and set the decision threshold of the statement classifier to K.

[0025] Furthermore, the error correction training model includes the statement classifier and the mask completion module;

[0026] Multi-path correction data is generated based on the dictionary corresponding to the correction scenario and the Chinese training text. Adaptive dynamic mask correction training is then performed on the correction training model using this multi-path correction data, including:

[0027] The Chinese training text is segmented based on a dictionary corresponding to the aforementioned correction scenario to obtain segmented text;

[0028] The word-segmented text is used to train the error correction training model in multiple rounds. In each round of training, training sentence pairs are obtained from the word-segmented text and the following steps are performed:

[0029] Read the training sentence pairs from the segmented text for this round of training, copy the training sentence pairs multiple times, and send each copy into a random mask channel;

[0030] Each of the random mask channels selects its own mask ratio from the current mask ratio range, and performs full-word masking on the training statement pair according to the mask ratio to obtain its own full-word masked statement pair;

[0031] Each full-word masked statement pair is fed into the statement classifier of the bias correction training model to determine its position and semantic relevance attributes, thereby obtaining the corresponding relevance attributes. The full-word masked statement pair and the corresponding relevance attributes are then fed into the mask completion module of the bias correction training model, and the mask-completed statement is obtained through a forward propagation algorithm. Based on the comparison results of all the mask-completed statements, the correct statement for this round of training is determined, and the mask completion module of the bias correction training model is optimized through a backward propagation algorithm.

[0032] The error correction accuracy of each random mask channel is calculated based on the correct statements obtained in this round of training, and the current mask ratio range is corrected. The corrected mask ratio range is used as the mask ratio range for the next round.

[0033] Furthermore, the Chinese training text is segmented based on a dictionary corresponding to the aforementioned correction scenario to obtain segmented text, including:

[0034] Based on the dictionary corresponding to the aforementioned correction scenario, each sentence of the Chinese training text is converted into a corresponding phase-acyclic graph.

[0035] The maximum possible path is calculated based on the directed acyclic graph corresponding to each sentence, and the word segmentation text is obtained based on the calculation result of the maximum possible path.

[0036] Furthermore, based on the comparison results of all the masked completion statements, the correct statements for this round of training are determined, including:

[0037] All masked completion statements are aligned in word position. For each word position, the corresponding word is compared. The word with the highest probability of occurrence at each word position is determined as the target word. Based on the word position, the target words are combined to form the correct statement.

[0038] Furthermore, based on the correct statements obtained in this round of training, the error correction accuracy of each random mask channel is calculated, and the current mask ratio range is corrected. The corrected mask ratio range is used as the mask ratio range for the next round, including:

[0039] Align the mask completion statement of the t-th random mask channel with the correct statement by word position, compare the words corresponding to each word position, and determine the error correction accuracy C of each mask completion statement. t t = 1, 2, ..., N3, where N3 is the number of the random mask channels;

[0040] Upper limit of the current mask ratio range and lower limit Make corrections, adjusting the mask ratio range. The mask ratio range to be used in the next round; the corrected formula is as follows:

[0041]

[0042] p t For the t-th random mask channel, in the current training round, the mask ratio range is... The proportion of the mask randomly selected in the middle.

[0043] Furthermore, each of the random masking channels selects its own masking ratio from the current masking ratio range, and performs full-word masking on the training statement pair according to the masking ratio to obtain its own full-word masked statement pair, including:

[0044] In the current training round, the t-th random mask channel selects from the mask ratio range The proportion of the mask randomly selected in the middle p t ;

[0045] Calculate the number of words s that need full word masking in the current round of training, s = Roundup(N4*p t N4 is the total number of words in the training statement, and Roundup() is the round-up function;

[0046] Randomly generate s random integers d1, d2, ..., dn in the interval [1, N4]. s The training statement pairs are numbered d1, d2, ..., d... s The words are fully masked to obtain the fully masked sentence pairs.

[0047] According to a second aspect of the present invention, a Chinese character correction device is provided, comprising:

[0048] The scenario determination module is used to determine the correction scenario based on the initial Chinese text to be corrected;

[0049] The classifier building module is used to construct a sentence classification dataset based on Chinese training text corresponding to the correction scenario, and to construct a sentence classifier using the sentence classification dataset. The sentence classifier is used to determine the semantic correlation attributes between input sentences.

[0050] The error correction model construction module is used to construct an error correction training model based on the sentence classifier, generate multi-path error correction data according to the dictionary corresponding to the error correction scenario and the Chinese training text, and use the multi-path error correction data to perform adaptive dynamic mask error correction training on the error correction training model to construct a Chinese error correction model.

[0051] The error correction execution module is used to feed the initial Chinese text into the trained Chinese error correction model to obtain the corrected target Chinese text.

[0052] The embodiments of the present invention have the following advantages or beneficial effects:

[0053] This invention extracts features from the text to be corrected to determine the correction scenario, selectively selects Chinese training texts that are suitable for the correction scenario, improves the semantic recognition ability of the Chinese correction model in specific correction scenarios, ensures the correct recognition of the semantics of the text to be corrected, and improves the correction accuracy and reliability of the obtained Chinese correction model.

[0054] This invention improves the ability of the error correction training model to identify semantic correlations between sentences by constructing a sentence classifier dataset and a sentence classifier adapted to the error correction scenario using Chinese training texts corresponding to the error correction scenario, reduces the interference of irrelevant sentences on semantic recognition, and improves the error correction stability of the final Chinese error correction model.

[0055] This invention provides an embodiment of adaptive dynamic masking correction training for a correction training model by generating multiple correction data. It obtains a mask ratio range corresponding to the correction scenario based on the semantic features of the text to be corrected. Based on this range, different random masks are applied to multiple sentences, ensuring that the same sentence is trained with different mask ratios and different words. This increases the training speed of the correction training model while simultaneously expanding the training parameter set, improving the error correction accuracy and generalization ability of the Chinese correction model while reducing training costs.

[0056] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained through the details specifically pointed out in the description and drawings. Attached Figure Description

[0057] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0058] Figure 1 This is a schematic diagram of the main process of a Chinese character correction method according to an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram illustrating an application scenario of the Chinese character correction method in another embodiment of the present invention.

[0060] Figure 3 yes Figure 2 The main flowchart of the Chinese character correction method in the scenario shown.

[0061] Figure 4 yes Figure 2 The main flowchart for classifier construction in the scenario shown.

[0062] Figure 5 This is a specific example of a directed acyclic graph used for word segmentation in an embodiment of the present invention.

[0063] Figure 6 This is a schematic diagram of the word segmentation results in an embodiment of the present invention.

[0064] Figure 7 yes Figure 2 The main process of each round of training for the bias correction training model in the illustrated embodiment.

[0065] Figure 8 This is a schematic diagram of a multi-channel random mask in an embodiment of the present invention.

[0066] Figure 9 This is a schematic diagram of the main modules of the Chinese character correction device according to an embodiment of the present invention; Detailed Implementation

[0067] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0068] Figure 1 This is a schematic diagram of the main flow of a Chinese character correction method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the Chinese character correction method in this embodiment includes the following steps S101 to S104.

[0069] Step S101: Determine the correction scenario based on the initial Chinese text to be corrected.

[0070] Understandably, in Chinese text processing, the correction scenario refers to the specific context or domain in which error correction or optimization is required, determined based on the text's characteristics and contextual semantics. Different correction scenarios may involve different language styles, terminology, and semantic structures. For example, official documents, contracts, and literary texts all have different linguistic features and common error types. Therefore, identifying the correction scenario of the text to be corrected is beneficial for adopting targeted correction strategies and correction training data (containing specific error types) for correction processing.

[0071] Specifically, in some embodiments of the present invention, determining the correction scenario based on the initial Chinese text to be corrected involves first classifying the initial Chinese text to be corrected and determining the corresponding classification identifier, and then determining the correction scenario corresponding to the initial Chinese text based on the preset mapping relationship between the classification identifier and the scenario. In other embodiments of the present invention, determining the correction scenario based on the initial Chinese text to be corrected involves extracting text features from the initial Chinese text to be corrected and determining the correction scenario based on the text features.

[0072] Step S102: Construct a sentence classification dataset based on the Chinese training text corresponding to the correction scenario, and construct a sentence classifier using the sentence classification dataset. The sentence classifier is used to determine the positional and semantic correlation attributes between the input sentences.

[0073] Understandably, positional and semantic relevance refers to the degree of correlation between two or more statements, words, or text fragments in terms of position and semantics within a paragraph or text. If two statements are adjacent in position and related in content, theme, or logic, they are considered to have positional and semantic relevance; conversely, if they are not related in these aspects, they are considered not adjacent in position or not semantically related. Therefore, positional and semantic relevance can include two values: adjacent in position and semantically related (true), and not adjacent in position or not semantically related (false). In some embodiments of this invention, the output of the corresponding statement classifier is the probability of these two values, and the relevance of statements is determined by the magnitude of the probability.

[0074] Understandably, the statements fed into the statement classifier can be one, two (statement pairs), or multiple statements. Two value classes can still be used to label the positional and semantic correlation attributes between the statements for classification. Specifically, for a single statement, it can be directly marked as false; for multiple statements, true can be used to indicate that the statements are semantically related and geographically close, while false can be used to represent other cases, such as at least one statement being not adjacent to other statements or semantically unrelated, thus avoiding the introduction of noise due to semantically unrelated statements used for training.

[0075] Step S103: Construct a correction training model based on the sentence classifier, generate multi-path correction data according to the dictionary corresponding to the correction scenario and the Chinese training text, and use the multi-path correction data to perform adaptive dynamic mask correction training on the correction training model to construct a Chinese correction model.

[0076] Specifically, in some embodiments of the present invention, the error correction training model built based on the statement classifier includes a statement classifier and a mask completion module. The statement classifier is a pre-module of the mask completion module, and the output of the statement classifier is sent to the mask completion module.

[0077] Specifically, in this embodiment and some embodiments of the present invention, the Chinese training text is segmented based on a dictionary corresponding to the correction scenario to obtain segmented text; the segmented text is used to train the correction training model in multiple rounds. In each round of training, training sentences are obtained from the segmented text and multi-way random masks are applied to obtain multiple different masked sentences. The correction training model is trained using the multi-way masked sentences and the mask ratio for the next round is adjusted according to the accuracy until the training ends.

[0078] Step S104: The initial Chinese text is fed into the trained Chinese text correction model to obtain the corrected target Chinese text.

[0079] Specifically, in this embodiment and some embodiments of the present invention, in step S104, the initial Chinese text is segmented based on the dictionary corresponding to the correction scenario to obtain the corresponding segmented text to be corrected. The correction sentences are read from the segmented text to be corrected in the order of sentence position and multi-way random masking is performed. The resulting multi-way different correction mask sentences are sent to the trained Chinese correction model. The Chinese correction model classifies the correction mask sentences to be corrected through its sentence classifier, identifies the positional and semantic related attributes between the input correction mask sentences, and marks them. The correction sentences and their related attributes are input into the mask completion module to obtain the corrected target Chinese text.

[0080] As is understood, the embodiments of the present invention determine the specific correction scenario by extracting features of the text to be corrected, and selectively select Chinese training texts that are suitable for the scenario. This not only improves the Chinese correction model's ability to recognize semantics in specific scenarios, but also ensures the correct understanding of the text semantics, thereby improving the model's correction accuracy and reliability.

[0081] Understandably, the embodiments of the present invention utilize training texts corresponding to the correction scenario to construct a sentence classifier dataset and construct a sentence classifier adapted to the scenario, thereby enhancing the correction training model's ability to identify semantic relevance between sentences, effectively reducing the interference of irrelevant sentences on semantic understanding, and thus enhancing the stability of the Chinese correction model.

[0082] Understandably, this invention generates multi-path correction data to perform adaptive dynamic mask correction training on the correction training model. Based on the semantic features of the text to be corrected, a mask ratio range corresponding to the correction scenario is obtained. Based on this value, multiple paths of text are trained with random masks at different ratios. This not only improves the model's training speed but also enhances its error correction accuracy and generalization ability by increasing the diversity of the training parameter set, while effectively reducing training costs.

[0083] Figure 2 This is a schematic diagram illustrating an application scenario of the Chinese character correction method in another embodiment of the present invention. Figure 3 yes Figure 2 The main flowchart of the Chinese character correction method in the scenario shown.

[0084] like Figure 2 As shown, the Chinese character correction method and knowledge base in this embodiment include, specifically, scene identifiers, scene features, and multiple Chinese training texts corresponding to each scenario. For example... Figure 3 As shown, the method in this embodiment includes the following steps S201 to S208.

[0085] Step S201: Extract text features from the initial Chinese text to be corrected, and determine the correction scenario based on the text features.

[0086] Step S202: Obtain multiple Chinese training texts, and label the sentences in the Chinese training texts in chronological order to obtain the sequence label corresponding to each sentence.

[0087] Step S203: Construct positive samples of the sentence classification dataset using continuous sentences and their sequence identifiers from the same Chinese training text; construct negative samples of the sentence classification dataset using non-continuous sentences and their sequence identifiers from different Chinese training texts.

[0088] Step S204: Calculate the correlation of each positive sample and each negative sample in the statement classification dataset, and construct the statement classifier based on the statistical results of the correlation between the positive and negative samples.

[0089] Step S205: Construct a correction training model based on the statement classifier.

[0090] Step S206: Segment the Chinese training text based on the dictionary corresponding to the correction scenario to obtain segmented text.

[0091] Step S207: The word segmented text is used to train the correction training model in multiple rounds. In each round of training, training sentence pairs are obtained from the word segmented text and multi-way random masks are applied to obtain multiple different masked sentence pairs. The correction training model is trained using the multi-way masked sentence pairs and the mask ratio for the next round is adjusted according to the accuracy until the training ends.

[0092] Step S208: The initial Chinese text is fed into the trained Chinese text correction model for multi-path mask correction to obtain the corrected target Chinese text.

[0093] For example, in this embodiment and some embodiments of the present invention, the text features include lexical features, grammatical features, and semantic features. Specifically, in this embodiment and some embodiments of the present invention, the text features are high-frequency word features. In step S201, the TF-IDF algorithm is used to obtain the high-frequency word features of the initial Chinese text; the high-frequency word features are matched with the scene features corresponding to each scene identifier to obtain the target scene identifier, which is used to determine the correction scene in subsequent processing. For example, if the high-frequency words extracted from the initial Chinese text to be corrected include words such as "stocks," "market," and "economy," then the preset correction scene "financial news" is matched, and the scene identifier corresponding to "financial news" is Context01, and the target scene identifier is Context01; if it includes words such as "experiment," "research," and "method," then the preset correction scene "academic paper" is matched, and the scene identifier corresponding to "academic paper" is Context02, and the target scene identifier is Context02.

[0094] Understandably, the statement sequence identifier is used to mark the order of statements. In this embodiment and some embodiments of the present invention, in step S202, the statement sequence identifier uses a globally unique identifier. Specifically, it uses the Chinese training text number plus the statement's sequence number in the text. For example, the sequence identifier of the 11th sentence in text A is "A011". The first digit of the sequence identifier identifies the text number, and the last three digits represent the statement's sequence number in text A. If the same statement appears in other texts, the assigned number is used.

[0095] Specifically, in this embodiment and some embodiments of the present invention, in step S203, each positive sample includes a first statement and its sequence identifier, and a second statement and its sequence identifier, wherein the first and second statements are the consecutive statements; each negative sample includes a third statement and its sequence identifier, and a fourth statement and its sequence identifier, wherein the third and fourth statements are the non-consecutive statements. For example, consecutive statements can be selected from the same paragraph of the same text, while non-consecutive statements are statements from different texts with different and non-consecutive sequence identifiers.

[0096] Specifically, Figure 4 yes Figure 2 The flowchart illustrates the main process of classifier construction in the scenario shown. Figure 4 As shown, in this embodiment and some embodiments of the present invention, step S204 includes steps S204a to S204e.

[0097] Step S204a: Using the Word2Vec model, each statement and its sequence identifier in the positive and negative samples are mapped into a multidimensional space vector, resulting in positive sample multidimensional space vector pairs and negative sample multidimensional space vector pairs. For example, mapping the positive sample and its sequence identifier to an n-dimensional vector space, the n-dimensional space vector corresponding to the first statement and its sequence identifier of the i-th positive sample is denoted as... The n-dimensional vector corresponding to the second statement and its sequence identifier of the i-th positive sample is denoted as . The n-dimensional space vector corresponding to the third statement and its sequence identifier of the j-th negative sample is denoted as . The n-dimensional vector corresponding to the fourth statement and its sequence identifier of the j-th negative sample is denoted as . The first n-1 dimensions of the vector correspond to the statements, and the nth dimension corresponds to the sequence identifier.

[0098] Step S204b: Calculate the correlation p between the two multidimensional space vectors in the i-th positive sample multidimensional space vector pair. i And calculate the mean correlation of positive samples. Where i = 1, 2, ..., N1, and N1 is the number of the positive sample multidimensional space vector pairs.

[0099] Step S204c: Calculate the correlation n between the two multidimensional space vectors in the j-th negative sample multidimensional space vector pair. j And calculate the mean correlation of negative samples. Where j = 1, 2, ..., N2, and N2 is the number of the negative sample multidimensional space vector pairs.

[0100] Specifically, in this embodiment and some embodiments of the present invention, the relevance p is calculated using the following formula. i and relevance n j ,

[0101] Step S204d: The decision threshold K of the statement classifier is obtained based on the mean relevance of the positive samples and the mean relevance of the negative samples, where...

[0102] Step S204e: Construct the statement classifier and set the decision threshold of the statement classifier to K.

[0103] Understandably, a pre-built statement classifier can be used to classify and label whether statements are relevant. For example, if two statements, "Today is an important day" and "An important moment in the company's development," are fed into the classifier, the classifier calculates the relevance between the two statements. If the relevance is greater than or equal to the aforementioned decision threshold K, the two statements are labeled as relevant, for example, using the English word "true". If the relevance is less than the aforementioned decision threshold K, the two statements are labeled as unrelated, for example, using the English word "false".

[0104] Specifically, in this embodiment and some embodiments of the present invention, the statement classifier includes a relevance calculation module and a category discrimination module. The relevance calculation module is used to calculate the positional and semantic relevance of two statements. The category discrimination module is used to compare the relevance obtained by the relevance calculation module with a decision threshold K, and determine the positional and semantic relevance attribute (true or false) between the two statements based on the comparison result. Setting the decision threshold K of the statement classifier is setting the decision threshold of the category discrimination module. Specifically, in this embodiment and some embodiments of the present invention, in step S205, the bias correction training model constructed based on the statement classifier includes a statement classifier and a mask completion module, wherein the mask completion module adopts the BERT model.

[0105] The BERT model itself uses the NSP task to enhance its understanding of context. Specifically, the NSP task generates positive and negative samples by: selecting the actual next sentence with a 50% probability to construct sentence pairs (positive samples), and randomly selecting sentences from different documents with a 50% probability to construct sentence pairs (negative samples). The `is_random_next` flag is set to indicate whether sentence B was randomly selected (true) and non-random selection (false). However, when determining whether two sentences are adjacent, the NSP task relies on thematic understanding of the sentences, ignoring the sentence's position within the paragraph or text. When generating negative samples using a large corpus, this may introduce semantically related sentences, thus affecting the model's performance and effectiveness.

[0106] Therefore, in this embodiment and some embodiments of the present invention, a sentence classifier is constructed by considering the position information and semantic relevance of sentences in the entire paragraph and passage of the Chinese training text. Positive and negative samples for training are constructed using the sentence classifier. The is_random_next flag of the original bert model is utilized, but the essential meaning it represents is changed. After adopting the classifier in the embodiments of the present invention, its meaning is the position and semantic related attributes of the sentence. True means the sentence pairs are adjacent and semantically related, and false means the semantics are not related or the sentences are not adjacent. When training the bert model with the positive and negative samples and the sentence pairs of the is_random_next flag, the original loss function including the MLM loss and the NSP loss can be used. Since the classified positive and negative samples are used during training, the classifier has essentially replaced the NSP task, thereby avoiding the noise introduced by the bert model performing the NSP task. Further, in some other embodiments of the present invention, when training the bert model, the NSP task may not be performed, and only the MLM task may be performed.

[0107] Specifically, in this embodiment and some embodiments of the present invention, step S206 includes: converting each sentence of the Chinese training text into a corresponding directed acyclic graph (DAG graph) based on the dictionary corresponding to the rectification scenario; calculating the maximum probability path based on the directed acyclic graphs corresponding to each sentence, and obtaining the segmented text based on the calculation result of the maximum possible path. For example, for the sentence "Another sunny day", the numbers from 0 to 8 are used to represent each character in the sentence in order. Each character takes itself as the end position of the corresponding segmentation, even if the character is not in the dictionary. According to the dictionary, a directed acyclic graph as shown in Figure 5 is obtained. Based on the DAG graph shown in Figure 5 , the maximum probability path is calculated, and the segmentation of the maximum probability path is used as the segmentation result.

[0108]

[0109] Among them, W is a segmentation method of the input sentence, w1, w2, w3... w n are n words obtained by dividing the input sentence according to the segmentation method W, P(W) is the probability of the segmentation method W of the input sentence, and P(w i ) is the probability obtained according to the occurrence frequency of the word w i in various segmentation methods in the input sentence. The result of dividing "Another sunny day" using the maximum probability path is as shown in Figure 6 .

[0110] Figure 7 is Figure 2 the main process of each round of training of the rectification training model in the shown embodiment. Figure 8This is a schematic diagram of a multi-channel random mask in an embodiment of the present invention. Specifically, as shown... Figure 7 and Figure 8 As shown, in some embodiments of the present invention, step S207 includes steps S207a to S207e.

[0111] Step S207a: Read the training sentence pairs for this round of training from the segmented text, for example, "It's another sunny day, I hope the weather will be like this tomorrow", copy the training sentence pairs multiple times, and send each copy into a random mask channel.

[0112] Step S207b, as follows Figure 8 As shown, each random masking channel selects its own masking ratio from the current masking ratio range, and performs full-word masking on the training sentence pairs according to the masking ratio to obtain their respective full-word masked sentence pairs. It can be understood that the masking ratio range used in the first round can adopt a preset initial value, such as 10% to 30%. For example, after performing full-word masking on "It's another sunny day, I hope the weather will be like this tomorrow", the full-word masked sentence pairs obtained include "[Mask][Mask]A sunny day, [Mask][Mask]The weather will be like this tomorrow", "It's another sunny [Mask] day, I hope the weather will be like this tomorrow", and "It's another sunny [Mask][Mask], I hope the weather will be like this tomorrow".

[0113] In step S207c, each of the full-word masked sentence pairs is fed into the sentence classifier of the correction training model to determine the position and semantic related attributes, obtain the corresponding related attributes, and then the full-word masked sentence pairs and the corresponding related attributes are fed into the mask completion module of the correction training model to obtain the mask completion sentence through the forward propagation algorithm.

[0114] Step S207d: Determine the correct statement for this round of training based on the comparison results of all the mask completion statements, and optimize the mask completion module of the correction training model through the backpropagation algorithm.

[0115] Specifically, in this embodiment and some embodiments of the present invention, the correct statement for this round of training is determined based on the comparison results of all the masked completion statements, including: aligning the word positions of all the masked completion statements, comparing the words corresponding to each word position, determining the word with the highest probability of occurrence at each word position as the target word, and combining the target words based on the word positions to form the correct statement.

[0116] Exemplarily, for sentences like "[Mask][Mask]A sunny day, [Mask][Mask]and the weather will be the same tomorrow", "Another sunny [Mask]day, hope the weather will be [Mask][Mask] tomorrow", "Another sunny [Mask][Mask], hope [Mask][Mask] will be the same tomorrow", the masked completion sentences are shown in Table 1 as follows.

[0117] Table 1

[0118]

[0119] For the position of the second-to-last character, the character with the highest occurrence probability is "this". Therefore, the correct sentence can be obtained as "Another sunny day, hope the weather will be the same tomorrow".

[0120] Step S207e: Calculate the error correction accuracy of each random masking channel based on the correct sentence in this round of training and correct the current masking ratio interval. The corrected masking ratio interval is used as the masking ratio interval for the next round.

[0121] Specifically, in this embodiment and some embodiments of the present invention, step S207e includes:

[0122] (1) Align the masked completion sentence of the t-th random masking channel with the correct sentence in terms of character positions, compare the characters corresponding to each character position, and determine the error correction accuracy rate C t of each masked completion sentence, where t = 1, 2,..., N3, and N3 is the number of random masking channels. For example, the accuracy rate of the 3rd channel in Table 1 is 1 / 18.

[0123] (2) Correct the upper limit and the lower limit of the current masking ratio interval, and use the corrected masking ratio interval as the masking ratio interval for the next round; the correction formula is as follows:

[0124]

[0125] p t is the masking ratio randomly selected from the masking ratio interval for the t-th random masking channel in the current round of training.

[0126] It is understood that in these embodiments of the present invention, by adaptively adjusting the mask ratio range, the Chinese training text in different correction scenarios can be adapted to perform correction training with an appropriate mask ratio, avoiding excessive or insufficient mask, which would affect the correction training effect and improve the generalization ability of the obtained Chinese correction model.

[0127] Furthermore, in some embodiments of the present invention, each random masking channel selects its own masking ratio from the current masking ratio range, and performs full-word masking on the training statement pair according to the masking ratio to obtain its own full-word masked statement pair, including:

[0128] (1) In the current round of training, the t-th random mask channel starts from the mask ratio range. The proportion of the mask randomly selected in the middle p t ;

[0129] (2) Calculate the number of words s that need to be masked in the current round of training, s = Roundup(N4*p t N4 is the total number of words in the training statement, and Roundup() is the round-up function;

[0130] (3) Randomly generate s random integers d1, d2, ..., dn in the interval [1, N4]. s The training statement pairs are numbered d1, d2, ..., d... s The words are fully masked to obtain the fully masked sentence pairs.

[0131] Furthermore, in some other embodiments of the present invention, the s random integers d1, d2, ..., d3 generated by each mask channel are also used. s The model is only trained if the random integers from each random channel are not identical, thus avoiding the input of duplicate masked statements. Figure 8 As shown, taking the first sentence of the above full-word mask statement pair as an example, the random integers corresponding to the three random mask channels are [1], [3], and [2, 4]. One of the sentences in the corresponding full-word mask statement pair is "[Mask][Mask]A sunny day", "Another sunny [Mask] day", and "Another [Mask][Mask][Mask][Mask] of [Mask] day".

[0132] Specifically, in this embodiment and some embodiments of the present invention, step S208 includes:

[0133] (1) The initial Chinese text is segmented based on a dictionary corresponding to the correction scenario to obtain the corresponding segmented text to be corrected;

[0134] (2) Read the pairs of sentences to be corrected from the text to be corrected in the order of sentence position, perform multi-way random masking on the sentences to be corrected, each random masking channel selects its own masking ratio from the last corrected masking ratio range, and performs full word masking on the pairs of sentences to be corrected according to the masking ratio to obtain multiple different pairs of sentences to be corrected by masking.

[0135] (3) The multi-path masked sentence pairs to be corrected are fed into the trained Chinese correction model. The Chinese correction model classifies each masked sentence pair to be corrected through its sentence classifier, identifies the positional and semantic related attributes between the input masked sentences to be corrected, and marks them (true and false). True indicates that the sentences are adjacent in position and related in meaning, and false indicates that the sentences are not related in meaning or are not adjacent in position.

[0136] (4) Input each pair of masked statements to be corrected and their related attributes into the mask completion module of the Chinese correction model, obtain the mask completion statement through the forward propagation algorithm, and determine the correct statement corresponding to the statement to be corrected based on the comparison results of all the mask completion statements.

[0137] (5) Concatenate all the correct statements in the corresponding position order to obtain the target Chinese text after the initial Chinese text has been corrected.

[0138] As can be seen from the above, the embodiments of the present invention perform multi-path replication on the Chinese word segmentation data and obtain the corresponding mask ratio range value according to the sentence features. Based on the mask ratio range value, different random masks are applied to multiple sentences to ensure that the same sentence is trained with different mask ratios and different words. This can not only improve the error correction accuracy of the model, but also increase the amount of training parameter data by a factor of two, enhance the model training effect, and reduce the training cost.

[0139] According to another aspect of the embodiments of the present invention, such as Figure 9 As shown, a Chinese character correction device is provided, comprising:

[0140] The scenario determination module is used to determine the correction scenario based on the initial Chinese text to be corrected;

[0141] The classifier building module is used to construct a sentence classification dataset based on Chinese training text corresponding to the correction scenario, and to construct a sentence classifier using the sentence classification dataset. The sentence classifier is used to determine the positional and semantic correlation attributes between the input sentences.

[0142] The error correction model construction module is used to construct an error correction training model based on the sentence classifier, generate multi-path error correction data according to the dictionary corresponding to the error correction scenario and the Chinese training text, and use the multi-path error correction data to perform adaptive dynamic mask error correction training on the error correction training model to construct a Chinese error correction model.

[0143] The error correction execution module is used to feed the initial Chinese text into the trained Chinese error correction model to obtain the corrected target Chinese text.

[0144] Based on the detailed description of the technical solution, it can be concluded that the technical solution of the embodiments of the present invention has the following advantages:

[0145] This invention extracts features from the text to be corrected to determine the correction scenario, selectively selects Chinese training texts that are suitable for the correction scenario, improves the semantic recognition ability of the Chinese correction model in specific correction scenarios, ensures the correct recognition of the semantics of the text to be corrected, and improves the correction accuracy and reliability of the obtained Chinese correction model.

[0146] This invention improves the ability of the error correction training model to identify semantic correlations between sentences by constructing a sentence classifier dataset and a sentence classifier adapted to the error correction scenario using Chinese training text corresponding to the error correction scenario, reduces the interference of irrelevant sentences on semantic recognition, and improves the error correction stability of the obtained Chinese error correction model.

[0147] This invention provides an embodiment of adaptive dynamic masking correction training for a correction training model by generating multiple correction data. It obtains a mask ratio range corresponding to the correction scenario based on the semantic features of the text to be corrected. Based on this range, different random masks are applied to multiple sentences, ensuring that the same sentence is trained with different mask ratios and different words. This increases the training speed of the correction training model while simultaneously expanding the training parameter set. This improves the error correction accuracy and generalization ability of the obtained Chinese correction model while reducing training costs.

[0148] This invention combines a Chinese knowledge base with a Chinese word segmentation model and, based on a Chinese word segmentation algorithm, can further improve the model's Chinese semantic recognition capabilities.

[0149] This invention improves the ability to understand the semantic relevance between sentences during Chinese error correction by constructing a sentence classifier. When applied to the BERT model, it can enhance the BERT model's ability to learn the semantic relevance of sentences, solve the problem of noise inherent in the NSP task, improve the training accuracy of the BERT model, and also remove the NSP task, thereby increasing the training speed.

[0150] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0151] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A Chinese character correction method, characterized in that, include: Determine the correction scenario based on the initial Chinese text to be corrected; A sentence classification dataset is constructed based on the Chinese training text corresponding to the correction scenario. A sentence classifier is constructed using the sentence classification dataset. The sentence classifier is used to determine the positional and semantic correlation attributes between the input sentences. Based on the sentence classifier, a correction training model is constructed. Multi-path correction data is generated according to the dictionary corresponding to the correction scenario and the Chinese training text. The multi-path correction data is used to perform adaptive dynamic mask correction training on the correction training model to construct a Chinese correction model. The initial Chinese text is fed into the trained Chinese text correction model to obtain the corrected target Chinese text. The method is based on a knowledge base, which includes scene identifiers and scene features corresponding to each scene, as well as multiple Chinese training texts corresponding to the correction scene; The correction scenario is determined based on the initial Chinese text to be corrected, including: The TF-IDF algorithm was used to obtain the high-frequency word features of the initial Chinese text; The high-frequency word features are matched with the scene features corresponding to each scene identifier to obtain the target scene identifier, which is used to determine the correction scene; A sentence classification dataset is constructed based on the Chinese training text corresponding to the aforementioned correction scenario. A sentence classifier is then built using this dataset. The sentence classifier is used to determine the positional and semantic correlation attributes between sentences, including: Multiple Chinese training texts are obtained, and the sentences in the Chinese training texts are identified in chronological order to obtain the sequence identifiers corresponding to each sentence; Positive samples of the sentence classification dataset are constructed using consecutive sentences and their sequence identifiers from the same Chinese training text; negative samples of the sentence classification dataset are constructed using non-consecutive sentences and their sequence identifiers from different Chinese training texts; wherein each positive sample includes a first sentence and its sequence identifier, a second sentence and its sequence identifier, the first sentence and the second sentence being consecutive sentences; each negative sample includes a third sentence and its sequence identifier, a fourth sentence and its sequence identifier, the third sentence and the fourth sentence being non-consecutive sentences; Calculate the relevance of each positive sample and each negative sample in the statement classification dataset, and construct the statement classifier based on the statistical results of the relevance of the positive and negative samples.

2. The method according to claim 1, characterized in that, Calculating the relevance of each positive sample and each negative sample in the statement classification dataset, and constructing the statement classifier based on the statistical results of the relevance of the positive and negative samples, including: The Word2Vec model is used to map each statement and its sequence identifier in the positive and negative samples into multi-dimensional space vectors, resulting in positive sample multi-dimensional space vector pairs and negative sample multi-dimensional space vector pairs. Calculate the first The correlation between two multidimensional space vectors in the positive sample multidimensional space vector pair And calculate the mean correlation of positive samples. ;in, The number of the positive sample multidimensional space vector pairs; Calculate the first The correlation between two multidimensional space vectors in the negative sample multidimensional space vector pair And calculate the mean correlation of negative samples. ;in, The number of the negative sample multidimensional space vector pairs; The decision threshold of the statement classifier is obtained based on the mean relevance of the positive samples and the mean relevance of the negative samples. ,in, ; Construct the statement classifier and set the decision threshold of the statement classifier to be [value missing]. .

3. The method according to claim 2, characterized in that, The error correction training model includes the sentence classifier and the mask completion module; Multi-path correction data is generated based on the dictionary corresponding to the correction scenario and the Chinese training text. Adaptive dynamic mask correction training is then performed on the correction training model using this multi-path correction data, including: The Chinese training text is segmented based on a dictionary corresponding to the aforementioned correction scenario to obtain segmented text; The word-segmented text is used to train the error correction training model in multiple rounds. In each round of training, training sentence pairs are obtained from the word-segmented text and the following steps are performed: Read the training sentence pairs from the segmented text for this round of training, copy the training sentence pairs multiple times, and send each copy into a random mask channel; Each of the random mask channels selects its own mask ratio from the current mask ratio range, and performs full-word masking on the training statement pair according to the mask ratio to obtain its own full-word masked statement pair; Each full-word masked sentence pair is fed into the sentence classifier of the correction training model to determine the position and semantic related attributes, obtain the corresponding related attributes, and then the full-word masked sentence pair and the corresponding related attributes are fed into the mask completion module of the correction training model to obtain the mask completion sentence through the forward propagation algorithm. The correct statement for this round of training is determined based on the comparison results of all the mask completion statements, and the mask completion module of the correction training model is optimized by the backpropagation algorithm. The error correction accuracy of each random mask channel is calculated based on the correct statements obtained in this round of training, and the current mask ratio range is corrected. The corrected mask ratio range is used as the mask ratio range for the next round.

4. The method according to claim 3, characterized in that, The Chinese training text is segmented based on a dictionary corresponding to the aforementioned correction scenario to obtain segmented text, including: Based on the dictionary corresponding to the aforementioned correction scenario, each sentence of the Chinese training text is converted into a corresponding phase-acyclic graph. The maximum possible path is calculated based on the directed acyclic graph corresponding to each sentence, and the word segmentation text is obtained based on the calculation result of the maximum possible path.

5. The method according to claim 3, characterized in that, The correct statements for this round of training are determined based on the comparison results of all the masked completion statements, including: All masked completion statements are aligned in word position. For each word position, the corresponding word is compared. The word with the highest probability of occurrence at each word position is determined as the target word. Based on the word position, the target words are combined to form the correct statement.

6. The method according to claim 5, characterized in that, Based on the correct statements from this training round, the error correction accuracy of each random mask channel is calculated, and the current mask ratio range is corrected. The corrected mask ratio range is used as the mask ratio range for the next round, including: The first The mask completion statements of each random mask channel are aligned with the correct statements by word position. A word comparison is performed on the words corresponding to each word position to determine the error correction accuracy of each mask completion statement. , , It is the number of the random mask channels; Upper limit of the current mask ratio range and lower limit Make corrections, and adjust the mask ratio range accordingly. The mask ratio range to be used in the next round; the corrected formula is as follows: ， ， ， ， For the first Each random mask channel is selected from the mask ratio range in the current training round ( The proportion of the mask randomly selected in the ().

7. The method according to any one of claims 3-6, characterized in that, Each of the aforementioned random mask channels selects its own mask ratio from the current mask ratio range, and performs full-word masking on the training statement pair according to the mask ratio to obtain its own full-word masked statement pair, including: In the current round of training, the first A random mask channel from a mask ratio range ( The proportion of the mask randomly selected in ) ; Calculate the number of words that require full word masking in the current round of training. , , The total number of words in the training statement. () is the floor function; Randomly generated interval random integers The sequence number of the training statement pair is The words are fully masked to obtain the fully masked sentence pairs.

8. A Chinese character error correction device, characterized in that, include: The scenario determination module is used to determine the correction scenario based on the initial Chinese text to be corrected; The classifier building module is used to construct a sentence classification dataset based on Chinese training text corresponding to the correction scenario, and to construct a sentence classifier using the sentence classification dataset. The sentence classifier is used to determine the positional and semantic correlation attributes between the input sentences. The error correction model construction module is used to construct an error correction training model based on the sentence classifier, generate multi-path error correction data according to the dictionary corresponding to the error correction scenario and the Chinese training text, and use the multi-path error correction data to perform adaptive dynamic mask error correction training on the error correction training model to construct a Chinese error correction model. The error correction execution module is used to feed the initial Chinese text into the trained Chinese error correction model to obtain the corrected target Chinese text. The device is based on a knowledge base, which includes scene identifiers and scene features corresponding to each scene, as well as multiple Chinese training texts corresponding to the correction scene. The correction scenario is determined based on the initial Chinese text to be corrected, including: The TF-IDF algorithm was used to obtain the high-frequency word features of the initial Chinese text; The high-frequency word features are matched with the scene features corresponding to each scene identifier to obtain the target scene identifier, which is used to determine the correction scene; A sentence classification dataset is constructed based on the Chinese training text corresponding to the aforementioned correction scenario. A sentence classifier is then built using this dataset. The sentence classifier is used to determine the positional and semantic correlation attributes between sentences, including: Multiple Chinese training texts are obtained, and the sentences in the Chinese training texts are identified in chronological order to obtain the sequence identifiers corresponding to each sentence; Positive samples of the sentence classification dataset are constructed using consecutive sentences and their sequence identifiers from the same Chinese training text; negative samples of the sentence classification dataset are constructed using non-consecutive sentences and their sequence identifiers from different Chinese training texts; wherein each positive sample includes a first sentence and its sequence identifier, a second sentence and its sequence identifier, the first sentence and the second sentence being consecutive sentences; each negative sample includes a third sentence and its sequence identifier, a fourth sentence and its sequence identifier, the third sentence and the fourth sentence being non-consecutive sentences; Calculate the relevance of each positive sample and each negative sample in the statement classification dataset, and construct the statement classifier based on the statistical results of the relevance of the positive and negative samples.

Images