Model training method, text error correction method and related products

Abstract

The embodiment of the invention provides a model training method, a text error correction method and a related product, and relates to the technical field of machine learning. According to the model training method provided by the invention, by introducing the model training data including various error types such as spelling errors, grammar errors, logic errors and format errors, various errors possibly existing in the to-be-corrected text can be comprehensively covered and recognized. Moreover, the word segmentation result of the text data is subjected to masking processing through a first sub-model of the machine learning model, the word segmentation result of the text data is subjected to disordering processing through a second sub-model, and the machine learning model is trained according to the annotation data, the result after masking processing and the result after disordering processing; the recognition capability of the machine learning model for the error mode is enhanced, the learning capability of the machine learning model for the correct text structure is also promoted, and the text error correction model shows higher accuracy during text error correction.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a model training method, a text error correction method, and related products. Background Art

[0002] In the financial industry, texts such as financial documents and financial reports play a vital role. They are the basis of financial operations such as investment decisions, financial planning and market forecasts. The accuracy and consistency of these texts have a decisive impact on ensuring the smooth progress of financial operations. Once errors occur in the text, it may lead to misunderstanding of the text, which in turn triggers a series of chain reactions, such as wrong investment decisions, inaccurate financial planning or market forecasts that deviate from reality.

[0003] In order to meet the challenges of accuracy and consistency of texts in the financial industry, natural language processing (NLP) algorithms are usually used to train machine learning models. Machine learning models learn a large number of correct text examples to master the language rules and characteristics of texts, and then form a text correction model. When it is necessary to correct new texts to be corrected, just input the text to be corrected into the text correction model, and the text correction model can automatically identify and correct the spelling errors in it.

[0004] However, this text correction method is mainly used to identify and correct spelling errors, but cannot comprehensively cover all possible types of text errors, resulting in a low accuracy rate of text correction. Summary of the invention

[0005] Based on the above problems, the present application provides a model training method, a text error correction method and related products, which can improve the accuracy of text error correction.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a model training method, the method comprising:

[0008] Acquire model training data, wherein the model training data includes text data and annotation data, the annotation data indicates an error type, an error start position, and an error end position of the text data, the error type includes at least one of a spelling error, a grammatical error, a logic error, and a format error;

[0009] Inputting the model training data into a machine learning model, performing masking processing on the word segmentation results of the text data through a first sub-model of the machine learning model, and performing shuffling processing on the word segmentation results of the text data through a second sub-model of the machine learning model;

[0010] The machine learning model is trained according to the labeled data, the result after masking processing and the result after scrambling processing to obtain a text error correction model.

[0011] Optionally, the first sub-model includes any one of an ELECTRA model, a BERT model, and a BART model; and the masking process of the word segmentation result of the text data by the first sub-model of the machine learning model includes:

[0012] Generate a first arbitrary value for each word segmentation result of the text data by using the first submodel of the machine learning model;

[0013] Masking is performed on the word segmentation results of the text data whose first arbitrary value is greater than a masking processing threshold.

[0014] Optionally, the performing masking processing on the word segmentation result of the text data by using the first sub-model of the machine learning model includes:

[0015] The first sub-model of the machine learning model performs masking processing on the target word segmentation result of the text data and all sub-words corresponding to the target word segmentation result, wherein all sub-words are all components of the target word segmentation result.

[0016] Optionally, the second sub-model includes a PERT model; and the step of shuffling the word segmentation results of the text data using the second sub-model of the machine learning model includes:

[0017] Generate a second arbitrary value for each sentence segmentation result of the text data by using the first submodel of the machine learning model;

[0018] All word segmentation results in the sentence results of the text data whose second arbitrary value is greater than the shuffling processing threshold are shuffled.

[0019] Optionally, the acquiring model training data includes:

[0020] Acquire original sample data, where the original sample data is text data with at least one error type among spelling errors, grammatical errors, logical errors and format errors;

[0021] Cleaning the original sample data to obtain cleaned original sample data;

[0022] Model training data is obtained by marking the error type, error start position and error end position of the cleaned original sample data.

[0023] In a second aspect, an embodiment of the present application provides a text error correction method, the method comprising:

[0024] Acquire data to be corrected, where the data to be corrected is text data;

[0025] The data to be corrected is input into a text correction model, and error detection is performed on the data to be corrected through the text correction model. When the error detection result indicates that the data to be corrected has errors, at least one of component redundancy correction, wrong word correction, component incomplete correction and word order error correction is performed on the data to be corrected through the text correction model, so that the text correction model outputs a text correction result, wherein the text correction model is trained by the model training method described in the first aspect.

[0026] In a third aspect, an embodiment of the present application provides a model training device, the device comprising: an acquisition module, an input module and a training module;

[0027] The acquisition module is used to acquire model training data, wherein the model training data includes text data and annotation data, the annotation data indicates an error type, an error start position, and an error end position of the text data, and the error type includes at least one of a spelling error, a grammatical error, a logic error, and a format error;

[0028] The input module is used to input the model training data into the machine learning model, perform masking processing on the word segmentation results of the text data through the first sub-model of the machine learning model, and perform scrambling processing on the word segmentation results of the text data through the second sub-model of the machine learning model;

[0029] The training module is used to train the machine learning model according to the labeled data, the result after masking processing and the result after scrambling processing to obtain a text error correction model.

[0030] Optionally, the first sub-model includes any one of an ELECTRA model, a BERT model, and a BART model;

[0031] The input module is specifically used to: generate a first arbitrary value for each word segmentation result of the text data through the first submodel of the machine learning model; and perform masking processing on the word segmentation results of the text data whose first arbitrary value is greater than a masking processing threshold.

[0032] Optionally, the input module is specifically used to: perform masking processing on a target word segmentation result of text data and all sub-words corresponding to the target word segmentation result through the first sub-model of the machine learning model, wherein all sub-words are all components of the target word segmentation result.

[0033] Optionally, the second sub-model includes a PERT model; the input module is specifically used to: generate a second arbitrary value for each sentence result of the text data through the first sub-model of the machine learning model; and shuffle all word segmentation results in the sentence results of the text data whose second arbitrary value is greater than a shuffle processing threshold.

[0034] Optionally, the acquisition module includes: a first acquisition submodule, a second acquisition submodule and a third acquisition submodule;

[0035] The first acquisition submodule is used to acquire original sample data, where the original sample data is text data with at least one error type among spelling errors, grammatical errors, logical errors and format errors;

[0036] The second acquisition submodule is used to clean the original sample data to obtain cleaned original sample data;

[0037] The third acquisition submodule is used to obtain model training data by marking the error type, error start position and error end position of the cleaned original sample data.

[0038] In a fourth aspect, an embodiment of the present application provides a text error correction device, the device comprising: a data acquisition module and a data input module;

[0039] The data acquisition module is used to acquire the data to be corrected, wherein the data to be corrected is text data;

[0040] The data input module is used to input the data to be corrected into a text correction model, and perform error detection on the data to be corrected through the text correction model. When the error detection result indicates that the data to be corrected has errors, the text correction model is used to perform at least one of component redundancy correction, wrong word correction, component incomplete correction and word order error correction on the data to be corrected, so that the text correction model outputs a text correction result, wherein the text correction model is trained by the model training method described in the first aspect.

[0041] In a fifth aspect, an embodiment of the present application provides a computer device, the device comprising: a memory and a processor;

[0042] The memory is used to store programs;

[0043] The processor is used to execute the program to implement the model training method as described in the first aspect, or the various steps of the text error correction method as described in the second aspect.

[0044] In a sixth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the model training method as described in the first aspect, or the various steps of the text error correction method as described in the second aspect.

[0045] Compared with the prior art, this application has the following beneficial effects:

[0046] The embodiment of the present application provides a model training method, a text error correction method and related products, the model training method comprising: obtaining model training data, wherein the model training data comprises text data and annotation data, the annotation data indicates the error type, error start position and error end position of the text data, and the error type comprises at least one of spelling error, grammatical error, logical error and format error; inputting the model training data into a machine learning model, masking the word segmentation result of the text data through the first sub-model of the machine learning model, and scrambling the word segmentation result of the text data through the second sub-model of the machine learning model; training the machine learning model according to the annotation data, the masked result and the scrambled result to obtain a text error correction model. Therefore, the model training method of the present application can comprehensively cover and identify various types of errors that may exist in the text to be corrected by introducing model training data containing multiple error types such as spelling errors, grammatical errors, logical errors and format errors. Furthermore, the word segmentation results of the text data are masked through the first sub-model of the machine learning model, and the word segmentation results of the text data are shuffled through the second sub-model, and the machine learning model is trained based on the labeled data, the masked results and the shuffled results. This not only enhances the machine learning model's ability to recognize error patterns, but also promotes the machine learning model's ability to learn the correct text structure, so that the text correction model shows higher accuracy when correcting text errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0049] Figure 2 A flowchart of a text error correction method provided in an embodiment of the present application;

[0050] Figure 3A schematic diagram of a model training device provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of a text error correction device provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of a computer-readable medium provided for an embodiment of the present application. DETAILED DESCRIPTION

[0053] As described above, in order to meet the challenges of accuracy and consistency of texts in the financial industry, natural language processing (NLP) algorithms are usually used to train machine learning models. Machine learning models learn a large number of correct text examples to master the language rules and characteristics of texts, and then form a text correction model. When it is necessary to correct new texts to be corrected, just input the text to be corrected into the text correction model, and the text correction model can automatically identify and correct spelling errors in it.

[0054] However, this text correction method is mainly used to identify and correct spelling errors, but cannot comprehensively cover all possible types of text errors, resulting in a low accuracy rate of text correction.

[0055] After research, the inventor proposed a model training method, a text error correction method and related products, the model training method comprising: obtaining model training data, wherein the model training data comprises text data and annotation data, the annotation data indicates the error type, error start position and error end position of the text data, and the error type comprises at least one of spelling error, grammatical error, logical error and format error; inputting the model training data into a machine learning model, masking the word segmentation result of the text data through the first sub-model of the machine learning model, and scrambling the word segmentation result of the text data through the second sub-model of the machine learning model; training the machine learning model according to the annotation data, the masked result and the scrambled result to obtain a text error correction model. Therefore, the model training method of the present application can comprehensively cover and identify various types of errors that may exist in the text to be corrected by introducing model training data containing multiple error types such as spelling errors, grammatical errors, logical errors and format errors. Furthermore, the word segmentation results of the text data are masked through the first sub-model of the machine learning model, and the word segmentation results of the text data are shuffled through the second sub-model, and the machine learning model is trained based on the labeled data, the masked results and the shuffled results. This not only enhances the machine learning model's ability to recognize error patterns, but also promotes the machine learning model's ability to learn the correct text structure, so that the text correction model shows higher accuracy when correcting text errors.

[0056] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] See also Figure 1 , which is a flow chart of a model training method provided in an embodiment of the present application. The method includes:

[0058] S101: Acquire model training data, wherein the model training data includes text data and annotation data, the annotation data indicates an error type, an error start position, and an error end position of the text data, and the error type includes at least one of a spelling error, a grammatical error, a logical error, and a format error.

[0059] First, model training data for training a text error correction model is obtained. The model training data includes text data and corresponding annotation data.

[0060] Text data refers to the original text that needs to be corrected, which can come from various sources, such as banks, securities companies, insurance companies and other financial institutions. Text data can include documents, contracts, emails, reports, etc.

[0061] The annotation data indicates in detail the error type in the text data (including at least one of spelling errors, grammatical errors, logical errors and format errors), the error start position (indicating the starting position of the error in the text, usually the index position of the first character of the error) and the error end position (indicating the ending position of the error in the text, usually the index position of the last character of the error plus 1).

[0062] Among them, spelling errors refer to incorrect spelling of words or terms, that is, the alphabetical order or composition of words does not conform to standard spelling rules. For example, "The previous natural day of November 8, 2024 is November 9, 2024. The next working day of November 8, 2024 is November 11, 2024." is the original text that needs to be corrected. Since "working day" should be "working day", "working day" is a spelling error. Therefore, the error type corresponding to this text data is a spelling error.

[0063] A grammatical error refers to a sentence structure that does not conform to grammatical rules, including subject-verb inconsistency, missing subject or predicate, incorrect tense, incorrect word order, etc. For example, "The previous natural day of November 8, 2024 is November 9, 2024. The next working day of November 8, 2024 is November 11, 2024." is the original text that needs to be corrected. Since the second sentence lacks the predicate "is", the sentence structure is incomplete, which is a grammatical error. Therefore, the error type corresponding to the text data is a grammatical error.

[0064] Logical errors refer to inconsistent or unreasonable arguments, data, or reasoning processes in the text, which lead to inaccurate or misleading information. For example, "The previous natural day of November 8, 2024 is November 9, 2024. The next working day of November 8, 2024 is November 10, 2024." is the original text that needs to be corrected. Since "the previous natural day of November 8, 2024" in the first sentence should be "November 7, 2024", and "the next working day of November 8, 2024" in the second sentence should be "November 11, 2024", because November 10 is a weekend, not a working day, these errors are logical errors. Therefore, the error type corresponding to the text data is a logical error.

[0065] Format error means that the format in the text is not uniform or does not meet the standard format requirements, including inconsistent use of dates, currency symbols, punctuation marks, etc. For example, "②The next working day after November 8, 2024 is November 11, 2024." is the original text that needs to be corrected. Since "②" in "②024" is a format error, the error type corresponding to the text data is format error.

[0066] Understandably, understanding these error types can help to better perform text error correction and improve the quality and accuracy of text data.

[0067] S102: Input the model training data into the machine learning model, perform masking processing on the word segmentation results of the text data through the first sub-model of the machine learning model, and perform scrambling processing on the word segmentation results of the text data through the second sub-model of the machine learning model.

[0068] The purpose of inputting model training data into the machine learning model is to input the model training data that has been acquired and labeled into the machine learning model, and to perform specific processing (masking processing and scrambling processing) on ​​the text data through the sub-models of the two machine learning models to enhance the learning ability and error correction effect of the machine learning model.

[0069] Masking the word segmentation results of text data refers to randomly replacing certain words (tokens) with masking tags (masks). The purpose of masking is to enable the machine learning model to learn how to predict the masked words based on the contextual information in the model training data, thereby enhancing the contextual understanding and prediction capabilities of the trained text error correction model.

[0070] Scrambling the word segmentation results of text data refers to randomly scrambling the word order of the word segmentation results of text data. The purpose of scrambling is to enable the machine learning model to learn how to restore the original word order based on the disordered words in the model training data, thereby enhancing the robustness and adaptability of the trained text error correction model.

[0071] It should be noted that this application does not limit the specific first sub-model and the second sub-model.

[0072] S103: Train a machine learning model based on the labeled data, the masking result, and the scrambled result to obtain a text error correction model.

[0073] By combining the labeled data, the masking results, and the scrambled results, the machine learning model is trained to obtain a model that can effectively correct text errors. Specifically, the machine learning model uses the labeled data to learn to identify and correct various types of errors (spelling errors, grammatical errors, logical errors, and format errors). In addition, the machine learning model learns how to predict masked words based on contextual information through the masking results, enhancing contextual understanding and prediction capabilities. In addition, the machine learning model learns how to restore the original word order based on the disordered words through the scrambled results, enhancing the ability to adapt to changes in text structure.

[0074] After training, the text error correction model can detect and correct errors based on the input text data and output the corrected text results.

[0075] It should be noted that after obtaining the text error correction model, the model parameters of the text error correction model can be optimized by minimizing the loss function (such as cross entropy loss) to make the text error correction model perform better on the model training data.

[0076] In summary, the model training method of the present application can comprehensively cover and identify various types of errors that may exist in the text to be corrected by introducing model training data containing various error types such as spelling errors, grammatical errors, logical errors and format errors. In addition, the word segmentation results of the text data are masked by the first sub-model of the machine learning model, and the word segmentation results of the text data are scrambled by the second sub-model, and the machine learning model is trained based on the labeled data, the masked results and the scrambled results, which not only enhances the machine learning model's ability to recognize error patterns, but also promotes the machine learning model's ability to learn the correct text structure, so that the text correction model shows a higher accuracy rate when performing text correction.

[0077] See also Figure 2 , which is a flow chart of a text error correction method provided by an embodiment of the present application. The method comprises:

[0078] S201: Acquire original sample data, where the original sample data includes text data with errors.

[0079] First, collect original sample data from banks, securities companies, insurance companies and other financial institutions. The original sample data includes unstructured text data (such as documents, contracts, emails, reports, etc.) with various errors (such as spelling errors, grammatical errors, logical errors, format errors, etc.). Among them, spelling errors refer to incorrect spelling of words, grammatical errors refer to sentence structures that do not conform to grammatical rules (such as missing subjects, etc.), logical errors refer to inconsistent arguments or data in text data, or unreasonable reasoning processes, and format errors refer to inconsistent text formats, such as inconsistent use of dates and currency symbols.

[0080] It should be noted that the text correction method provided in the embodiment of the present application needs to ensure that the acquisition of all original sample data complies with relevant laws and regulations, especially information involving personal privacy and business secrets.

[0081] S202: Clean the original sample data to obtain cleaned original sample data.

[0082] Cleaning the original sample data refers to removing special non-text characters in the original sample data to ensure that the cleaned original sample data only contains common non-text characters such as valid numbers and common punctuation marks.

[0083] Exemplarily, special non-text characters may include control characters, special symbols, HyperText Markup Language (HTML) tags, and extra spaces. Control characters include line feed (\n), tab (\t), carriage return (\r), etc., and control characters can be removed by regular expressions or string processing functions. Special symbols include @, #, $, %, ^, &, *, (,), {,}, [,], |, \, / , ~, ! , ? , +, =, >, etc., and common special symbols can be removed by regular expressions. HTML tags include , , HTML tags can be removed by using libraries such as BeautifulSoup. Redundant spaces can be removed by using string processing functions. It should be noted that this application does not limit the specific special non-text characters and cleaning methods.

[0084] It is understandable that before performing any data cleaning operations on the original sample data, it is recommended to back up the original sample data to prevent accidental loss of important information.

[0085] S203: Annotate the error type, error starting position and error ending position in the cleaned original sample data to obtain model training data.

[0086] First, the cleaned raw sample data is annotated, including the error type (such as spelling errors, grammatical errors, logical errors, format errors, etc.), the error start position (i.e., the index position of the first character of the error) and the error end position (i.e., the index position of the last character of the error plus 1). Then, the labeled raw sample data is segmented into sentences to generate model training data.

[0087] It should be noted that model training data is usually stored in JSON or CSV format, and each model training data includes text data and annotation data.

[0088] In some specific implementations, the annotation step can be performed using a text editor or a professional annotation tool (such as Label Studio, Prodigy, etc.). For example, if the original sample data after cleaning is "The previous natural day of November 8, 2024 is November 9, 2024. ②The next working day of November 8, 2024 is November 11, 2024." Then the annotation results are "Logical error: 9 (error start position: 27, error end position: 28)" and "Format error: ② (error start position: 30, error end position: 31)".

[0089] In some specific implementations, the segmentation step may be to use punctuation marks such as periods, question marks, exclamation marks, etc. as sentence separators, thereby splitting the marked original sample data into multiple sentences.

[0090] For example, if the original sample data after labeling is "The previous natural day of November 8, 2024 is November 9, 2024. ②The next working day of November 8, 2024 is November 11, 2024." Then the segmentation result is "Sentence 1: The previous natural day of November 8, 2024 is November 9, 2024. Sentence 2: ②The next working day of November 8, 2024 is November 11, 2024."

[0091] S204: A text error correction model is obtained by training the model training data into a machine learning model, wherein the machine learning model is composed of multiple sub-models, including an ELECTRA model, a BERT model, a BART model, and a PERT model.

[0092] Specifically, for the ELECTRA model and the BERT model, a combination of random masking and full-word masking is needed to enable the ELECTRA model and the BERT model to fully learn the text data of the model training data.

[0093] Among them, random masking means that each word group (token) has a 15% probability of being replaced by a masking token (Mask). Specifically, the ELECTRA model and the BERT model can generate a first arbitrary value based on the word segmentation result of each text data; the word segmentation results of the text data whose first arbitrary value is greater than the masking processing threshold are masked. The purpose of random masking is to enable the ELECTRA model and the BERT model to learn how to predict the masked word group based on context information.

[0094] Full word masking means that when a phrase is masked, all subwords corresponding to the phrase will be masked. The purpose of full word masking is to ensure that the ELECTRA model and the BERT model can understand the meaning of the entire phrase, rather than just focusing on some subwords.

[0095] Exemplarily, assuming that the text data of the model training data is: "The previous natural day of November 8, 2024 is November 9, 2024.", then the word segmentation result of the WordPiece word segmenter of the BERT model or the BPE word segmenter of the ELECTRA model is: ["2024","year","11","month","8","day","of","last","one","one","natural","day","is","2024","year","11","month","9","day","."]. Then, random masking means that "2024" has a 15% probability of being replaced by a masked token. Full-word masking means that if "November 8, 2024" is randomly selected for masking, all subwords (i.e. ["2024","year","11","month","8","day"]) will be replaced with masking markers, and the word segmentation result will become: ["[MASK]","[MASK]","[MASK]","[MASK]","[MASK]","[MASK]","'s","last","one","one","natural","day","is","[MASK]","[MASK]","[MASK]","[MASK]","[MASK]","."].

[0096] Specifically, for the BART model, you need to first use the BART model's word segmentation tool to segment the text data of the model training data, and each word group (token) has a 10% probability of being replaced by a masking tag (Mask). The BART model can generate a first arbitrary value based on the word segmentation result of each text data; mask the word segmentation results of the text data whose first arbitrary value is greater than the masking processing threshold.

[0097] Specifically, for the PERT model, it is necessary to first use the word segmentation tool of the PERT model to segment the text data of the model training data, and then shuffle the order of multiple phrases in a sentence, and add "<>" marks at the beginning and end of the sentence. Specifically, the PERT model generates a second arbitrary value for each sentence segmentation result of the text data; all word segmentation results in the sentence results of the text data whose second arbitrary value is greater than the shuffle processing threshold are shuffled. The purpose of the above random shuffling strategy is to enable the PERT model to learn how to restore the original word order based on the shuffled words.

[0098] Exemplarily, assuming that the text data of the model training data is: "The previous natural day of November 8, 2024 is November 9, 2024.", then the result of word segmentation using the BOE word segmenter of the PERT model is: ["2024","year","11","month","8","day","of","last","one","one","natural","day","is","2024","year","11","month","9","day","."]. Then, traverse each sentence and generate a random number. If the random number is less than 0.05, randomly shuffle the order of words in the sentence and add "<>" marks at the beginning and end positions. For example, the result of shuffling the above text data may be: "<>9th is the previous natural day of November 2024, November 8, 2024<>". Therefore, the PERT model can fully learn how to restore the original word order based on the disordered words and improve the performance in the text error correction task.

[0099] S205: Acquire the data to be corrected, where the data to be corrected is text data.

[0100] The data format of the data to be corrected may include .txt, .csv, .json, etc.

[0101] It should be noted that after obtaining the data to be corrected, the data to be corrected can be preprocessed, such as cleaning, so as to provide more reliable input for the subsequent text correction model, and the trained text correction model can be better utilized to correct the data to be corrected, thereby improving the quality and accuracy of the obtained text correction results.

[0102] S206: Input the data to be corrected into a text correction model, and perform error detection on the data to be corrected through the text correction model. When the error detection result indicates that the data to be corrected has errors, perform at least one of component redundancy correction, wrong word correction, component incomplete correction, and word order error correction on the data to be corrected through the text correction model, so that the text correction model outputs a text correction result.

[0103] First, the data to be corrected is input into the text correction model, so that the text correction model performs error detection on the word segmentation results of the data to be corrected, and classifies the errors into four types: component redundancy, component incompleteness, typos, and word order errors.

[0104] Specifically, the data to be corrected is input into the ELECTRA model of the text correction model. The ELECTRA model uses a sequence labeling method to predict the word segmentation results of the data to be corrected. First, the ELECTRA model vectorizes the word segmentation results of the data to be corrected, and inputs the vectorized word segmentation results into the encoder for encoding. Subsequently, each vocabulary is classified through the softmax layer, and a label sequence is output. Among them, the labels are divided into five categories, including four types of text errors (component redundancy, typos, incomplete components, word order errors) and correct. Exemplary, if the word segmentation result is ""2024","year","11","month","8","day","of","on","one","a","self","natural","day","is","2024","year","11","month","9","day",". "", then the label sequence is "["correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","correct","logical error","correct","correct"]". Subsequently, if the error detection indicates that the data to be corrected is incorrect, error correction is performed in the order of component redundancy, wrong words, missing components and incorrect word order.

[0105] Specifically, first, the detected redundant component errors are directly deleted. Secondly, the BERT model is used to correct the wrong words and replace the wrong words. Subsequently, the BART model is used to correct the missing components, predict and supplement the missing parts. Finally, the PERT model is used to correct the word order errors, predict and restore the correct word order.

[0106] In summary, the embodiment of the present application provides a text error correction method, which first collects original sample data containing multiple error types, and then cleans the original sample data by removing special non-text characters, and marks the cleaned original sample data to obtain model training data, and then inputs the model training data into a machine learning model jointly trained by multiple sub-models such as ELECTRA, BERT, BART and PERT to construct a text error correction model. Finally, the word segmentation result of the data to be corrected is input into the text error correction model, and error detection is performed by the text error correction model. When the error detection characterizes that the data to be corrected has an error, the text error correction model is used to correct the data to be corrected. At least one of component redundancy correction, typo correction, component incomplete correction and word order error correction is performed to the data to be corrected, so that the text error correction model outputs a text error correction result. Thus, the text error correction method provided by the embodiment of the present application improves the generalization ability and robustness of the text error correction model obtained by training, and also improves the accuracy and consistency of the output text error correction results.

[0107] See also Figure 3 , which is a schematic diagram of a model training device provided in an embodiment of the present application. The model training device 300 includes: an acquisition module 301, an input module 302 and a training module 303;

[0108] An acquisition module 301 is used to acquire model training data, wherein the model training data includes text data and annotation data, the annotation data indicates an error type, an error start position, and an error end position of the text data, and the error type includes at least one of a spelling error, a grammatical error, a logic error, and a format error;

[0109] An input module 302 is used to input the model training data into the machine learning model, mask the word segmentation results of the text data through the first sub-model of the machine learning model, and shuffle the word segmentation results of the text data through the second sub-model of the machine learning model;

[0110] The training module 303 is used to train a machine learning model based on the labeled data, the masking processing results and the scrambling processing results to obtain a text error correction model.

[0111] In some specific implementations, the first sub-model includes any one of an ELECTRA model, a BERT model, and a BART model;

[0112] The input module 302 is specifically used to: generate a first arbitrary value for the word segmentation result of each text data through the first sub-model of the machine learning model; and perform masking processing on the word segmentation results of the text data whose first arbitrary value is greater than the masking processing threshold.

[0113] In some specific implementations, the input module 302 is specifically used to: perform masking processing on the target word segmentation result of the text data and all sub-words corresponding to the target word segmentation result through the first sub-model of the machine learning model, wherein all sub-words are all components of the target word segmentation result.

[0114] In some specific implementations, the second sub-model includes a PERT model; the input module 302 is specifically used to: generate a second arbitrary value for each sentence result of the text data through the first sub-model of the machine learning model; and shuffle all word segmentation results in the sentence results of the text data whose second arbitrary value is greater than the shuffle processing threshold.

[0115] In some specific implementations, the acquisition module 301 includes: a first acquisition submodule, a second acquisition submodule, and a third acquisition submodule;

[0116] The first acquisition submodule is used to acquire original sample data, where the original sample data is text data with at least one error type among spelling errors, grammatical errors, logical errors and format errors;

[0117] The second acquisition submodule is used to clean the original sample data to obtain the cleaned original sample data;

[0118] The third acquisition submodule is used to obtain model training data by marking the error type, error start position and error end position of the cleaned original sample data.

[0119] In summary, the model training device of the present application can comprehensively cover and identify various types of errors that may exist in the text to be corrected by introducing model training data containing various error types such as spelling errors, grammatical errors, logical errors and format errors. In addition, the word segmentation results of the text data are masked by the first sub-model of the machine learning model, and the word segmentation results of the text data are scrambled by the second sub-model, and the machine learning model is trained based on the labeled data, the masked results and the scrambled results, which not only enhances the machine learning model's ability to recognize error patterns, but also promotes the machine learning model's ability to learn the correct text structure, so that the text correction model shows a higher accuracy rate when performing text correction.

[0120] See also Figure 4 , which is a schematic diagram of a text error correction device provided in an embodiment of the present application. The text error correction device 400 includes: a data acquisition module 401 and a data input module 402;

[0121] The data acquisition module 401 is used to acquire the data to be corrected, where the data to be corrected is text data;

[0122] The data input module 402 is used to input the data to be corrected into the text correction model, and perform error detection on the data to be corrected through the text correction model. When the error detection result indicates that the data to be corrected has errors, the text correction model is used to perform at least one of component redundancy correction, wrong word correction, component incomplete correction and word order error correction on the data to be corrected, so that the text correction model outputs a text correction result, wherein the text correction model is trained by the model training method of the first aspect.

[0123] In summary, the embodiments of the present application provide a text correction device. The text correction device provided by the embodiments of the present application improves the generalization ability and robustness of the trained text correction model, and also improves the accuracy and consistency of the output text correction results.

[0124] The embodiments of the present application also provide corresponding computer devices and computer-readable media for implementing the text error correction method provided in the embodiments of the present application.

[0125] Among them, the computer device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute instructions or codes so that the device executes a text correction method of any embodiment of the present application.

[0126] See also Figure 5 , which is a schematic diagram of a computer readable medium provided in an embodiment of the present application. The computer readable medium 500 stores a computer program 511, which implements the above-mentioned Figure 1 Steps of the text correction method.

[0127] It should be noted that in the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0128] It should be noted that the machine-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0129] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0130] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0131] Although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present application. Certain features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0132] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A model training method, characterized in that: The method comprises: Acquire model training data, wherein the model training data includes text data and annotation data, the annotation data indicates an error type, an error start position, and an error end position of the text data, the error type includes at least one of a spelling error, a grammatical error, a logic error, and a format error; Inputting the model training data into a machine learning model, performing masking processing on the word segmentation results of the text data through a first sub-model of the machine learning model, and performing shuffling processing on the word segmentation results of the text data through a second sub-model of the machine learning model; The machine learning model is trained according to the labeled data, the result after masking processing and the result after scrambling processing to obtain a text error correction model.

2. The method according to claim 1, characterized in that The first sub-model includes any one of an ELECTRA model, a BERT model, and a BART model; and the masking process of the word segmentation result of the text data by the first sub-model of the machine learning model includes: Generate a first arbitrary value for each word segmentation result of the text data by using the first submodel of the machine learning model; Masking is performed on the word segmentation results of the text data whose first arbitrary value is greater than a masking processing threshold.

3. The method according to claim 2, characterized in that The masking process of the word segmentation result of the text data by using the first sub-model of the machine learning model includes: The first sub-model of the machine learning model performs masking processing on the target word segmentation result of the text data and all sub-words corresponding to the target word segmentation result, wherein all sub-words are all components of the target word segmentation result.

4. The method according to claim 1, characterized in that: The second sub-model includes a PERT model; and the second sub-model of the machine learning model is used to shuffle the word segmentation results of the text data, including: Generate a second arbitrary value for each sentence segmentation result of the text data by using the first submodel of the machine learning model; All word segmentation results in the sentence results of the text data whose second arbitrary value is greater than the shuffling processing threshold are shuffled.

5. The method according to claim 1, characterized in that The obtaining of model training data comprises: Acquire original sample data, where the original sample data is text data with at least one error type among spelling errors, grammatical errors, logical errors and format errors; Cleaning the original sample data to obtain cleaned original sample data; Model training data is obtained by marking the error type, error start position and error end position of the cleaned original sample data.

6. A text error correction method, characterized in that: The method comprises: Acquire data to be corrected, where the data to be corrected is text data; The data to be corrected is input into a text correction model, and error detection is performed on the data to be corrected by the text correction model. When the error detection result indicates that the data to be corrected has errors, at least one of component redundancy correction, wrong word correction, component incomplete correction and word order error correction is performed on the data to be corrected by the text correction model, so that the text correction model outputs a text correction result, wherein the text correction model is trained by the model training method described in any one of claims 1 to 5.

7. A model training device, characterized in that: The device comprises: an acquisition module, an input module and a training module; The acquisition module is used to acquire model training data, wherein the model training data includes text data and annotation data, the annotation data indicates the error type, error start position and error end position of the text data, and the error type includes at least one of spelling error, grammatical error, logical error and format error; The input module is used to input the model training data into the machine learning model, perform masking processing on the word segmentation results of the text data through the first sub-model of the machine learning model, and perform scrambling processing on the word segmentation results of the text data through the second sub-model of the machine learning model; The training module is used to train the machine learning model according to the labeled data, the result after masking processing and the result after scrambling processing to obtain a text error correction model.

8. A text error correction device, characterized in that: The device comprises: a data acquisition module and a data input module; The data acquisition module is used to acquire the data to be corrected, wherein the data to be corrected is text data; The data input module is used to input the data to be corrected into a text correction model, and perform error detection on the data to be corrected through the text correction model. When the error detection result indicates that the data to be corrected has errors, the text correction model is used to perform at least one of component redundancy correction, wrong word correction, component incomplete correction and word order error correction on the data to be corrected, so that the text correction model outputs a text correction result, wherein the text correction model is trained by the model training method described in any one of claims 1 to 5.

9. A computer device, characterized in that: The device comprises: a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement the model training method as described in any one of claims 1 to 5, or the various steps of the text error correction method as described in claim 6.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the model training method as described in any one of claims 1 to 5, or the various steps of the text error correction method as described in claim 6.

Images