Text processing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the application provides a text processing method and device, electronic equipment and readable storage medium, and relates to the technical field of computers. In the embodiment of the application, the target model obtained by training according to the original sample of the first predetermined proportion is used to screen out the target sample, and then the syntax detection model is trained by using the target sample. Therefore, the embodiment of the application can accurately screen out the target sample of a specific type, and then construct a target training set according to the target sample of the specific type, so that the syntax detection model can be more effectively trained. Therefore, through the embodiment of the application, the syntax detection model trained based on the target training set can make the syntax detection model more accurately perform syntax detection, thereby improving the accuracy of syntax error detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a text processing method and device, electronic equipment and readable storage medium. BACKGROUND

[0002] At present, with the development of computer technology, the text can be detected for grammatical errors in the process of processing the text. The grammatical error detection is a detection for identifying whether there is a grammatical error in the input text.

[0003] In the related art, there are still problems such as error detection, which leads to low accuracy of grammatical error detection. SUMMARY

[0004] Therefore, the embodiments of the present application provide a text processing method and device, electronic equipment and readable storage medium to improve the accuracy of grammatical error detection.

[0005] In a first aspect, a text processing method is provided, and the method comprises:

[0006] obtaining a target text.

[0007] inputting the target text into a pre-trained grammar detection model to determine a grammar detection result corresponding to the target text, wherein the grammar detection model is trained based on a target training set, the target training set at least includes a target sample, the target sample is obtained by screening a target model, and the target model is trained according to a first predetermined proportion of original samples.

[0008] In some embodiments, the method further comprises:

[0009] obtaining a pre-training model and original samples, the original samples including a first predetermined proportion of positive samples and first negative samples, the positive samples being used to represent samples without grammatical errors, the first negative samples being used to represent samples with grammatical errors, and the proportion of the first negative samples being greater than the proportion of the positive samples.

[0010] training the pre-training model according to the original samples to determine the target model.

[0011] In some embodiments, the method further comprises:

[0012] obtaining a to-be-screened data set.

[0013] inputting the text in the to-be-screened data set into the target model to determine an identification result output by the target model, wherein the identification result includes text without grammatical errors and text with grammatical errors.

[0014] The text without grammatical errors in the recognition results is identified as the target sample.

[0015] In some embodiments, the method further includes:

[0016] A target training set is constructed based on the target samples. The target training set includes a second predetermined proportion of target samples and a second negative sample. The second negative sample is used to represent samples with grammatical errors. The proportion of the target samples is greater than the proportion of the second negative sample.

[0017] The target model is trained a second time based on the target training set to determine the syntax detection model.

[0018] In some embodiments, the step of performing secondary training on the target model based on the target training set to determine the syntax detection model includes:

[0019] Each sample in the target training set is input into the target model to determine the output of the target model.

[0020] Based on the training labels corresponding to each sample in the target training set and the output results, the model parameters of the target model are adjusted to determine the syntax detection model.

[0021] In some embodiments, the pre-trained model is a Chinese pre-trained unbalanced transformer model.

[0022] In some embodiments, the syntax detection model consists of a shared encoder and a understanding decoder in the pre-trained model, and the target training set is used to train at least the shared encoder and the understanding decoder.

[0023] Secondly, a text processing apparatus is provided, the apparatus comprising:

[0024] The target text acquisition module is configured to acquire the target text.

[0025] The syntax detection module is configured to perform a syntax detection model by inputting the target text into a pre-trained syntax detection model to determine the syntax detection result corresponding to the target text. The syntax detection model is trained based on a target training set, which includes at least target samples. The target samples are selected by the target model, which is trained based on a first predetermined proportion of the original samples.

[0026] In some embodiments, the apparatus further includes:

[0027] The first acquisition module is configured to acquire a pre-trained model and original samples. The original samples include a first predetermined proportion of positive samples and a first negative sample. The positive samples are used to represent samples without grammatical errors, and the first negative samples are used to represent samples with grammatical errors. The proportion of the first negative samples is greater than the proportion of the positive samples.

[0028] The first training module is configured to train the pre-trained model based on the original samples to determine the target model.

[0029] In some embodiments, the apparatus further includes:

[0030] The second acquisition module is configured to acquire the dataset to be filtered.

[0031] The input module is configured to input text from the dataset to be filtered into the target model to determine the recognition result output by the target model, wherein the recognition result includes text without grammatical errors and text with grammatical errors.

[0032] The filtering module is configured to identify grammatically correct text from the recognition results as target samples.

[0033] In some embodiments, the apparatus further includes:

[0034] The target training set construction module is configured to construct a target training set based on the target samples. The target training set includes a second predetermined proportion of target samples and a second negative sample. The second negative sample is used to represent samples with grammatical errors. The proportion of the target samples is greater than the proportion of the second negative sample.

[0035] The second training module is configured to perform secondary training on the target model based on the target training set to determine the syntax detection model.

[0036] In some embodiments, the second training module is specifically configured to perform:

[0037] Each sample in the target training set is input into the target model to determine the output of the target model.

[0038] Based on the training labels corresponding to each sample in the target training set and the output results, the model parameters of the target model are adjusted to determine the syntax detection model.

[0039] In some embodiments, the pre-trained model is a Chinese pre-trained unbalanced transformer model.

[0040] In some embodiments, the syntax detection model consists of a shared encoder and a understanding decoder in the pre-trained model, and the target training set is used to train at least the shared encoder and the understanding decoder.

[0041] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the method described in the first aspect.

[0043] In this embodiment, the target training set for training the grammar detection model includes target samples, which are selected by the target model. The target model is trained based on a first predetermined proportion of original samples. In this process, because this embodiment first selects target samples using the target model trained on the first predetermined proportion of original samples, and then trains the grammar detection model using these target samples, it can accurately select specific types of target samples before constructing a target training set, thereby training the grammar detection model more effectively. Therefore, through this embodiment, the grammar detection model trained on the target training set can perform grammar detection more accurately, thus improving the accuracy of grammar error detection. Attached Figure Description

[0044] The above and other objects, features and advantages of the present application will become clearer from the following description of embodiments of the present application with reference to the accompanying drawings, in which:

[0045] Figure 1 This is a flowchart illustrating the text processing method according to an embodiment of this application;

[0046] Figure 2 This is a flowchart of the text processing method according to an embodiment of this application;

[0047] Figure 3 This is a flowchart of another text processing method according to an embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the pre-training process of the pre-trained model in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the syntax detection model in an embodiment of this application;

[0050] Figure 6 This is a flowchart of another text processing method according to an embodiment of this application;

[0051] Figure 7 This is a flowchart of another text processing method according to an embodiment of this application;

[0052] Figure 8 This is a schematic diagram of the structure of the text processing device according to an embodiment of this application;

[0053] Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0054] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0055] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0056] Unless the context explicitly requires it, words such as "including" or "contains" in the instruction manual should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0057] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0058] Currently, due to the limited number of grammar correction datasets, and the fact that much of the data in these datasets is synthesized through methods such as random addition, deletion, replacement, or shuffling, the amount of data closely resembling real-world writing scenarios is very small. Consequently, problems still exist in the detection of grammar errors using related technologies, resulting in low accuracy in grammar error detection. Therefore, improving the accuracy of grammar error detection is an urgent problem to be solved.

[0059] To address the aforementioned problems, this application provides a text processing method to improve the accuracy of syntax error detection. This method can be applied to electronic devices, which can be terminals or servers. Terminals can be smartphones, tablets, or personal computers (PCs), etc. Servers can be single servers, distributed server clusters, or cloud servers.

[0060] like Figure 1 As shown, the electronic device 13 can be a terminal or a server. The user 11 can input the target text 12 through external input devices such as a mouse or keyboard, or through the input unit of the electronic device 13 itself (such as a keyboard or touch screen). Correspondingly, the electronic device 13 can receive the target text 12 and perform grammatical error detection on the target text 12 based on the above-described text processing method. Specifically, the grammar detection model in the electronic device 13 can be trained in a specific manner. Specifically, the target training set used to train the grammar detection model can include target samples, which are selected by the target model. The target model is trained based on a first predetermined proportion of original samples. In this process, since the embodiment of this application first selects target samples using the target model trained based on the first predetermined proportion of original samples, and then trains the grammar detection model using the target samples, the embodiment of this application can accurately select specific types of target samples first, and then construct a target training set based on the specific types of target samples, thereby training the grammar detection model more effectively. Therefore, through the embodiment of this application, the grammar detection model trained based on the target training set can perform grammar detection more accurately, thereby improving the accuracy of grammar error detection.

[0061] Furthermore, this application will provide a detailed description of a text processing method provided in conjunction with specific implementation methods, such as... Figure 2 As shown, a text processing method may specifically include the following steps:

[0062] In step S110, the target text is obtained.

[0063] The target text may include multiple characters, words, and sentences.

[0064] In one optional implementation, after obtaining the target text, the present application embodiment can preprocess the target text to improve the efficiency of syntax error detection.

[0065] The preprocessing may include noise filtering, text segmentation, space detection, and symbol detection. Specifically, the target text may contain meaningless text noise such as garbled characters. In this case, the embodiments of this application can identify, delete, and mark the aforementioned text noise, thereby preserving the valid information in the target text while also incorporating error marking as part of the detection process to alert the user to writing errors in the target text.

[0066] In cases where the target text includes multiple sentences, embodiments of this application can further divide the target text into sentence-level segments based on punctuation marks in the target text, so as to mark, record and return the sentence set corresponding to the target text, thereby enabling further individual grammatical error detection for each sentence in the sentence set.

[0067] This application embodiment can also perform word recognition on the target text, thereby dividing the target text or its various sentences into word-level segments to annotate, record, and return the word set corresponding to the target text. Furthermore, based on this word set, this application embodiment determines the part of speech and meaning of each word, and thus, based on the part of speech and meaning of each word, determines whether there are grammatical errors in the target text.

[0068] This application embodiment can also perform space detection or symbol detection on the target text, thereby recording or deleting spaces or invalid symbols in the target text, thus avoiding the impact of the aforementioned spaces or invalid symbols on syntax error detection.

[0069] Therefore, by preprocessing the target text, the embodiments of this application can remove noise from the target text, segment the target text, and detect spaces or symbols in the target text, thereby improving the efficiency of subsequent syntax error detection.

[0070] In step S120, the target text is input into the pre-trained syntax detection model to determine the syntax detection result corresponding to the target text.

[0071] The syntax detection model is trained on a target training set, which includes at least target samples. The target samples are selected by the target model, which is trained on a first predetermined proportion of the original samples.

[0072] Specifically, the original samples include samples with grammatical errors and samples without grammatical errors. A first predetermined ratio is used to characterize the ratio between samples with grammatical errors and samples without grammatical errors in the original samples. In the embodiments of this application, the ratio between various samples can be adjusted by changing the first predetermined ratio, so that the target model has a stronger bias when processing text, for example, making it easier for the target model to classify text into samples with grammatical errors or samples without grammatical errors.

[0073] Furthermore, embodiments of this application can modify the target model's bias in processing text, enabling the target model to accurately filter out target samples of specific types. Taking text types A and B as an example, when the target model is more inclined to classify text as type A samples and less inclined to classify text as type B samples, the target model will classify most text as type A. Among the text classified as type A, there may be a certain number of type B samples. In other words, when the target model is more inclined to classify text as type A samples, the target model's recognition of type A text is inaccurate.

[0074] Correspondingly, since the target model is not prone to classifying text into type B, when the target model outputs a type B recognition result, the result is accurate. In other words, the embodiments of this application can adjust the first predetermined proportion corresponding to the original samples so that the target model is more inclined to select a certain type when filtering text, thereby enabling the target model to accurately select another type of text as the target sample during the text filtering process.

[0075] Furthermore, in this embodiment of the application, after accurately selecting the target samples, a target model can be trained based on a training set including the target samples to obtain a syntax detection model, and syntax error detection can be performed on the target text based on the syntax detection model.

[0076] In this embodiment, the target training set for training the grammar detection model includes target samples, which are selected by the target model. The target model is trained based on a first predetermined proportion of original samples. In this process, because this embodiment first selects target samples using the target model trained on the first predetermined proportion of original samples, and then trains the grammar detection model using these target samples, it can accurately select specific types of target samples before constructing a target training set, thereby training the grammar detection model more effectively. Therefore, through this embodiment, the grammar detection model trained on the target training set can perform grammar detection more accurately, thus improving the accuracy of grammar error detection.

[0077] In one alternative implementation, embodiments of this application can train a pre-trained model to determine a target model. Specifically, such as... Figure 3 As shown, the process may include the following steps:

[0078] In step S210, the pre-trained model and the original samples are obtained.

[0079] The original sample includes a first predetermined proportion of positive samples and a first negative sample. The positive samples are used to represent samples without grammatical errors, and the first negative samples are used to represent samples with grammatical errors. The proportion of the first negative samples is greater than the proportion of the positive samples.

[0080] In other words, the first predetermined ratio can be used to characterize the ratio between samples with grammatical errors and samples without grammatical errors. For example, the ratio between the first negative sample and the positive sample (i.e., the first predetermined ratio) can be a suitable ratio such as 6:3, 7:3, 8:3, 8:2 or 9:4.

[0081] In step S220, the pre-trained model is trained based on the original samples to determine the target model.

[0082] In the first predetermined proportion of the embodiments of this application, if the proportion of the first negative sample is higher, the pre-trained model (i.e., the target model) after training is more likely to identify the text as having grammatical errors. In other words, since the proportion of the first negative sample is greater than the proportion of the positive sample, the target model after training has a stronger ability to identify the first negative sample (i.e., the text with grammatical errors).

[0083] Furthermore, because the target model has a stronger ability to identify text with grammatical errors, it tends to classify text as grammatically incorrect rather than as grammatically correct. In other words, the target model will classify most text as grammatically incorrect, but among those classified as grammatically incorrect, there may be a certain number of grammatically correct texts.

[0084] Correspondingly, since the target model is not easy to classify text as grammatically correct, when the target model outputs a grammatically correct recognition result, the result is accurate. In other words, the embodiments of this application can adjust the first predetermined proportion corresponding to the original samples so that the target model is more inclined to select text with grammatical errors when screening text, thereby enabling the target model to accurately select grammatically correct text as target samples during the text screening process.

[0085] In one alternative implementation, the pre-trained model is a Chinese pre-trained unbalanced transformer (CPT) model.

[0086] The CPT model is a Transformer encoder-decoder structure model that can be used for Chinese text processing. Specifically, the CPT model includes a shared encoder (S-Enc), an understanding decoder (U-Dec), and a generation decoder (G-Dec).

[0087] The shared encoder is a bidirectional attention-based encoder that receives target text and encodes it. The understanding decoder is also a bidirectional attention-based decoder that receives the encoded result from the shared encoder and outputs the text understanding result. The generation decoder receives the encoded result from the shared encoder and outputs the generated text.

[0088] Therefore, the CPT model is an asymmetric structure model with a single input and multiple outputs. Furthermore, since the CPT model is a single-input, multiple-output structure model, a multi-task training approach can be used during the pre-training process to fully train each module in the CPT model.

[0089] like Figure 4 As shown, Figure 4 Five training methods are included: (a) CPTu, (b) CPTg, (c) CPTug, (d) CPTu+p, and (e) CPTg+p. Among them, training methods (a) CPTu, (b) CPTg, and (c) CPTug correspond to traditional external classifier training methods, with adjustments made to the understanding decoder, the generative decoder, and their combination, respectively. Training method (d) CPTu+p employs Masked Language Modeling (MLM), taking text with a prompt as input and having the model predict the word corresponding to the training label at the mask position. Training method (e) CPTg+p uses text generation, allowing the model to generate a prompt with the corresponding training label based on the input training samples. Here, V(y) is the mapping function that maps the model's prediction to the training label.

[0090] according to Figure 4The various training methods shown in this application embodiment can obtain a relatively mature pre-trained model. Further, this application embodiment can obtain a pre-trained CPT model as the aforementioned pre-trained model, and train the pre-trained CPT model based on the original samples to determine the target model. Subsequently, this application embodiment can train the target model using the target training set to determine the syntax detection model, and perform syntax error detection based on the syntax detection model. Since this application embodiment first selects target samples using the target model obtained by training on the original samples according to a first predetermined ratio, and then trains the syntax detection model using the target samples, this application embodiment can accurately select specific types of target samples first, and then construct a target training set based on the specific types of target samples, thereby training the syntax detection model more effectively. Therefore, through this application embodiment, using the syntax detection model trained on the target training set, the syntax detection model can perform syntax detection more accurately, thereby improving the accuracy of syntax error detection.

[0091] In an optional implementation, the syntax detection model of this application embodiment may consist of a shared encoder and a comprehension decoder in a pre-trained model, wherein the target training set is used to train at least the shared encoder and the comprehension decoder.

[0092] Since the understanding decoder can be used to perform text understanding tasks and output text understanding results, this application embodiment can be used to understand target text and perform syntax error detection on the target text to determine the syntax detection result corresponding to the target text.

[0093] In addition, during the pre-training process of the CPT model, since the shared encoder needs to be pre-trained along with both the understanding decoder and the generating decoder, the embodiments of this application extract the shared encoder from the pre-trained CPT model for syntax error detection, which can effectively improve the accuracy of text information extraction.

[0094] For example, such as Figure 5 As shown, in this embodiment of the application, the target text 12 can be input into the shared encoder 41 of the trained syntax detection model, and the shared encoder 41 can extract information from the target text 12. Then, the shared encoder 41 can send the extracted information to the understanding decoder 42, so that the understanding decoder 42 can perform syntax error detection on the target text 12, thereby outputting a syntax detection result 14. The understanding decoder 42 can perform syntax error detection by classification, that is, the syntax detection result 14 can include "there is a syntax error" and "there is no syntax error".

[0095] It should be noted that since the syntax detection model is trained from the target model, and the target model is trained from the pre-trained model, in this embodiment of the application, if the pre-trained model is a pre-trained CPT model, then the target model is also a model obtained based on the CPT model.

[0096] In one scenario, the target model can have the same structure as the CPT model, meaning the target model includes a shared encoder, a comprehension decoder, and a generative decoder. In this case, embodiments of this application can extract the shared encoder and comprehension decoder from the trained target model to form a syntax detection model.

[0097] In another scenario, the target model can have the same structure as the syntax detection model, i.e., the target model includes a shared encoder and a comprehension decoder. In this case, embodiments of this application can first extract the shared encoder and comprehension decoder from the CPT model, and then train the shared encoder and comprehension decoder based on the original samples to obtain the target model. Further, embodiments of this application can perform secondary training on the target model based on the target training set to determine the syntax detection model.

[0098] Furthermore, embodiments of this application can perform syntax error detection using a syntax detection model trained on a target training set, enabling the syntax detection model to perform syntax detection more accurately, thereby improving the accuracy of syntax error detection.

[0099] In one optional implementation, after determining the target model, the embodiments of this application can filter target samples based on the target model. Specifically, for example... Figure 6 As shown, embodiments of this application may further include the following steps:

[0100] In step S310, the dataset to be filtered is obtained.

[0101] As can be seen from the above embodiments, the present application embodiments can adjust the first predetermined ratio corresponding to the original samples so that the target model tends to favor a certain type when filtering text, thereby enabling the target model to accurately filter out another type of text as the target sample during the text filtering process. In other words, the target model of the present application embodiments has a better recognition ability for a certain type of text.

[0102] In this embodiment, a target model can be used to filter text without grammatical errors as target samples. That is, this embodiment can adjust the first predetermined proportion corresponding to the original samples so that the proportion of the first negative samples is greater than the proportion of the positive samples. This makes the target model more inclined to filter text with grammatical errors, and consequently, the target model can accurately filter out text without grammatical errors as target samples.

[0103] In addition, in order to make the target samples more closely resemble real writing scenarios, this application embodiment can add publicly available data such as actual news, magazines, and encyclopedias to the dataset to be screened, avoiding data synthesized by methods such as random addition, deletion, replacement, or disorder in the dataset to be screened, so as to improve the authenticity of the target samples and thereby increase the text recognition capability of the grammar detection model.

[0104] In step S320, the text in the dataset to be filtered is input into the target model to determine the recognition result output by the target model.

[0105] The identification results include texts without grammatical errors and texts with grammatical errors.

[0106] In step S330, the text without grammatical errors in the recognition results is identified as the target sample.

[0107] Since the target model is not prone to classifying text as grammatically correct, when the target model outputs a grammatically correct recognition result, the result is accurate. In other words, this embodiment of the application can adjust the first predetermined proportion corresponding to the original samples so that the target model is more inclined to select text with grammatical errors when filtering text, thereby enabling the target model to accurately select grammatically correct text as target samples during the text filtering process.

[0108] In one optional implementation, embodiments of this application can perform secondary training on the target model based on the target training set, specifically, as follows: Figure 7 As shown, embodiments of this application may further include the following steps:

[0109] In step S410, a target training set is constructed based on the target samples.

[0110] The target training set includes a second predetermined proportion of target samples and a second negative sample. The second negative sample is used to represent samples with grammatical errors, and the proportion of target samples is greater than the proportion of the second negative sample.

[0111] In this embodiment, since a sufficient number of target samples are obtained through the target model screening, the proportion of target samples in the target training set can be increased so that the proportion of target samples is greater than the proportion of the second negative samples. For example, the ratio of target samples to second negative samples can be 10:1, 20:1, or 25:1, etc.

[0112] In step S420, the target model is trained a second time based on the target training set to determine the syntax detection model.

[0113] In one optional implementation, step S420 may include the following steps:

[0114] In step S421, each sample in the target training set is input into the target model to determine the output of the target model.

[0115] The target training set may include target samples and second negative samples, with the proportion of target samples being greater than the proportion of second negative samples.

[0116] In step S422, the model parameters of the target model are adjusted according to the training labels and output results corresponding to each sample in the target training set to determine the syntax detection model.

[0117] In this embodiment, the target training set for training the grammar detection model includes target samples, which are selected by the target model. The target model is trained based on a first predetermined proportion of original samples. In this process, because this embodiment first selects target samples using the target model trained on the first predetermined proportion of original samples, and then trains the grammar detection model using these target samples, it can accurately select specific types of target samples before constructing a target training set, thereby training the grammar detection model more effectively. Therefore, through this embodiment, the grammar detection model trained on the target training set can perform grammar detection more accurately, thus improving the accuracy of grammar error detection.

[0118] Based on the same technical concept, embodiments of this application also provide a text processing device, such as... Figure 8 As shown, the device includes a target text acquisition module 81 and a syntax detection module 82.

[0119] The target text acquisition module 81 is configured to acquire the target text.

[0120] The syntax detection module 82 is configured to perform a syntax detection model that inputs the target text into a pre-trained syntax detection model to determine the syntax detection result corresponding to the target text. The syntax detection model is trained based on a target training set, which includes at least target samples. The target samples are selected by the target model, and the target model is trained based on a first predetermined proportion of the original samples.

[0121] In some embodiments, the apparatus further includes:

[0122] The first acquisition module is configured to acquire a pre-trained model and original samples. The original samples include a first predetermined proportion of positive samples and a first negative sample. The positive samples are used to represent samples without grammatical errors, and the first negative samples are used to represent samples with grammatical errors. The proportion of the first negative samples is greater than the proportion of the positive samples.

[0123] The first training module is configured to train the pre-trained model based on the original samples to determine the target model.

[0124] In some embodiments, the apparatus further includes:

[0125] The second acquisition module is configured to acquire the dataset to be filtered.

[0126] The input module is configured to input text from the dataset to be filtered into the target model to determine the recognition result output by the target model, wherein the recognition result includes text without grammatical errors and text with grammatical errors.

[0127] The filtering module is configured to identify grammatically correct text from the recognition results as target samples.

[0128] In some embodiments, the apparatus further includes:

[0129] The target training set construction module is configured to construct a target training set based on the target samples. The target training set includes a second predetermined proportion of target samples and a second negative sample. The second negative sample is used to represent samples with grammatical errors. The proportion of the target samples is greater than the proportion of the second negative sample.

[0130] The second training module is configured to perform secondary training on the target model based on the target training set to determine the syntax detection model.

[0131] In some embodiments, the second training module is specifically configured to perform:

[0132] Each sample in the target training set is input into the target model to determine the output of the target model.

[0133] Based on the training labels corresponding to each sample in the target training set and the output results, the model parameters of the target model are adjusted to determine the syntax detection model.

[0134] In some embodiments, the pre-trained model is a Chinese pre-trained unbalanced transformer model.

[0135] In some embodiments, the syntax detection model consists of a shared encoder and a understanding decoder in the pre-trained model, and the target training set is used to train at least the shared encoder and the understanding decoder.

[0136] In this embodiment, the target training set for training the grammar detection model includes target samples, which are selected by the target model. The target model is trained based on a first predetermined proportion of original samples. In this process, because this embodiment first selects target samples using the target model trained on the first predetermined proportion of original samples, and then trains the grammar detection model using these target samples, it can accurately select specific types of target samples before constructing a target training set, thereby training the grammar detection model more effectively. Therefore, through this embodiment, the grammar detection model trained on the target training set can perform grammar detection more accurately, thus improving the accuracy of grammar error detection.

[0137] Figure 9 This is a schematic diagram of an electronic device according to an embodiment of this application. For example... Figure 9 As shown, Figure 9 The illustrated electronic device is a general address lookup device, comprising a general computer hardware architecture, including at least a processor 91 and a memory 92. The processor 91 and memory 92 are connected via a bus 93. The memory 92 is adapted to store instructions or programs executable by the processor 91. The processor 91 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 91 executes the instructions stored in the memory 92, thereby performing the method flow described in the embodiments of this application as above to process data and control other devices. The bus 93 connects the aforementioned components together, and also connects these components to a display controller 94, a display device, and an input / output (I / O) device 95. The input / output (I / O) device 95 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 95 is connected to the system via an input / output (I / O) controller 96.

[0138] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.

[0140] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.

[0141] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.

[0142] Another embodiment of this application relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.

[0143] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program specifying the relevant hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0144] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A text processing method, characterized in that, The method includes: Obtain the target text; and The target text is input into a pre-trained grammar detection model to determine the grammar detection result corresponding to the target text. The grammar detection model is trained based on a target training set, which includes at least target samples. The target samples are selected by the target model. The target model is trained based on a first predetermined proportion of original samples. The first predetermined proportion is used to characterize the ratio between samples with grammatical errors and samples without grammatical errors. Obtain a pre-trained model and original samples, wherein the original samples include a first predetermined proportion of positive samples and a first predetermined proportion of negative samples, the positive samples being used to represent samples without grammatical errors, and the first predetermined proportion of negative samples being used to represent samples with grammatical errors, the proportion of the first predetermined proportion of negative samples being greater than the proportion of positive samples; and The pre-trained model is trained based on the original samples to determine the target model; Obtain the dataset to be filtered; The text from the dataset to be filtered is input into the target model to determine the recognition result output by the target model, wherein the recognition result includes text without grammatical errors and text with grammatical errors; and The text without grammatical errors in the recognition results is identified as the target sample.

2. The method according to claim 1, characterized in that, The method further includes: A target training set is constructed based on the target samples. The target training set includes a second predetermined proportion of target samples and a second negative sample, where the second negative sample represents samples with grammatical errors. The proportion of the target samples is greater than the proportion of the second negative sample. The target model is trained a second time based on the target training set to determine the syntax detection model.

3. The method according to claim 2, characterized in that, The step of performing secondary training on the target model based on the target training set to determine the syntax detection model includes: Each sample in the target training set is input into the target model to determine the output of the target model; and Based on the training labels corresponding to each sample in the target training set and the output results, the model parameters of the target model are adjusted to determine the syntax detection model.

4. The method according to any one of claims 1-3, characterized in that, The pre-trained model is a Chinese pre-trained unbalanced transformer model.

5. The method according to claim 4, characterized in that, The syntax detection model consists of a shared encoder and a understanding decoder in the pre-trained model, and the target training set is used to train at least the shared encoder and the understanding decoder.

6. A text processing device, characterized in that, The device includes: The target text acquisition module is configured to acquire the target text; and A grammar detection module is configured to input the target text into a pre-trained grammar detection model to determine the grammar detection result corresponding to the target text. The grammar detection model is trained based on a target training set, which includes at least target samples. The target samples are selected by the target model. The target model is trained based on a first predetermined proportion of original samples. The first predetermined proportion is used to characterize the ratio between samples with grammatical errors and samples without grammatical errors. A first acquisition module is configured to acquire a pre-trained model and original samples, wherein the original samples include a first predetermined proportion of positive samples and a first negative sample, the positive samples representing samples without grammatical errors, and the first negative samples representing samples with grammatical errors, the proportion of the first negative samples being greater than the proportion of the positive samples; and The first training module is configured to train the pre-trained model based on the original samples to determine the target model; The second acquisition module is configured to acquire the dataset to be filtered. An input module is configured to input text from the dataset to be filtered into the target model to determine the recognition results output by the target model, wherein the recognition results include grammatically correct text and grammatically incorrect text; and The filtering module is configured to identify text without grammatical errors in the recognition results as target samples.

7. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-5.

Images