A grammatical error correction method and device

Abstract

The present invention discloses a grammatical error correction method and device. The method comprises: upon receiving a sentence text, detecting the sentence text based on a trained error detection model to determine whether the sentence text contains errors; if the sentence text contains errors, searching a pre-established search library for text most relevant to the erroneous portion of the sentence text; and processing the sentence text and the most relevant text using a preset grammatical error correction model to obtain a corrected text corresponding to the sentence text. This retrieval-enhanced error correction method achieves improved error correction results, overcoming the drawbacks of related technologies, such as the difficulty and poor error correction results.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a grammatical error correction method and device. Background Art

[0002] The grammatical error correction task requires the model to correct grammatically incorrect input sentences into sentences without them before outputting them. A serious challenge facing grammatical error correction tasks is the scarcity of labeled data. Fortunately, unlabeled correct data is plentiful. Pre-training is a common approach to leveraging unlabeled data to improve the performance of downstream tasks. However, pre-training typically involves learning from a significant amount of unlabeled data. This approach implicitly incorporates knowledge from this unlabeled data into the model parameters, making it impossible to explicitly control or modify the knowledge referenced by the model.

[0003] In related technologies, the task of grammatical error correction is difficult and the effect needs to be improved. Summary of the Invention

[0004] The present application provides a grammatical error correction task and device, and an electronic device to solve the problems existing in the related art.

[0005] In a first aspect, the present invention provides a grammatical error correction method, comprising: after receiving a sentence text, detecting the sentence text based on a trained error detection model to determine whether the sentence text contains errors; if the sentence text contains errors, retrieving the text most relevant to the erroneous part of the sentence text in a pre-established retrieval library; and using a preset grammatical error correction model to process the sentence text and the most relevant text to obtain the correct text corresponding to the sentence text.

[0006] Optionally, when establishing the retrieval library, the method includes: obtaining unannotated correct corpus, and segmenting the sentences in the corpus according to fixed length to obtain correct phrases; inputting the correct phrases into a pre-trained neural network model to encode the correct phrases to obtain the encoding of the phrases; and storing the encoding of the phrases and the key-value pairs of the correct phrases in the retrieval library.

[0007] Optionally, the correct phrase is input into a pre-trained neural network model to encode the correct phrase, and the encoding of the phrase is obtained including: H=AvgPool(Encoder(s)[start:end]); wherein Encoder is a neural network model, s is a sentence containing the phrase, and start and end are the starting and ending positions of the phrase in the sentence.

[0008] Optionally, pre-training the neural network model includes: processing incorrect sentences and correct sentences based on an edit distance algorithm to obtain the content of the correct part corresponding to the incorrect part in the incorrect sample; using the incorrect part as a query text and the content of the correct part as a positive sample, and using the BM25 algorithm to retrieve phrases close to the query text from a large number of correct phrases as negative samples; and training the neural network model based on the query text, positive samples and negative samples.

[0009] Optionally, the training of the neural network model based on the query text positive sample and the negative sample includes: training the neural network model using contrastive learning loss: Among them, p1, p2 are query and positive example, n i is a negative example, sim is the function for calculating similarity, and τ is the temperature parameter.

[0010] Optionally, training the error detection model includes: obtaining the original sentence and marking the part of the original sentence that has grammatical errors; obtaining the vector representation of each position of the sentence based on the encoder in the error detection model, and then predicting its correctness through a linear layer, that is, D = Linear(Encoder(s)).

[0011] Optionally, the use of a preset grammatical error correction model to process the sentence text and the most relevant text to obtain the correct text corresponding to the sentence text includes: splicing the sentence text and the most relevant text and inputting them into the encoder to obtain the corresponding hidden state; splicing the encoding corresponding to the sentence text and the hidden state and inputting them into the decoder for prediction to obtain a predicted correct sentence.

[0012] In a second aspect, the present invention provides a grammatical error correction device, comprising an error detection unit, configured to, after receiving a sentence text, detect the sentence text based on a trained error detection model to determine whether the sentence text contains errors; a retrieval unit, configured to retrieve the text most relevant to the erroneous part of the sentence text from a pre-established retrieval library if the sentence text contains errors; and a prediction unit, configured to process the sentence text and the most relevant text using a preset grammatical error correction model to obtain a correct text corresponding to the sentence text.

[0013] In a third aspect, the present invention provides a storage medium storing a computer program, which, when executed by a processor, implements the method described in any one of the implementation methods of the first aspect.

[0014] In a fourth aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for selecting a three-dimensional reconstructed texture view provided in the first aspect is implemented.

[0015] The present invention discloses a grammatical error correction method and device. The method comprises: upon receiving a sentence text, detecting the sentence text based on a trained error detection model to determine whether the sentence text contains errors; if the sentence text contains errors, searching a pre-established search library for text most relevant to the erroneous portion of the sentence text; and processing the sentence text and the most relevant text using a preset grammatical error correction model to obtain a corrected text corresponding to the sentence text. This retrieval-enhanced error correction method achieves improved error correction results, overcoming the drawbacks of related technologies, such as the difficulty and poor error correction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A flowchart of a grammatical error correction method for this application;

[0018] Figure 2 This is a schematic diagram of an application of a grammatical error correction method provided in this application;

[0019] Figure 3 A method corresponding to the Figure 1 Schematic diagram of the electronic device. DETAILED DESCRIPTION

[0020] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] The following is combined with Figure 1 The grammatical error correction method of the present application is exemplified. The method comprises the following steps:

[0022] Step 101: After receiving a sentence text, the sentence text is detected based on a trained error detection model to determine whether there are errors in the sentence text.

[0023] In this embodiment, the error detection model is used to identify the input sentence text to determine whether there are grammatical errors and the part with grammatical errors. Exemplarily, the sequence tagging model is used to tag the grammatical errors that may be contained in the sentence.

[0024] As an optional implementation method of this embodiment, training the error detection model includes: obtaining the original sentence and marking the part of the original sentence that has grammatical errors; obtaining the vector representation of each position of the sentence based on the encoder in the error detection model, and then predicting its correctness through a linear layer, that is, D = Linear(Encoder(s)).

[0025] In this optional implementation, an encoder based on a pre-trained model is used to obtain the vector representation of each position in the sentence, and then its correctness is predicted through a linear layer, that is, D = Linear(Encoder(s)).

[0026] Step 102: If there is an error in the sentence text, search a pre-established search library for the text most relevant to the error portion of the sentence text.

[0027] In this implementation, reference Figure 2 If there is an error in the sentence text, the phrase at the error detection result will be intercepted and entered into the search library for search, and the phrase closest to its expression will be obtained as the search result for recall.

[0028] As an optional implementation method of this embodiment, when establishing the retrieval library, it includes: obtaining unannotated correct corpus, and segmenting the sentences in the corpus according to fixed length to obtain correct phrases; inputting the correct phrases into a pre-trained neural network model to encode the correct phrases to obtain the encoding of the phrases; and storing the encoding of the phrases and the key-value pairs of the correct phrases in the retrieval library.

[0029] In this optional implementation, correct phrases for retrieval are extracted from a large amount of correct texts, and after a phrase library is constructed, they are encoded using a phrase encoding module, and a retrieval library for retrieval is constructed using key-value pairs of the correct phrases and their phrase encodings.

[0030] As an optional implementation method of this embodiment, the correct phrase is input into a pre-trained neural network model to encode the correct phrase, and the encoding of the phrase is obtained, including: H = AvgPool(Encoder(s)[start:end]); wherein Encoder is a neural network model, s is a sentence containing the phrase, and start and end are the starting and ending positions of the phrase in the sentence.

[0031] In this optional implementation, refer to Figure 2 , obtain a large amount of unlabeled correct corpus data set, split the sentences in it according to fixed length, and obtain the correct phrase library for retrieval. When encoding the text sequence, considering that the same phrase should be represented differently in different contexts, a context-based encoding method is used to obtain the encoding of the phrase. First, after encoding the sentence, obtain the hidden layer representation of each position, and then take the representation of the position corresponding to the phrase, and take the mean pooling as the encoding of the phrase, that is: H = AvgPool(Encoder(s)[start:end]), where Encoder is the DPR model obtained in step A, s is the sentence containing the phrase, and start and end are the starting and ending positions of the phrase in the sentence.

[0032] Furthermore, considering the issues of retrieval accuracy and retrieval time, the FAISS library can be used to construct a retrieval library for the acquired phrases and their representations.

[0033] As an optional implementation of this embodiment, pre-training the neural network model includes: processing incorrect sentences and correct sentences based on an edit distance algorithm to obtain the content of the correct part corresponding to the incorrect part in the incorrect sample; using the incorrect part as a query text and the content of the correct part as a positive sample, and using the BM25 algorithm to retrieve phrases close to the query text from a large number of correct phrases as negative samples; and training the neural network model based on the query text, the positive sample and the negative sample.

[0034] In this optional implementation, a fine-tuned DPR model can be used to obtain phrase encoding results. The DPR model requires training data consisting of a query, a positive example, and a set of negative examples. Therefore, the DPR model first uses the edit distance algorithm to find the incorrect and correct parts of the incorrect and correct sentences, which serve as the query and positive examples. The BM25 algorithm is then used to search for phrases close to the query within a large set of correct phrases, which serve as negative examples.

[0035] In an optional implementation of the present embodiment, the training of the neural network model based on the query text positive sample and the negative sample includes: training the neural network model using contrastive learning loss: Among them, p1, p2 are query and positive example, n i is a negative example, sim is the function for calculating similarity, and τ is the temperature parameter.

[0036] Furthermore, when training the encoding model, after obtaining the training data, the DPR model is trained using the contrastive learning loss, namely: Where p1, p2 are query and positive example, ni is a negative example, sim is the function for calculating similarity, cosine similarity is used here, and τ is the temperature parameter.

[0037] Step 103: Process the sentence text and the most relevant text using a preset grammar error correction model to obtain a correct text corresponding to the sentence text.

[0038] In this embodiment, a sequence-to-sequence model is used to integrate the error detection results and the retrieval results into the input end, and the correct sentence is predicted.

[0039] This example transforms the model's implicit "error detection" and "error correction" into an explicit three-step process of "error detection" + "retrieval" + "error correction." This improves interpretability while also leveraging a vast amount of correct data through retrieval to enhance model performance. Unlike other tasks, the grammatical error correction task, in which the most relevant phrases within an erroneous sentence are fed into an error correction model for correction, focuses more on the erroneous portion and requires contextual analysis to clarify the meaning of the phrase. Therefore, enhancing grammatical error correction through retrieval can effectively improve its effectiveness.

[0040] As an optional implementation method of this embodiment, the use of a preset grammatical error correction model to process the sentence text and the most relevant text to obtain the correct text corresponding to the sentence text includes: splicing the sentence text and the most relevant text into an encoder to obtain a corresponding hidden state; splicing the encoding corresponding to the sentence text and the hidden state and inputting them into a decoder for prediction to obtain a predicted correct sentence.

[0041] In this optional implementation, an encoder-decoder architecture can be used to design a grammatical error correction model. At the input end, the phrase containing the error in the sentence text is concatenated with the original sentence and fed into the encoder to obtain its hidden state representation, i.e.: H1 = Encoder([s,d]); where s is the original sentence and d is the error detection result. Subsequently, the obtained hidden state is concatenated with the vector obtained from the most relevant text and fed into the decoder for decoding prediction, i.e.: T = Decoder([H1,H r ]); where H r is the vector representation of the recall result, and T is the correct sentence finally predicted by the model.

[0042] This embodiment utilizes a retrieval system to enhance an error correction model. This involves using a separate error detection model to identify erroneous phrases, then searching for relevant corrected fragments within a retrieval database using the erroneous phrase along with its partial context. The retrieval results are then fed into an enc-dec model using an attention mechanism. A supervised training model is then used to generate corrected sentences based on the original erroneous sentence, the erroneous phrase, and the retrieval results. Experimental results demonstrate the effectiveness of this error correction model, demonstrating its high practicality.

[0043] The above is a method provided for one or more embodiments of the present application. Based on the same idea, the present application also provides a corresponding grammatical error correction device, including an error detection unit, which is configured to detect the sentence text based on a trained error detection model after receiving the sentence text to determine whether the sentence text has errors; a retrieval unit, which is configured to retrieve the text most relevant to the erroneous part of the sentence text in a pre-established retrieval library if there are errors in the sentence text; and a prediction unit, which is configured to use a preset grammatical error correction model to process the sentence text and the most relevant text to obtain the correct text corresponding to the sentence text.

[0044] As an optional implementation method of this embodiment, when establishing the retrieval library, it includes: obtaining unannotated correct corpus, and segmenting the sentences in the corpus according to fixed length to obtain correct phrases; inputting the correct phrases into a pre-trained neural network model to encode the correct phrases to obtain the encoding of the phrases; and storing the encoding of the phrases and the key-value pairs of the correct phrases in the retrieval library.

[0045] As an optional implementation method of this embodiment, the correct phrase is input into a pre-trained neural network model to encode the correct phrase, and the encoding of the phrase is obtained, including: H = AvgPool(Encoder(s)[start:end]); wherein Encoder is a neural network model, s is a sentence containing the phrase, and start and end are the starting and ending positions of the phrase in the sentence.

[0046] As an optional implementation method of this embodiment, pre-training the neural network model includes: processing incorrect sentences and correct sentences based on an edit distance algorithm to obtain the content of the correct part corresponding to the incorrect part in the incorrect sample; using the incorrect part as a query text and the content of the correct part as a positive sample, and using the BM25 algorithm to retrieve phrases close to the query text from a large number of correct phrases as negative samples; and training the neural network model based on the query text, positive samples and negative samples.

[0047] As an optional implementation of this embodiment, the training of the neural network model based on the query text positive sample and the negative sample includes: training the neural network model using contrastive learning loss: Among them, p1, p2 are query and positive example, n i is a negative example, sim is the function for calculating similarity, and τ is the temperature parameter.

[0048] As an optional implementation method of this embodiment, training the error detection model includes: obtaining the original sentence and marking the part of the original sentence that has grammatical errors; obtaining the vector representation of each position of the sentence based on the encoder in the error detection model, and then predicting its correctness through a linear layer, that is, D = Linear(Encoder(s)).

[0049] As an optional implementation method of this embodiment, the sentence text and the most relevant text are processed using a preset grammatical error correction model to obtain the correct text corresponding to the sentence text, including: determining the hidden state corresponding to the sentence text based on the sentence text and the erroneous part detected by the error detection model; and predicting the correct sentence based on the hidden state and the most relevant text.

[0050] The present application also provides a computer readable medium, which stores a computer program, which can be used to execute the above Figure 1 Provided method.

[0051] This application also provides Figure 3 The one shown corresponds to Figure 1 Schematic diagram of the electronic equipment. Figure 3 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0052] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0053] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0054] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0055] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0056] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0057] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0060] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0061] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0062] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0063] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0064] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0065] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer media, including storage devices.

[0066] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment.

[0067] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A grammatical error correction method, characterized in that: include: After receiving the sentence text, the sentence text is checked based on the trained error detection model to determine whether the sentence text contains errors, wherein training the error detection model includes: obtaining the original sentence and marking the parts of the original sentence with grammatical errors; obtaining the vector representation of each position of the sentence based on the encoder in the error detection model, and then predicting its correctness through a linear layer, that is, D = Linear(Encoder(s)); If there is an error in the sentence text, the text most relevant to the erroneous part of the sentence text is searched in a pre-established search library, wherein, if there is an error in the sentence text, the phrase at the error detection result is intercepted and input into the search library for search, and the phrase closest to the erroneous part is obtained as the search result for recall, wherein, when establishing the search library, the following steps are included: obtaining unannotated correct corpus, and segmenting the sentences in the corpus according to a fixed length to obtain correct phrases; inputting the correct phrases into a pre-trained neural network model to encode the correct phrases to obtain the encoding of the phrases; and storing the key-value pair of the encoding of the phrases and the correct phrases in the search library; Processing the sentence text and the most relevant text using a preset grammatical error correction model to obtain a correct text corresponding to the sentence text, including splicing the sentence text and the most relevant text and inputting them into an encoder to obtain a corresponding hidden state; splicing the encoding corresponding to the sentence text and the hidden state and inputting them into a decoder for prediction to obtain a predicted correct sentence; The correct phrase is input into a pre-trained neural network model to encode the correct phrase, and the encoding of the phrase is obtained by: H = AvgPool (Encoder (s) [start: end]); wherein Encoder is a neural network model, s is a sentence containing the phrase, and start and end are the starting and ending positions of the phrase in the sentence; Pre-training the neural network model includes: processing incorrect sentences and correct sentences based on an edit distance algorithm to obtain the content of the correct part corresponding to the incorrect part in the incorrect sample; using the incorrect part as a query text and the content of the correct part as a positive sample, and using the BM25 algorithm to retrieve phrases close to the query text from a large number of correct phrases as negative samples; and training the neural network model based on the query text, the positive sample, and the negative sample; The training of the neural network model based on the query text positive sample and the negative sample includes: training the neural network model using contrastive learning loss: Among them, p1, p2 are query and positive example, n i is a negative example, sim is the function for calculating similarity, and τ is the temperature parameter.

2. A grammatical error correction device, characterized in that: include: The error detection unit is configured to, after receiving a sentence text, detect the sentence text based on a trained error detection model to determine whether the sentence text contains errors, wherein training the error detection model includes: obtaining an original sentence and marking the parts of the original sentence that contain grammatical errors; obtaining a vector representation of each position in the sentence based on an encoder in the error detection model, and then predicting its correctness through a linear layer, that is, D = Linear(Encoder(s)); a retrieval unit configured to, if the sentence text contains an error, search a pre-established retrieval library for text most relevant to the erroneous portion of the sentence text, wherein, if the sentence text contains an error, extract a phrase from the error detection result, input it into the retrieval library for search, and obtain a phrase closest to the erroneous portion as a retrieval result for recall; The prediction unit is configured to process the sentence text and the most relevant text using a preset grammatical error correction model to obtain a correct text corresponding to the sentence text, including splicing the sentence text and the most relevant text into an encoder to obtain a corresponding hidden state; splicing the code corresponding to the sentence text and the hidden state into a decoder for prediction to obtain a predicted correct sentence; The method for establishing the search library includes: obtaining unlabeled correct corpus and segmenting sentences in the corpus according to fixed lengths to obtain correct phrases; inputting the correct phrases into a pre-trained neural network model to encode the correct phrases to obtain the encoding of the phrases; and storing the encoding of the phrases and the key-value pairs of the correct phrases in the search library; Inputting the correct phrase into a pre-trained neural network model to encode the correct phrase, obtaining an encoding of the phrase includes: H = AvgPool(Encoder(s)[start:end]); wherein Encoder is a neural network model, s is a sentence containing the phrase, and start and end are the starting and ending positions of the phrase in the sentence; Pre-training the neural network model includes: processing incorrect sentences and correct sentences based on an edit distance algorithm to obtain the content of the correct part corresponding to the incorrect part in the incorrect sample; using the incorrect part as a query text and the content of the correct part as a positive sample, and using the BM25 algorithm to retrieve phrases close to the query text from a large number of correct phrases as negative samples; and training the neural network model based on the query text, the positive sample, and the negative sample; The training of the neural network model based on the query text positive sample and the negative sample includes: training the neural network model using contrastive learning loss: Among them, p1, p2 are query and positive example, n i is a negative example, sim is the function for calculating similarity, and τ is the temperature parameter.

3. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to claim 1 is implemented.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to claim 1 is implemented.

Images