Grammar error correction method and related device

Abstract

The invention discloses a grammar error correction method and a related device in the field of artificial intelligence. The method comprises the following steps: performing up-sampling processing on an obtained to-be-corrected text to obtain a to-be-corrected text sequence corresponding to the to-be-corrected text; through the grammar error correction model, hidden layer features corresponding to each character in the to-be-corrected text sequence are sequentially determined according to the character sequence of the to-be-corrected text sequence, a target output text sequence is determined according to the hidden layer features corresponding to each character in the to-be-corrected text sequence, and further through the grammar error correction model, the target output text sequence is obtained. And determining a correct text corresponding to the to-be-corrected text according to the target output text sequence. Through the grammar error correction model, the target output text sequence is determined according to the hidden layer feature corresponding to each character in the to-be-corrected text sequence, the correct text can be determined only by calling the grammar error correction model once, and the grammar error correction efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of natural language processing, and in particular, to a grammar correction method and related devices. Background Art

[0002] Currently, the text generated during writing or speech-to-text conversion may have grammar errors, resulting in low text accuracy.

[0003] In related technologies, in order to correct grammar errors in text, a grammar correction model based on sequence labeling or a grammar correction model based on sequence-to-sequence is generally used to correct the text with grammar errors to obtain the correct text.

[0004] However, the grammar correction model based on sequence labeling can only correct one grammar error in the text at a time. If there are multiple grammar errors in the text, multiple iterations are required to obtain the correct text; for a sentence with a length of m, the grammar correction model based on sequence-to-sequence needs to call the model m times to obtain the correct text. That is, both grammar correction models in related technologies need to be processed multiple times to obtain the correct text, reducing the efficiency of grammar correction. Summary of the Invention

[0005] Embodiments of this application provide a grammar correction method and related devices, aiming to solve the problem that the text with grammar errors needs to be processed multiple times, reducing the grammar correction efficiency.

[0006] The first aspect of this application provides a grammar correction method, including:

[0007] Obtain the text to be corrected;

[0008] Perform upsampling processing on the text to be corrected to obtain a sequence of texts to be corrected corresponding to the text to be corrected;

[0009] Through the grammar correction model, in the order of the characters in the sequence of texts to be corrected, sequentially determine the hidden layer features corresponding to each character in the sequence of texts to be corrected, and determine the target output text sequence according to the hidden layer features corresponding to each character in the sequence of texts to be corrected;

[0010] Through the grammar correction model, determine the correct text corresponding to the text to be corrected according to the target output text sequence.

[0011] The second aspect of this application provides a grammar correction device, including:

[0012] An acquisition module, configured to obtain the text to be corrected;

[0013] A processing module, configured to perform upsampling processing on the text to be corrected to obtain a sequence of texts to be corrected corresponding to the text to be corrected;

[0014] A syntax error correction module, which is used to determine the hidden layer features corresponding to each character in the text sequence to be corrected in sequence according to the character order of the text sequence to be corrected through a syntax error correction model, and determine the target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be corrected;

[0015] A target text output module, which is used to determine the correct text corresponding to the text to be corrected according to the target output text sequence through a syntax error correction model.

[0016] A third aspect of the present application provides a syntax error correction device, which includes a processor and a memory:

[0017] The memory is used to store program codes and transmit the program codes to the processor;

[0018] The processor is used to execute the steps of the syntax error correction method provided in the first aspect according to the instructions in the program code.

[0019] A fourth aspect of the present application provides a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the steps of the syntax error correction method provided in the first aspect.

[0020] A fifth aspect of the present application provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed, it realizes the steps of the syntax error correction method provided in the first aspect.

[0021] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0022] In the technical solution of the present application, the obtained text to be corrected is upsampled to obtain a text sequence to be corrected corresponding to the text to be corrected; through a syntax error correction model, the hidden layer features corresponding to each character in the text sequence to be corrected are determined in sequence according to the character order of the text sequence to be corrected, and the target output text sequence is determined according to the hidden layer features corresponding to each character in the text sequence to be corrected. Further, through a syntax error correction model, the correct text corresponding to the text to be corrected is determined according to the target output text sequence. Among them, by determining the target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be corrected through a syntax error correction model, it is possible to determine the correct text by only calling the syntax error correction model once, improving the syntax error correction efficiency. Description of the Drawings

[0023] Figure 1 A schematic diagram of a syntax error correction model based on sequence annotation provided for the related art;

[0024] Figure 2Schematic diagram of an application scenario of a grammar error correction method provided by an embodiment of the present application;

[0025] Figure 3 Schematic flowchart of a grammar error correction method provided by an embodiment of the present application;

[0026] Figure 4 Schematic flowchart of a training method of a grammar error correction model provided by an embodiment of the present application;

[0027] Figure 5 Schematic diagram of a mask matrix provided by an embodiment of the present application;

[0028] Figure 6 Path diagram representing the correspondence between an input text sequence and an output text sequence provided by an embodiment of the present application;

[0029] Figure 7 Schematic structural diagram of a grammar error correction device provided by an embodiment of the present application;

[0030] Figure 8 Schematic structural diagram of a server in an embodiment of the present application;

[0031] Figure 9 Schematic structural diagram of a terminal device in an embodiment of the present application. Detailed implementation manners

[0032] Currently, the text generated during writing or speech - to - text conversion may have grammar errors, resulting in low text accuracy. Taking writing as an example, during the writing process, the correct text "A little boy goes to school" may be wrongly written as "An little boy go school".

[0033] In the related art, in order to correct the grammar errors in the text, a grammar error correction model is generally used to correct the text with grammar errors. As an example, the input text with grammar errors is X = {x1, x2, …, x n}, where x i represents the i - th word in X; the grammar error correction model is used to modify all the grammar errors in the input text X and generate a grammatically correct output text Y = {y1, y2, …, y m}. Since a word can be modified into zero or more words, the length n of the input text and the length m of the output text may be different.

[0034] Currently, grammar error correction models generally use grammar error correction models based on sequence annotation or sequence - to - sequence grammar error correction models to correct the text with grammar errors to obtain the correct text.

[0035] See Figure 1 , which is a schematic diagram of a grammar error correction model based on sequence annotation provided by the related art.

[0036] Combined with Figure 1 shown, the grammar error correction model based on sequence annotation requires that the input text X = {x1, x2, …, x n} and the output text Y = {t1, t2, …, t n} have the same length n. To solve the problem that the input text and the output text are different, generally, by comparing the differences between the input text sequence and the output text sequence, the output labels are converted into a mapping relationship sequence T = {t1, t2, …, t n} from X to Y; where, t i represents the transformation operation performed on the word x i , and each operation change t i belongs to the operation set Operation set includes operations such as "keep", "delete", "replace with word y t '", "insert word y t '", "case conversion", "singular / plural conversion", "part-of-speech conversion", etc.

[0037] However, since the grammar error correction model based on sequence annotation requires the input text and the output text to have the same length, it causes the grammar error correction model based on sequence annotation to not be able to correct all grammar errors in the text at once. As an example, assume the input text is "An old boy go school", then the grammar error correction model based on sequence annotation needs to replace "go" with "goes" in the first round of iteration, and then perform the operation of inserting "to" on "goes" in the second round of iteration. In this way, two rounds of iteration are required to correct it to "An old boy goes to school". Secondly, since the output text does not appear in the pre-training stage of the grammar error correction model based on sequence annotation, it is necessary to readjust the output part parameters in the grammar error correction model based on sequence annotation. In addition, the number of operation elements included in the operation set cannot be too large, otherwise it may cause the grammar error correction model based on sequence annotation to not be able to generate non-high-frequency words.

[0038] The grammar error correction model based on sequence-to-sequence directly inputs the input text X = {x1, x2, …, x n} with grammar errors into the model, and uses autoregressive decoding to generate the correct text Y = {y1, y2, …, y m}, however, since the sequence-to-sequence grammar error correction model uses autoregressive decoding for generation, for a sentence of length m, the model needs to be called m times to generate grammatically correct text, resulting in an overly long model call latency when used online.

[0039] That is, the grammar error correction model based on sequence labeling can only correct one grammar error in the text at a time. If there are multiple grammar errors in the text, multiple iterations are required to obtain the correct text; for a sentence of length m, the sequence-to-sequence grammar error correction model needs to call the model m times to obtain the correct text. That is, both grammar error correction models in the related art need to be processed multiple times to obtain the correct text, reducing the efficiency of grammar error correction.

[0040] In view of the above problems, a grammar error correction method and related device are provided in this application. The method includes: performing upsampling processing on the obtained text to be corrected to obtain a text sequence to be corrected corresponding to the text to be corrected; through a grammar error correction model, sequentially determining the hidden layer features corresponding to each character in the text sequence to be corrected according to the character order of the text sequence to be corrected, and determining a target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be corrected. Further, through the grammar error correction model, determining the correct text corresponding to the text to be corrected according to the target output text sequence.

[0041] In this way, by determining the target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be corrected through the grammar error correction model, it is possible to determine the correct text by only calling the grammar error correction model once, improving the efficiency of grammar error correction. In addition, in the embodiments of this application, by performing upsampling processing on the text to be corrected to obtain a text sequence to be corrected, more detailed information in the text to be corrected can be obtained, reducing the learning difficulty of the grammar error correction model, so that the grammar error correction model can better output the correct text.

[0042] See Figure 2 , which is a schematic diagram of an application scenario of a grammar error correction method provided by an embodiment of this application. The application scenario includes a grammar error correction device 201 or a server 202.

[0043] The grammar error correction device 201 or the server 202 obtains the text to be corrected. As an example, the text to be corrected can be Chinese, English, a mixture of Chinese and English, etc., and is not limited here.

[0044] The grammar error correction device 201 or the server 202 performs upsampling processing on the text to be corrected to obtain a text sequence to be corrected corresponding to the text to be corrected. As an example, assuming the text to be corrected is "abc", performing upsampling processing on the text to be corrected can obtain the text sequence [aabbcc].

[0045] The grammar error correction device 201 or the server 202 determines the hidden layer features corresponding to each character in the text sequence to be corrected in sequence according to the character order of the text sequence to be corrected through the grammar error correction model, and determines the target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be corrected. As an example, assume that the text sequence to be corrected is [aabbcc]. Through the grammar error correction model, the hidden layer features [h1, h2, h3, h4, h5, h6] corresponding to each character in the text sequence to be corrected [aabbcc] can be determined in sequence according to the character order of the text sequence to be corrected [aabbcc], and the target output sequence can be determined according to the hidden layer features [h1, h2, h3, h4, h5, h6].

[0046] The grammar error correction device 201 or the server 202 determines the correct text corresponding to the text to be corrected according to the target output text sequence through the grammar error correction model. As an example, assume that the target output text sequence is [a <blank>b], the grammar error correction model can be used to generate the target output text sequence [a <blank>b] Determine that the correct text is "ab".

[0047] The grammar error correction method provided by the embodiments of this application obtains a sequence of texts to be corrected corresponding to the text to be corrected through upsampling the obtained text to be corrected; through a grammar error correction model, sequentially determines the hidden layer features corresponding to each character in the sequence of texts to be corrected according to the character order of the sequence of texts to be corrected, and determines a target output text sequence according to the hidden layer features corresponding to each character in the sequence of texts to be corrected. Further, through the grammar error correction model, determines the correct text corresponding to the text to be corrected according to the target output text sequence.

[0048] In this way, by determining the target output text sequence according to the hidden layer features corresponding to each character in the sequence of texts to be corrected through the grammar error correction model, it is possible to determine the correct text by only calling the grammar error correction model once, improving the grammar error correction efficiency. In addition, in the embodiments of this application, by performing upsampling processing on the text to be corrected to obtain a sequence of texts to be corrected, more detailed information in the text to be corrected can be obtained, reducing the learning difficulty of the grammar error correction model, so that the grammar error correction model can better output the correct text.

[0049] The grammar error correction method provided by the embodiments of this application can be applied to a terminal device or a server with data processing capabilities. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device includes, but is not limited to, mobile phones, tablets, computers, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here.

[0050] The grammar error correction method provided by the embodiments of this application involves artificial intelligence, computer vision technology, and natural language processing.

[0051] Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making.

[0052] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, the pre-trained model, also known as the large model or the foundation model, can be widely applied to downstream tasks in various directions of artificial intelligence after fine-tuning. Artificial intelligence software technology mainly includes several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0053] Computer Vision (CV) is a science that studies how to enable machines to "see". More specifically, it refers to using cameras and computers to replace human eyes for tasks such as object recognition and measurement in machine vision, and further performing graphics processing to make the images processed by the computer more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to build artificial intelligence systems that can obtain information from images or multi-dimensional data. The large model technology has brought important changes to the development of computer vision technology. Pre-trained models in the visual field such as swin-transformer, ViT, V-MOE, and MAE can be quickly and widely applied to downstream specific tasks after fine-tuning. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc., and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0054] Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural languages. Natural language processing involves natural languages, that is, the languages used by people in daily life, and is closely related to linguistics research; at the same time, it involves computer science and mathematics. The pre-trained model, an important technology for model training in the field of artificial intelligence, evolved from the large language model in the NLP field. After fine-tuning, the large language model can be widely applied to downstream tasks. Natural language processing technology usually includes technologies such as text processing, semantic understanding, machine translation, robot question answering, and knowledge graphs.

[0055] In this application, when collecting and processing relevant data in practical applications, the informed consent or separate consent of the subject of personal information should be obtained in strict accordance with the requirements of relevant national laws and regulations, and subsequent data use and processing behaviors should be carried out within the scope authorized by laws and regulations and the personal information subject.

[0056] See Figure 3 , which is a schematic flowchart of a grammar error correction method provided by an embodiment of this application.

[0057] Combined with Figure 3 As shown, the grammar error correction method provided by an embodiment of this application may include:

[0058] S301: Obtain the text to be corrected.

[0059] The text to be corrected refers to the text containing grammar errors. The text to be corrected includes but is not limited to the text converted from speech, the text after OCR image recognition, Chinese text, English text, Chinese-English mixed text, Chinese-numeric mixed text, English-numeric mixed text, text containing characters, etc., and no specific limitation is made here.

[0060] Grammar errors include but are not limited to orthographic errors, morphological errors, syntactic errors, grammatical errors, etc.

[0061] S302: Perform upsampling processing on the text to be corrected to obtain a sequence of the text to be corrected corresponding to the text to be corrected.

[0062] Upsampling processing is used to amplify the detailed information of the text to be corrected, including but not limited to processing the text to be corrected using interpolation method, deconvolution method, anti-pooling method, etc. to obtain a sequence of the text to be corrected, so as to reduce the learning difficulty during subsequent model training and inference. As an example, if the text to be corrected is "abcde", then performing upsampling processing on this text to be corrected can obtain a sequence of the text to be corrected [aabbccddee].

[0063] S303: Through the grammar error correction model, in the character order of the sequence of the text to be corrected, sequentially determine the hidden layer features corresponding to each character in the sequence of the text to be corrected, and determine the target output text sequence according to the hidden layer features corresponding to each character in the sequence of the text to be corrected.

[0064] The grammar error correction model refers to a model used to correct grammar errors in the text to be corrected, and this grammar error correction model is a model obtained by further training based on a pre-trained initial model. The grammar error correction model provided by an embodiment of this application can, through training, achieve determining the correct text corresponding to the text to be corrected by only calling the model once, without calling or iterating the model once for each grammar error correction text, realizing end-to-end grammar error correction.

[0065] Hidden features refer to abstracting characters into another dimensional space to show more abstract features of characters, which can better classify different types of characters. As an example, assuming that the text sequence to be corrected is "我我今今日天体体息息", where the character "体" is a grammatical error in the text sequence to be corrected, then when determining the hidden features corresponding to each character in the text sequence to be corrected, the hidden features corresponding to the character "体" will be different from the hidden features of other characters, so that the model pays more attention to this hidden feature.

[0066] Among them, in some possible implementations, the text sequence to be corrected can be encoded by the feature encoding structure in the grammatical error correction model to obtain the hidden features corresponding to each character in the text sequence to be corrected.

[0067] It should be understood that in the embodiments of the present application, since the hidden features corresponding to each character in the text sequence to be corrected can represent the deeper features corresponding to each character, characters with grammatical errors can be distinguished from correct characters. That is, the hidden features corresponding to each character in the text sequence to be corrected can be determined through the grammatical correction model, and the hidden features corresponding to each character in the text sequence to be corrected can be used to determine the output sequence corresponding to the correct text, that is, the target output text sequence.

[0068] S304: Determine the correct text corresponding to the text to be corrected according to the target output text sequence through the grammatical error correction model.

[0069] The target output text sequence refers to the text sequence corresponding to the correct text. Therefore, it can be processed by a grammatical error correction model to determine the correct text corresponding to the text to be corrected based on the target output text sequence.

[0070] In the technical solution of the present application, the acquired text to be corrected is upsampled to obtain a text sequence to be corrected corresponding to the text to be corrected; the hidden features corresponding to each character in the text sequence to be corrected are determined in sequence according to the character order of the text sequence to be corrected through a grammatical correction model, and the target output text sequence is determined according to the hidden features corresponding to each character in the text sequence to be corrected, and the correct text corresponding to the text to be corrected is further determined according to the target output text sequence through the grammatical correction model. Among them, by using the grammatical correction model to determine the target output text sequence according to the hidden features corresponding to each character in the text sequence to be corrected, it is possible to determine the correct text by calling the grammatical correction model only once, thereby improving the efficiency of grammatical correction.

[0071] Based on the grammar correction method provided in the above embodiment, the grammar correction model provided in the embodiment of the present application can be fine-tuned based on the pre-trained large model to obtain a pre-trained grammar correction model. To illustrate the training process of the grammar correction model, see Figure 4 The present application also provides a method for training a grammatical error correction model, which can be:

[0072] S401: Obtain input training text and output training text.

[0073] The input training text refers to the text used to train the grammatical error correction model, and the input training text may include at least two grammatical errors. The output training text is the correct text corresponding to the input training text.

[0074] It should be noted that in order to improve the training effect of the grammatical error correction model for multiple grammatical errors, therefore, in the embodiment of the present application, the input training text includes at least two grammatical errors. In some possible implementations, the input training text may also include one grammatical error, which is not specifically limited here. As an example, the input training text may be "Although I rest today, I did not go to school", where the grammatical errors in the input training text are glyph errors and syntactic errors, that is, "体" should be "休", and "虽" should be "因".

[0075] S402: Perform upsampling processing on the input training text to determine an input text sequence corresponding to the input training text.

[0076] The upsampling process of step S402 is the same as the upsampling process of step S302 in the above embodiment. The explanation of the relevant upsampling process can be found above and will not be repeated here.

[0077] S403: Using the initial grammatical error correction model, according to the character order of the input text sequence, determine the training hidden layer features corresponding to each character in the input text sequence in turn, and obtain a training hidden layer feature set corresponding to the input text sequence.

[0078] The initial grammatical error correction model refers to the initial model to be trained, and the initial grammatical error correction model may be, but is not limited to, a pre-trained language model such as a BERT model, a RoBERTa model, or a DeBERTa model.

[0079] BERT (Bidirectional Encoder Representation from Transformers) is a pre-trained language representation model. It emphasizes that it no longer uses the traditional unidirectional language model or the shallow concatenation of two unidirectional language models for pre-training as in the past, but uses a new masked language model (MLM) to generate deep bidirectional language representation.

[0080] RoBERTa (Robustly Optimized BERT Pretraining Approach) is a pre-trained language model based on BERT (Bidirectional Encoder Representations from Transformers). RoBERTa has made a series of optimizations on the basis of BERT, thus further improving the performance of the model.

[0081] DeBERTa (Decoding-enhanced-BERT-with-disentangled attention) is a neural language model based on Transformer, which is pre-trained on a large corpus of raw text using self-supervised learning. Like other pre-trained language models, DeBERTa aims to learn general language representations and can be adapted to various downstream NLU tasks. Among them, Natural Language Understanding (NLU) is the general term for all method models or tasks that support machines to understand the content of text.

[0082] Training hidden layer features refer to the features corresponding to each character in the input text sequence. Training hidden layer features can represent deeper features of the corresponding characters and improve the learning speed of the initial grammar error correction model.

[0083] The set of training hidden layer features refers to the set that includes the training hidden layer features corresponding to each character in the input text sequence respectively. Among them, the order of multiple training hidden layer features in the set of training hidden layer features is arranged according to the order of each character in the input text sequence. As an example, assuming the input text sequence is [a, a, b, b, c, c], then the corresponding set of training hidden layer features is [h1, h2, h3, h4, h5, h6], where h1 corresponds to the first character a, h2 corresponds to the second character a, h3 corresponds to the third character b, h4 corresponds to the fourth character b, h5 corresponds to the fifth character c, and h6 corresponds to the sixth character c.

[0084] In some possible implementation manners, it can be encoded through the feature encoding structure of the initial grammar error correction model to obtain the set of training hidden layer features corresponding to the input text sequence.

[0085] S404: Determine the predicted output text sequence through the initial grammar error correction model according to the set of training hidden layer features and the training output text sequence.

[0086] Among them, the training output text sequence is obtained based on the output training text. The predicted output text sequence means the output text sequence obtained by processing through the initial grammar correction model. In some possible implementation manners, the training output text sequence can be obtained in the following way:

[0087] Insert a blank character before the first character in the training output text, and insert a blank character after the last character in the training output text to obtain a first output sequence; insert blank characters between two adjacent characters in the first output sequence to obtain the training output text sequence corresponding to the training output text.

[0088] It should be understood that during the process of model training, the model cannot directly distinguish consecutive identical characters. For example, if the output text sequence is [h, h, e, l, l, l, o], the finally obtained output text is 'helo' instead of 'hello', which may reduce the accuracy of grammar correction.

[0089] To solve the above technical problems, the embodiments of the present application introduce blank characters. These blank characters have no meaning and will be simply removed when converting from the output text sequence to the output text. Therefore, when training the initial grammar correction model in the embodiments of the present application, blank characters can be set before and after each character of the training output text, and only one blank character is set between two adjacent characters. As an example, assuming the training output text is "abcd", then the training output text sequence can be <blank> a <blank> b <blank> c <blank> d <blank>, where <blank>Represents a blank character. In the embodiments of the present application, by setting the blank character, it is possible to avoid the inability to distinguish between two consecutive identical characters, ensuring the accuracy of grammar error correction.

[0090] In some possible implementation manners, the initial grammar error correction model may determine the predicted output text sequence according to the relationship between each hidden layer feature in the training hidden layer feature set and each character in the training output text sequence. Specifically, the process of determining the predicted output text sequence may include:

[0091] A1: Determine the output probability of each training hidden layer feature in the training hidden layer feature set for each character in the training output text sequence.

[0092] The output probability means the probability that the output result corresponding to the training hidden layer feature is a certain character in the training output text sequence. As an example, assume that the training hidden layer feature set is [h1, h2, h3], and the training output text sequence is <blank> a <blank> b <blank>, then, it can be determined that the output of the trained hidden layer feature h1 is <blank>The probability is 0.9, the probability that h1 outputs a is 0.5, and the probability that h1 outputs <blank>The probability is 0.9, the probability that h1 outputs b is 0.1, and the output of h1 is <blank>The probability is 0.2.

[0093] A2: From the multiple output probabilities corresponding to each training hidden layer feature, determine the maximum output probability corresponding to each training hidden layer feature.

[0094] Among them, by determining the maximum output probability among the multiple output probabilities corresponding to each training hidden layer feature, the output result corresponding to the training hidden layer feature can be determined. That is, it can be considered that the character corresponding to the maximum output probability is the optimal solution of the output result of the corresponding training hidden layer feature.

[0095] A3: Determine the predicted output text sequence according to the maximum output probabilities corresponding to each obtained training hidden layer feature.

[0096] Among them, by determining the maximum output probabilities corresponding to each training hidden layer feature, that is, determining the optimal output results of each training hidden layer feature, and obtaining the predicted output text sequence according to each output result.

[0097] It should be understood that in the embodiments of the present application, by determining the maximum output probabilities of each training hidden layer feature, and using the characters corresponding to each maximum output probability as the optimal output results of the training hidden layer features, and further obtaining the predicted output text sequence according to multiple output results, so as to filter the grammar errors in the input text sequence, map each training hidden layer feature to the optimal solution, so as to obtain the correct predicted output text sequence. That is, through the training method of the embodiments of the present application, the initial grammar error correction model can be effectively trained end to end.

[0098] Further, in some possible implementation manners, step A3 may include:

[0099] B1: In the order from the last training hidden layer feature to the first training hidden layer feature in the training hidden layer feature set, sequentially determine the characters corresponding to the maximum output probabilities of each training hidden layer feature as the target characters.

[0100] It should be understood that in the embodiments of the present application, since the maximum output probability corresponding to each training hidden layer feature is determined in the order of the training hidden layer feature set (from the first training hidden layer feature to the last hidden layer feature), if the output characters are still determined in the order of the training hidden layer feature set when determining each output character, then it is necessary to re-determine the maximum output probability corresponding to each training hidden layer feature and determine the character corresponding to the maximum output probability.

[0101] In order to enable the initial grammar error correction model to save training time, after determining the maximum output probability corresponding to each training hidden layer feature, the character corresponding to the maximum output probability of each hidden layer feature can be determined in reverse order (that is, in the order from the last training hidden layer feature to the first training hidden layer feature) as the target character.

[0102] B2: Determine the predicted output text sequence according to the target characters corresponding to each training hidden layer feature.

[0103] In the embodiment of the present application, the predicted output text sequence is determined by a recursive method, that is, the maximum output probability corresponding to each training hidden layer feature is saved as the prediction result, and the saved prediction result is directly called in reverse order to determine the target character corresponding to each training hidden layer feature, without repeatedly determining the maximum output probability corresponding to each training hidden layer feature, which can save the training time of the model.

[0104] It should be understood that since the two adjacent characters determined may be repeated, the repeated characters need to be merged. In some possible implementation manners, step B2 may include:

[0105] C1: Arrange multiple target characters in the order of each training hidden layer feature in the training hidden layer feature set to obtain a target character sequence.

[0106] It should be understood that after determining the target characters corresponding to each training hidden layer feature in reverse order, since it is in reverse order, the order of each target character is also incorrect. Therefore, multiple target characters need to be arranged in the order of each training hidden layer feature in the training hidden layer feature set to obtain a target character sequence.

[0107] C2: If two adjacent target characters in the target character sequence are the same, merge the two target characters to obtain a merged character.

[0108] It should be understood that since the output text sequence is upsampled, it may cause two training hidden layer features to output the same output result, that is, two adjacent target characters in the target character sequence are the same. Therefore, the same characters need to be merged to avoid affecting subsequent training. As an example, assume that the target character sequence is <blank> <blank> b <blank>c cd <blank>, then it is necessary to put " <blank> <blank>"Merge into" <blank>", combine "c c" into "c".

[0109] C3: If two adjacent target characters in the target character sequence are different, then keep both target characters.

[0110] It should be understood that if two adjacent target characters are different, it is considered that the output results of two adjacent training hidden layer features are different. Therefore, both target characters need to be kept. Assume the target character sequence is <blank> <blank> b <blank>c cd <blank>, then, "b <blank>” are two different target characters and need to be retained.

[0111] C4: Obtain the predicted output text sequence based on the merged character and multiple target characters.

[0112] As an example, assume the target character sequence is <blank> <blank> b <blank>c cd <blank>, then the corresponding predicted output text sequence can be <blank> b <blank>c d <blank>.

[0113] S405: Determine the loss function value according to the predicted output text sequence and the input text sequence.

[0114] Among them, in the embodiments of the present application, the loss function for training the grammar correction model can be the negative log-likelihood function. That is, for a given sample (X, Y), the loss function is -log p(Y|X), where X represents the input of the grammar correction model and Y represents the output of the grammar correction model. Among them, by using the sentence-level loss function to train the grammar correction model, different from the original word-level loss function, the loss function provided by the embodiments of the present application helps the initial grammar correction model to more effectively learn the correspondence between the input X and the output Y, especially considering <blank>In the case of a separator, it should be understood that in the embodiments of the present application, the loss function value can be determined according to the input text sequence, the predicted output text sequence, and the loss function.

[0115] S406: Train the initial grammar error correction model using the loss function value.

[0116] It should be understood that in the embodiments of the present application, the initial grammar error correction model can be trained and fine-tuned according to the determined loss function value to obtain a trained grammar error correction model.

[0117] Among them, in the embodiments of the present application, the above operations can be iteratively executed based on different training samples until the trained grammar error correction model meets the training end condition. For example, until the number of training times for the grammar error correction model reaches a preset number of times, or the model performance of the grammar error correction model reaches a preset performance requirement.

[0118] In the embodiments of the present application, training the grammar error correction model through knowledge distillation can make the trained grammar error correction model achieve better performance and accuracy. That is, by training a pre-trained language model (referring to the initial grammar error correction model in the present application), the supervision information of a pre-trained language model with better performance can be used to train the grammar error correction model, so that the trained grammar error correction model can achieve better performance and accuracy.

[0119] Among them, knowledge distillation is a commonly used method for model compression. Different from pruning and quantization in model compression, knowledge distillation is to train a lightweight small model by using the supervision information of a large model with better performance, in order to achieve better performance and accuracy. As an example, the knowledge distillation involved in the embodiments of the present application includes but is not limited to offline distillation, semi-supervised distillation, and self-supervised distillation, which are not specifically limited here.

[0120] Among them, in some possible implementation manners, the initial grammar error correction model provided in the embodiments of the present application includes a masked attention mechanism layer; the processing object of the masked attention mechanism layer is a mask matrix;

[0121] The mask matrix is composed of a first sub-matrix and a second sub-matrix; the first sub-matrix is constructed based on the training hidden layer feature set, and the first sub-matrix is not masked; the second sub-matrix is constructed based on the text features of each of the n characters in the predicted output text sequence. When determining the i-th character in the predicted output text sequence, the text features of the first i - 1 characters in the predicted output text sequence are not masked, and the text features of the i-th character to the n-th character in the predicted output text sequence are masked, where n is an integer greater than 1.

[0122] As an example, taking the first i - 1 characters as historical tokens, it can be understood that they have been decoded through the feature decoding structure of the initial grammar error correction model to obtain the decoding result (i.e., the characters in the predicted output text sequence). When decoding the text features of the i-th character, the text features of the i-th character can be used as the current token. The current token can see the historical tokens but cannot see the token information after the current token to ensure the correlation between the current token and the historical tokens, so as to realize the result prediction of the current token.

[0123] Among them, referring to Figure 5 , this figure is a schematic diagram of a mask matrix provided by an embodiment of the present application. The text features of the first i - 1 characters in the predicted output text sequence are used as sequence Q, and the text features of the i-th text unit to the n-th text unit in the predicted output text sequence are used as sequence A. The tokens in sequence Q are visible to each other, that is, the current token can see all other tokens in Q. The internal of sequence A uses a causal mask (the current token can only see historical tokens and cannot see future tokens), and all tokens in sequence A can see all tokens in sequence Q. As Figure 5 shown, that is, all tokens in Q are unmasked (not masked), and all tokens in sequence Q cannot see any tokens in sequence A, that is, all tokens in A are masked (masked).

[0124] Based on the grammar error correction method provided in the above embodiment, the embodiment of the present application also provides another training method for a grammar error correction model. The method may include: training an initial grammar error correction model using the CTC (Connectionist Temporal Classification) loss function.

[0125] Among them, the initial grammar error correction model can capture the input text X = {x1, x2,..., x n} and the output text Y = {y1, y2,..., y m} The corresponding relationship between them. In the model training of the embodiments of this application, the pre-trained RoBERTa can be used as the initial grammar correction model. In the RoBERTa model, each input text sequence is dynamically re-masked at different training steps, enabling the model to learn richer representations. The RoBERTa model contains 24 layers of transformer layers, with a hidden layer size of 1024 and a total number of parameters of approximately 335 million. That is, the pre-trained RoBERTa model has already learned rich language representation capabilities, so it can be fine-tuned on the pre-trained RoBERTa model to converge to the grammar correction model faster.

[0126] The CTC loss function is a loss function used for sequence-to-sequence tasks and is applicable to cases where there is a many-to-one relationship between the input and the output. To model this many-to-one relationship, " <blank> ”， <blank>For placeholder and separation, <blank>It has no practical meaning in itself.

[0127] Given an input training text X, the initial CTC-based grammar error correction model will consider all possible output training texts Y and calculate the probability distribution for each Y. In the inference stage, the most likely output training text can be found based on this probability distribution.

[0128] Since <blank>Due to the existence of multiple different paths, the same output text may be decoded. For example, "goes <blank>to”, "goes goes <blank>Both "to” and "goes to” can be decoded as "goes to”, and their probability scores can be combined for calculation.

[0129] To illustrate the process by which the initial grammar correction model determines the output training text, taking the input training text "a b c” and the corresponding output training text "a b” as an example, this determination process may include:

[0130] Step 1: Upsample the input training text to obtain the input text sequence [a a b b c c]; and insert at the beginning, end, and between adjacent characters of the output training text <blank>, construct the output text sequence Z = { <blank> a <blank> b <blank>}}。

[0131] Step 2: The input text sequence passes through the encoder of the initial grammar error correction model to obtain the set of training hidden layer features H = {h1, h2, h3, h4, h5, h6} corresponding to each character in the input text sequence [a a bb cc].

[0132] Among them, in the embodiments of the present application, a path graph can be constructed to represent the correspondence between the input text sequence and the output text sequence. For details, please refer to Figure 6 。 Figure 6 The row elements in represent each character in the output text sequence, Figure 6 The column elements in represent each training hidden layer feature in the training hidden layer features. The intersection of the row element and the column element represents the conversion from the training hidden layer feature to the character in the output text sequence.

[0133] Combined with Figure 6 As shown, in the embodiments of the present application, it is necessary to determine an optimal path from the starting point to the ending point of the path graph. This optimal path corresponds to the most likely output text sequence, that is, the predicted output text sequence. However, if all paths are calculated and the probability scores of each path are calculated, and the path corresponding to the maximum probability score is used as the optimal path, it will make the computational complexity of the entire determination process relatively large, and there may be problems of repeated calculation. Therefore, in this embodiment, a dynamic programming algorithm (such as the Viterbi algorithm) can be used to find the optimal path and calculate the probability score corresponding to the optimal path.

[0134] Among them, the Viterbi algorithm is a dynamic programming algorithm used to find the Viterbi path (that is, the optimal path) that is most likely to generate the observed event sequence. As an example, the intersection of the row element and the column element in Figure 6 can be used as a node and organized by column. The number of nodes in each column can be different, but the nodes in each column can only be connected to the nodes in the adjacent column and cannot be connected across columns. By retaining the most likely state at each time step, the complexity of traversing all possible paths is avoided. The Viterbi algorithm can be divided into two steps: 1) Forward calculation, calculating the local optimal solution at each time step; 2) Backtracking, starting from the last time step, constructing the global optimal path according to the local optimal solution. The advantage of the Viterbi algorithm is that it can significantly reduce the computational complexity and accelerate the decoding process, so as to efficiently find the optimal solution in the sequence-to-sequence task. Through the Viterbi algorithm, the initial grammar error correction model can be effectively trained to capture the correspondence between the input text sequence and the output text sequence, and generate the predicted output text sequence corresponding to the optimal path in the inference stage.

[0135] As an example, combined with Figure 6 As shown, assume that Path 1 and Path 2 are two optimal paths. If the outputs at each time step are independent of each other, that is, the output results corresponding to each training hidden layer feature are independent of each other, then the probability of the optimal path can be modeled as the product of the probabilities at each time step.

[0136] Specifically, for example, assume that at time step t, after the logistic regression operation, the maximum output probability corresponding to each training hidden layer feature is p_t(a_t|X). The predicted output text sequence can be modeled as where A X,Y represents the optimal path, such as Figure 6 Path 1 and Path 2 in the above.

[0137] In the test stage, since each time step can be performed independently, the model will select the target character corresponding to the maximum output probability at each time step for decoding. That is, it can be expressed by the formula as

[0138] Combined with Table 1 shown below, Table 1 shows the test results of the syntax error correction model based on sequence annotation, the syntax error correction model based on sequence-to-sequence, and the syntax error correction model provided by the embodiments of the present application:

[0139]

[0140] where p represents the precision of the correct output text, r represents the recall rate, and f represents the comprehensive probability combining p and r.

[0141] It should be understood that in the embodiments of the present application, when using clang88 as the training set and bea2019 as the test set, the precision and recall rate of the syntax error correction model of the embodiments of the present application are both better than those of the two syntax error correction models in the related art. By calling the model once, the correct text can be determined, improving the efficiency of syntax error correction.

[0142] In the embodiments of the present application, by using the CTC algorithm and the dynamic programming algorithm, the initial syntax error correction model can quickly find the predicted output text sequence with the highest output probability, without multiple iterations or multiple calls to the model for decoding, improving the decoding efficiency.

[0143] Based on the syntax error correction method provided in the above embodiments, referring to Figure 7 , this figure is a schematic structural diagram of a syntax error correction device provided by the embodiments of the present application. Combined with Figure 7 shown, the syntax error correction device 700 provided by the embodiments of the present application may include:

[0144] An acquisition module 701, configured to acquire the text to be error-corrected;

[0145] A processing module 702 is configured to perform upsampling processing on the text to be error-corrected to obtain an error-correction text sequence corresponding to the text to be error-corrected;

[0146] A syntax error correction module 703 is configured to, through a syntax error correction model, sequentially determine hidden layer features corresponding to each character in the error-correction text sequence according to the character order of the error-correction text sequence, and determine a target output text sequence according to the hidden layer features corresponding to each character in the error-correction text sequence;

[0147] A target text output module 704 is configured to, through a syntax error correction model, determine the correct text corresponding to the text to be error-corrected according to the target output text sequence.

[0148] As an example, the syntax error correction model is trained in the following manner:

[0149] An acquisition unit is configured to acquire an input training text and an output training text; the input training text includes at least two syntax errors; the output training text is the correct text corresponding to the input training text;

[0150] A processing unit is configured to perform upsampling processing on the input training text to determine an input text sequence corresponding to the input training text;

[0151] A training hidden layer feature determination unit is configured to, through an initial syntax error correction model, sequentially determine training hidden layer features corresponding to each character in the input text sequence according to the character order of the input text sequence, and obtain a training hidden layer feature set corresponding to the input text sequence;

[0152] A prediction unit is configured to, through an initial syntax error correction model, determine a predicted output text sequence according to the training hidden layer feature set and the training output text sequence; the training output text sequence is obtained based on the output training text;

[0153] A loss function value determination unit is configured to determine a loss function value according to the predicted output text sequence and the input text sequence;

[0154] A training unit is configured to train the initial syntax error correction model by using the loss function value.

[0155] As an example, the initial syntax error correction model includes a masked attention mechanism layer; the processing object of the masked attention mechanism layer is a mask matrix;

[0156] The mask matrix is composed of a first sub-matrix and a second sub-matrix; the first sub-matrix is constructed based on the training hidden layer feature set and is not masked; the second sub-matrix is constructed based on the text features of each of the n characters in the predicted output text sequence. When determining the i-th character in the predicted output text sequence, the text features of the first i - 1 characters in the predicted output text sequence are not masked, and the text features of the i-th to n-th characters in the predicted output text sequence are masked, where n is an integer greater than 1.

[0157] As an example, the prediction unit includes:

[0158] A first determination subunit, configured to determine the output probabilities of each training hidden layer feature in the training hidden layer feature set for each character in the training output text sequence;

[0159] A second determination subunit, configured to determine the maximum output probability corresponding to each training hidden layer feature from the multiple output probabilities corresponding to each training hidden layer feature;

[0160] A prediction subunit, configured to determine the predicted output text sequence according to the maximum output probabilities corresponding to the obtained training hidden layer features.

[0161] As an example, the prediction subunit is specifically configured to:

[0162] In the order from the last training hidden layer feature to the first training hidden layer feature in the training hidden layer feature set, determine the characters corresponding to the maximum output probabilities of each training hidden layer feature in turn as the target characters;

[0163] Determine the predicted output text sequence according to the target characters corresponding to the training hidden layer features.

[0164] As an example, the prediction subunit is specifically configured to:

[0165] Arrange the multiple target characters in the order of the training hidden layer features in the training hidden layer feature set to obtain a target character sequence;

[0166] If two adjacent target characters in the target character sequence are the same, merge the two target characters to obtain a merged character;

[0167] If two adjacent target characters in the target character sequence are different, retain the two target characters;

[0168] Determine the predicted output text sequence according to the merged character and the multiple target characters.

[0169] As an example, the training output text sequence is obtained by the following method:

[0170] A first spacing unit, configured to insert blank characters before the first character in the training output text and after the last character in the training output text, to obtain a first output sequence;

[0171] A second spacing unit, configured to insert blank characters between adjacent characters in the first output sequence, to obtain a training output text sequence corresponding to the training output text.

[0172] The grammar error correction device provided in the embodiments of the present application has the same beneficial effects as the grammar error correction method provided in the above embodiments, and thus will not be elaborated herein.

[0173] The structures will be separately introduced below in the forms of a server and a terminal device.

[0174] Figure 8 FIG. is a schematic structural diagram of a server provided in the embodiments of the present application. The server 900 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 922 (for example, one or more processors) and a memory 932, and one or more storage media 930 (for example, one or more mass storage devices) for storing application programs 942 or data 944. Among them, the memory 932 and the storage media 930 may be transient storage or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Further, the central processing unit 922 may be configured to communicate with the storage media 930 and execute a series of instruction operations in the storage media 930 on the server 900.

[0175] The server 900 may further include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM and so on.

[0176] Among them, the CPU 922 is configured to perform the following steps:

[0177] Obtain the text to be corrected;

[0178] Perform upsampling processing on the text to be corrected to obtain a text sequence to be corrected corresponding to the text to be corrected;

[0179] Through the syntax error correction model, in the character order of the text sequence to be corrected, the hidden layer features corresponding to each character in the text sequence to be corrected are determined in sequence, and a target output text sequence is determined according to the hidden layer features corresponding to each character in the text sequence to be corrected;

[0180] Through the syntax error correction model, according to the target output text sequence, the correct text corresponding to the text to be corrected is determined.

[0181] The embodiment of the present application also provides another terminal device structure, as Figure 9 shown. For the sake of convenience of description, only the parts related to the embodiment of the present application are shown. For those specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), a point of sales (English full name: Point of Sales, English abbreviation: POS), an in-vehicle computer, etc. Taking the terminal as a mobile phone as an example:

[0182] Figure 9 Shown is a block diagram of a part of the structure of a mobile phone related to the terminal provided by the embodiment of the present application. Refer to Figure 9 , the mobile phone includes: a radio frequency (English full name: Radio Frequency, English abbreviation: RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (English full name: wirelessfidelity, English abbreviation: WiFi) module 1070, a processor 1080, and a power supply 1090 and other components. Those skilled in the art can understand that Figure 9 the mobile phone structure shown in does not limit the mobile phone, and it may include more or fewer components than shown in the figure, or combine some components, or arrange different components.

[0183] The following will combine Figure 9 to specifically introduce each component of the mobile phone:

[0184] The RF circuit 1010 can be used for receiving and transmitting information or signals during a call. In particular, after receiving the downlink information from the base station, it is sent to the processor 1080 for processing. Additionally, the uplink data designed is sent to the base station. Generally, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (full English name: Low Noise Amplifier, English abbreviation: LNA), a duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (full English name: Global System of Mobile communication, English abbreviation: GSM), General Packet Radio Service (full English name: General Packet Radio Service, GPRS), Code Division Multiple Access (full English name: Code Division Multiple Access, English abbreviation: CDMA), Wideband Code Division Multiple Access (full English name: Wideband Code Division Multiple Access, English abbreviation: WCDMA), Long Term Evolution (full English name: Long Term Evolution, English abbreviation: LTE), email, Short Messaging Service (full English name: Short Messaging Service, SMS), etc.

[0185] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory 1020 can include high-speed random access memory and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0186] The input unit 1030 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the mobile phone. Specifically, the input unit 1030 can include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations of the user thereon or nearby (such as operations of the user using any suitable object or accessory such as a finger, a stylus, etc. on or near the touch panel 1031), and drive corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 can include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch orientation of the user, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 1080, and can receive and execute the commands sent by the processor 1080. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch panel 1031. In addition to the touch panel 1031, the input unit 1030 can also include other input devices 1032. Specifically, the other input devices 1032 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0187] The display unit 1040 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 can include a display panel 1041. Optionally, the display panel 1041 can be configured in the form of a liquid crystal display (English full name: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (English full name: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 1031 can cover the display panel 1041. When the touch panel 1031 detects a touch operation thereon or nearby, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 9 the touch panel 1031 and the display panel 1041 are implemented as two independent components to realize the input and input functions of the mobile phone, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0188] The mobile phone may further include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary, and can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the mobile phone can also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be elaborated here.

[0189] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the mobile phone. The audio circuit 1060 can transmit the electrical signal converted from the received audio data to the speaker 1061, and the speaker 1061 converts it into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 and then converted into audio data. After the audio data is output to the processor 1080 for processing, it is sent through the RF circuit 1010 to, for example, another mobile phone, or the audio data is output to the memory 1020 for further processing.

[0190] WiFi belongs to short - range wireless transmission technology. The mobile phone can help users send and receive emails, browse the web, and access streaming media through the WiFi module 1070, which provides users with wireless broadband Internet access. Although Figure 9 the WiFi module 1070 is shown, it can be understood that it does not belong to the essential components of the mobile phone and can be omitted entirely within the scope of not changing the essence of the invention according to needs.

[0191] The processor 1080 is the control center of the mobile phone, connecting various parts of the entire mobile phone using various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 1020, and by calling the data stored in the memory 1020, it executes various functions of the mobile phone and processes data, thereby collecting overall data and information of the mobile phone. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above - mentioned modem processor may not be integrated into the processor 1080 either.

[0192] The mobile phone further includes a power supply 1090 (such as a battery) for powering each component. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system.

[0193] Although not shown, the mobile phone may further include a camera, a Bluetooth module, etc., which will not be elaborated here.

[0194] In the embodiment of the present application, the processor 1080 included in the terminal further has the following functions:

[0195] Obtain the text to be error-corrected;

[0196] Perform upsampling processing on the text to be error-corrected to obtain a text sequence to be error-corrected corresponding to the text to be error-corrected;

[0197] Through the syntax error correction model, in the character order of the text sequence to be error-corrected, sequentially determine the hidden layer features corresponding to each character in the text sequence to be error-corrected, and determine a target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be error-corrected;

[0198] Through the syntax error correction model, according to the target output text sequence, determine the correct text corresponding to the text to be error-corrected.

[0199] The embodiment of the present application further provides a computer-readable storage medium for storing program codes, and the program codes are used to execute any one of the implementation manners of a syntax error correction method described in the foregoing various embodiments.

[0200] The embodiment of the present application further provides a computer program product including instructions, and when it runs on a computer, it causes the computer to execute any one of the implementation manners of a syntax error correction method described in the foregoing various embodiments.

[0201] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated here.

[0202] In several embodiments provided by the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the division of the system is only a logical function division, and there may be other division methods in actual implementation. For example, multiple systems can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.

[0203] The system described as a separation component may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] In addition, in each embodiment of this application, each functional unit can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0205] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (English full name: Read-Only Memory, English abbreviation: ROM), random access memories (English full name: Random Access Memory, English abbreviation: RAM), magnetic disks, or optical discs that can store program codes.

[0206] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of this application.< / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank> < / blank>

Claims

1. A grammar error correction method, characterized in that, The method includes: Obtain the text to be corrected; Perform upsampling processing on the text to be corrected to obtain a text sequence to be corrected corresponding to the text to be corrected; Through the grammar correction model, in the order of the characters in the text sequence to be corrected, sequentially determine the hidden layer features corresponding to each character in the text sequence to be corrected, and determine the target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be corrected; Through the grammar correction model, determine the correct text corresponding to the text to be corrected according to the target output text sequence.

2. The method according to claim 1, characterized in that, The grammar correction model is trained in the following manner: Obtain the input training text and the output training text; at least two grammar errors are included in the input training text; the output training text is the correct text corresponding to the input training text; Perform upsampling processing on the input training text to determine an input text sequence corresponding to the input training text; Through the initial grammar correction model, in the order of the characters in the input text sequence, sequentially determine the training hidden layer features corresponding to each character in the input text sequence, and obtain a set of training hidden layer features corresponding to the input text sequence; Through the initial grammar correction model, determine a predicted output text sequence according to the set of training hidden layer features and the training output text sequence; the training output text sequence is obtained based on the output training text; Determine the loss function value according to the predicted output text sequence and the input text sequence; Use the loss function value to train the initial grammar correction model.

3. The method according to claim 2, wherein The initial grammar correction model includes a masked attention mechanism layer; the processing object of the masked attention mechanism layer is a mask matrix; The mask matrix is composed of a first sub-matrix and a second sub-matrix; the first sub-matrix is constructed based on the set of training hidden layer features, and the first sub-matrix is not masked; the second sub-matrix is constructed based on the text features of n characters in the predicted output text sequence. When determining the i-th character in the predicted output text sequence, the text features of the first i-1 characters in the predicted output text sequence are not masked, and the text features of the i-th character to the n-th character in the predicted output text sequence are masked, where n is an integer greater than 1.

4. The method according to claim 2, wherein The determining the predicted output text sequence according to the set of training hidden layer features and the training output text sequence includes: Determine the output probability of each training hidden layer feature in the set of training hidden layer features for each character in the training output text sequence; Determine the maximum output probability corresponding to each training hidden layer feature from the multiple output probabilities corresponding to each training hidden layer feature; Determine the predicted output text sequence according to the maximum output probabilities corresponding to the obtained training hidden layer features.

5. The method according to claim 4, characterized in that, The determining the predicted output text sequence according to the obtained maximum output probabilities corresponding to the training hidden layer features includes: In the order from the last training hidden layer feature to the first training hidden layer feature in the set of training hidden layer features, sequentially determine the characters corresponding to the maximum output probabilities corresponding to the training hidden layer features as the target characters; Determine the predicted output text sequence according to the target characters corresponding to each of the training hidden layer features.

6. The method according to claim 5, wherein The determining the predicted output text sequence according to the target characters corresponding to each of the training hidden layer features includes: Arrange a plurality of the target characters in the order of each training hidden layer feature in the training hidden layer feature set to obtain a target character sequence; If two adjacent target characters in the target character sequence are the same, merge the two target characters to obtain a merged character; If two adjacent target characters in the target character sequence are different, retain the two target characters; Obtain the predicted output text sequence according to the merged character and the plurality of target characters.

7. The method according to claim 2, wherein The training output text sequence is obtained by the following method: Insert a blank character before the first character in the training output text, and insert the blank character after the last character in the training output text to obtain a first output sequence; Insert the blank character between two adjacent characters in the first output sequence to obtain the training output text sequence corresponding to the training output text.

8. A grammar error correction device, characterized in that, The device includes: An acquisition module, configured to acquire a text to be corrected; A processing module, configured to perform upsampling processing on the text to be corrected to obtain a text sequence to be corrected corresponding to the text to be corrected; A grammar error correction module, configured to, through a grammar error correction model, sequentially determine hidden layer features corresponding to each character in the text sequence to be corrected according to the character order of the text sequence to be corrected, and determine a target output text sequence according to the hidden layer features corresponding to each character in the text sequence to be corrected; A target text output module, configured to, through the grammar error correction model, determine a correct text corresponding to the text to be corrected according to the target output text sequence.

9. A grammar error correction device, characterized in that, The device includes a processor and a memory: The memory is configured to store program code and transmit the program code to the processor; The processor is configured to execute the steps of the grammar error correction method according to any one of claims 1 to 7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store program code, and the program code is used to execute the steps of the grammar error correction method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes a computer program or instruction, and when the computer program or instruction is executed, the steps of the grammar error correction method according to any one of claims 1 to 7 are implemented.

Images