Text error correction model training method and device, medium and product

Abstract

The invention provides a text error correction model training method and device, a medium and a product, and the method comprises the following steps: obtaining a sample text list, and segmenting any same text in the sample text list to obtain sample fragments; inputting any sample segment into a preset text error correction model, obtaining a first loss function value and a second loss function value, and obtaining a total loss function value based on the first loss function value and the second loss function value; the text error correction model can be adjusted according to the total loss function value until the text error correction model converges, so that the text error correction accuracy of the text error correction model is improved, error correction of the text by the error correction model is reduced and copying of a correct text is enhanced while correct error correction of the text is ensured, and the error correction efficiency of the text is improved. Therefore, false alarms during text error correction are reduced.

Description

technical field [0001] The invention relates to the technical field of text processing, in particular to a training method, device, medium and product of a text error correction model. Background technique [0002] Text error correction is a fundamental problem in Natural Language Processing (NLP), and it can usually be placed in front of other NLP tasks such as text retrieval, text classification, machine translation or sequence labeling to improve the validity of input text and prevent The bad effects of spelling mistakes. [0003] In the prior art, by training a neural network model, wherein the neural network model includes a memory bank gate control layer, an encoding layer and an output layer, the encoding vector of the semantic unit selected from the historical text of the sample text can be added to the memory. In this way, the error correction of the sample text is realized by combining the coding vector of the semantic unit in the memory library, but it only enric...

AI Technical Summary

Problems solved by technology

[0003] In the prior art, through the training of the neural network model, wherein the neural network model includes a memory bank gating layer, a coding layer and an output layer, the coding vector of the semantic unit selected from the historical text of the sample text can be added to the memory Library, so as to realize the error correction of the sample text by combining the coding vector of the semantic unit in the memory library, but it only enriches the error correction information used by the text error correction model, and does not solve the problem of generating sample sentences in the text error correction model of the copying attention mechanism When changing more words, resulting in lower accuracy of error correction results

Images