English sentence simplification algorithm based on pre-trained Transformer language model

Abstract

The invention discloses an English sentence simplification algorithm based on a pre-trained Transformer language model, and the algorithm comprises the following steps: 1, carrying out the statisticsof word frequency through a public Wikipedia corpus; 2, utilizing a public pre-trained word embedding model to obtain vectorized representation of words; 3, preprocessing sentences needing to be simplified to obtain content words; 4, for the content words in the sentences, utilizing a public pre-training Transformer language model Bert to obtain a candidate alternative word set of the words; 5, sorting the candidate alternative word set of each content word by utilizing a plurality of features; 6, comparing the word frequencies of the candidate words with the highest sequence with the word frequencies of the original content words, and determining a final substitute word; and 7, processing other content words in the sentence according to the steps 4 to 6 in sequence to obtain a final simplified sentence. According to the method, the pre-trained Transformer language model is fully utilized without utilizing any labeled parallel corpus, so that the English sentence simplification accuracy is effectively improved.

Description

technical field [0001] The invention relates to the field of English text simplification, in particular to an English sentence simplification algorithm based on a pre-trained Transformer language model. Background technique [0002] In recent years, there are more and more English materials on the Internet. For example, many professional papers are written in English and published in English journals. Many people begin to like to read English materials directly instead of translating English materials into Chinese first and reading them in Chinese. Material. However, because English is not our mother tongue, the lack of vocabulary has seriously affected the understanding of English materials. Many studies have confirmed that if you can understand 90% of the words in the text, even if it is a long and complex text, the meaning expressed in the text is easier to understand. Additionally, English text simplification can be of great help to native English speakers, especially ...

AI Technical Summary

Problems solved by technology

The first two types of algorithms have great constraints. First, the construction of dictionaries is expensive and high-quality parallel corpus extraction is very difficult. Second, the coverage of these two types of algorithms for complex words is also limited.

The main problem of the above three types of algorithms is that in the process of generating candidate words, only the complex word itself is considered, and the context of the complex word is ignored, which will inevitably produce many unsuitable candidate words, which will bring great inconvenience to the subsequent steps of the system. big disturbance