Corpus expansion method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a corpus expansion method and device, computer equipment and a storage medium. The method comprises the steps of obtaining a parallel seed set, a first corpus and a second corpus, training a selection model according to the parallel seed set, the first corpus and the second corpus, and for each first text in the first corpus, determining a matched second text from the second corpus through the trained selection model; forming a plurality of groups of pseudo-parallel text pairs according to the plurality of first texts in the first corpus and the respective matched second texts; and screening the multiple groups of pseudo-parallel text pairs, and adding the screened multiple groups of pseudo-parallel text pairs into the parallel seed set. According to the embodiment of the invention, the mapping relation between the plurality of first texts and the plurality of second texts is learned by using the selection model, so that the problem of style conversion failure caused by the fact that the content style cannot be completely decoupled in related technologies is avoided, and the subsequent corpus generation effect is ensured.

Description

technical field [0001] The present disclosure relates to the field of computer technology, and in particular to a corpus expansion method, device, computer equipment and storage medium. Background technique [0002] Text style transfer is a technique that automatically converts text expression styles while preserving text content. [0003] In related technologies, due to the lack of a large number of parallel corpora with similar content but different styles, the mainstream technology of text style transfer is carried out in the way of unsupervised learning. Usually, a style-independent content representation vector is learned in the training stage, and the input sentence is reconstructed by combining the style representation of the original sentence. In the inference stage, sentences with the target style and specified content are generated based on the style-independent content representation vector and the target style representation. During training, GANs are often use...

AI Technical Summary

Problems solved by technology

[0004] However, in the above methods, unsupervised learning cannot learn a completely decoupled style and content representation, that is, the content representation always contains part of the style information, causing the generator to sometimes fail to generate sentences with the specified style, resulting in ineffective sentence generation. good

Images