Text processing method and device, text feature extraction method and device, equipment and medium

Abstract

The invention provides a text processing method, and relates to the field of artificial intelligence. The method comprises the following steps: inputting a first text into a pre-trained feature extractor to obtain a first text feature; inputting a second text into the pre-trained feature extractor to obtain a second text feature; obtaining a similarity prediction result between the first text feature and the second text feature; wherein the feature extractor comprises N cyclic encoders, each cyclic encoder is constructed and obtained according to a cyclic memory mechanism and an encoder in a Transform model, and the cyclic memory mechanism is used for processing data according to state information at the previous moment and input information at the current moment. The invention also provides a text feature extraction method, which comprises the following steps of: inputting a third text into the pre-trained feature extractor to obtain a third text feature. The invention further provides a text processing device, a text feature extraction device, equipment, a storage medium and a program product.

Description

technical field [0001] The present disclosure relates to the field of artificial intelligence, and more particularly, to a text processing method, text feature extraction method, apparatus, device, medium and program product. Background technique [0002] Using deep learning methods to process text can be implemented based on a recurrent neural network model or a convolutional neural network model. However, due to the problem of gradient dispersion, the structure of the recurrent neural network cannot be too deep in the vertical deep stack, resulting in the loss of the regularization and strong coding ability brought by the deep network, and the upper limit of the generalization performance is limited. . While convolutional neural networks are often used for images, they are not suitable for processing serialized data structures of text. SUMMARY OF THE INVENTION [0003] In view of the above problems, the present disclosure provides a text processing method, text feature...

AI Technical Summary

Problems solved by technology

However, due to the problem of gradient dispersion, the structure of the recurrent neural network cannot be too deep in the vertical deep stack, resulting in the loss of the regularization and strong coding capabilities brought by the deep network, and the upper limit of generalization performance is limited, which cannot be improved with the help of large data sets.

While convolutional neural networks are often used for images, they are not suitable for processing serialized data structures of text

Images