Position information fusion method and device and computer readable storage medium

Abstract

The invention provides a position information fusion method and device and a computer readable storage medium. The position information fusion method provided by the invention comprises the following steps: performing word segmentation processing on an input statement to obtain a first sequence of words in the input statement, and generating absolute position information of the words in the first sequence; splitting words in the first sequence to obtain a second sequence comprising sub-words, and generating position information of the sub-words in the second sequence according to absolute position information of the words to which the sub-words belong in the first sequence; and fusing the position information of the sub-words in the second sequence into a self-attention model, and carrying out model training or model prediction. After the sub-word splitting algorithm is introduced into the self-attention model, the word position information based on the word absolute position is fused in the model input features, so that the subsequent task can obtain more accurate position information, and the accuracy of the model is improved.

Description

technical field [0001] The present invention relates to the technical field of natural language processing (NLP, Natural Language Processing), in particular to a location information fusion method, device and computer-readable storage medium. Background technique [0002] In recent years, the attention mechanism has been widely used in deep learning-based natural language processing tasks. With the in-depth study of the attention mechanism, various attention models have been proposed by researchers. In June 2017, the paper "Attention is all you need" published by the google machine translation team on arXiv has received widespread attention in the industry. The self-attention mechanism has become a research hotspot in the neural network attention mechanism. also achieved good results. [0003] Self-attention models (also called self-attention network models) generate context-sensitive representations by applying attention to each pair of elements in the input sequence. Co...

AI Technical Summary

Problems solved by technology

The traditional word representation method has the problem of out-of-vocabulary (OOV, Out Of Vocabulary), and it is difficult to deal with unknown or rare vocabulary. If a smaller vocabulary is used, the OOV problem will be more prominent

Images