Text summarization generation method and device based on self-attention and relative position encoding

Abstract

This invention provides a text summarization method and apparatus based on self-attention and relative positional encoding. The text summarization model includes a semantic feature representation module, a positional encoding module, a masking module, and a self-attention mechanism module. By employing relative positional encoding of subwords with respect to the word segmentation sequence, this method is better suited for text summarization tasks with character limit constraints and exhibits stronger extrapolation capabilities. Because each word segment of the input text is soft-masked based on importance weights, the model can focus its attention on key subwords while ignoring relatively unimportant information, thus improving the model's prediction accuracy. Furthermore, by fusing global information through the self-attention mechanism to calculate the importance and positional encoding of each word segment and integrating keyword and positional information into the decoded text, this method is better suited for text summarization tasks with character limit constraints.

Images