Natural language semantic matching method for sequence global attention and local dynamic attention

Abstract

The invention discloses a natural language semantic matching method for sequence global attention and local dynamic attention. Global understanding and representation of sentence semantics are achieved through a stacked gating recurrent neural network and a self-attention mechanism; dynamic analysis of important information of the sentences is achieved through a local dynamic attention mechanism,and dynamic local representation of sentence semantics is obtained. According to the method, more comprehensive understanding of sentence semantics is achieved through global attention and local dynamic attention of the sentence sequence, then semantic interaction between two sentences is accurately modeled, finally accurate judgment of the sentence semantic matching relation is achieved, and thedefects of an existing method in the aspect of attention mechanism use are overcome.

Description

technical field [0001] The invention relates to the technical fields of deep learning and natural language understanding, and in particular to a natural language semantic matching method of sequence global attention and local dynamic attention. Background technique [0002] Natural language sentence semantic matching (Sentence Semantic Matching) is a very important part of the field of natural language processing, and it is a common method to evaluate whether the semantic representation of sentences is accurate. The main problem it solves is to judge the semantic relationship between two sentences. Depending on the specific task, the semantic relationship between sentences is also defined differently. For example, in Natural Language Inference (NLI), sentence semantic matching is mainly used to determine whether the semantics of the premise sentence (Premise Sentence) implies the semantics of the Hypothesis Sentence (Hypothesis Sentence). In Information Retrieval (IR), sen...

AI Technical Summary

Problems solved by technology

[0005] The above research on the semantic representation of natural language using the attention mechanism is mainly to select all the important information in one selection, ignoring two important phenomena: 1) The important part of the sentence may change with the in-depth understanding of the sentence semantics. (or in-depth understanding of the surrounding context); 2) the same words in different positions in the sentence have different influences on the semantics of the sentence, and the local structure corresponding to these same words helps to distinguish this difference.

Images