Co-Attention Text Matching for Sentence Pairs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current text matching methods, particularly in natural language processing, face challenges such as sensitivity to word segmentation in rule-based systems and the need for large training data in deep-learning-based systems, leading to inefficient inference times and performance issues in various application scenarios.
Innovation Solution
A neural network-based system that utilizes a multi-layer co-attention mechanism with pre-trained word embeddings to generate correlation factors and feature vectors, enabling efficient text matching with reduced training data requirements and faster inference times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep-learning-based models (LSTM, BERT, DSSM, Match Pyramid) are used for text matching, then matching accuracy is improved, but inference time increases significantly
Solution Approach 1:
The patent segments the text matching task into multiple attention layers, where each layer focuses on different aspects of sentence pair relationships. This multi-layer co-attention mechanism divides the complex matching problem into manageable components, achieving high accuracy without requiring excessively large models that would increase inference time
Solution Approach 2:
The patent introduces a multi-layer attention architecture that adds depth to the matching process. By stacking multiple attention layers, the system processes sentence pairs through different transformation dimensions, capturing complex relationships more efficiently than single-layer models while maintaining reasonable computational complexity
2Productivity
If rule-based systems are used for text matching, then inference speed is improved, but matching accuracy deteriorates due to sensitivity to word segmentation
Solution Approach 1:
The patent employs pre-trained word embeddings that have already captured semantic relationships during training. This preliminary action of pre-training allows the model to process new sentence pairs quickly without requiring extensive feature engineering or segmentation rules, achieving both speed and accuracy
Solution Approach 2:
The patent replaces the mechanical rule-based word segmentation and matching system with a neural network-based co-attention mechanism. This substitution eliminates the sensitivity to segmentation rules while maintaining fast inference through efficient attention computations
Data Source
AI summary
A method, computer program, and computer system is provided for query matching of sentences based on co-attention scores. Two or more query inputs to a neural network are received. A correlation factor between the two or more query input is identified based on attention weights. A feature vector is generated based on the attention weights. A probability value corresponding to the two or more query inputs is determined based on the generated feature vector.