Dual-Encoder Neural Network for Semantic Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex machine-trained models for interpreting linguistic expressions are often too slow and resource-intensive, particularly in search engines where fast results are required, and on user devices with limited resources.
Innovation Solution
A dual-encoder neural network system processes query and target items separately using transformer-based encoders, with one encoder processing the query item and another processing the target item offline, allowing for efficient comparison and reducing latency and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a complex machine-trained model is used to interpret linguistic expressions, then accuracy is improved, but processing speed deteriorates
Solution Approach 1:
The patent divides the semantic similarity assessment into two separate encoder models that process query and target items independently, rather than using a single complex model. This segmentation allows each encoder to be optimized for speed while maintaining accuracy through their combined output.
Solution Approach 2:
The patent pre-processes target items using the second encoder and stores their embeddings offline before query time. This preliminary action moves computational work from the critical query response path, significantly improving processing speed for incoming queries while maintaining accuracy through the pre-computed embeddings.
2Measurement precision
If a complex machine-trained model is used to interpret linguistic expressions, then accuracy is improved, but resource consumption increases
Solution Approach 1:
The patent splits the computational workload across two separate encoder models rather than running one complex model. This allows the system to use smaller, more resource-efficient encoders that consume less memory and computational power while achieving the same accuracy through their combined similarity assessment.
Solution Approach 2:
By pre-computing and storing embeddings for target items offline, the patent moves resource-intensive processing away from real-time query handling. This reduces the resource consumption during active query processing while maintaining accuracy through the pre-computed representations.
3Loss of time
If semantic similarity assessment is performed in real-time for all items, then response time is improved, but resource consumption increases
Solution Approach 1:
The patent performs preliminary processing of target items by computing their embeddings offline using the second encoder and storing them for later use. This eliminates the need to re-process target items for each query, dramatically reducing real-time resource consumption while maintaining fast response times through the pre-computed data.
Data Source
AI summary
A technique is described herein for processing a given query item in a latency-efficient and resource-efficient manner. The technique uses a first transformer-based encoder to transform the given query item into an encoded query item. In one case, the given query item is an expression that includes one or more query-expression linguistic tokens. The technique includes a second transformer-based encoder for transforming a given target item into an encoded target item. The given target item may likewise correspond to an expression that includes one or more target-expression linguistic tokens. A similarity-assessing mechanism then assesses the semantic similarity between the given query item and the given target item based on the encoded query item and the encoded target item. Each transformer-based encoder uses one or more self-attention mechanisms. The second transformer-based encoder can optionally perform its work in an offline manner, prior to receipt of the given query item.