Interaction Layer Neural Network for Search Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text retrieval systems face a vocabulary gap problem due to mismatches between query and document terms, which affects retrieval performance, and existing solutions like IBM Model 1 require large query sets and are not effective for monolingual English queries.
Innovation Solution
A neural conditional translation probability network is used to generate similarity scores for query and document embeddings, with a product-of-sum aggregation to produce ranking scores, allowing for efficient and interpretable ranking of documents, even when queries are significantly shorter than documents, and enabling execution on lower-performance systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If simple term/token-matching techniques are used for retrieval, then the system is computationally efficient and easy to implement, but retrieval performance deteriorates due to vocabulary gap between query and document terms
Solution Approach 1:
The patent introduces an interaction layer neural network as an intermediary component between the query embedding and document embedding. This interaction layer computes pairwise similarity scores between query tokens and document tokens, acting as a mediator that bridges the vocabulary gap without requiring complex end-to-end models. The interaction layer uses a product-of-sum aggregation mechanism that combines similarity scores efficiently, providing improved retrieval performance while maintaining reasonable system complexity.
2Measurement precision
If BERT-based models are used for document ranking, then retrieval accuracy is improved, but computational efficiency deteriorates and the system requires high-performance hardware
Solution Approach 1:
The patent segments the ranking task into independent pairwise comparisons between query tokens and document tokens through the interaction layer. Instead of processing the entire query-document pair as a single unit like BERT, the interaction layer computes similarity scores for individual token pairs and aggregates them using product-of-sum. This segmentation enables parallel computation and reduces the computational burden, improving productivity while maintaining ranking accuracy.
Solution Approach 2:
The patent changes the computational parameters by using pre-computed embeddings and similarity scores instead of full BERT model inference. The interaction layer uses fixed-dimensional embedding vectors and computes similarity using efficient operations (cosine similarity or dot product), rather than requiring the heavy transformer architecture. This parameter change allows the system to achieve good ranking accuracy with significantly reduced computational requirements.
3Reliability
If IBM Model 1 is used to reduce vocabulary gap, then retrieval effectiveness is improved for cross-lingual and QA tasks, but the model requires large query sets and does not work well for monolingual English queries
Solution Approach 1:
The patent creates a universal interaction layer that can handle multiple query types (monolingual, cross-lingual, QA queries) through the same architectural framework. The interaction layer uses language-agnostic embedding representations and similarity computation mechanisms that work across different query-document scenarios. This universal approach eliminates the need for separate models like IBM Model 1 for different task types, improving adaptability while maintaining retrieval effectiveness across diverse query types.
Data Source
AI summary
A linguistic system includes a controller. The controller may be configured to receive a query and document, tokenize the query into a sequence of query tokens and tokenize the document into a sequence of document tokens, generate a matrix of token pairs for each of the query and the document tokens, retrieve for each entry in the matrix of token pairs, a precomputed similarity score produced by a neural conditional translation probability network, wherein the neural network has been trained in a ranking task using a corpus of paired queries and respective relevant documents, produce a ranking score for each document with respect to each query via a product-of-sum aggregation of each of the similarity scores for the respective query; and output the document and associated ranking score of the document.