Passage Re-Ranking via Tournament Sorting for Query Relevance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing re-ranking systems for extracting passages relevant to a query suffer from inefficiencies due to high computational complexity and poor relevance comparison, particularly in cross-encoder and listwise re-ranking models, leading to position bias and reduced efficiency.
Innovation Solution
A re-ranking system utilizing tournament sortings to evaluate relevance through an encoder-decoder structure, specifically the Fusion-in-Decoder (FiD) or T5-base architecture, where passages are grouped and evaluated in multiple stages to extract top-ranked relevant passages efficiently, reducing computational complexity to O(n + k * log m n).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cross-encoder models use pointwise re-ranking for each document, then the model can process documents individually, but the capability in comparing relevance is poor
Solution Approach 1:
The patent combines multiple documents into a single concatenated input sequence that is processed by the encoder-decoder model. This merging approach allows the model to compare relevance across multiple documents simultaneously rather than processing them individually, thereby improving relevance comparison capability while maintaining processing efficiency.
2Measurement precision
If listwise re-ranking models using LLMs are used, then the model can improve relevance comparison, but efficiency is reduced due to large model size
Solution Approach 1:
The patent employs a smaller encoder-decoder model (such as T5-base or T5-small) instead of large language models. This smaller model achieves comparable relevance comparison capability while significantly reducing computational cost and improving processing efficiency. The model is used temporarily for re-ranking and then discarded, aligning with the disposable principle.
3Measurement precision
If pairwise re-ranking is used to compare relevance between documents, then relevance comparison improves, but time complexity increases to O(n^2)
Solution Approach 1:
The patent merges multiple document processing operations into a single batch processing step. By concatenating multiple documents and their corresponding queries into one input sequence, the model processes all comparisons simultaneously in O(n) time complexity rather than performing O(n^2) pairwise comparisons sequentially.
Solution Approach 2:
The patent transforms the problem from sequential pairwise comparison (1D processing) to parallel batch processing (adding a batch dimension). This dimensional change allows multiple relevance comparisons to be performed simultaneously across different document-query pairs, reducing overall computational time.
4Measurement precision
If tournament sorting is performed multiple times to extract top-ranked passages, then extraction precision improves, but computational complexity increases
Solution Approach 1:
The patent segments the re-ranking process into multiple tournament sorting stages. In each stage, documents are divided into groups and processed separately, with winners advancing to the next round. This segmentation allows precise extraction of top-ranked passages while managing computational complexity through structured organization of the processing steps.
Solution Approach 2:
The patent performs tournament sorting only for the necessary number of rounds to extract the desired number of top-ranked passages, rather than processing all documents to completion. This partial action approach achieves sufficient extraction precision while avoiding unnecessary computational overhead from excessive processing rounds.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A re-ranking system, method, and program for extracting passages having higher relevance to a query are disclosed. The system may include at least one processor, and a memory storing one or more commands, wherein the at least one processor may execute the one or more commands stored in the memory to perform re-ranking of a plurality of passages by performing a plurality of tournament sortings, wherein the tournament sorting may include pairing first passages included in the passages in a specific number to divide the first passages into a plurality of groups, evaluating relevance between the first passages included in a group and a query for each group, and extracting passages having higher relevance up to a predetermined rank among the first passages to output the extracted first passages as second passages, and pairing the second passages in a specific number to divide the second passages into a plurality of groups, evaluating relevance between the second passages included in a group and the query for each group, and extracting passages having higher relevance up to a predetermined rank among the second passages to output the extracted second passages as third passages.