Document Query Compression via Importance-Based Segment Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document query systems face inefficiencies due to high computing costs and reduced accuracy caused by long prompt inputs for language models, especially when dealing with large documents.
Innovation Solution
The method involves determining importance degrees of document segments relative to a target question, calculating respective compression ratios, compressing feature representations of these segments based on their importance, and using a trained model to determine the target answer from the compressed feature representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the complete document is input to the language model, then the query accuracy is improved, but the computing cost and input size increase significantly
Solution Approach 1:
The patent divides the document into multiple segments and processes them separately. The encoder processes each segment independently to generate segment embeddings, which are then aggregated. This segmentation allows the system to handle long documents without overwhelming the model with excessive input tokens, thus maintaining query accuracy while controlling input size.
Solution Approach 2:
The patent extracts only the relevant features from document segments using an encoder, rather than inputting the complete raw text. The encoder extracts semantic representations (embeddings) from each segment, which are then used for query matching. This extraction process preserves important information while significantly reducing the input size required for the language model.
2Measurement precision
If the complete document is input to the language model, then the query accuracy is improved, but the computing cost increases
Solution Approach 1:
By segmenting the document and processing segments independently through the encoder, the system reduces the computational burden on the language model. Only the compressed segment embeddings are passed to the language model for query matching, significantly reducing the number of tokens processed and thus lowering computing costs and energy consumption.
Solution Approach 2:
The encoder extracts essential semantic features from document segments before passing them to the language model. This extraction process creates compact representations that retain the important information needed for accurate query matching while requiring far fewer computational resources to process than the original full document text.
3Quantity of substance
If document segments are compressed, then the model input size is reduced, but information loss may occur
Solution Approach 1:
The encoder extracts semantic embeddings from each document segment, capturing the essential meaning and information. These embeddings serve as compressed representations that retain the critical information needed for query matching while significantly reducing the input size. The extraction process is designed to preserve semantic content rather than simply reducing data volume.
Solution Approach 2:
The system transforms the document segments from their original text form into embedding representations through the encoder. This parameter transformation changes the data from discrete text tokens to continuous vector representations, which compress the information more efficiently while preserving the semantic relationships needed for accurate query answering.
Data Source
AI summary
Embodiments of the disclosure provide a method and apparatus for document query, a device, and a readable medium. The method includes: determining, for a candidate document of a target question, a plurality of importance degrees of a plurality of document segments in the candidate document relative to the target question; determining, based on the respective importance degrees of the plurality of document segments, respective compression ratios for the plurality of document segments; compressing respective feature representations of the plurality of document segments based on the respective compression ratios for the plurality of document segments to obtain a compressed feature representation of the candidate document; and determining a target answer to the target question based on the compressed feature representation of the candidate document using a trained target model.