Semantic Document Extraction Using Targeted Segment Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting semantic information from long documents are inefficient as they process unnecessary information, leading to increased computational load and resource consumption.
Innovation Solution
A system and method that selectively processes only segments of a document determined to be most likely to contain useful information, using a retriever model to generate embeddings and an extraction model to extract semantic information, reducing the amount of processed data through targeted sequence selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a machine learning model processes all segments of a long document to extract semantic information, then extraction completeness is improved, but computational load and processing time increase
Solution Approach 1:
The patent divides a long document into multiple segments and processes only selected segments that are most likely to contain useful information. This segmentation approach allows the system to maintain extraction completeness for relevant information while reducing overall processing time by excluding irrelevant segments from full model processing.
Solution Approach 2:
The patent extracts and identifies specific segments from the full document that are most likely to contain useful information for the extraction model to process. By taking out only the relevant segments rather than processing the entire document, the system reduces computational load and processing time while maintaining extraction effectiveness.
2Measurement precision
If a machine learning model processes all segments of a long document, then information extraction accuracy is improved, but resource consumption increases
Solution Approach 1:
The patent segments the document and applies the extraction model only to selected segments, reducing computational resource consumption while maintaining accuracy for relevant information extraction.
Solution Approach 2:
The patent applies partial action by processing only a subset of segments that are most likely to contain useful information. This partial processing approach reduces resource consumption while maintaining sufficient extraction accuracy for the target information.
3Loss of information
If all segments of a document are processed by the extraction model, then semantic information completeness is improved, but device complexity increases
Solution Approach 1:
The patent introduces a segment selection mechanism that divides the document processing into identification and extraction phases, managing system complexity by handling only relevant segments while maintaining semantic information completeness.
Solution Approach 2:
The patent introduces an intermediary segment identification process that selects which segments to process. This intermediary layer manages the complexity by filtering segments before they reach the extraction model, maintaining information completeness while reducing system complexity.
Data Source
AI summary
Systems and methods to extract semantic information from documents are disclosed. Exemplary implementations may obtain target-specific aggregated embeddings representing generalized semantic contexts of sequences of text included in segments pertinent to targets and a set of sequences of text included in segments included in a document; provide the set of sequences of text as input for a retriever model configured to take as input sequences of text and to output embeddings representing semantic meanings of the sequences of text; obtain output embeddings from the retriever model, generate a set of targeted sequences of text in accordance with the output embeddings, provide the set of targeted sequences of text as input for an extraction model configured to take as input sequences of text and output semantic information extracted from the document; and obtain output semantic information from the extraction model.