Entity and Evidence-Guided Relation Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural models for relation extraction primarily focus on intra-sentence relations and lack effective methods for extracting relations between entities across sentences, limiting their ability to perform document-level relation extraction and evidence prediction.
Innovation Solution
The proposed system employs a computing device with a language model and bilinear layers to process entities and documents, predicting relations and evidence sentences by generating entity-guided input sequences and utilizing internal attention probabilities for fine-tuning, thereby enhancing the model's ability to focus on relevant entities and supporting sentences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural models focus on intra-sentence relations, then relation extraction accuracy within sentences is improved, but the ability to extract relations between entities across sentences deteriorates
Solution Approach 1:
The patent segments the document into multiple sentences and processes each sentence individually through the language model, then aggregates the results to achieve document-level relation extraction. This allows the model to maintain focus on intra-sentence relations while capturing cross-sentence entity relationships through the aggregation of sentence-level predictions.
Solution Approach 2:
The patent extends the relation extraction task from a single-sentence dimension to a multi-sentence document dimension by processing each sentence separately and then combining results. This dimensional expansion enables the model to capture relations between entities that appear in different sentences while maintaining the effectiveness of intra-sentence relation extraction.
2Loss of information
If language models are used for relation extraction, then understanding of contextual relations is improved, but computational complexity and training difficulty increase
Solution Approach 1:
The patent employs pre-trained language models that have already undergone extensive training on large corpora before being applied to relation extraction. This preliminary training captures rich contextual information and linguistic patterns, allowing the model to achieve high contextual understanding while requiring minimal additional training for the specific relation extraction task.
Solution Approach 2:
The patent uses the pre-trained language model as an intermediary that bridges the gap between raw text and relation extraction. The language model processes the input text and transforms it into contextualized representations that can be effectively used for relation prediction, simplifying the overall training process while maintaining high contextual understanding.
3Measurement precision
If attention probabilities are extracted from multiple layers, then evidence prediction accuracy is improved, but computational time and processing overhead increase
Solution Approach 1:
The patent extracts attention probabilities from a specific subset of layers (last 2 to 5 layers) rather than all layers of the language model. This partial extraction provides sufficient information for accurate evidence prediction while significantly reducing the computational overhead compared to extracting attention from every layer.
Solution Approach 2:
The patent applies different processing to different parts of the model output by extracting attention probabilities specifically from the later layers of the language model, which contain the most relevant contextual information for evidence prediction. This localized extraction optimizes the balance between accuracy and computational efficiency.
Data Source
AI summary
System and method multitask prediction. The system include a computing device. The computing device has a processer and a storage device storing computer executable code. The computer executable code is configured to: provide a head entity and a document containing the head entity; process the head entity and the document by a language model to obtain head extraction corresponding to the head entity, tail extractions corresponding to tail entities in the document, and sentence extraction corresponding to sentences in the document; predict a head-tail relation between the head extraction and the tail extractions using a first bilinear layer; combine the sentence extraction and a relation vector corresponding to the predicted head-tail relation using a second bilinear layer to obtain a sentence-relation combination; and predict an evidence sentence supporting the head-tail relation using a third bilinear layer based on the sentence-relation combination and attention extracted from the language model.