Relation Prediction Using Entity-Focused and Mid-Context Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current natural language processing techniques for relation extraction lack entity type and contextual information between entities, leading to sub-optimal relation prediction accuracy and inefficient batch processing.
Innovation Solution
A computer-implemented method using a pre-trained machine learning model generates embedding representations for target spans and mid-context spans, integrating entity type information and mid-context embeddings to predict relations, which are then processed through fully connected and softmax layers for improved accuracy and reduced latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional relation extraction methods are used, then the system is simpler to implement, but relation prediction accuracy is sub-optimal due to lack of entity type and contextual information
Solution Approach 1:
The patent segments the context into three distinct parts: subject entity span, mid-context span (words between entities), and object entity span. Each segment is processed separately to extract specific features, allowing the model to capture different types of information systematically without overwhelming complexity
Solution Approach 2:
The patent adds entity type information as an additional dimension to the relation extraction problem. By incorporating entity type embeddings alongside contextual embeddings, the model operates in a higher-dimensional space that captures both linguistic context and semantic category information, improving prediction accuracy
2Productivity
If traditional batch processing is used, then computing resources are consumed inefficiently, but processing multiple relations in a single pass increases productivity
Solution Approach 1:
The patent performs preliminary processing by generating embeddings for all entity spans and mid-context spans simultaneously before relation prediction. This preliminary embedding generation enables batch processing of multiple relations in a single pass, as the contextual representations are already prepared and can be reused across multiple relation predictions
Solution Approach 2:
The embedding representations generated by the model serve multiple functions: they capture contextual information for relation prediction, encode entity type information, and provide mid-context features. This multi-functionality allows the same computational output to be used across multiple relation extraction tasks, reducing redundant computation and improving batch processing efficiency
Data Source
AI summary
Method includes: accessing text, where spans are identified within the text and include one or more pairs of target spans and one or more mid-context spans; generating embedding representations of tokens associated with each target span, tokens associated with the entity types of each target span, and tokens associated with each mid-context span; generating, for each target span, entity-focused span embedding representation based on embedding representations of tokens associated with each target span and embedding representations of tokens associated with entity type of target span; generating, for each mid-context span, mid-context embedding representation based on the embedding representations of tokens associated with each mid-context span; and generating probability distribution of each relation of set of relations based on entity-focused span embedding representations of subject span and object span that are included in each target pair and mid-context embedding representation for mid-context span appearing between subject span and object span.