Composite Embeddings for Hierarchical Relationship Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models trained on specific labelled datasets struggle to handle nuances and generic examples of entities and relationships when encountering unknown text data, leading to biased performance and limited generalizability in recognizing relationships between biological entities such as diseases, treatments, and genes.
Innovation Solution
A system and method for generating composite embeddings of text portions that include both specific and generic representations of entities and relationships, allowing for the creation of a more robust training dataset that enhances the ability of ML models to recognize relationships in a wider range of input data by combining embeddings of separable entities and their associated entities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ML models are trained on specific labelled datasets, then they achieve high accuracy on training data, but they fail to generalize to unknown text data and become biased towards training instances
Solution Approach 1:
The patent combines specific entity embeddings with generic hierarchical embeddings to create composite embeddings. This merging allows the model to retain specific knowledge from training data while incorporating generic patterns from hierarchical structures (ontologies, taxonomies), thereby resolving the contradiction between maintaining training accuracy and achieving generalization to unknown data.
Solution Approach 2:
The patent introduces hierarchical embeddings that represent generic categories and relationships applicable across multiple specific instances. These universal embeddings enable the model to handle unseen entities and relationships by leveraging shared hierarchical structures, thus providing multi-functionality that works both for known training instances and unknown test data.
2Loss of information
If ML models are trained on specific labelled datasets, then they learn detailed patterns for known entities, but they cannot handle nuances or generic examples of entities and relationships
Solution Approach 1:
The patent embeds specific entity representations within a nested hierarchical structure where generic categories contain multiple specific instances. The composite embedding nests both the specific entity vector and the hierarchical category vectors, allowing the model to access both granular specific knowledge and broader generic patterns simultaneously, thus resolving the contradiction between retaining specific information and handling generic examples.
3Reliability
If composite embeddings combining specific and generic representations are generated, then model robustness and generalization improve, but computational complexity and processing time increase
Solution Approach 1:
The patent pre-computes and stores hierarchical embeddings for generic categories and relationships before they are needed for inference. This preliminary action allows the embeddings to be cached and reused across multiple queries, reducing the computational burden during actual model operation while still providing the robustness benefits of composite embeddings.
Data Source
AI summary
Methods, apparatus, system and computer-implemented method are provided for embedding a portion of text describing one or more entities of interest and a relationship. The portion of text describes a relationship for the one or more entity(ies) of interest, where the portion of text includes multiple separable entities describing the relationship and the entity(ies). The multiple separable entities including the one or more entity(ies) of interest and one or more relationship entity(ies). A set of embeddings for each of the separable entities is generated, where the set of embeddings for a separable entity includes an embedding for the separable entity and an embedding for at least one entity associated with the separable entity. One or more composite embeddings may be formed based on at least one embedding from each of the sets of embeddings. The composite embedding(s) may be sent for input to a machine learning model or classifier.