Dynamic Embedding Training for AI Model Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems struggle to handle new information and nodes or relations introduced after the initial training, leading to reduced performance and the need for re-training from scratch, which is resource-intensive and inefficient.
Innovation Solution
A dynamic embedding-based machine learning training mechanism that allows for continuous training of AI models by generating new computational graphs with updated embedding dimensions, migrating weights from initial to new optimizers, and iteratively refining embeddings with new data sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the AI model is re-trained from scratch to handle new information, then the model can incorporate new nodes or relations, but the computational resources and time required increase significantly
Solution Approach 1:
The patent performs preliminary actions by pre-processing new data to identify new nodes and relations before full model re-training. This allows the system to prepare embedding updates and structural modifications in advance, reducing the actual re-training time when new information needs to be incorporated.
Solution Approach 2:
The patent segments the re-training process into distinct phases: identifying new nodes and relations, updating computational graphs, modifying embedding dimensions, and performing selective re-training. This segmentation allows only the affected portions of the model to be re-trained rather than the entire model, significantly reducing computational resources and time.
2Adaptability or versatility
If the AI model is re-trained from scratch to handle new information, then the model can incorporate new nodes or relations, but the computational resources required increase significantly
Solution Approach 1:
The patent segments the re-training process into distinct phases: identifying new nodes and relations, updating computational graphs, modifying embedding dimensions, and performing selective re-training. This segmentation allows only the affected portions of the model to be re-trained rather than the entire model, significantly reducing computational resources and time.
Solution Approach 2:
The patent dynamically changes model parameters such as embedding dimensions and computational graph structure to accommodate new information. By adjusting these parameters selectively rather than re-training with fixed parameters, the system reduces the computational burden while maintaining adaptability to new data.
3Stability of the object's composition
If the embedding dimensions are fixed during training, then the model structure remains stable, but the model cannot adapt to new nodes or relations with different dimensions
Solution Approach 1:
The patent implements dynamic embedding dimensions that can change during the model's lifecycle. The system allows embedding dimensions to be modified when new nodes or relations are introduced, transitioning from a static to a dynamic structure. This enables the model to adapt to varying data dimensions while maintaining structural integrity through controlled updates to the computational graph.
Data Source
AI summary
A computer-implemented, machine learning method for a dynamic embedding-based machine learning training mechanism includes training, using an initial optimizer, an embedding-based neural network model based on a dataset to generate an initial computational graph having trainable variables. Based on receiving a new dataset: a new computational graph is generated, instantiated with new embedding dimensions migrated from the initial computational graph; a new optimizer is generated based on a weight matrix that fits to the trainable variables of the new computational graph; and weights of the trainable variables from the initial optimizer are migrated to the new optimizer. The embedding-based neural network model is trained with the new dataset by updating embeddings and learning new embeddings of the new dataset. The present invention can be used in a variety of applications including, but not limited to, several anticipated use cases in drug development, public safety, and medical/healthcare.