AI Training Method Merging Feature Values to Reduce Retraining Time
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional AI training methods require substantial time for retraining AI models due to changes in configuration or conditions, even when the model is used in similar environments, which hampers efficiency.
Innovation Solution
An AI training method that merges trained-layer feature values from existing and training-target AI models to generate a new trained AI model, reducing the need for extensive retraining by using a cross-attention mechanism or averaging/suming feature values to expedite the training process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional AI training methods are used to train a new model from scratch, then the model can be trained to achieve required recognition accuracy, but the training time becomes substantially long
Solution Approach 1:
The patent applies preliminary action by pre-training a source AI model on source domain data before using it to train the target AI model. The source model's learned features and weights are transferred to the target model, so that when training begins on target domain data, the model already has a foundation of learned representations. This preliminary training on source data eliminates the need to learn basic features from scratch during target domain training, substantially reducing training time while maintaining required recognition accuracy.
Solution Approach 2:
The patent implements copying by creating a target AI model that replicates the architectural structure and initial parameters of a source AI model. The target model copies the source model's layer configurations, activation functions, and weight initialization schemes. This copying allows the target model to inherit the source model's learned representations and training dynamics, enabling rapid adaptation to the target domain without relearning fundamental patterns, thus reducing training time while preserving accuracy.
2Adaptability or versatility
If the AI model configuration or hyperparameters are changed, then the model can be adapted to new conditions, but retraining from scratch is required which increases training time
Solution Approach 1:
The patent applies dynamics by implementing a dynamic training approach where the model can switch between using pre-trained source domain features and learning from target domain data. The training system dynamically adjusts the degree of feature extraction and fine-tuning based on the specific adaptation requirements. When configuration changes are needed, the system can leverage the dynamic nature of the pre-trained model to quickly adapt to new conditions without complete retraining, reducing time loss while maintaining adaptability.
Solution Approach 2:
The patent implements parameter changes by modifying hyperparameters and model configurations during the fine-tuning phase rather than requiring complete retraining. The system changes parameters such as learning rates, batch sizes, and regularization strengths to optimize performance on target domain data while preserving the foundational weights from source domain training. This selective parameter adjustment enables model adaptation to new conditions with minimal time investment compared to full retraining.
3Loss of time
If feature values from trained models are merged with training-target layer feature values, then training time is reduced, but the complexity of the training process increases
Solution Approach 1:
The patent applies segmentation by dividing the training process into distinct phases: source domain pre-training phase, feature extraction phase, and target domain fine-tuning phase. The segmentation separates the complex tasks of learning general features from learning domain-specific features, allowing each phase to be optimized independently. During target domain training, only the final fine-tuning phase is executed, which is simpler and faster because the heavy lifting of feature learning was completed during the source domain pre-training phase. This segmentation reduces overall training time while managing process complexity through structured organization.
Solution Approach 2:
The patent uses an intermediary approach by introducing a feature extraction layer that acts as a mediator between the source domain pre-trained model and the target domain training process. This intermediary layer extracts and merges feature values from the source model with the target model's layer outputs, creating a bridge that enables knowledge transfer without requiring direct integration of the entire source model. The intermediary simplifies the training process by providing a standardized interface for feature merging, reducing complexity while achieving time savings through efficient feature utilization.
Data Source
AI summary
An AI training method includes inputting training data to an existing trained AI model to acquire a trained-layer feature value output from a trained layer in the existing trained AI model, merging the trained-layer feature value with a training-target layer feature value output from a training-target layer in the training-target AI model to generate a merged feature value, and inputting the merged feature value to a training-target layer subsequent to the training-target layer to generate a new trained AI model.


