Custom Model Training With Frozen Base Model and Adaptation Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pre-trained base models, such as Grounding DINO, lack specific task knowledge and are costly to train for tasks like circuit substrate defect detection due to their extensive weight parameters, while partial freezing methods compromise performance or cost savings.
Innovation Solution
A custom model is trained using a pre-trained base model with a configuration that includes feature extraction blocks, adaptation blocks, residual units, and a task head block, where only the adaptation and task head blocks are updated through backpropagation, fusing residual and converted features to enhance inference performance without updating base model parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Full Fine-Tuning method is used to train all weight parameters of the base model, then the performance for specific task is improved, but the training cost becomes astronomically expensive
Solution Approach 1:
The model is segmented into three distinct parts: frozen base model (feature extraction), adaptor layers (domain adaptation), and task head (specific task processing). This segmentation allows selective training of only necessary components while freezing the majority of parameters, dramatically reducing training cost while maintaining performance.
Solution Approach 2:
Instead of training all parameters (excessive action), the method trains only the adaptor layers and task head (partial action). This partial training approach is sufficient to achieve task-specific performance while avoiding the astronomical cost of full fine-tuning.
2Use of energy by stationary object
If weight parameters of front layers are frozen to reduce training cost, then training cost is reduced, but the performance is greatly reduced since specific primitive features cannot be generated
Solution Approach 1:
Adaptor layers are introduced as intermediary components between the frozen base model and the task head. These adaptor layers receive features from the frozen base model, perform domain adaptation, and pass transformed features to the task head, enabling specific task performance without training the front layers.
Solution Approach 2:
The method creates a copy of the base model architecture but only instantiates and trains the adaptor layers and task head portions. The frozen base model serves as a pre-trained feature extractor, and the copied/adapted components handle task-specific processing, avoiding the need to train the entire model.
3Use of energy by stationary object
If adaptor layers are added to frozen base model to reduce training cost, then training cost is reduced, but it is difficult to enhance the performance on advanced inference for specific task
Solution Approach 1:
The method transforms the problem from training in the original feature space (frozen base model) to training in an adapted feature space (adaptor layers). By introducing this new dimensional layer for domain adaptation, the system can achieve advanced inference performance without training the base model parameters.
Solution Approach 2:
The model architecture becomes a composite structure combining frozen base model components (providing general features), adaptor layers (providing domain-specific transformation), and task head (providing task-specific processing). This composite architecture enables advanced inference by integrating multiple functional components with different training states.
Data Source
Figure 1(a)~1(d)
Figure 2
Figure 3
AI summary
The present disclosure provides a method for training a custom model based on a pre-trained base model. The custom model is configured to include a base model having at least one pre-trained feature extraction block, at least one adaptation block, at least one residual unit, and a task head block, and update at least part of the task head block, the at least one residual unit and the at least one adaptation block through a backpropagation using a task loss, while not updating parameters of the base model. Further, at least one residual feature (generated through the at least one residual unit) and at least one converted feature (generated by converting the at least one intermediate feature through the at least one adaptation block) are fused to thereby allow a learning device to achieve an advanced inference.