AI Model Domain Adaptation Using a New Covariates Block
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models for forecasting future values of a target variable face challenges due to large numbers of neural network parameters, particularly when training datasets are limited, leading to underfitting and unsatisfactory forecasts, especially when covariates influencing the target variable are not adequately considered.
Innovation Solution
Adapting an AI model to a new domain by replacing the covariates block with a new covariates block specific to the new domain, training this new block using a new-domain-specific dataset, and fine-tuning the main block of the AI model using the same dataset to accommodate the covariates of the new domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the AI model uses a large number of neural network parameters to forecast future values, then the model capacity increases, but the model underfits the training data when the training dataset size is limited
Solution Approach 1:
The patent segments the AI model into distinct components: a pre-trained main block and a domain-specific adaptation block. This segmentation allows the model to have large capacity through the pre-trained block while avoiding underfitting by using a smaller, focused adaptation block that specializes in domain-specific patterns without requiring extensive training data.
Solution Approach 2:
The patent applies preliminary action by pre-training the main block on a source domain before adapting it to the target domain. This pre-training establishes a solid foundation of general forecasting capabilities, allowing the model to start with learned patterns and only require minimal domain-specific adjustments, thereby avoiding underfitting when target domain data is limited.
2Loss of time
If the AI model is pre-trained on a source domain, then the model can leverage existing patterns, but the model fails to capture domain-specific characteristics of the target domain
Solution Approach 1:
The patent divides the model into a reusable pre-trained main block and a domain-specific adaptation block. This segmentation enables the main block to retain general forecasting knowledge from source domain pre-training while the adaptation block captures target domain characteristics, resolving the conflict between training efficiency and domain adaptability.
Solution Approach 2:
The patent applies local quality by making only the adaptation block domain-specific while keeping the main block general. This allows the model to efficiently reuse pre-trained components across domains while locally adapting only the necessary parts to capture domain-specific characteristics, balancing training efficiency with adaptability.
3Device complexity
If the AI model uses many parameters, then the model can represent complex patterns, but the model requires large training datasets to prevent underfitting
Solution Approach 1:
The patent segments the complex model into a large pre-trained main block and a smaller domain-specific adaptation block. This segmentation allows the system to use high model complexity only where necessary (in the pre-trained block with access to source domain data) while keeping the target domain adaptation block simpler, thereby reducing the overall training data size requirement for the target domain.
Solution Approach 2:
The pre-trained main block serves as a universal component that can be reused across multiple domains. This multi-functionality allows the same complex model structure to be deployed in different domains without requiring full retraining, thereby reducing the total training data size needed across multiple applications.
Data Source
AI summary
A method for adapting to a new domain an AI model pre-trained for a current domain. Having at least one main block of the AI model for modeling a target variable and having at least one covariates block of the AI model for modeling covariates effect on the target variable in the current domain, the method comprises: replacing the covariates block with a new covariates block adapted to the new domain, the new covariates block modifying one or more first layers compared to the covariate block, the target variable in the new domain being affected differently by at least one of the one or more covariates; training the new covariates block of the AI model using a new-domain-specific dataset from the new domain; and fine-tuning the at least one main block of the AI model using the new-domain-specific dataset from the new domain.