Dual-Head Neural Network Training for Target Label Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing domain adaptation methods in transfer learning fail to prioritize target labels over source labels, leading to reduced accuracy in classification tasks due to equal mixing of target and source labels, especially when source data is abundant.
Innovation Solution
A neural network model is split into two heads, where the second head prioritizes target labels by updating parameters based on calculated consistency between the first and second head results, using methods like Jensen-Shannon divergence or mean square error to ensure target data is prioritized during training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If naive mixing of target and source labels is used, then the training process is simple, but the network cannot prioritize target label patterns leading to reduced accuracy
Solution Approach 1:
The neural network is divided into two separate heads: a source head for processing source domain data and a target head for processing target domain data. This segmentation allows each head to specialize in its respective domain while sharing the same feature extractor, enabling the network to prioritize target labels without complex training procedures
Solution Approach 2:
Different parts of the network (source head and target head) are given different functions and training objectives. The source head focuses on source domain patterns while the target head focuses on target domain patterns, allowing local optimization for each domain while maintaining overall system coherence through shared features
2Quantity of substance
If more source labels are available, then the source domain training is robust, but the target domain accuracy suffers due to label mixing
Solution Approach 1:
By segmenting the network into separate source and target heads, the patent enables the network to process abundant source data through the source head without it interfering with target domain learning. The target head receives only target domain signals, ensuring that even with limited target data, the network can prioritize and accurately learn target patterns
Solution Approach 2:
The shared feature extractor acts as an intermediary between the source and target heads. It learns representations from both domains but provides domain-specific outputs to the respective heads, allowing the network to leverage source data abundance while maintaining target domain accuracy through the mediating feature representation
Data Source
AI summary
The invention relates to a computer-implemented method (100) for configuring a neural network model, wherein the method comprises the following steps: providing (102) a neural network model; splitting (104) the neural network model into a first portion and a second portion, the second portion comprising a first head for classifying a first type of classification data and a second head for classifying the second type of classification data; pre-processing (106), in a training phase, the second type of classification data in the first portion, processing (108) the pre-processed second type of classification data in the first and second heads and determining a first result of the processing of first type of classification data in the first head and a second result of the processing of first type of classification data in the second head; calculating (110) the consistency between the first result and the second result; and configuring (112) the neural network model by updating a value of at least one parameter of the second head based on the calculated consistency.