Multi-Task Encoder Consistency Learning for Partial Annotations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-task learning methods face challenges in maintaining performance stability when dealing with partially annotated data and tasks with different input types, often leading to biased learning towards tasks with ground truth data.
Innovation Solution
A multi-task learning method that utilizes pretrained encoders with the same structure for both tasks, generates consistency losses based on scaled features, and learns the multi-task encoder and decoders while fixing the pretrained encoder parameters, ensuring balanced performance across tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If multi-task learning is performed using a single network, then computing resource efficiency is improved, but learning performance becomes biased towards tasks with ground truth data
Solution Approach 1:
The patent introduces consistency loss as an intermediary mechanism that mediates between tasks with and without ground truth data. The consistency loss compares features from the multi-task encoder with features from a single-task encoder, serving as a bridge to guide the multi-task encoder when ground truth is unavailable, thus preventing learning bias while maintaining computing efficiency
Solution Approach 2:
The patent changes the loss function parameters by adding consistency loss (L_consis) to the traditional ground truth loss (L_gt). This parameter change allows the learning process to incorporate feature consistency constraints, balancing the optimization between tasks with and without ground truth data while using the same network
2Quantity of substance
If partially annotated data is used for multi-task learning, then data efficiency is improved, but learning stability deteriorates due to bias towards tasks with ground truth
Solution Approach 1:
The patent implements a feedback mechanism where the consistency loss continuously monitors and compares feature representations between multi-task and single-task encoders. This feedback loop provides corrective signals to the multi-task encoder during training, ensuring stable learning even when ground truth data is incomplete or biased towards certain tasks
3Adaptability or versatility
If different input types are processed through a single network, then versatility is improved, but feature consistency becomes difficult to maintain
Solution Approach 1:
The patent designs a universal multi-task encoder that processes different input types (RGB images, depth maps, normal maps) through the same network architecture. This universal encoder extracts features consistent across different modalities, maintaining feature consistency while achieving versatility in handling diverse input types
Data Source
AI summary
A multi-task learning method according to an embodiment of the present disclosure may include generating, by a generation device, a first feature based on a first input image through a multi-task encoder, generating, by the generation device, a first output image based on the first feature through a first decoder for a first task, generating, by the generation device, a first loss based on the first output image and a first ground truth (GT) for the first task, generating, by the generation device, a second feature based on the first input image through a pretrained first encoder for a second task, generating, by the generation device, a second loss based on the first feature and the second feature, and learning, by a learning device, the multi-task encoder and the first decoder based on the first loss and the second loss.