Multi-Task Network Training for Detection and Segmentation with Missing Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-task networks for semantic segmentation and target detection require high-quality training data, leading to data waste and increased difficulty in training due to missing labels, unbalanced class quantities, and uncertain correlations between subtasks.
Innovation Solution
Implement a method for training a multi-task network that includes feature extraction at different scales, balanced sampling, and a multi-task loss function to adapt to misaligned subtask annotations and incomplete data, using existing data efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multi-task network is trained with synchronous target detection and semantic segmentation, then scene understanding capability is improved, but training data requirement increases
Solution Approach 1:
The training process is segmented into two stages: first training the semantic segmentation subtask, then training the target detection subtask using the pre-trained segmentation model. This segmentation allows each subtask to be trained independently with its own data requirements, rather than requiring all data to meet both subtasks' requirements simultaneously.
Solution Approach 2:
The semantic segmentation subtask is trained in advance before the target detection subtask. The pre-trained segmentation model provides initial features and predictions that are then used to guide the target detection training, reducing the data requirement for the second subtask.
2Measurement precision
If high-quality training data is required for multi-task network, then recognition accuracy is improved, but data annotation difficulty increases
Solution Approach 1:
The annotation requirement is segmented and differentiated for each subtask. Semantic segmentation requires pixel-level annotations while target detection requires object-level annotations. By training subtasks sequentially rather than simultaneously, the system can use data with different annotation types and qualities for different training stages, reducing the overall annotation burden.
Solution Approach 2:
The system accepts and utilizes data with partial annotations - data that may be complete for one subtask but incomplete for another. By training subtasks in sequence and using the output of the first subtask to inform the second, the system can effectively use data that would be insufficient for full multi-task training, thereby reducing the need for complete high-quality annotations across all tasks.
3Productivity
If missing labels in training data are present, then data utilization is reduced, but data waste increases
Solution Approach 1:
The training data is effectively segmented into different usage phases. Data with complete annotations can be used for both subtasks, while data with partial annotations is strategically used for the appropriate subtask training phase. This segmentation of data usage ensures that no data is wasted due to missing labels for tasks it cannot support.
Solution Approach 2:
The system dynamically adapts its data usage strategy based on annotation completeness. During semantic segmentation training, all available data with segmentation labels is used. During target detection training, the system leverages both the pre-trained segmentation model and detection-specific data. This dynamic approach maximizes data utilization across different training phases.
4Quantity of substance
If resources for annotating data are limited, then annotation requirement increases, but training difficulty increases
Solution Approach 1:
The training process is segmented into two distinct phases with different data requirements, allowing limited annotation resources to be allocated strategically to each phase rather than requiring all data to meet both phases' requirements simultaneously. This reduces the total annotation burden while maintaining training effectiveness.
Solution Approach 2:
By performing semantic segmentation training in advance with potentially fewer resources, the system creates a solid foundation that simplifies the subsequent target detection training. This preliminary action reduces the overall training difficulty by breaking down a complex multi-task training problem into two more manageable sequential tasks.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Embodiments of the present disclosure disclose a method for training a multi-task network, and a related device. The method includes: obtaining a data training set, the data training set including a plurality of images; performing feature extraction on the plurality of images in the data training set, to obtain a first feature and a second feature; determining a comprehensive feature based on the first feature and the second feature; determining a target-detection prediction result based on the second feature; determining a semantic-segmentation prediction result based on the comprehensive feature and the target-detection prediction result; and determining a multi-task loss function based on the target-detection prediction result and the semantic-segmentation prediction result, and optimizing training of the multi-task network based on the multi-task loss function. In the present disclosure, a function of multi-task information exchange can be implemented based on a training framework of the multi-task network, so that the multi-task network implements accurate recognition in a lack of data annotation.