Occluded Instance Segmentation via Perspective Mask Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image recognition technologies struggle to accurately deduce the invisible parts of instances that are occluded in images, which is crucial for tasks like self-driving vehicle navigation.
Innovation Solution
A large-scale perspective instance dataset is finely annotated with semantic labels, perspective masks, and relative occlusion orders, and a multitasking framework with multi-branch coding is proposed to infer the invisible parts of occluded instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional instance segmentation is used to obtain visible parts of instances, then segmentation of visible areas is achieved, but invisible occluded parts cannot be deduced
Solution Approach 1:
The patent applies preliminary action by training the neural network model in advance with specially constructed training data that includes occlusion relationships. The model learns to predict invisible parts before actual segmentation is performed, enabling it to deduce occluded regions without additional computation during inference.
Solution Approach 2:
The patent introduces an intermediary mechanism by using a neural network model as a mediator between visible instance parts and invisible occluded parts. The model acts as an intelligent intermediary that infers missing information based on learned patterns from training data, bridging the gap between visible and invisible regions.
2Measurement precision
If a multitasking framework with multi-branch coding is proposed to infer invisible parts, then occluded instance prediction is enhanced, but computational complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the neural network into multiple specialized branches, each responsible for specific tasks such as instance segmentation, occlusion detection, and visibility prediction. This modular segmentation allows each branch to focus on specific aspects of the problem, improving overall accuracy while maintaining manageable complexity through functional specialization.
Solution Approach 2:
The patent implements universality by designing a multitasking framework where a single neural network model performs multiple functions simultaneously: instance segmentation, occlusion detection, visibility prediction, and incomplete instance completion. This multi-functionality reduces the need for separate models and optimizes computational resources by sharing features across tasks.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
Disclosed is an image processing method. The method comprises: acquiring an image to be predicted, and performing feature extraction on the image to be predicted; generating at least one candidate area of the image to be predicted, and mapping the at least one candidate area to a feature map of the image to be predicted, wherein one candidate area includes one instance; and processing the mapped feature map on the basis of a target network, using a perspective mask to indicate an overall structure of a shielded instance in the image to be predicted, and using a non-perspective mask to indicate an invisible part of the shielded instance, wherein the perspective mask and the non-perspective mask represent a relative shielding relationship of the shielded instance.