Instance Segmentation via Affinity Graph Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing instance segmentation methods face challenges with complex occlusions, difficulty in detecting non-rigid shapes, accidental splits and merges, slow convergence during training, and long computational duration, particularly in image processing for self-driving vehicles and automated systems.
Innovation Solution
The method involves using a first neural network to generate affinity vectors for each pixel, forming an affinity graph, and a second neural network to expand a seed-map using this graph, ensuring accurate instance segmentation by iteratively determining pixel affiliations and producing a binary mask, with the option of using a deep neural network and a recurrent neural network for processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional instance segmentation methods are used, then object detection can be performed, but accuracy deteriorates due to complex occlusions, non-rigid shapes, and accidental splits and merges
Solution Approach 1:
The method segments the instance segmentation task into two distinct stages: (1) generating an affinity graph that captures pairwise relationships between pixels, and (2) propagating instance labels from seed pixels through the affinity graph. This two-stage segmentation approach allows the system to handle complex occlusions and non-rigid shapes more effectively by decoupling affinity computation from label propagation.
Solution Approach 2:
The affinity graph serves as an intermediary structure between the input image and the final instance segmentation mask. Instead of directly segmenting pixels, the system first computes affinity relationships between pixel pairs and stores them in the affinity graph, then uses this graph as a mediator to propagate instance labels from seed pixels. This intermediary representation captures spatial and semantic relationships that improve segmentation reliability under occlusion.
2Measurement precision
If deep neural networks are used for instance segmentation, then detection capability improves, but computational duration increases
Solution Approach 1:
The computational task is segmented into two efficient stages: affinity graph generation using a lightweight neural network, and label propagation through the affinity graph using iterative relaxation. This segmentation avoids the need for complex end-to-end deep networks while maintaining detection accuracy, thereby reducing computational duration.
Solution Approach 2:
Instead of computing affinities for all pixel pairs in the image (which would be computationally expensive), the system computes affinities only for pixel pairs that are spatially close or potentially related based on local image features. This partial computation approach maintains detection accuracy while significantly reducing computational duration.
3Measurement precision
If comprehensive pixel relationship analysis is performed, then segmentation accuracy improves, but device complexity increases
Solution Approach 1:
The complex task of determining pixel affiliations is segmented into two simpler sub-tasks: (1) computing local affinity scores between pixel pairs based on image features, and (2) propagating instance labels through the affinity graph using iterative relaxation. This segmentation reduces the complexity of the neural network architecture while maintaining comprehensive pixel relationship analysis.
Solution Approach 2:
The affinity graph and label propagation mechanism serve themselves by automatically capturing pixel relationships and propagating instance labels without requiring complex external processing. The iterative relaxation algorithm self-adjusts the label assignments based on the affinity relationships, reducing the need for complex device architecture.
Data Source
AI summary
A system and a method for processing an image so as to perform instance segmentation. The system/method includes: a—inputting (S1) the image (IMG) to a first neural network configured to output an affinity graph (AF), and b—inputting (S2), to a second neural network, the affinity graph and a predefined seed-map (SM), so as to determine whether other pixels belong to a same instance, and set at a first value the value of the other pixels determined as belonging to the same instance.