Double-Embedding Segmentation for Multi-Camera Road User Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional instance segmentation methods face challenges in precisely recognizing object classes and extracting edge information, particularly when these processes are disjoint and performed separately, leading to poor performance in detecting road user events like pedestrian and vehicle events.
Innovation Solution
A learning method employing a double embedding configuration, involving similarity and distance embedding processes, is used to generate instance segmentation results without a clustering process. This method includes acquiring network output features, applying similarity and distance convolution operations, and backpropagating similarity and distance losses to refine the neural network parameters, thereby enhancing inter-class differences and reducing intra-class variance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional instance segmentation methods are used to detect objects in images, then the segmentation process can be performed, but the precision of recognizing object classes and extracting edge information deteriorates due to disjoint processes
Solution Approach 1:
The patent combines class recognition and edge extraction into a unified instance segmentation process. The encoder-decoder architecture integrates feature extraction, class identification, and boundary detection in a single coherent framework, eliminating the disjoint processes of conventional methods and improving recognition precision while maintaining manageable system complexity.
Solution Approach 2:
The patent employs encoder-decoder segmentation to divide the image processing task into distinct functional stages: the encoder extracts hierarchical features and identifies object classes, while the decoder reconstructs precise edge information and generates segmentation masks. This structured segmentation of the processing pipeline enables both accurate class recognition and sharp edge extraction.
2Measurement precision
If conventional instance segmentation methods are used to detect objects in images, then the segmentation process can be performed, but the precision of extracting edge information deteriorates due to disjoint processes
Solution Approach 1:
The patent merges edge extraction with class recognition in a unified instance segmentation framework. The decoder component specifically targets edge information recovery by reconstructing boundary features from encoded representations, ensuring precise edge extraction is achieved simultaneously with class identification rather than as a separate post-processing step.
Solution Approach 2:
The encoder-decoder architecture segments the processing function where the encoder focuses on semantic feature extraction and class identification, while the decoder specializes in spatial reconstruction and edge refinement. This functional segmentation allows each component to optimize for its specific task, improving edge extraction precision without proportionally increasing overall system complexity.
3Device complexity
If disjoint processes are used for class recognition and edge extraction, then the processing can be simplified, but the performance in detecting road user events deteriorates
Solution Approach 1:
The patent merges class recognition and edge extraction into a single integrated instance segmentation process that jointly optimizes both functions. This unified approach ensures that class labels and boundary information are derived from the same feature representations, improving the reliability of road user event detection by ensuring consistency between what objects are identified and where their boundaries are located.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A learning method for improving segmentation performance to be used for detecting road user events including pedestrian events and vehicle events using double embedding configuration in a multi-camera system is provided. The learning method includes steps of: a learning device instructing similarity convolutional layer to generate similarity embedding feature by applying similarity convolution operations to a feature outputted from a neural network; instructing similarity loss layer to output a similarity loss by referring to a similarity between two points sampled from the similarity embedding feature, and its corresponding GT label image; instructing distance convolutional layer to generate distance embedding feature by applying distance convolution operations to the similarity embedding feature; instructing distance loss layer to output a distance loss for increasing inter-class differences among mean values of instance classes and decreasing intra-class variance values of the instance classes; backpropagating at least one of the similarity loss and the distance loss.