Latent Heatmap Training for Occluded Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) used in object detection for autonomous driving struggle with challenging data samples, such as occluded objects, leading to missed detections and hazardous events.
Innovation Solution
A method for training neural networks for object detection that involves generating latent heatmaps from neural network activations, calculating a heatmap loss, and backpropagating this loss for training. This approach aims to produce meaningful activation maps with minimal noise, improving the detection of occluded objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standard object detection methods are used, then detection speed is maintained, but detection precision for occluded objects deteriorates
Solution Approach 1:
The patent introduces latent heatmaps as an intermediary representation between the input image and final detection output. These heatmaps serve as a mediator that highlights potential object locations and guides the detection process, improving both precision and reliability for occluded objects without sacrificing detection speed
Solution Approach 2:
The detection process is segmented into multiple components: backbone feature extraction, latent heatmap generation, and detection head processing. This segmentation allows each component to specialize - the backbone handles general feature extraction while the latent heatmap specifically addresses occluded object detection, improving overall reliability
2Measurement precision
If more complex detection algorithms are applied, then detection precision improves, but computational complexity increases
Solution Approach 1:
The patent merges the latent heatmap generation with the existing backbone feature extraction process. By reusing features from the backbone and integrating heatmap generation within the same computational framework, the system improves detection precision without proportionally increasing computational complexity
Solution Approach 2:
Instead of completely redesigning the detection algorithm, the patent applies partial action by adding latent heatmap generation to specific parts of the existing pipeline. This selective enhancement improves precision for critical cases (occluded objects) while maintaining efficiency for standard detection scenarios
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The present invention is related to a method, a computer program, and an apparatus for training a neural network for object detection, and to a respective trained neural network. The invention is further related to a method, a computer program, and an apparatus for object detection with a trained neural network, and to a vehicle equipped with a perception system, which makes use of such a method or apparatus. In a first step, an input image is received (10), which is then processed (11) with the neural network. Latent activations for the image are extracted (12) from one or more layers of the neural network and a latent heatmap is generated (13) from these latent activations. If not yet available, a ground truth mask for the input image may be generated (14) by converting bounding boxes representing object instances into a binary mask. A heatmap loss between the latent heatmap and the ground truth mask is then determined (15). At least the heatmap loss is backpropagated (16) for training the neural network.