Deformable-DETR Feature Map Selection for Real-Time Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deformable-DETR object detection models require high computational load, leading to increased latency and are less suitable for real-time or near real-time object detection tasks.
Innovation Solution
Optimize object detection models by reducing the number of feature maps used to generate a vector input into the transformer, using a select combination of feature maps, such as the lowest and highest resolution maps, to reduce computational resources and processing time while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all feature maps are used to generate the vector input into the transformer, then detection accuracy is improved, but computational load increases leading to increased latency
Solution Approach 1:
The patent extracts and uses only a select subset of feature maps (specifically the lowest resolution and highest resolution feature maps) to generate the vector input into the transformer, rather than using all feature maps. This extraction principle reduces the computational load and latency while maintaining detection accuracy by focusing on the most informative feature maps.
Solution Approach 2:
The patent applies partial action by using only a portion of the available feature maps (specifically 2 out of 4 feature maps) to generate the vector. This partial usage reduces computational requirements and latency while still achieving the necessary detection accuracy for real-time object detection tasks.
2Productivity
If a select number of feature maps are used to generate the vector, then computational requirements are reduced, but detection accuracy may be compromised
Solution Approach 1:
The patent applies local quality by selectively choosing specific feature maps (lowest resolution and highest resolution) that have different local characteristics and information content. This strategic selection ensures that the most informative feature maps are used, maintaining detection accuracy while reducing computational requirements.
Solution Approach 2:
The patent changes the parameter of feature map selection by using a specific configuration (lowest and highest resolution feature maps) rather than all feature maps. This parameter change optimizes the balance between computational efficiency and detection accuracy for real-time streaming applications.
3Measurement precision
If traditional deformable-DETR models are used, then high detection accuracy is achieved, but real-time performance is not suitable due to high computational load
Solution Approach 1:
The patent extracts and uses only a select subset of feature maps (specifically the lowest resolution and highest resolution feature maps) to generate the vector input into the transformer, rather than using all feature maps. This extraction principle reduces the computational load and latency while maintaining detection accuracy by focusing on the most informative feature maps.
Solution Approach 2:
The patent applies partial action by using only a portion of the available feature maps (specifically 2 out of 4 feature maps) to generate the vector. This partial usage reduces computational requirements and latency while still achieving the necessary detection accuracy for real-time object detection tasks.
Data Source
AI summary
In various examples, techniques for optimizing object detection models are described herein. Systems and methods are disclosed that process sensor data using a backbone of a machine learning model(s) in order to generate feature maps at different resolutions. The systems and methods then use the machine learning model(s) to generate a vector based at least in part on one or more of the feature maps. For example, if the backbone generates four feature maps, then the machine learning model(s) may generate the vector using two feature maps from the four feature maps. The systems and methods then process the vector using a transformer of the machine learning model(s) in order to generate data representing a class label(s) for an object(s) depicted by an image represented by the sensor data and/or a location(s) of the object(s) within the image.