Object Detection via NMS and Mean Shift Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object detection and tracking methods in image data, such as non-maxima suppression (NMS) and mean shift grouping, face challenges in achieving both accurate identification and real-time computational efficiency, with NMS being suitable for single-frame detections but lacking temporal consistency, and mean shift being computationally inefficient for multi-frame tracking.
Innovation Solution
The proposed solution combines initial non-maxima suppression (NMS) grouping with subsequent mean shift clustering to identify and track objects across multiple frames, using a multistep process that first determines initial clusters with NMS, then refines them using mean shift processing for spatial and temporal consistency, and further refines with NMS to remove overlapping detection windows.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If non-maxima suppression (NMS) grouping is used for single frame detections, then detection accuracy is improved, but temporal consistency deteriorates causing jittery and lacking smooth movement in multi-frame tracking
Solution Approach 1:
The patent segments the window grouping process into two distinct phases: first applying NMS grouping to achieve accurate object detection, then applying mean shift clustering to refine group locations and ensure temporal consistency. This segmentation allows each method to excel at its strengths while compensating for the other's weaknesses.
Solution Approach 2:
The patent performs preliminary NMS grouping to identify candidate object locations before applying mean shift clustering. This preliminary action reduces the computational complexity of the subsequent mean shift process by limiting it to only the most relevant candidate windows, while still achieving smooth temporal tracking.
2Stability of the object's composition
If mean shift grouping is used to achieve good temporal consistency, then smooth movement is improved, but computational efficiency deteriorates requiring many iterative techniques
Solution Approach 1:
Instead of applying mean shift clustering to all detected windows, the patent applies it only to the subset of windows that survive the initial NMS grouping. This partial action significantly reduces computational requirements while maintaining the temporal consistency benefits of mean shift for the most relevant objects.
Solution Approach 2:
The patent performs preliminary NMS grouping to reduce the number of candidate windows before applying the computationally intensive mean shift clustering. This preliminary reduction in data volume makes the subsequent mean shift process feasible for real-time applications while preserving smooth temporal tracking.
3Measurement precision
If iterative mean shift clustering is applied to all detection windows, then accurate object identification is improved, but processing time increases making real-time applications difficult
Solution Approach 1:
The patent segments the processing pipeline into two stages: a fast NMS grouping stage that quickly identifies candidate objects, followed by a more accurate but slower mean shift clustering stage applied only to those candidates. This segmentation achieves accurate object identification while minimizing overall processing time.
Solution Approach 2:
The patent applies the computationally expensive iterative mean shift clustering only to a partial set of detection windows that have already been filtered by NMS grouping. This selective application maintains high object identification accuracy for relevant objects while dramatically reducing total processing time.
Data Source
AI summary
Disclosed examples include image processing methods and systems to process image data, including computing a plurality of scaled images according to input image data for a current image frame, computing feature vectors for locations of the individual scaled images, classifying the feature vectors to determine sets of detection windows, and grouping detection windows to identify objects in the current frame, where the grouping includes determining first clusters of the detection windows using non-maxima suppression grouping processing, determining positions and scores of second clusters using mean shift clustering according to the first clusters, and determining final clusters representing identified objects in the current image frame using non-maxima suppression grouping of the second clusters. Disclosed examples also include methods and systems to track identified objects from one frame to another using feature vectors and overlap of identified objects between frames to minimize computation intensive operations involving feature vectors.