Multi-Scale Hard Example Mining for CNN Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object detection methods using neural networks face challenges due to the imbalance between positive and negative examples in training sets, and are limited by the sensitivity of feature maps to object sizes, leading to suboptimal performance in recognizing objects across various scales.
Innovation Solution
The proposed solution involves a multi-scale hard example mining technique that enriches feature maps by using combinations of multiple layers, selecting hard examples within each mini-batch for feature maps of different scales, and combining these results to improve the robustness of object detection systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional object detection methods are used with imbalanced training sets, then the system can process images quickly, but the detection accuracy deteriorates due to the imbalance between positive and negative examples
Solution Approach 1:
The patent segments the training process by separately identifying and processing hard positive examples and hard negative examples through distinct mining processes. Hard positives are mined from regions containing objects, while hard negatives are mined from background regions, allowing targeted optimization of detection accuracy without requiring a uniformly large training set
Solution Approach 2:
The patent performs preliminary hard example mining to identify challenging positive and negative samples before the main training process. This preliminary action creates a curated set of difficult examples that are then used to train the detector, improving accuracy by focusing computational resources on the most informative samples rather than processing all examples equally
2Adaptability or versatility
If feature maps are used for object detection, then the system can identify objects efficiently, but the detection performance deteriorates when objects vary in size due to sensitivity to scale
Solution Approach 1:
The patent introduces a scale dimension by processing feature maps at multiple resolutions. Instead of relying on a single feature map scale, the system extracts hard examples from feature maps at different scales, enabling the detector to learn scale-invariant features and improve accuracy across objects of varying sizes
Solution Approach 2:
The patent changes the parameter of feature map resolution by generating and processing feature maps at multiple scales. This parameter change allows the system to capture objects at different sizes effectively, as each scale's feature maps are optimized for detecting objects of corresponding sizes, thereby improving overall scale invariance
3Measurement precision
If hard example mining is applied to improve detection accuracy, then the system can achieve better precision, but the training process complexity increases
Solution Approach 1:
The patent extracts only the most informative and challenging examples from the training data through hard mining processes. By taking out and focusing on hard positives and hard negatives separately, the system achieves high detection accuracy with a smaller, more targeted training set, reducing the overall complexity of the training process compared to using all examples
Data Source
AI summary
An example apparatus for mining multi-scale hard examples includes a convolutional neural network to receive a mini-batch of sample candidates and generate basic feature maps. The apparatus also includes a feature extractor and combiner to generate concatenated feature maps based on the basic feature maps and extract the concatenated feature maps for each of a plurality of received candidate boxes. The apparatus further includes a sample scorer and miner to score the candidate samples with multi-task loss scores and select candidate samples with multi-task loss scores exceeding a threshold score.