Neural Network Object Detection via Bipartite Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural networks face challenges in accurately predicting multiple bounding boxes in images and generating confidence scores that reflect the likelihood of object presence, while being agnostic to object categories, with existing methods not effectively minimizing localization and confidence loss terms.
Innovation Solution
The method involves training a deep neural network using a system that receives training images with object location data, generates candidate bounding boxes with confidence scores, and determines optimal assignments through bipartite matching to minimize a loss function combining localization and confidence loss terms, allowing for accurate prediction of bounding boxes and confidence scores regardless of object category.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional object detection methods are used, then the system can identify objects in images, but the accuracy of predicting multiple bounding boxes and generating confidence scores is insufficient
Solution Approach 1:
The patent segments the object detection task into distinct components: generating multiple candidate bounding boxes, assigning object locations to candidate boxes, and calculating confidence scores. This segmentation allows each component to be optimized independently, improving overall accuracy without proportionally increasing complexity.
Solution Approach 2:
The patent introduces new parameters including confidence scores for each candidate bounding box and uses bipartite matching to optimize assignments between object locations and candidate boxes. By changing the parameter space to include these additional dimensions, the system achieves more precise bounding box predictions and reliable confidence scoring.
2Reliability
If the neural network is trained to predict multiple bounding boxes, then object detection capability is enhanced, but the loss function minimization becomes more difficult
Solution Approach 1:
The patent introduces bipartite matching as an intermediary step between generating candidate bounding boxes and calculating final loss. This intermediary optimization layer assigns object locations to the most appropriate candidate boxes, making the loss function minimization more tractable by reducing the search space and providing better initialization for gradient-based optimization.
Solution Approach 2:
The patent performs preliminary action by generating multiple candidate bounding boxes and computing confidence scores before the actual training optimization. This preliminary structuring of the prediction space allows the loss function to work with pre-organized data, reducing the difficulty of optimization during training.
3Measurement precision
If confidence scores are generated for each candidate bounding box, then detection accuracy improves, but computational complexity increases
Solution Approach 1:
The patent generates multiple candidate bounding boxes (excessive action) but only assigns a subset of them to actual object locations through bipartite matching (partial action). This approach allows confidence scores to be computed for all candidates while ensuring computational resources are focused on the most relevant assignments, balancing accuracy with energy consumption.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to detect object in images. One of the methods includes receiving a training image and object location data for the training image; providing the training image to a neural network and obtaining bounding box data for the training image from the neural network, wherein the bounding box data comprises data defining a plurality of candidate bounding boxes in the training image and a respective confidence score for each candidate bounding box in the training image; determining an optimal set of assignments using the object location data for the training image and the bounding box data for the training image, wherein the optimal set of assignments assigns a respective candidate bounding box to each of the object locations; and training the neural network on the training image using the optimal set of assignments.