Scene Rectification Via Homography for Retail Product Bounding Boxes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision methods struggle to accurately detect and identify products on retail shelves due to issues such as mutual occlusion, rotation, distortion, and restricted shooting angles, which result in imperfect bounding boxes that include irrelevant background or cut out parts of the products.
Innovation Solution
A system and method that utilize planar homography to correct the pose of objects in images by fitting corner endpoints to different configurations, providing a more frontal view of the objects before determining bounding boxes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If axis-aligned bounding boxes (AABB) are used to detect products in retail scenes, then the detection process is simple and fast, but the bounding boxes cannot be perfectly aligned with products of arbitrary poses, resulting in irrelevant background inclusion or product parts being cut out
Solution Approach 1:
The patent applies pose correction through homography estimation before performing object detection and bounding box generation. By pre-aligning the image to a frontal view of the shelf, the system ensures that subsequent AABB detections are more accurate without requiring complex rotated bounding box algorithms during the main detection process.
Solution Approach 2:
The patent introduces an intermediate homography transformation step that acts as a mediator between the arbitrary-angle input image and the standard AABB detection process. This intermediate step warps the image to a canonical frontal view, allowing simple AABB detectors to achieve better alignment accuracy without directly handling rotated objects.
2Adaptability or versatility
If cameras are mounted to capture retail scenes from various positions, then the system is more flexible and adaptable, but the shooting angles become restricted and products appear at arbitrary poses
Solution Approach 1:
Instead of requiring the camera to be positioned at a specific angle to capture products frontally, the patent inverts the approach by keeping the camera flexible in position but applying a computational transformation (homography) to virtually reposition the view to a frontal perspective. This allows flexible camera mounting while achieving precise product boundary detection.
Solution Approach 2:
The patent changes the parameter space by transforming the image from an arbitrary-angle view to a canonical frontal view through homography estimation. This parameter transformation (change of coordinate system and perspective) allows the system to maintain camera flexibility while achieving consistent, accurate product detection across different shooting angles.
3Measurement precision
If images are captured at restricted angles due to mutual occlusion and rotation, then the system can work with limited camera positions, but the bounding boxes include noisy background context
Solution Approach 1:
The patent performs pose correction through homography estimation before bounding box detection, pre-aligning the image to minimize background inclusion. This preliminary alignment ensures that subsequent AABB detections tightly bound the products without capturing excessive noisy context, improving feature extraction quality for downstream tasks.
Data Source
AI summary
Disclosed herein is a system and method for performing pose-correction on images containing objects within a scene, or the entire scene, to compensate for off-centered camera views. The system and method generate a more frontal view of the object or scene by applying planar homography by identifying corner endpoints of the object or the scene and repositioning the corner endpoints to provide a more frontal view. The pose-corrected scene may then be input to an object detector to determine a location of a bounding box of an object-of-interest which would be more accurate than a bounding box from the original off-centered image.