Retail Product Detection With Quadrilateral Boxes and Text Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing retail scene product recognition (SPR) systems face challenges due to densely-packed, low-shot, and fine-grained product characteristics, leading to improper image alignment, occlusion, and inconsistent recognition, especially with occlusions and text variations, which degrade detection and classification performance.
Innovation Solution
The United Retail Datasets (Unitail) provides quadrilateral annotations for product detection and text recognition, using RetailDet for quadrilateral product detection and spatially encoded text features with the Hungarian Algorithm for matching, enhancing detection and recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If axis-aligned rectangular bounding boxes (AABB) are used for product detection, then detection coverage is simplified, but alignment accuracy deteriorates due to occlusion and product orientation variations
Solution Approach 1:
The patent transitions from symmetric axis-aligned rectangular bounding boxes to asymmetric rotated rectangular bounding boxes (RBOX) that can be rotated to match the actual orientation of products. This allows the detection boxes to align with products at various angles, significantly improving alignment accuracy while maintaining manageable computational complexity through efficient rotation transformation algorithms.
Solution Approach 2:
The patent introduces an additional rotational dimension to the bounding box representation. Instead of only translating boxes in 2D space, the system now includes rotation angle as a parameter, enabling boxes to orient themselves according to product placement. This dimensional enhancement resolves the alignment issue caused by occlusion and varied product orientations.
2Adaptability or versatility
If traditional object detection methods are used, then general detection capability is achieved, but detection performance deteriorates due to densely-packed products and occlusion
Solution Approach 1:
The patent applies segmentation by dividing the detection task into multiple stages: first detecting individual products, then performing non-maximum suppression (NMS) to eliminate redundant detections, and finally applying rotation transformations to refine box positions. This segmented approach handles densely-packed products more effectively by processing them in discrete, manageable steps rather than attempting simultaneous detection of all objects.
Solution Approach 2:
The patent implements feedback mechanisms through iterative refinement processes. The NMS algorithm uses feedback from overlapping box comparisons to suppress redundant detections, and the rotation transformation uses feedback from alignment quality metrics to optimize box orientations. This feedback loop continuously improves detection reliability in dense product arrangements.
3Productivity
If visual classifiers are used for product recognition, then recognition speed is improved, but recognition accuracy deteriorates when products look similar but vary in text information
Solution Approach 1:
The patent segments the product recognition process into two distinct stages: first, visual classification for rapid initial categorization based on appearance features, and second, text recognition for precise identification of product-specific details. This segmentation allows the system to leverage the speed of visual classifiers while compensating for their inability to distinguish visually similar products with different text information through subsequent OCR processing.
Data Source
AI summary
Disclosed herein are designs for two baselines to detect products in a retail setting. A novel detector, referred to herein as RetailDet, detects quadrilateral products. To match products using visual texts on 2D space, text features are encoded with spatial positional encoding and the Hungarian Algorithm that calculates optimal assignment plans between varying text sequences is used.