Multi-Item Checkout Recognition Using 3D Point Clouds to Resolve Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-item recognition during checkouts is challenging due to occlusion issues, especially in environments like convenience stores where space is limited and self-service terminals are not feasible, leading to inefficiencies and potential loss of sales.
Innovation Solution
A system that uses multiple cameras to capture images from different angles, creates synchronized point clouds, removes background pixels, clusters items, and employs a machine-learning model to identify item codes, enabling accurate recognition and elimination of manual scanning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple items are placed in one area for checkout, then the quantity of items processed increases, but occlusion blocks the view of some items making recognition difficult
Solution Approach 1:
The patent transitions from 2D image analysis to 3D point cloud processing by capturing depth information from multiple cameras. This dimensional change allows the system to reconstruct the spatial arrangement of items and identify occluded items by analyzing their 3D positions and relationships, effectively solving the occlusion problem in multi-item checkouts
Solution Approach 2:
The patent introduces point clouds as an intermediary representation between the captured images and the final item recognition. By converting image data into point clouds with depth information, the system creates a intermediate spatial model that facilitates better item separation and identification, especially for occluded items that are difficult to recognize in standard 2D images
2Productivity
If self-service terminals are installed to improve checkout efficiency, then automated item recognition is enabled, but the physical space required is not available in convenience stores
Solution Approach 1:
The patent makes existing surveillance cameras multi-functional by enabling them to perform both their traditional security monitoring function and the new function of item recognition for automated checkout. This eliminates the need for dedicated self-service terminal equipment, achieving space efficiency while maintaining productivity improvement
Solution Approach 2:
The system enables automated item recognition and checkout processing without requiring physical self-service terminals or cashier assistance. The multi-camera system with point cloud processing autonomously identifies items, retrieves their prices, and completes the transaction,实现ing self-service functionality within the existing store infrastructure
3Measurement precision
If traditional barcode scanning is used for item identification, then manual processing is required, but this increases queue times and reduces checkout speed
Solution Approach 1:
The patent replaces the mechanical barcode scanning process with an automated computer vision system. Instead of requiring manual barcode scanning, the system uses multiple cameras to capture images, converts them to point clouds, and automatically identifies items through image processing and pattern recognition, eliminating the need for manual intervention and significantly reducing queue times
Data Source
AI summary
Multiple images of a designated area are taken. The designated area comprises multiple items that are to be identified from the images. Depth information and Red, Green, Blue (RGB) data from each image is processed to create a point cloud for each image of the designated area. The point clouds are patched together or synchronized into a single point cloud for the designated area. Known background pixels associated with backgrounds for each image are removed from the single point cloud. The depth information and RGB data for the single point cloud is clustered together and bounding boxes are placed around each item in the single point cloud. At least the RGB data for each bounding box is provided to a machine-learning model (MLM) and the MLM returns an item code for the corresponding item. The item codes are fed to a transaction manager for a transaction associated with a customer.