RGB-D Self-Checkout Counter for Accurate Frictionless Retail Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional retail store checkouts require extensive manual labor, are prone to errors, and are inefficient, leading to long queues and increased operational costs, which deter customers and limit the number of transactions per hour.
Innovation Solution
A 3D computer vision-assisted self-service checkout system using RGB-D sensors captures RGB and depth images, performs volumetric analysis with a two-stream Convolutional Neural Network (CNN), and generates unique product IDs to automate the checkout process, eliminating the need for human supervision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual scanning of each product is performed by store executives, then product identification can be achieved, but the number of customers served per hour decreases and operational costs increase
Solution Approach 1:
The system enables self-service checkout by automatically detecting products on the conveyor belt using RGB-D sensors and deep learning models, eliminating the need for manual scanning by store executives. The automated product identification and billing process allows multiple customers to be served simultaneously, dramatically increasing the number of customers served per hour while maintaining accurate product identification.
Solution Approach 2:
The patent replaces the mechanical manual scanning process with an automated computer vision system using RGB-D sensors, deep learning models, and automated billing software. This substitution eliminates human labor requirements at checkout counters while maintaining or improving product identification accuracy, thereby increasing overall checkout throughput and customers served per hour.
2Productivity
If additional skilled manual labour is hired to serve more customers, then the number of customers served per hour increases, but operational costs overshoot the budget
Solution Approach 1:
The automated checkout system performs self-service functionality, eliminating the need to hire additional skilled manual labor. The system uses RGB-D sensors, deep learning-based product recognition, and automated billing to handle checkout processes independently, allowing the store to serve more customers without increasing labor costs or overshooting the operational budget.
Solution Approach 2:
The patent substitutes human labor with an automated system comprising RGB-D sensors, processing units running deep learning models, and automated billing software. This replacement eliminates the need to hire additional staff while increasing customer service capacity, thereby improving productivity without increasing operational costs.
3Extent of automation
If 2D camera with digital watermark detection is used, then product identification is automated, but products that are visually similar but volumetrically different cannot be distinguished
Solution Approach 1:
The patent transitions from 2D camera-based detection to 3D RGB-D sensing, adding depth information as a new dimension. This enables the system to capture volumetric data of products, allowing differentiation between products that are visually similar but differ in volume or shape. The depth maps generated by RGB-D sensors provide the additional dimensional information needed for accurate product differentiation.
Solution Approach 2:
The system changes the detection parameters by incorporating depth information alongside visual features. The deep learning model processes both 2D visual data and 3D depth data, utilizing volumetric parameters to distinguish between similar products. This parameter expansion from purely 2D to 3D feature space enables accurate differentiation of products with identical visual appearance but different physical dimensions.
4Reliability
If traditional checkout process is maintained, then human supervision ensures accuracy, but checkout queues become long and customer experience deteriorates
Solution Approach 1:
The system implements self-service checkout with automated product recognition, tracking, and billing processes. The RGB-D sensors continuously monitor the conveyor belt, automatically identify products, track their movement, and generate bills without human intervention. This maintains reliability through automated verification while eliminating waiting times associated with manual checkout processes.
Solution Approach 2:
The automated system enables continuous operation of the checkout process without interruptions for manual scanning or billing. The RGB-D sensors continuously capture depth and visual data, the deep learning model continuously processes product identification, and the billing system continuously updates transactions. This continuous automated operation eliminates idle time and queue waiting, maintaining accuracy while dramatically reducing customer waiting time.
Data Source
AI summary
This present disclosure proposes a system and method for providing a 3D computer vision-assisted frictionless self-checkout experience in a traditional retail store by using two RGB-D camera sensors and a conveyer belt. The disclosure provides a self-checkout counter (106) that enables a customer (102) to go through a self-service checkout process by simply placing the collected products one by one on a conveyer belt. One vertical and another horizontal RGB-D sensor (108) mounted in a housing frame attached towards the end of the conveyer belt capture RGB and depth image of each product passing through the housing and pass it to a product recognition engine (216). The engine (216) identifies the unique product along with its volumetric attributes processing the RGB-D data that is further compared with a master product database and processed for invoicing. The customer wallet and payment may be integrated with the customer phone number at the self-checkout counter (106) for providing a completely automated checkout experience.