Multistage Object Detection for Cashierless Checkout
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated checkout systems in retail environments face challenges in ensuring security, efficiency, and reducing operational costs while enhancing the customer experience, as existing self-checkout systems may lack accuracy and reliability in detecting and tracking items without manual intervention.
Innovation Solution
A multistage object detection and tracking system utilizing a mobile device with on-board sensors, including digital cameras and time-of-flight imaging sensors, and an ML detection and tracking engine that captures and processes video frames to detect items entering or leaving a shopping container, with multiple stages of processing to achieve high accuracy and a 'go/no-go' decision for checkout.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single-stage object detection system is used for cashierless checkout, then the system complexity is low, but the measurement precision and reliability of item detection are insufficient
Solution Approach 1:
The object detection system is divided into multiple stages: first stage performs initial detection and tracking of items in the shopping container, second stage performs verification and refinement of detection results, and third stage makes final go/no-go checkout decisions. This segmentation allows each stage to specialize in specific detection tasks, improving overall accuracy while distributing computational complexity across multiple processing steps rather than requiring one extremely complex single-stage system.
Solution Approach 2:
The first stage performs preliminary detection and tracking of items before final verification. By pre-processing and identifying potential items in advance, the system reduces the computational burden on subsequent stages and improves overall detection accuracy through progressive refinement rather than attempting to achieve high precision in a single processing step.
2Productivity
If manual checkout processes are maintained, then operational costs are high, but automation reliability is insufficient
Solution Approach 1:
The system enables automated self-service checkout by equipping shopping containers with sensors and detection systems that automatically track items added to or removed from the container. The multistage detection system autonomously monitors shopping activities, identifies items, and determines checkout eligibility without requiring manual intervention, thereby improving both operational efficiency and reliability through consistent automated processing.
Solution Approach 2:
The system continuously receives feedback from sensors monitoring the shopping container, processes this information through multiple detection stages, and adjusts tracking and detection algorithms in real-time. This feedback loop ensures high reliability by constantly verifying item detection accuracy and maintaining up-to-date information about container contents, enabling trustworthy automated checkout decisions.
3Reliability
If simple detection algorithms are used, then processing speed is fast, but detection accuracy is insufficient for secure checkout
Solution Approach 1:
The detection process is segmented into multiple stages with increasing computational complexity. The first stage uses faster, simpler algorithms for initial item identification to maintain processing speed, while subsequent stages apply more complex verification algorithms to ensure detection accuracy and security. This segmentation allows the system to achieve both fast initial processing and high final accuracy without requiring all processing to be slow.
Solution Approach 2:
Simpler detection algorithms are applied in preliminary stages to quickly identify potential items and narrow down candidates. This preliminary action reduces the number of items requiring intensive verification in later stages, thereby reducing overall processing time while still maintaining high security through subsequent rigorous verification of the reduced candidate set.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system enables a robust, secure, and efficient cashierless checkout process by accurately detecting and tracking items, reducing the need for manual checkout lines and enhancing operational efficiency and customer experience through improved accuracy and reduced operational costs.
Implementation Method 1
The on-board sensors include digital cameras and/or time of flight imaging sensors for viewing the interior of the shopping container
Data Source
AI summary
A system for multistage object detection and tracking for realizing a cashierless checkout includes a mobile device. The mobile device includes a set of on-board sensors together installed to the shopping container. The on-board sensors could include multiple digital cameras viewing interior of the shopping container from different angles and positions. The on-board sensors detect and provide user activity data with respect to placement or removal of the items into or from the shopping container. The digital camera obtains a set of digital image frames associated with the unique signatures associated with various shopping items entering and leaving the interior region of the shopping container and communicates them to a mobile device comprising an on-device machine learning (ML) detection and tracking engine; and relay them to subsequent stages for improved accuracy and confidence to arrive at a go/no-go decision.