Self-Checkout Merchandise Detection Using Zero-Shot ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In self-checkout systems, it is challenging to detect unauthorized activities such as cheating the number of purchases due to the difficulty in analyzing the depth relationship between bounding boxes and identifying interactions between a person and objects, leading to potential manual input errors or scans that do not accurately represent the actual number of purchased merchandise.
Innovation Solution
A computer-readable storage medium storing an alert generation program that uses machine learning models, specifically a zero-shot image classifier, to analyze videos of self-checkout processes, identify merchandise candidates, and generate alerts for abnormalities in merchandise registration, thereby detecting and preventing unauthorized activities like label switches or banana tricks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If self-checkout systems are used to reduce manual input, then productivity is improved, but measurement precision of merchandise registration deteriorates
Solution Approach 1:
The system captures video of the person holding merchandise, uses machine learning to identify the actual merchandise, and compares it with the registered merchandise to generate alerts for discrepancies. This feedback loop ensures registration accuracy while maintaining self-checkout efficiency.
Solution Approach 2:
The patent replaces manual verification with automated machine learning-based image recognition. The system automatically identifies merchandise from video frames and compares it with registration data, substituting human judgment with automated optical and computational systems.
2Measurement precision
If machine learning models are used to identify merchandise, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The machine learning model serves multiple functions: identifying merchandise type, verifying registration accuracy, and triggering alerts. This multi-functionality reduces the need for separate verification systems, managing complexity while improving precision.
Solution Approach 2:
The system performs self-verification by automatically comparing video-based merchandise identification with registration data without requiring additional manual intervention. The machine learning model autonomously detects discrepancies and generates alerts.
3Reliability
If video analysis is performed to detect unauthorized activities, then reliability is improved, but use of energy increases
Solution Approach 1:
The system performs video analysis selectively - processing frames only when merchandise is detected or discrepancies are suspected. This partial processing approach maintains detection reliability while reducing overall energy consumption compared to continuous full-video analysis.
Data Source
AI summary
A non-transitory computer-readable storage medium storing an alert generation program that causes at least one computer to execute a process, the process includes acquiring a video of a person who holds a merchandise to be registered in a checkout machine; specifying merchandise candidates corresponding to merchandises included in the video and a number of the merchandise candidates by inputting the acquired video to a machine learning model; acquiring items of merchandises registered by the person and a number of the items of the merchandises; and generating an alert indicating an abnormality of merchandises registered in the checkout machine based on the acquired items of the merchandises and the number of the items of the merchandises, and the specified merchandise candidates and the number of the merchandise candidates.