Self-Checkout Alert Generation for Product Misregistration Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing self-checkout systems face challenges in detecting fraudulent activities such as the 'banana trick' where a low-priced item is scanned instead of a high-priced item, and there is difficulty in analyzing the depth relationship between objects in images for accurate product recognition.
Innovation Solution
A system utilizing machine learning models, including Human-Object Interaction Detection (HOID) and a zero-shot image classifier, to identify products and generate alerts for abnormal registrations in self-checkout machines, leveraging a hierarchical product structure to enhance fraud detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If self-checkout machines are introduced to reduce labor costs and address labor shortages, then productivity and labor efficiency are improved, but the system becomes vulnerable to fraudulent activities such as the 'banana trick' where product misregistration occurs
Solution Approach 1:
The patent introduces an information processing device as an intermediary between the self-checkout machine and the monitoring system. This device acquires video data, identifies products using machine learning models, compares them with registered products, and generates alerts for discrepancies. The intermediary enables automated fraud detection without requiring additional manual labor at the checkout counter.
Solution Approach 2:
The patent replaces manual monitoring and product verification mechanisms with automated machine learning-based image recognition systems. Instead of relying on cashiers to verify products, the system uses HOID models and zero-shot image classifiers to automatically identify products from video data and compare them against the checkout registry, substituting human mechanical verification with automated computational analysis.
2Ease of operation
If traditional image recognition methods are used to identify products in self-checkout systems, then basic object detection is achieved, but accurate product identification and depth relationship analysis remain difficult
Solution Approach 1:
The patent changes the parameters and approaches of image recognition by employing multiple machine learning models with different capabilities. It transitions from basic object detection to using HOID (Human-Object Interaction Detection) models that can identify relationships between persons and objects, and further to zero-shot image classifiers that can recognize products without requiring extensive training data. This parameter change in the recognition approach enables accurate product identification even with limited training data.
Solution Approach 2:
The patent combines multiple machine learning models (HOID models and zero-shot image classifiers) to create a composite recognition system. The HOID model detects human-object interactions and identifies products being handled, while the zero-shot image classifier provides additional verification and identification capabilities. This composite approach leverages the strengths of different models to achieve higher product recognition accuracy than any single model could achieve alone.
3Measurement precision
If machine learning models are implemented to detect fraudulent activities, then fraud detection accuracy is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent segments the fraud detection system into distinct functional modules: a video acquisition unit that captures footage, a machine learning unit that performs product identification using HOID and zero-shot classification, and an alert generation unit that compares identified products with registered products and generates alerts for discrepancies. This segmentation allows each component to be optimized independently and simplifies the overall system architecture by dividing complex functions into manageable, specialized units.
Data Source
AI summary
A non-transitory computer-readable storage medium storing an alert generation program that causes at least one computer to execute a process, the process includes acquiring a video of a person who holds a product to be registered in an accounting machine; specifying, by inputting the acquired video to a machine learning model, a product candidate that corresponds to the product included in the video from a plurality of product candidates; acquiring an item of the product input by the person from a plurality of product candidates output by the accounting machine; and generating an alert that indicates an abnormality of the product registered in the accounting machine based on the acquired item of the product and the specified product candidate.