Self-Checkout Fraud Detection Using Video Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image recognition technologies struggle to analyze the depth relationship between objects in a two-dimensional space, making it difficult to detect errors or fraud in self-checkout machines, such as scan omissions or barcode concealment, without the need for costly weight sensors.
Innovation Solution
An information processing program and device that uses video data to identify products by specifying regions of interest, such as a user's hand and the product, and applies machine learning models to detect anomalies, generating alerts when the registered product does not match the product being scanned.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If weight sensors are introduced in each self-checkout machine to automatically count products and detect fraud, then detection accuracy is improved, but cost increases excessively
Solution Approach 1:
The patent replaces the mechanical weight sensor system with a computer vision system using cameras and machine learning models. The image recognition technology analyzes video data to detect products, user actions, and potential fraud without physical contact or additional hardware sensors at each checkout machine.
Solution Approach 2:
The patent uses image data as a copy or representation of the physical product and user actions. By analyzing visual copies (images) of products on shelves and in user hands, the system infers product information and detects fraud without needing physical sensors to measure weight or presence directly.
2Device complexity
If image recognition technology is used to identify products, then cost is reduced, but the ability to detect depth relationships and product positioning is insufficient
Solution Approach 1:
The patent transitions from two-dimensional image analysis to three-dimensional spatial understanding by analyzing the positional relationships between multiple detected objects (user hand, product in hand, product on shelf). By comparing the relative positions and depths of these objects in the image space, the system infers three-dimensional product placement and user actions.
3Ease of manufacture
If traditional image recognition is used to detect products, then implementation is simple, but detection of user actions and product relationships is inaccurate
Solution Approach 1:
The patent segments the image analysis task into distinct components: detecting the user hand, detecting products in the hand, detecting products on shelves, and analyzing the relationships between these detected elements. This segmentation allows each component to be optimized independently while maintaining overall system simplicity.
Data Source
AI summary
A storage medium storing an information processing program that causes a computer to execute a process that includes acquiring video data that includes a registration machine; extracting image data that include products;specifying a timing when first information regarding a first product registered to the registration machine; specifying certain image data of the image data that includes a second product held in the hand of the user within a certain time period from the timing and placed in a place in an angle of view of the video data that is not a place where a product that has been registered to the registration machine is placed for most of the certain time period; specifying second information regarding the second product by inputting the certain image data to a machine learning model; and generating an alert when the first information and the second information do not match.