Fraud Detection via Human-Object Interaction Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for detecting fraudulent behavior at self-checkouts, such as skip scanning, face challenges in accurately identifying items using surveillance cameras due to the vast variety of products and frequent changes in inventory, leading to issues with item tracking and false counting.
Innovation Solution
A fraud detection system that employs human-object interaction detection (HOID) technology to track item actions by distinguishing between actions of holding and releasing items, using bounding boxes and ID management to accurately count items entering and leaving specific regions, thereby preventing erroneous counting and improving detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a machine learning model is used to identify each item on the basis of captured images, then item identification accuracy may be improved, but it becomes unrealistic to learn each and every item sold in the store because there are many types of items and switching to new items is fast
Solution Approach 1:
The patent uses bounding boxes as simplified copies or representations of items instead of attempting to identify and learn every specific item type. The system captures the spatial position and size of objects using bounding boxes, which serve as abstract representations that can be tracked without requiring detailed item classification. This approach replaces complex item identification with simpler spatial tracking.
2Reliability
If surveillance cameras are used to detect suspicious behavior, then fraud detection capability is improved, but accurate item tracking becomes difficult due to the vast variety of products and frequent inventory changes
Solution Approach 1:
The patent segments the fraud detection process into distinct phases: item pickup detection, item placement detection, and scanning verification. By dividing the continuous monitoring process into discrete action segments, the system can track items more accurately through specific behavioral patterns rather than attempting to continuously identify every item type throughout the entire process.
Solution Approach 2:
The patent introduces bounding boxes as an intermediary representation between the surveillance camera and the fraud detection logic. Instead of directly identifying items, the system uses bounding boxes to track object positions and actions, serving as a mediator that simplifies the detection process while maintaining accuracy in tracking item movements and verifying scanning behavior.
3Measurement precision
If the system attempts to identify and track each item individually, then detection accuracy may be improved, but false counting and redundant counting occur leading to erroneous results
Solution Approach 1:
The patent performs preliminary detection of item pickup actions before attempting to track items through the scanning process. By detecting the pickup action first and establishing tracking from that point, the system ensures that only items that were actually picked up are counted, preventing false counting of items that were never taken. This preliminary action establishes the foundation for accurate subsequent tracking.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A n information processing program is provided. The information processing program causes a computer to execute processing including tracking an action taken by a user on an object on the basis of a captured image, distinguishing, on the basis of the action, between a first action in which the user holds the object and a second action in which the user releases the object held by the user, storing, on a first storage unit, first identification information and first location information with respect to the first action, storing, on a second storage unit, second identification information and second location information with respect to the second action, and in a case where the first action is detected at a position indicated in the second location information, storing the second identification information and the second location information on the first storage unit.