Self-Checkout Fraud Detection via Skeleton Action Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing self-checkout systems are unable to detect missed checkout of products, where users may not properly scan items or pretend to scan them, leading to potential fraud and errors.

Innovation Solution

A computer-readable recording medium storing a program that acquires product information from a self-checkout machine, identifies feature amounts related to the number of products and the frequency of user actions, and generates alerts based on these features to detect missed checkout.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional barcode scanning detection is used, then the system can determine when a barcode reading action is performed, but it cannot detect missed checkout or fraudulent scanning behavior

Engineering Contradiction:

Improvedetection accuracy of barcode reading actionVSAvoidability to detect missed checkout and fraud

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent transitions from two-dimensional image data analysis to three-dimensional skeleton-based action recognition. By extracting skeletal information and temporal sequences of user movements, the system gains an additional dimension of analysis that enables detection of fraudulent scanning behaviors and missed checkouts that cannot be identified through traditional barcode detection alone

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The system implements feedback by continuously monitoring user actions through skeleton tracking and comparing them against expected scanning patterns. The detection results are fed back to identify discrepancies between actual user behavior and proper scanning procedures, enabling real-time detection of fraud and missed checkouts

Inventive Principle:

Principle #23Feedback

2Productivity

If image data analysis is used to track user actions, then the system can identify when products are moved to the scan region, but it cannot distinguish between legitimate scanning and fraudulent or missed scanning

Engineering Contradiction:

Improveautomation of barcode reading detectionVSAvoidaccuracy in distinguishing legitimate vs fraudulent scanning

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent replaces traditional mechanical/image-based detection systems with a skeleton-based action recognition system. By substituting image pixel analysis with skeletal joint tracking and movement pattern analysis, the system achieves higher precision in distinguishing between legitimate scanning actions and fraudulent or missed scanning behaviors

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12266183B2Non-transitory computer-readable recording medium, notification method, and information processing device

Publication Date: 2025.04.01 FUJITSU LTD

AI summary

An information processing device identifies a first feature amount related to the number of the product registered on the specific machine based on an acquired product information, generating, from an image capturing a user in front of the specific machine, a first area information, a second area information and an interaction between the first class and the second class, specifies an action of the user of registering the product to the specific machine based on the first area information, the second area information, and the interaction, identifies a second feature amount related to the number of time of carrying out the action; and generates an alert based on the first feature amount and the second feature amount.