Shopper Action Detection for Cashier-Less Item Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current devices require network connections and power to exchange data, which complicates their setup and operation, especially in retail environments where seamless data exchange over networks is needed for cashier-less transactions.
Innovation Solution
The implementation of wireless coded communication (WCC) devices with energy harvesting capabilities, using sensors and machine learning algorithms to track item interactions and user behavior, enabling cashier-less transactions by detecting item takes and returns, and managing shopping carts electronically.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If wireless coded communication devices with energy harvesting capabilities are implemented, then device complexity and network setup requirements are reduced, but measurement precision and reliability of item interaction detection may be compromised
Solution Approach 1:
The patent combines multiple sensing modalities (weight sensors, computer vision cameras, depth sensors, ultrasonic sensors, infrared sensors) into an integrated sensor system that works together to detect item interactions. This merging of sensors compensates for the limited capabilities of wireless coded communication devices while maintaining system simplicity.
Solution Approach 2:
The patent introduces intermediary processing components that bridge the gap between simple wireless coded communication devices and the complex task of item interaction detection. These intermediaries process sensor data and translate it into actionable information for the wireless devices.
2Reliability
If sensors and machine learning algorithms are used to track item interactions, then transaction accuracy is improved, but device complexity and power requirements increase
Solution Approach 1:
The patent implements periodic sampling of sensor data rather than continuous monitoring. The system takes snapshots of the shopping environment at intervals, reducing power consumption while still capturing sufficient information for accurate transaction tracking.
Solution Approach 2:
The patent uses machine learning models that are pre-trained offline to recognize item interaction patterns. During actual operation, the system only needs to feed preprocessed sensor data into these trained models, significantly reducing the computational power and energy required during transactions.
3Measurement precision
If multiple sensors are deployed to monitor shopper behavior, then measurement precision of item interactions is improved, but device complexity and cost increase
Solution Approach 1:
The patent divides the monitoring system into multiple independent sensor zones and functional modules. Each sensor type (weight, vision, depth, ultrasonic, infrared) operates independently and can be selectively activated based on the specific detection task, reducing overall system complexity while maintaining high measurement precision.
Solution Approach 2:
The patent designs the sensor system with multi-functional sensors that can perform multiple detection tasks. For example, computer vision cameras can simultaneously track item locations, recognize product labels, and monitor shopper movements, reducing the total number of devices needed while maintaining comprehensive monitoring capability.
Data Source
AI summary
Method of identifying actions of a shopper to account for taken items by the shopper in a cashierless checkout includes sampling a shopping environment using one or more video cameras to generate video features related a shopper in connection to an item and sampling using one or more supplemental sensors to generate supplemental sensor feature data, receiving output of the sampled video and supplemental sensor features as feature inputs to a deep learning model used for making inferences related to the state of a scenario involving shopper action of taking the item into their possession or held and other actions including moving outside a zone initially associated with the item.