Depth Imaging for Shelf Interaction Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing detection technologies in brick-and-mortar settings struggle to reliably determine whether a moving object, such as a customer or robot, interacts with articles on a rack, necessitating a solution that does not require pressure sensors at each shelf to reduce maintenance costs.
Innovation Solution
An interaction behavior detection system comprising a depth photographing device and a processing device, where the depth photographing device captures depth images of the rack and surrounding area, and the processing device extracts spatial coordinate information of the moving object to determine which article is touched, using background depth images to compare and identify shelf edges and object positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pressure sensors are installed at each shelf to detect article interactions, then detection reliability is improved, but device complexity and maintenance costs increase
Solution Approach 1:
The patent extracts the detection function from individual shelf-level sensors and consolidates it into a single overhead depth photographing device. This removes the need for pressure sensors at each shelf position, reducing device complexity while maintaining detection capability through centralized imaging and coordinate analysis
Solution Approach 2:
The patent replaces the mechanical pressure sensor system with an optical-depth imaging system. Instead of using physical sensors to detect contact forces, the system uses a depth photographing device to capture spatial coordinates and determine interactions through image processing and coordinate comparison
2Measurement precision
If pressure sensors are installed at each shelf to detect article interactions, then detection accuracy is improved, but maintenance costs increase
Solution Approach 1:
The patent merges multiple detection functions into a single overhead depth photographing device. By combining the detection capabilities that would otherwise require multiple distributed pressure sensors into one centralized imaging system, the patent reduces the number of components that need maintenance while preserving detection accuracy through coordinate analysis
Solution Approach 2:
The patent uses the depth photographing device to create a digital copy of the spatial environment, including shelf positions and article locations. This virtual model allows for accurate interaction detection without physical contact sensors, reducing maintenance requirements while maintaining measurement precision
3Device complexity
If a depth photographing device is used to detect article interactions, then device complexity is reduced, but measurement precision may worsen
Solution Approach 1:
The patent transitions from two-dimensional image data to three-dimensional spatial coordinate information by incorporating depth data from the photographing device. This dimensional enhancement allows accurate determination of whether moving objects touch articles by comparing Z-axis depth coordinates with shelf positions, maintaining measurement precision while using a simpler single-device setup
Solution Approach 2:
The patent introduces spatial coordinate information as an intermediary between the depth image data and the interaction detection result. By converting image coordinates to real-world spatial coordinates and comparing them with predefined shelf and article positions, the system achieves precise interaction detection despite using a single overhead device
Data Source
AI summary
Implementations of the present specification provide an interaction behavior detection method, apparatus, system, and device. The method includes the following: obtaining a to-be-detected depth image photographed by a depth photographing device, extracting a foreground image used to represent a moving object from the to-be-detected depth image, obtaining spatial coordinate information of the moving object based on the foreground image, comparing the spatial coordinate information of the moving object with spatial coordinate information of a shelf in a rack, and determining an article touched by the moving object based on a comparison result and one or more articles on the shelf.