Produce Tray Quantity Estimation Using Empty-Area Image Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting product quantity in retail shelves, particularly for perishable items like fruits and vegetables, are inefficient, manpower-intensive, and prone to errors, and require high-resolution image capturing devices and complex computational processes.
Innovation Solution
A system and method using deep learning models to estimate produce quantity in trays by identifying tray edges and empty areas through image processing, employing first and second deep learning models trained on images of different tray colors, textures, and shapes, and calculating gap percentages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensor-based product detection (conductive contact sensors, inductance sensors, weight sensors, optical sensors) is used to detect out-of-shelf products, then detection capability is provided, but the system becomes complex and requires specific product arrangements that do not work for perishable items like fruits and vegetables
Solution Approach 1:
The patent replaces mechanical sensor-based detection systems with an optical imaging system combined with deep learning algorithms. Instead of using conductive contact sensors, inductance sensors, or weight sensors that require specific product arrangements, the system uses cameras to capture images and processes them through trained neural networks to detect product quantities and identify empty spaces, making it universally applicable to perishable items like fruits and vegetables regardless of their shape or packaging
Solution Approach 2:
The patent changes the detection parameter from physical contact or electromagnetic field interaction to optical image analysis. By transforming the detection approach from sensor-based physical measurement to image processing, the system can handle irregularly shaped perishable items that cannot be measured by traditional sensors
2Reliability
If cameras are used to capture shelf images and process them to detect missing products, then product detection is achieved, but the process requires proper product arrangement according to planogram and fails for products with irregular shapes and colors
Solution Approach 1:
The patent changes the detection approach from planogram-based comparison to direct image analysis. Instead of requiring products to be arranged according to a predefined planogram and comparing against it, the system uses deep learning models trained to directly identify products and empty spaces in images, eliminating the need for strict arrangement compliance and enabling detection of irregularly shaped products
Solution Approach 2:
The patent creates a digital copy of the shelf state through image capture and uses deep learning models to analyze this copy. The models are trained on extensive datasets of product images to recognize patterns and identify products without requiring physical contact or strict arrangement, allowing accurate detection of perishable items with irregular shapes and colors
3Measurement precision
If image processing is used to detect product quantity, then detection is achieved, but higher resolution image capturing devices are required and the process becomes computationally intensive making the application bulky
Solution Approach 1:
The patent changes the computational approach by training deep learning models to perform detection directly on standard resolution images rather than requiring high-resolution captures. The models are optimized to extract meaningful features and estimate product quantities from images captured by ordinary cameras, significantly reducing hardware requirements while maintaining detection accuracy
Solution Approach 2:
The patent uses image processing to create a digital representation of the shelf state that can be analyzed by deep learning models. Instead of requiring high-resolution physical measurement devices, the system captures images at standard resolution and uses computational algorithms to extract the necessary information, reducing both hardware and software complexity
4Reliability
If continuous or frequent monitoring by employees is performed to ensure product availability, then product availability is maintained, but the process is time consuming and requires more manpower reducing efficiency
Solution Approach 1:
The patent implements a self-monitoring system where the deep learning-based image processing system automatically detects product quantities and identifies empty spaces without human intervention. The system continuously captures images and processes them to provide real-time stock status, eliminating the need for employees to manually monitor shelves and significantly improving productivity while maintaining reliable product availability tracking
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A system and method for estimating a quantity of a produce in a tray is disclosed. The system comprises a server (105) which receives an image from a camera (115), identifies a tray in it, using a first deep learning model (270) trained using a plurality of images of trays not containing any produce. For identifying empty areas in the tray, the server (105) estimates a total area of the tray and identifies one or more areas in the image of the tray in which the top surface of the bottom of the tray is exposed by using a second deep learning model (275) trained using the plurality of images of areas exposed in trays. Then, using these the server (105) estimates the quantity of the produce in the tray as a ratio of the area of the tray covered by the produce and the total area of the tray.