Container-Aware Computer Vision for Sorting Error Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer-vision machine-learning models struggle to identify items that are partially obscured or stored in physical containers, leading to inefficiencies in sorting and monitoring user interactions, particularly when items are partially or fully occluded.
Innovation Solution
An online system employs a contained-item identification model to recognize both physical containers and visible items within images, using machine-learning techniques to associate items with their respective containers and detect sorting errors by comparing images before and after delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional computer-vision machine-learning models are used to identify items, then items that are fully visible can be accurately identified, but items that are partially obscured or stored in containers cannot be reliably identified
Solution Approach 1:
The patent introduces physical containers as intermediary objects that hold items. Instead of attempting to identify obscured items directly, the system identifies the container first, then uses the container as a proxy to represent the items it contains. The container becomes a mediator that bridges the gap between the camera's limited view and the complete set of items that need to be tracked.
Solution Approach 2:
The system creates a digital representation or copy of the physical container and its contents in the virtual monitoring environment. When items are placed in a container, the system creates a corresponding digital container object that tracks the items indirectly. This digital copy allows the system to monitor items even when they are not directly visible to the camera.
2Productivity
If items are stored in physical containers to organize deliveries, then sorting efficiency is improved, but the ability to monitor and verify correct sorting deteriorates because obscured items cannot be identified
Solution Approach 1:
The system implements feedback mechanisms where the identified container information is continuously monitored and compared against the expected delivery composition. When a container is delivered, the system provides feedback by comparing the delivered container's identified items with the order requirements, alerting users to any sorting errors or mismatches.
Solution Approach 2:
The patent replaces the mechanical/physical system of visual inspection with an automated computer-vision-based identification system. Instead of requiring manual verification of container contents, the system uses machine learning models to automatically identify containers and their visible contents, substituting automated digital processing for manual mechanical inspection.
3Measurement precision
If the system only identifies fully visible items, then identification accuracy is maintained, but the system cannot detect sorting errors for items that are obscured by containers
Solution Approach 1:
The system segments the identification task into two distinct parts: first identifying the container as a separate object, and then identifying the visible items within or near the container. This segmentation allows the system to reliably identify containers even when items are obscured, and use the container identification as a basis for inferring information about the obscured items.
Solution Approach 2:
The system adds a new dimension to the identification process by introducing container-level identification as an intermediate layer between individual item identification and delivery verification. Instead of working only with individual items in three-dimensional space, the system operates at two levels: container level and item level, effectively adding a hierarchical dimension to the problem-solving approach.
Data Source
AI summary
An online system uses a computer-vision item identification model to identify items and physical containers storing those items to detect sorting errors of the physical containers. The online system receives a first image from a client device that depicts a set of physical containers that contain items for a batch of orders that the online system has received. The online system identifies items in those physical containers by applying a contained-item identification model to the first image. The online system uses the output of this model to determine which visible items are in each physical container and uses that information plus order data for the batch of orders to determine which physical containers are associated with each order. The online system compares this first image to a subsequently received image to determine whether the correct physical containers were delivered by the user.