Box Detection Using Monocular and Depth Image Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting boxes in logistics environments are inefficient and labor-intensive, relying heavily on human labor and often suffer from accuracy issues due to human fatigue, which can be improved by automating the process using robots equipped with advanced image processing capabilities.
Innovation Solution
A robot system that processes monocular and depth image frames to detect boxes by determining corners, performing edge detection, and matching faces, using machine learning models and gradient ascent optimizers, to generate accurate box estimations and adjust them over time, incorporating sensors like stereo cameras or LIDAR for precise detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human labor is used for box detection, then the process is simple to implement, but accuracy deteriorates due to human fatigue and efficiency is low
Solution Approach 1:
The patent replaces human visual inspection with an automated computer vision system that uses image processing algorithms to detect boxes. The system processes monocular images and depth images to automatically identify box locations, corners, and dimensions, eliminating human fatigue and improving both accuracy and efficiency.
Solution Approach 2:
The patent introduces depth images as an intermediary data source between the monocular images and the final box detection. By combining monocular image information with depth information, the system achieves more accurate box detection and measurement without requiring additional manual intervention.
2Productivity
If automated image processing is used, then productivity increases, but device complexity increases due to multiple sensors and processing steps
Solution Approach 1:
The patent designs a unified image processing system that handles multiple tasks using the same input data. The system processes monocular images and depth images to simultaneously detect box locations, determine corners, measure dimensions, and track box movements over time, reducing the need for separate specialized systems for each function.
Solution Approach 2:
The patent combines monocular image processing and depth image processing into a single integrated system. By merging the processing of multiple image types and combining detection algorithms, the system achieves high productivity while managing complexity through consolidation rather than separate independent systems.
3Measurement precision
If multiple image frames are processed over time, then measurement precision improves through aggregation, but processing time increases
Solution Approach 1:
The patent performs preliminary processing of individual image frames to extract corners, edges, and basic box information before aggregating results over time. By preparing and pre-processing the data structure in advance, the system can efficiently aggregate information from multiple frames without significant time penalty, as the aggregation operation works on pre-processed summaries rather than raw images.
Data Source
AI summary
A method for detecting boxes includes receiving a plurality of image frame pairs for an area of interest including at least one target box. Each image frame pair includes a monocular image frame and a respective depth image frame. For each image frame pair, the method includes determining corners for a rectangle associated with the at least one target box within the respective monocular image frame. Based on the determined corners, the method includes the following: performing edge detection and determining faces within the respective monocular image frame; and extracting planes corresponding to the at least one target box from the respective depth image frame. The method includes matching the determined faces to the extracted planes and generating a box estimation based on the determined corners, the performed edge detection, and the matched faces of the at least one target box.