Operator Assistance Vision System 3D Pop-Up Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image processing systems for machines fail to accurately depict the depth and positioning of objects in the environment, leading to distortion and distraction for operators, as they often lose information and fail to draw attention to detected objects effectively.
Innovation Solution
A vision system that captures images using an image capturing device, applies object detection processes to determine bounding boxes, calculates height and range, and generates a 3D view by reinserting pixels as vertical pop-up elements relative to a ground plane, enhancing object representation and visibility for the operator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional object detection processes are used to generate overhead view images, then the operator can see objects in the environment, but depth information is lost and vertical objects are distorted
Solution Approach 1:
The patent transforms 2D overhead view images into 3D visualizations by adding depth information. The system extracts depth data from the 2D images and reconstructs the spatial positions of objects in three-dimensional space, allowing the operator to perceive depth and vertical dimensions that were lost in the conventional 2D overhead view.
2Quantity of substance
If multiple types of information are presented on the display, then the operator receives comprehensive data, but the operator becomes distracted and attention is not drawn to detected objects
Solution Approach 1:
The patent extracts and highlights only the most critical information - the detected objects and their 3D positions - from the comprehensive set of environmental data. By isolating and emphasizing object locations in 3D space, the system draws the operator's attention to safety-critical elements without being overwhelmed by other environmental information.
Solution Approach 2:
The system uses visual differentiation through color and graphical representation to highlight detected objects against the background environment. This visual emphasis allows the operator to quickly identify objects of interest without being distracted by other information presented on the display.
3Illumination intensity
If two-dimensional images are converted to overhead view, then visibility and control are improved, but object depth perception becomes difficult
Solution Approach 1:
The patent adds the third dimension (depth) to the overhead view by generating 3D visualizations. This allows the operator to maintain the broad visibility benefits of overhead viewing while simultaneously perceiving depth information through three-dimensional rendering of objects and their spatial relationships.
Data Source
AI summary
A vision system for assisting an operator of a machine is provided. An image capturing device, mounted on the machine, is configured to capture an image of an environment of the machine. The vision system includes a controller configured to apply an object detection process to detect an object in the image. The controller determines a bounding box comprising one or more pixels associated with the object. The controller determines a height and a range associated with the object based on the bounding box. The controller extracts the one or more pixels within the bounding box. The controller is further configured to generate a three-dimensional (3D) view comprising the object based on the image captured by the image capturing device. The controller reinserts the one or more pixels as a vertical pop-up element with respect to a ground plane in the 3D view. A display is provided to display the 3D view to the operator of the machine.