Robot Stereo Vision ROI Selection for Detailed 3D Teleoperation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robot systems fail to allow operators to three-dimensionally view detailed parts of the work area, as they can only display a fixed stereo camera's field of view, limiting the operator's ability to manipulate the robot effectively.
Innovation Solution
A robot system equipped with a left-eye and right-eye camera capturing images of the work area, a stereoscopic display unit showing parallax images, and an area manipulation device allowing the operator to specify a stereoscopic vision target area within the common field of view, enabling detailed three-dimensional viewing of the work area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a fixed stereo camera displays the entire field of view, then the operator can see the overall work area, but the operator cannot three-dimensionally see a specific part of the work area in detail
Solution Approach 1:
The patent divides the wide field of view into multiple selectable regions of interest. Instead of displaying the entire field of view at once, the system segments it into manageable portions that can be viewed in detail. The operator can select different regions to be magnified and displayed stereoscopically, allowing detailed three-dimensional inspection of specific areas without requiring a complete overhaul of the camera system.
Solution Approach 2:
The patent transitions from a two-dimensional display to a three-dimensional stereoscopic display for the region of interest. By using parallax images from the stereo camera and displaying them with proper binocular disparity, the system adds the depth dimension to the viewing experience. This allows the operator to perceive height, width, and depth of the selected region simultaneously, enhancing detail observation capability.
2Area of stationary object
If the field of view is expanded to cover the entire work area, then the operator can see more area, but the detail of specific parts becomes less visible
Solution Approach 1:
The patent implements a dynamic display system that can adjust the displayed region and magnification level based on operator input. The display area is not fixed but can be dynamically changed by selecting different regions of interest. The system dynamically switches between showing the entire field of view and magnified views of specific areas, allowing the operator to adapt the visualization to the current task requirements.
Solution Approach 2:
The patent extracts and isolates the region of interest from the entire field of view for detailed examination. By extracting the selected region and displaying it separately with enhanced magnification and stereoscopic effect, the system allows the operator to focus on specific details without being distracted by the surrounding area. This extraction principle enables detailed viewing of critical regions while maintaining awareness of the overall context.
3Ease of operation
If parallax images are displayed without correction, then the display is simple, but significant distortion occurs in the images
Solution Approach 1:
The patent replaces mechanical image correction methods with computational image processing. Instead of using complex optical systems or mechanical adjustments to correct distortion, the system uses software-based image processing algorithms to detect and correct geometric distortion in the captured images. This substitution maintains operational simplicity while significantly improving image accuracy and reducing distortion.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables operators to manipulate the robot while three-dimensionally viewing specific parts of the work area in detail, improving operability and allowing for precise control, even in wide work areas, by displaying corrected parallax images without significant distortion.
Implementation Method 1
a left-eye camera and a right-eye camera configured to capture a left-eye capturing image and a right-eye capturing image of a work area
Implementation Method 2
capture a left-eye capturing image and a right-eye capturing image of a work area where the working part of the robot body performs the work
Implementation Method 3
a stereoscopic display unit configured to display parallax images seen three-dimensionally by the operator with both eyes
Data Source
AI summary
A vision system includes a robot body and a robot operation manipulator that receives inputs from an operator to manipulate the robot body. The vision system also includes a left-eye and right-eye cameras, and a display that displays parallax images for an operator. The vision system further includes an area operation manipulator that receives inputs by the operator to specify a target area to be seen three-dimensionally through the parallax images displayed on the display. The target area is located in an absolute space and is included in a portion of a field of view common between the left-eye and right-eye cameras. The vision system further includes a first controller that controls operation of the robot body, and a second controller that extracts and displays, as parallax images, images corresponding to the target area from a left-eye and right-eye capturing images captured by the left-eye and right-eye cameras, respectively.