Endoscope Bending CG View Alignment for Intuitive Manipulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Surgeons using motorized endoscopes face difficulty in intuitively aligning the operation direction of touchpads or remote controllers with the movement of the endoscope's bending portion due to the lack of haptic feedback, making it challenging to perform precise bending operations.
Innovation Solution
An image processing device that acquires shape information about the endoscope's variable bending portion and displays a computer graphics (CG) image aligned with the imaging direction, using a virtual camera whose viewing direction is rotated based on the bending angle to align the imaging directions in both the endoscope and CG images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a virtual 3D model of the bending portion is displayed together with an endoscope image, then the surgeon can visually recognize the bending state, but the operation direction on touchpads or remote controllers still does not intuitively align with the bending portion movement due to lack of haptic feedback
Solution Approach 1:
The patent creates a virtual 3D model that copies the physical bending portion's shape and behavior. This virtual model is displayed alongside the endoscope image, allowing surgeons to visually track the bending state. The virtual model responds to operation inputs in the same way the physical bending portion responds, providing visual feedback that mimics the expected physical behavior.
Solution Approach 2:
The system implements visual feedback by displaying the virtual 3D model that updates in real-time based on operation inputs. When the surgeon operates the touchpad or remote controller, the virtual model immediately reflects the intended bending direction and magnitude, providing continuous visual feedback about the relationship between operation inputs and bending portion movement.
2Ease of operation
If haptic feedback were provided to indicate bending direction, then operational intuitiveness would improve, but device complexity and lack of haptic capability in current endoscope systems would be worsened
Solution Approach 1:
The patent replaces the mechanical haptic feedback system with a visual information system. Instead of adding physical haptic feedback mechanisms to the touchpad or remote controller, the system uses visual display of the virtual 3D model to convey bending direction and magnitude information, substituting mechanical feedback with optical feedback.
Solution Approach 2:
The virtual 3D model serves as an intermediary between the operation inputs and the physical bending portion. It translates operation commands into visual representations of bending state, mediating the information flow without requiring direct haptic feedback mechanisms in the control device.
3Device complexity
If the viewing direction of the virtual camera is fixed, then the display system is simpler, but the imaging direction in the CG image does not align with the endoscope's imaging direction, reducing operational intuitiveness
Solution Approach 1:
The patent makes the virtual camera's viewing direction dynamic rather than fixed. The virtual camera automatically adjusts its viewing angle to follow the bending portion's orientation, ensuring that the imaging direction in the CG image always aligns with the endoscope's actual imaging direction. This dynamic adjustment occurs in real-time as the bending portion moves.
Data Source
AI summary
An image processing device includes a processor. The processor acquires shape information about a variable shape of a manipulator. The processor displays on a display a CG image of a manipulator model as viewed from a virtual camera in an object space, based on the shape information. The processor executes a rotation process of rotating a viewing direction of the virtual camera by a rotation angle according to the shape information.