Endoscope Image Shake Correction via Gyro Rotation Center Estimation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Endoscopic images taken during laparoscopic operations often suffer from shake due to the motion of the hand holding the head part of the endoscope, which is not effectively corrected by existing technologies despite the use of gyroscopes for angular velocity detection.

Innovation Solution

An image processing device estimates the approximate center of rotation of the endoscopic device based on motion detection, particularly the angular velocity of the head part, to correct image shake by adjusting the position of the image cutting area and removing rolling shutter distortion.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a gyo is built in the head part to detect moving angular velocity, then the direction of camera for three-dimensional display can be detected, but the detected angular velocity is not used for image shake correction

Engineering Contradiction:
Improveusage of angular velocity dataVSAvoidimage shake correction effectiveness
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The gyo detected angular velocity data is made multi-functional by using it for both three-dimensional display direction detection and image shake correction. The estimation unit leverages the existing angular velocity data from the gyo to estimate the rotation center position, thereby correcting image shake without requiring additional sensors or complex mechanisms.

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Reliability

If hand holding the head part is shaken, then image shake occurs due to transmission of shake motion through the trocar, but no effective correction mechanism exists

Engineering Contradiction:
Improveimage stabilityVSAvoidcorrection mechanism complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The system uses the endoscope's own structural characteristics (linear rod shape with trocar as rotation center) and its existing gyo data to automatically estimate the rotation center position and correct image shake. The estimation unit processes the angular velocity data to determine the rotation center, and the correction unit then uses this information to stabilize the image, making the system self-correcting without external intervention.

Inventive Principle:
Principle #25Self-service

3Reliability

If the rotation center position is estimated based on motion detection, then image shake can be corrected, but processing complexity increases

Engineering Contradiction:
Improveimage shake correction accuracyVSAvoidimage processing complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The system changes the parameter approach by estimating the rotation center position based on angular velocity data from the gyo rather than using complex motion detection algorithms. This parameter substitution simplifies the processing while maintaining correction accuracy, as the angular velocity data directly reflects the shaking motion characteristics.

Inventive Principle:
Principle #35Parameter changes

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This solution effectively corrects hand shake in endoscopic images, allowing for accurate 3D measurements and stitching of images to produce high-quality, wide-viewing angle images with reduced processing complexity.

Implementation Method 1

a gyo 26 built in the head part 24 to detect a moving angular velocity of the head part 24

Methodology Applied
Scientific EffectGyroscope: Gyroscope

Implementation Method 2

estimate an approximate center of rotation of the endoscopic device based on motion detection in response to a movement of an objective lens

Methodology Applied
Scientific EffectMotion detection:

Data Source

PatentUS10716458B2Endoscopic system, image processing device, image processing method, and program
Publication Date: 2020.07.21 SONY GROUP CORP
  • US10716458B2 patent drawing
  • US10716458B2 patent drawing
  • US10716458B2 patent drawing

AI summary

To solve the above problem, a first technology is an image processing device configured to estimate an approximate center of rotation of an endoscopic device based on motion detection in response to a movement of an objective lens at a distal portion of the endoscopic device by manipulating a proximal portion of the endoscopic device.