Head-Mounted Display Eye-Tracking for Endoscopic Image Switching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional head-mounted displays (HMDs) used in endoscopic surgery often require operators to manually rotate mirrors to view additional diagnosis images, which can be cumbersome, especially when hands need to remain clean, and may distract from the primary endoscopic images, affecting surgical precision and efficiency.

Innovation Solution

An endoscopic surgery assisting system that includes a head-mounted display, a motion detector, and a controller, allowing operators to control the display of endoscopic and other diagnosis images (such as ultrasonic or CT images) using intentional motions, without the need for manual rotation, thereby enabling hands-free image switching and manipulation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a mirror is manually rotated to view other diagnosis images, then the operator can consult additional diagnostic information, but the operator's hands become contaminated and surgical precision decreases

Engineering Contradiction:

Improveability to view multiple diagnosis imagesVSAvoidsurgical precision

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent replaces the mechanical mirror rotation system with a head-mounted display that uses eye-tracking technology. The HMD detects eye movement and automatically switches between endoscopic images and other diagnosis images without requiring manual intervention, thereby eliminating hand contamination while maintaining the ability to consult multiple diagnostic sources.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The head-mounted display acts as an intermediary device between the operator and multiple diagnosis images. Instead of directly manipulating mirrors, the operator simply moves their eyes, and the HMD mediates the image switching based on detected eye movement, providing a contactless interface that preserves surgical precision.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If a mirror is manually rotated for image switching, then additional diagnosis images can be accessed, but the operation becomes cumbersome and time-consuming

Engineering Contradiction:

Improveability to switch between imagesVSAvoidease of image switching

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent replaces the cumbersome manual mirror rotation mechanism with an automated head-mounted display system. The HMD uses eye-tracking sensors to detect natural eye movements and automatically switches images accordingly, transforming a complex mechanical operation into a simple, intuitive eye-based control system that greatly improves ease of operation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The head-mounted display system performs image switching automatically based on detected eye movement without requiring manual intervention. The system serves itself by using the operator's natural physiological response (eye movement) to trigger image switching, eliminating the need for separate control actions and making the operation effortless.

Inventive Principle:

Principle #25Self-service

3Measurement precision

If the display occupies the entire field of view, then endoscopic images are clearly visible, but the operator cannot see the outside environment

Engineering Contradiction:

Improveclarity of endoscopic imagesVSAvoidability to see outside environment

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The head-mounted display dynamically adjusts its display mode based on the operator's needs. The display can switch between full-screen mode for clear endoscopic image viewing and a mode that allows peripheral vision of the outside environment. This dynamic adaptability resolves the contradiction by providing both clear imaging and environmental awareness at different times during the procedure.

Inventive Principle:

Principle #15Dynamics

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This system enhances surgical precision and efficiency by allowing operators to focus on the procedure while easily accessing and switching between different diagnostic images without moving their line of sight, reducing fatigue and improving surgical accuracy and speed.

Implementation Method 1

a detector (4) worn by an operator and capable of detecting an motion of the operator

Methodology Applied

Scientific EffectMotion detection: Accelerometer

Data Source

PatentUS10134185B2Endoscopic surgery assisting system using head-mounted displays and associated methodology

Publication Date: 2018.11.20 SONY GROUP CORP

AI summary

An endoscopic system according to an embodiment of the present technology includes a head-mounted display, a detector, and a controller. The head-mounted display is worn by an operator. The detector is capable of detecting a motion of the operator. The controller causes each of the plurality of head-mounted displays to individually display an image. The controller includes an endoscopic image acquisition unit capable of obtaining endoscopic image data of an affected area of a patient and an image control unit capable of controlling the endoscopic image data based on an output from each of the plurality of detectors. The controller performs control to display the image based on an output from the image control unit.