Endoscope Inspection Assistance With Gaze-Directed Lesion Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing endoscope inspection methods face challenges in achieving both accuracy and efficiency due to the heavy processing load on computers for real-time lesion detection and the need to operate the endoscope with both hands, without considering how to select the region for analysis.
Innovation Solution
An inspection support device and method that uses visual-line detection to determine the operator's attention region, performing recognition processing only on this region, reducing the processing load and enhancing accuracy and efficiency by focusing on the visually attended area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real-time lesion detection and identification is performed on the entire captured image, then the accuracy of inspection is improved, but the processing load on the computer increases significantly
Solution Approach 1:
The captured image is divided into multiple regions based on the operator's visual attention. The system performs detailed recognition processing only on the attention region where the operator is looking, while other regions receive reduced or no processing. This segmentation approach maintains lesion detection accuracy in the critical attention region while significantly reducing the overall processing load on the computer.
2Productivity
If the operator focuses on a specific region for lesion detection, then the processing load is reduced, but the likelihood of overlooking lesions in other regions increases
Solution Approach 1:
The system continuously tracks the operator's visual attention and dynamically adjusts the recognition processing based on the current attention region. When the operator moves their gaze to a new region, the system immediately begins detailed analysis of that region. This feedback mechanism ensures that all regions are eventually examined with high accuracy while maintaining efficient real-time processing, as the system adapts to the operator's natural scanning pattern.
3Ease of operation
If the operator uses both hands to operate the endoscope, then the ease of operation is improved, but the ability to manually indicate regions of interest on the image is lost
Solution Approach 1:
The system replaces the mechanical method of region indication (such as using a pointer or manual marking tool) with an optical/visual tracking method. By using a visual-line detection device to automatically track where the operator is looking on the display screen, the system infers the attention region without requiring any additional manual input from the operator. This substitution maintains ease of operation with both hands on the endoscope while eliminating the loss of region indication capability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system reduces processing load, improves accuracy and efficiency by performing recognition processing only on the visually attended region, allowing for real-time lesion detection and identification, and reducing the likelihood of overlooking lesions.
Implementation Method 1
an imaging device 8 for capturing an image of a subject; a visual-line detection unit 44B that detects a visual line of an operator who operates the endoscope 1 on the basis of images of both eyes of a person captured by the imaging device 8
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are an inspection support device, an endoscope device, an inspection support method, and an inspection support program that can make both the accuracy and efficiency of inspection using an endoscope compatible. The system control unit 44 functions as a captured image data acquisition unit 44A that acquires captured image data obtained by imaging the inside of a subject with an endoscope 1; a visual-line detection unit 44B that detects a visual line directed to a display device 7 that displays a captured image based on the captured image data; a processing unit 44C that performs recognition processing for performing detection of a lesion site from the captured image data and identification of the detected lesion site on the captured image data; and a display control unit 44D for causing the display device 7 to display a result of the recognition processing by the processing unit 44C. The processing unit 44C controls the content of the recognition processing on the captured image data on the basis of the visual line detected by the visual-line detection unit 44B.