Endoscopic Gaze-Guided Target Tracking for Reliable Instrument Following
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing endoscopic surgery systems face challenges in maintaining accurate automatic tracking of surgical instruments due to insufficient image recognition and inadequate integration of user intent, leading to suboptimal tracking accuracy and reliability.
Innovation Solution
A medical observation system that incorporates a gaze information acquisition unit to enhance tracking target determination by combining recognition reliability with user gaze information, utilizing a target candidate recognition unit, reliability acquisition unit, and a tracking target determination unit to improve tracking accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If image recognition processing is used to automatically track surgical instruments, then tracking automation is improved, but tracking accuracy deteriorates when image recognition cannot be performed well
Solution Approach 1:
The patent introduces gaze information as an intermediary element that mediates between the automatic tracking system and the surgeon's actual attention. The gaze information acquisition unit captures where the surgeon is looking, and this information serves as a mediator to correct or supplement image recognition results, thereby improving tracking accuracy without reducing automation.
Solution Approach 2:
The system implements feedback by continuously monitoring the surgeon's gaze and using this information to adjust the tracking target selection. When image recognition identifies potential tracking targets, the system receives feedback from gaze information to determine which target the surgeon actually intends to track, creating a closed-loop control system that improves accuracy while maintaining automation.
2Adaptability or versatility
If multiple tracking target candidates are recognized, then tracking versatility is improved, but tracking reliability deteriorates due to insufficient recognition reliability
Solution Approach 1:
The patent makes the tracking target selection dynamic by integrating real-time gaze information with recognition reliability scores. Instead of relying solely on static recognition algorithms, the system dynamically adjusts target selection based on the surgeon's current attention, allowing it to adapt to changing surgical conditions while maintaining reliability through the dual-criteria approach.
Solution Approach 2:
The system changes the selection parameter from relying only on recognition reliability to a combined parameter that includes both recognition reliability and gaze information. This parameter change allows the system to distinguish between targets that are merely recognizable and those that are both recognizable and currently relevant to the surgeon's work, thereby improving reliability.
3Ease of operation
If autonomous tracking operation is implemented, then operational complexity is reduced, but tracking accuracy deteriorates due to insufficient cooperation with user intention
Solution Approach 1:
The system enables self-service by automatically acquiring gaze information and integrating it with image recognition results without requiring explicit surgeon input. The tracking system serves itself by autonomously determining the correct tracking target through the combination of automated recognition and automated gaze monitoring, maintaining ease of operation while improving accuracy.
Solution Approach 2:
Gaze information acts as an intermediary that bridges the autonomous tracking system and the surgeon's intentions. Rather than requiring direct communication or manual control from the surgeon, the system uses gaze as a natural, passive mediator that conveys intent, thereby maintaining operational simplicity while achieving accurate tracking that aligns with user intention.
Data Source
AI summary
A medical observation system includes: an imaging unit that acquires an operative field image including an abdominal cavity environment; a display unit that displays the operative field image acquired by the imaging unit; a gaze information acquisition unit that acquires information regarding gaze of a user on the abdominal cavity environment of the operative field image displayed by the display unit; a target candidate recognition unit that recognizes a tracking target candidate from the operative field image acquired by the imaging unit; a reliability acquisition unit that acquires recognition reliability of the tracking target candidate recognized by the target candidate recognition unit; and a target determination unit that determines a tracking target in the abdominal cavity environment from the tracking target candidate on the basis of the information regarding the tracking target candidate, the recognition reliability, and the information regarding the gaze.