Endoscope Learning Support Device for Training Image Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for automatically recognizing treatment instruments in endoscopic images require a large number of training images and struggle to effectively form training images for instruments with limited clinical data, leading to suboptimal recognition performance.
Innovation Solution
A learning support device and method that form a training image by superimposing a foreground image of treatment instruments, adjusted based on three-dimensional placement data, onto a background image, enabling the creation of realistic training images even for instruments with scarce clinical data, thereby enhancing recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of training images are required for deep learning, then recognition accuracy can be improved, but data acquisition time and cost increase significantly
Solution Approach 1:
The patent creates synthetic training images by copying and superimposing foreground images of treatment instruments onto background images of endoscopic scenes. This copying approach generates additional training data without requiring actual clinical images, thereby improving recognition accuracy while avoiding the time and ethical constraints of data acquisition from real surgeries.
Solution Approach 2:
The system performs preliminary actions by pre-acquiring and storing foreground images of treatment instruments and background images of endoscopic scenes before training is needed. These pre-prepared image libraries enable rapid generation of training images when required, eliminating the need for time-consuming data collection during the training phase.
2Ease of manufacture
If simple averaging of pixel intensities is used to form training images, then image creation is simple and fast, but the realism and recognition accuracy are insufficient
Solution Approach 1:
Instead of mathematically averaging pixel intensities, the patent copies actual foreground images of treatment instruments and superimposes them onto background images. This copying method preserves the realistic visual characteristics of instruments while maintaining ease of image generation through automated compositing.
Solution Approach 2:
The patent applies local quality by selectively superimposing foreground images only in regions where treatment instruments should appear, while preserving the original background quality in other areas. This localized approach ensures that training images have realistic instrument appearances with proper lighting and shading characteristics.
Data Source
AI summary
A learning support device includes a processor. The processor is configured to: form a foreground image containing at least one treatment instrument by placing an image of the at least one treatment instrument within an image region based on placement data; and form a training image by superimposing the foreground image on a background image. The placement data are data showing a three-dimensional placement of the at least one treatment instrument as viewed through an endoscope.