CNN-Based CLE Image Selection for Brain Surgery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Handheld Confocal Laser Endomicroscopy (CLE) devices during brain surgery generate a large number of images, many of which are not diagnostically useful due to artifacts such as motion and blood, making it time-consuming for surgeons or pathologists to sort through them for diagnostic purposes.
Innovation Solution
A method using a convolutional neural network (CNN) to selectively present images captured by CLE devices during brain surgery, where the CNN is trained on labeled images to classify images as diagnostic or non-diagnostic, allowing only diagnostically relevant images to be presented during surgery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If CLE devices capture images at a high rate during brain surgery, then real-time cellular-scale imaging is achieved, but the number of non-diagnostic images with artifacts increases significantly
Solution Approach 1:
The patent introduces an intermediary classification system (manual or automated) that mediates between the CLE device and the surgeon/pathologist. This intermediary filters out non-diagnostic images containing motion artifacts and blood artifacts, allowing the high capture rate to be maintained while preventing harmful artifacts from reaching the end user.
Solution Approach 2:
The patent extracts and removes non-diagnostic images with artifacts from the overall image stream. By identifying and separating problematic images (those with motion blur, blood artifacts, or insufficient histopathological features) from diagnostic images, the system maintains high capture rates while eliminating harmful artifacts from the final presentation.
2Measurement precision
If surgeons or pathologists manually sort through all captured images to identify diagnostic ones, then accurate diagnostic selection is achieved, but significant time is lost during the surgical procedure
Solution Approach 1:
The patent performs preliminary classification of images before they are presented to the surgeon or pathologist. By pre-sorting images into diagnostic and non-diagnostic categories (either manually or through automated algorithms), the system eliminates the time-consuming sorting step during surgery while maintaining accurate diagnostic selection.
Solution Approach 2:
The system enables self-service by implementing automated classification algorithms that independently identify and filter diagnostic images without requiring manual review of every captured frame. This allows the system to serve itself in the image selection process, dramatically reducing the time burden on surgical personnel.
3Loss of information
If all captured images are presented during surgery, then complete information is available, but the surgical process becomes inefficient and time-consuming
Solution Approach 1:
The patent extracts only the essential diagnostic information from the complete image set. By removing redundant non-diagnostic images (those with artifacts, motion blur, or insufficient features) while retaining all diagnostically useful images, the system maintains complete diagnostic information in a condensed, efficient format that improves surgical workflow.
Solution Approach 2:
The patent applies partial action by presenting a selective subset of captured images rather than all images. This partial presentation focuses on diagnostically relevant frames, eliminating excessive non-diagnostic content while ensuring all necessary diagnostic information is included, thereby optimizing surgical efficiency.
Data Source
AI summary
Systems, methods, and media for selectively presenting images captured by confocal laser endomicroscopy (CLE) are provided. In some embodiments, a method comprises: receiving images captured by a CLE device during brain surgery; providing the images to a convolution neural network (CNN) trained using at least a plurality of images of brain tissue captured by a CLE device and labeled diagnostic or non-diagnostic; receiving an indication, from the CNN, likelihoods that the images are diagnostic images; determining, based on the likelihoods, which of the images are diagnostic images; and in response to determining that an image is a diagnostic image, causing the image to be presented during the brain surgery.