Multi-frame Polyp Classification via Joint Embedding Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI-driven algorithms have not significantly improved the ability of physicians to accurately classify polyps during colonoscopy procedures, which is crucial for preventing colon cancer.
Innovation Solution
A system utilizing a combination of pretrained machine learning models, including a convolutional neural network and a transformer, to generate embedding vectors and classify polyps by jointly analyzing frames from medical videos, thereby improving classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single-frame classification approach is used, then the processing speed is fast, but the classification accuracy is insufficient
Solution Approach 1:
The system segments the video into multiple individual frames and processes each frame separately through the classification model, then aggregates the results. This allows the model to analyze temporal patterns across frames while maintaining efficient processing of individual frames, resolving the contradiction between accuracy and complexity.
Solution Approach 2:
The system transitions from analyzing a single frame in two dimensions (width and height) to analyzing multiple frames by adding a temporal dimension. This enables the model to capture temporal variations and patterns that improve classification accuracy without requiring a fundamentally more complex model architecture.
2Reliability
If multiple frames are analyzed individually and aggregated, then the processing is simpler, but the classification reliability is reduced due to low-quality frames and outliers
Solution Approach 1:
The system merges the analysis of multiple frames by processing them through the same classification model and aggregating the results. This combining approach allows the model to leverage information from multiple frames while maintaining consistent analysis criteria, improving reliability without excessive complexity.
Solution Approach 2:
The system incorporates feedback mechanisms where the classification results from individual frames are aggregated and used to improve the overall classification decision. The model can adjust its predictions based on patterns observed across multiple frames, enhancing reliability while managing complexity through iterative refinement.
Data Source
AI summary
Methods, systems, and devices for classifying a target feature in a medical video are presented herein. Some methods may include the steps of: receiving a plurality of frames of the medical video, where the plurality of frames include the target feature; generating, by a first pretrained machine learning model, an embedding vector for each frame of the plurality of frames, each embedding vector having a predetermined number of values; and generating, by a second pretrained machine learning model, a classification of the target feature using the plurality of embedding vectors, where the second pretrained machine learning model analyzes the plurality of embedding vectors jointly.