AI Surgical Image Annotation Using Feature-Point Video Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI-assisted surgical systems are inadequate for documenting surgical procedures, lacking in efficiency and effectiveness in identifying and labeling anatomical features during medical procedures.
Innovation Solution
Utilizing AI algorithms and machine learning techniques to annotate medical images and videos in real-time or post-operatively, identifying anatomical structures, surgical tools, and procedural actions, and generating annotated outputs for documentation and patient education.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI algorithms are used to annotate medical images in real-time, then annotation speed and productivity are improved, but computational complexity and processing time requirements increase
Solution Approach 1:
The patent segments the medical image annotation task into multiple processing stages: feature extraction, candidate region identification, classification, and annotation generation. This segmentation allows each stage to be optimized independently and processed in parallel where possible, improving overall annotation speed while managing computational complexity through modular architecture
Solution Approach 2:
The system performs preliminary actions by pre-processing medical images to extract key features and pre-identifying potential annotation candidates before the actual annotation process. This preliminary processing reduces the computational burden during real-time annotation, enabling faster processing without sacrificing accuracy
2Measurement precision
If comprehensive annotation of all anatomical features is performed, then measurement precision and documentation quality are improved, but processing time and computational resources increase
Solution Approach 1:
The patent applies local quality by focusing annotation efforts on specific regions of interest within medical images rather than uniformly processing entire images. The system identifies and prioritizes annotation of clinically significant areas, applying higher measurement precision where needed while reducing processing overhead in less critical regions
Solution Approach 2:
The system uses template-based annotation copying where previously annotated features and annotation patterns are reused as templates for similar structures. This allows rapid propagation of accurate annotations across multiple images or regions, maintaining high measurement precision while significantly reducing processing time
3Loss of information
If multiple annotation layers and detailed information are provided, then information completeness is improved, but data processing complexity and storage requirements increase
Solution Approach 1:
The patent implements multiple annotation layers by adding information in different dimensional layers rather than increasing complexity within a single layer. Each annotation layer operates at a different level of detail and abstraction, allowing comprehensive information storage while maintaining manageable processing complexity through hierarchical organization
Data Source
AI summary
AI-based systems and methods to annotate images and video from a surgical procedure may include extracting feature points (FP) from a still image (SI) of the procedure and each frame of a group of frames (GOF) from a procedure video and comparing FPs of the SI to FPs of each frame of the GOF to determine a match between the SI and a given frame from the video and a location in the video where the match occurred. Then the portion of the video containing the matched image is copied to create a video clip (VC) having a selected duration. Frames in the VC are analyzed to identify image features (IF), where a fidelity of IF identification is substantially unaffected when the IF is non-localizable or obscured. Then the SI and/or VC is annotated with information from the procedure, the information including or derived from the IFs.