Semi-Supervised Medical Image Tracking With Cycle Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning-based tracking methods for objects of interest in medical images face challenges due to the difficulty and expense of annotating medical images, and require assumptions on device shape or human intervention, leading to sub-optimal accuracy and prolonged exposure to radiation.
Innovation Solution
A semi-supervised tracking method using a machine learning-based location predictor network trained with trajectory consistency, which tracks objects forward and backward through sequences of medical images, reducing the need for extensive annotation and enabling real-time performance without contrast agents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning based approaches are trained on extensively annotated datasets of natural images, then tracking accuracy is improved, but the difficulty and expense of annotating medical images increases
Solution Approach 1:
The patent uses cycle-consistent tracking to create synthetic annotated trajectories by copying and reversing the tracking path. The network tracks objects forward through a sequence of images and then backward, using the reversed path as synthetic annotations for training, thereby avoiding the need for expensive manual annotation of every frame.
Solution Approach 2:
The system performs self-supervised learning where the tracking network itself generates the training data. By tracking objects forward and then backward through the image sequence, the network creates its own synthetic ground truth annotations, eliminating the need for external manual annotation resources.
2Measurement precision
If contrast agents are administered to navigate in the patient, then object tracking visibility is improved, but harmful effects to the patient and prolonged radiation exposure increase
Solution Approach 1:
The patent replaces the mechanical/chemical contrast agent system with a computational tracking system. Instead of using contrast agents to make objects visible in medical images, the system uses deep learning-based tracking networks to identify and track objects of interest, substituting physical enhancement methods with intelligent image processing.
Solution Approach 2:
The patent introduces an intermediary computational tracking layer between the medical images and the analysis process. Rather than relying on contrast agents to directly enhance visibility, the tracking network serves as an intermediary that learns to identify and follow objects through image sequences, eliminating the need for harmful contrast agents.
3Reliability
If methods requiring human intervention are used for images with contrast agents, then tracking reliability is improved, but procedure time increases
Solution Approach 1:
The patent implements fully automated tracking where the deep learning network independently performs object tracking without requiring human intervention. The cycle-consistent training enables the network to automatically handle various imaging conditions, including those with contrast agents, thereby eliminating manual review time while maintaining high tracking reliability.
Solution Approach 2:
The patent uses cycle-consistent feedback where the tracking network is trained by comparing forward and backward tracking paths. This feedback mechanism ensures that the network learns to produce consistent and reliable tracking results automatically, reducing the need for human verification and thereby decreasing procedure time.
Data Source
AI summary
Systems and methods for tracking a location of an object of interest through a sequence of medical images are provided. First and second input medical images of a patient are received. The first input medical image comprises an annotation of a location of an object of interest. Features are extracted from the first and the second input medical images. A location of the object of interest in the second input medical image is determined using a machine learning based location predictor network based on the annotation of the location of the object of interest in the first input medical image and the extracted features from the first and the second input medical images. The location of the object of interest in the second input medical image is output. The machine learning based location predictor network is trained based on a comparison between 1) locations of a particular object in a sequence of training images determined during a forward tracking of the particular object through the sequence of training images and 2) locations of the particular object determined during a backward tracking of the particular object through the sequence of training images.