Deep Learning Instance Segmentation via Partial Annotations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning-based models for nuclei segmentation in digital pathology face challenges due to the difficulty in obtaining large training datasets with precise annotations, leading to poor performance when applied to different tissue types, and require significant additional annotation efforts in transfer-learning scenarios.
Innovation Solution
Implementing instance segmentation using deep learning-based methods with partial annotations and a user interface that facilitates efficient user annotation, leveraging partial annotations to train an AI model for predicting object instances, and utilizing morphological and machine learning operations to enhance segmentation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models are trained with full manual annotations of nuclei boundaries, then segmentation accuracy is improved, but annotation time and labor cost increase significantly
Solution Approach 1:
The patent applies partial annotation by requiring annotators to mark only the center point of each nucleus rather than drawing complete boundary contours. This partial action (center point only) reduces annotation time significantly while the model learns to infer complete segmentation boundaries from these partial cues through the dual-regression framework that predicts both distance transform and boundary distance maps
Solution Approach 2:
The patent introduces an intermediary computational process between center point annotations and final segmentation masks. The dual-regression model acts as an intermediary that transforms simple center point coordinates into accurate segmentation boundaries by learning the relationship between center positions and boundary locations through training on partially annotated data
2Measurement precision
If deep learning models are trained on specific tissue types with full annotations, then performance on those tissue types is improved, but performance on different tissue types deteriorates
Solution Approach 1:
The patent creates a universal segmentation framework that works across multiple tissue types by training on diverse datasets containing different tissue types simultaneously. The dual-regression model learns tissue-agnostic features and relationships between center points and boundaries that generalize to unseen tissue types, making the model multi-functional rather than specialized for a single tissue type
3Reliability
If large datasets with precise annotations are collected for training, then model robustness is improved, but data collection complexity and cost increase
Solution Approach 1:
The patent enables robust model training by collecting large datasets with simplified partial annotations (center points only) rather than requiring complex complete boundary drawings. The dual-regression framework leverages this partial annotation efficiency to scale up data collection, accumulating thousands of training samples with minimal annotation effort per sample while maintaining model robustness through the power of large dataset training
Data Source
Figure 1
Figure 2A~2B
Figure 2C
AI summary
Novel tools and techniques are provided for implementing digital microscopy imaging using deep learning-based segmentation and/or implementing instance segmentation based on partial annotations. In various embodiments, a computing system might receive first and second images, the first image comprising a field of view of a biological sample, while the second image comprises labeling of objects of interest in the biological sample. The computing system might encode, using an encoder, the second image to generate third and fourth encoded images (different from each other) that comprise proximity scores or maps. The computing system might train an AI system to predict objects of interest based at least in part on the third and fourth encoded images. The computing system might generate (using regression) and decode (using a decoder) two or more images based on a new image of a biological sample to predict labeling of objects in the new image.