Automated EM Image Labeling via Secondary Detector Composition Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of generating training sets of labeled electron microscopy images is inefficient due to the need for expert operators to manually mark pixels, which is time-consuming and labor-intensive, especially when high-resolution images require hours to segment.
Innovation Solution
The method involves using a combination of microscope detector systems of different modalities to automatically generate labeled images by applying a focused charged beam to a sample, detecting emissions, and leveraging detector data from one system to label images with composition information, thereby expediting the process of creating training sets for deep learning algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If expert operators manually mark pixels to generate labeled EM images, then composition information can be accurately obtained, but the process takes huge amounts of time (hours to months)
Solution Approach 1:
The patent uses a secondary detector as an intermediary to obtain composition information that is then transferred to label the primary detector images. The secondary detector captures composition data through a different physical mechanism (e.g., EDS for elemental composition), which serves as a mediator to automatically generate labels for the primary imaging detector, eliminating the need for manual pixel-by-pixel marking while preserving accuracy.
Solution Approach 2:
The patent replaces the mechanical/manual process of expert operators marking pixels with an automated system that uses secondary detector data and machine learning algorithms. The automated labeling system substitutes human manual work with computational processes that analyze secondary detector signals and transfer that information to label primary detector images automatically.
2Productivity
If deep learning algorithms are used to automatically analyze EM images, then analysis speed increases, but training sets require extensive manual labeling time
Solution Approach 1:
The patent performs preliminary action by using secondary detectors to pre-obtain composition information during the imaging process itself. This composition data is prepared in advance and used to automatically generate labels for training sets, so that when deep learning models need training data, it is already available without requiring subsequent manual labeling efforts.
Solution Approach 2:
The system uses itself to generate training data by leveraging the secondary detector's composition information to automatically label primary detector images. The imaging system self-generates its own training data without external manual intervention, creating a self-sufficient workflow where the same instrument that captures images also provides the composition data needed for automatic labeling.
3Measurement precision
If EM systems obtain high resolution images, then imaging quality improves, but composition information cannot be acquired
Solution Approach 1:
The patent merges data from two different detector modalities: the primary detector that provides high-resolution structural imaging and the secondary detector that provides composition information. By combining these two data streams and using the secondary detector's composition data to label the primary detector's high-resolution images, the system achieves both high imaging quality and composition information acquisition simultaneously.
Solution Approach 2:
The patent adds another dimension of information by incorporating composition data from a secondary detector modality. Instead of relying on a single detector that must choose between resolution or composition, the system operates in a multi-dimensional data space where structural information from the primary detector is enriched with compositional information from the secondary detector, creating a more complete characterization of the sample.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach automates the generation of labeled electron microscopy images, significantly reducing the time and effort required to create training sets, enabling faster analysis, labeling, and correction of abnormalities using deep learning techniques.
Implementation Method 1
applying a focused charged beam to a sample, using a first microscope detector system of a first modality to detect emissions resultant from the focused charged beam being incident on the sample
Data Source
AI summary
Methods and systems for generating labeled images from a microscope detector by leveraging detector data from a different microscope detector of a different modality include applying a focused charged beam to a sample, using a first microscope detector to detect emissions resultant from the focused charged beam being incident on the sample, and then using detector data from the first microscope detector to automatically generate a first labeled image. Automatically generating the first labeled image includes determining composition information about portions of the sample based on the detector data, and then automatically labeling regions of the first image associated with the portions of the sample with corresponding composition information. A second image of the sample is generated using detector data from a second microscope detector system of a different modality, and then the first labeled image is used to automatically label regions of the second image with corresponding composition information.