Mass Image Quality Conversion Using Cross-Modality Transfer Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in improving the image quality of mass images is exacerbated by the small amount of ions acquired from each micro area, leading to low signal-to-noise ratios and difficulty in applying machine learning-based image quality conversion models, as high-definition mass images are hard to obtain.
Innovation Solution
A mass image processing apparatus comprising a pre-processor, converter, and post-processor that applies intensity scaling and noise correction processes to fit the image quality conversion model's input conditions, allowing for effective transfer learning from scanning electron microscope images to enhance mass image quality, even when high-definition mass images are not available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If a simple smoothing filter is applied to reduce noise in mass images, then noise prominence is reduced, but the entire mass image becomes blurred
Solution Approach 1:
The patent introduces a machine learning model as an intermediary between the noisy mass image and the final processed image. This model, trained on paired low-definition and high-definition images from other modalities, acts as a mediator that can suppress noise while preserving or enhancing fine details, avoiding the blurring effect of traditional smoothing filters.
Solution Approach 2:
The patent changes the fundamental parameter of image processing from simple spatial filtering to complex machine learning-based transformation. By training a neural network model to learn the mapping between low-definition and high-definition images, the system transforms the processing approach from local pixel manipulation to global pattern recognition, enabling noise reduction without detail loss.
2Measurement precision
If machine learning-based image quality conversion models are used to improve mass image quality, then image quality can be enhanced, but high-definition mass images are difficult to obtain for model training
Solution Approach 1:
The patent uses images from other modalities (such as optical microscopy or electron microscopy) as substitutes or copies for the unavailable high-definition mass images. These alternative images serve as proxies to train the machine learning model, allowing the system to learn the transformation to high-definition images without requiring actual high-definition mass spectrometry images for training.
Solution Approach 2:
The patent applies a universal machine learning model that can be trained on data from multiple sources and modalities. The model learns general image enhancement patterns that are transferable to mass images, making the training process more flexible and reducing dependency on specific high-definition mass image datasets.
3Measurement precision
If high-definition mass images are acquired to train image quality conversion models, then model training can be performed, but the small amount of ions from each micro area makes this extremely difficult
Solution Approach 1:
The patent uses images from other imaging modalities as substitutes for the unavailable high-definition mass images. These alternative images provide the necessary training data without requiring additional ion acquisition, effectively copying the structural information from one modality to train the model for another modality.
Solution Approach 2:
The patent introduces a machine learning model trained on alternative modality images as an intermediary that bridges the gap between low ion amount measurements and high-definition image requirements. This intermediary translates information from modalities with sufficient signal to the mass imaging domain.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A pre-processor (46) applies a pre-process to an original mass image produced through mass spectrometry of a sample, to produce a model input image. An image quality converter (48) has an image quality conversion model produced through machine learning based on a group of images produced by a scanning electron microscope, and produces a model output image through image quality conversion of the model input image. A post-processor (50) applies a post-process to the model output image, to produce a mass image after image quality conversion.