Few-Shot Microscopy Image Processing with Rapid Model Retraining

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Charged particle microscopy generates large datasets that require significant processing, and creating machine-learning models for image analysis often necessitates large datasets not readily available, leading to inefficiencies in training and inference times.

Innovation Solution

A scientific instrument support system that includes logic to receive microscopy images, process them through a general machine-learning model, retrain the model with labeled images, and generate processed images, reducing the need for extensive training data and time by employing a general model that can be fine-tuned with few samples, thereby improving inference performance and adaptability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a machine-learning model is trained using a large dataset of microscopy images, then the model's accuracy and reliability are improved, but the training time and computational resources required increase significantly

Engineering Contradiction:
Improvemodel accuracyVSAvoidtraining time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-training a general machine-learning model on a large, diverse dataset of microscopy images before deployment. This pre-trained model serves as a foundation that has already learned general features and patterns across multiple domains. When the model needs to be adapted to a specific application, it requires only fine-tuning with a small subset of data rather than training from scratch, thus significantly reducing training time while maintaining high accuracy.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements universality by creating a general machine-learning model that can perform multiple functions across different microscopy image analysis tasks. The model is trained on a diverse dataset encompassing various sample types, imaging modalities, and analysis objectives, enabling it to generalize to new tasks with minimal retraining. This multi-functional approach allows a single model to serve multiple purposes, reducing the need for separate large-scale training for each application.

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Measurement precision

If a machine-learning model is trained from scratch for each specific application, then the model achieves high precision for that task, but the time and data resources required for training increase

Engineering Contradiction:
Improvetask-specific accuracyVSAvoidtraining data required
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent uses preliminary action by pre-training a general model on a large, diverse dataset that encompasses multiple domains and tasks. This pre-trained model already possesses general knowledge and features that transfer across different applications. For specific tasks, only a small amount of task-specific data is needed for fine-tuning, dramatically reducing the quantity of training data required compared to training from scratch while maintaining high task-specific accuracy.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent applies copying by using the general pre-trained model as a template or foundation that can be copied and adapted to multiple specific applications. Rather than creating new models from scratch for each task, the system copies the general model and performs targeted fine-tuning with minimal data. This copying approach preserves the general knowledge learned during pre-training while adapting it to specific tasks efficiently.

Inventive Principle:
Principle #26Copying

3Adaptability or versatility

If a general machine-learning model is used for multiple tasks, then the system's adaptability improves, but the model may struggle to achieve high precision on specific tasks without extensive retraining

Engineering Contradiction:
Improvemulti-task capabilityVSAvoidtask-specific accuracy
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent applies preliminary action by pre-training the general model on a diverse dataset that includes multiple tasks and domains. This pre-training establishes a strong foundation of general features and patterns that transfer across tasks. When adapting to specific tasks, the model requires only fine-tuning with a small amount of task-specific data, achieving high precision without extensive retraining. The preliminary action of pre-training enables both versatility and precision simultaneously.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements dynamics by making the model adaptable through fine-tuning on task-specific data. The general pre-trained model serves as a flexible foundation that can be dynamically adjusted to specific tasks by updating its parameters with minimal data. This dynamic adaptation allows the model to maintain high versatility across multiple tasks while achieving high precision on individual tasks through targeted fine-tuning when needed.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS20240354924A1Few-shot learning for processing microscopy images
Publication Date: 2024.10.24 FEI CO
  • US20240354924A1 patent drawing
  • US20240354924A1 patent drawing
  • US20240354924A1 patent drawing

AI summary

Disclosed herein are scientific instrument support systems, as well as related methods, computing devices, and computer-readable media. For example, in some embodiments, a scientific instrument support apparatus may include: first logic to receive, from a charged particle microscope, a microscopy image of a sample; second logic to generate a first processed image by processing the microscopy image through a general machine-learning model trained using a plurality of previously processed microscopy images; third logic to retrain the general machine-learning model with a related microscopy image, wherein the related microscopy image includes a label of an object related to the sample and the related microscopy image is not included in the plurality of previously processed microscopy images; and fourth logic to generate a second processed image, different from the first processed image, by processing the microscopy image through the retrained general machine-learning model.