Microscope Workflow Automation Using Trained Models

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing microscope systems face inefficiencies in data processing, limited application support, and high costs due to preprogrammed schemes, loss of original image information during postprocessing, and unpredictable malfunctions, leading to increased costs and time for experiments.

Innovation Solution

Implementing a method and apparatus that utilize trained models, such as neural networks, to capture data and make decisions in real-time, allowing for adaptive workflows, improved accuracy, and continuous model updates through data aggregation and feedback loops.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If preprogrammed schemes are used to control microscope experiments, then the workflow can be automated, but the system cannot adapt to unexpected biological phenomena and requires complete termination to intervene

Engineering Contradiction:
Improveworkflow automationVSAvoidadaptability to biological phenomena
Core Design Contradiction:
Extent of automationVSAdaptability or versatility

Solution Approach 1:

The patent implements feedback mechanisms where trained models continuously analyze recorded images and provide real-time feedback to the control unit. This enables the system to automatically adapt the workflow based on detected biological phenomena without requiring complete termination or manual intervention, resolving the contradiction between automation and adaptability.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system dynamically adjusts the experimental workflow based on real-time analysis by trained models. The control parameters and acquisition conditions are continuously optimized according to the detected state of the specimen, transforming the static preprogrammed scheme into a dynamic adaptive process that maintains both automation and versatility.

Inventive Principle:
Principle #15Dynamics

2Reliability

If preprogrammed schemes record comprehensive data, then all possible biological phenomena can be captured, but excessive non-informative data increases storage and evaluation costs

Engineering Contradiction:
Improvecompleteness of data captureVSAvoiddata storage and evaluation costs
Core Design Contradiction:
ReliabilityVSLoss of substance

Solution Approach 1:

Instead of recording all possible data comprehensively, the system uses trained models to perform partial action by selectively recording only the most relevant and informative data. The models predict which data points are likely to contain meaningful biological phenomena, allowing the system to capture sufficient information for reliable analysis while significantly reducing storage and evaluation costs.

Inventive Principle:
Principle #16Partial or excessive action

Solution Approach 2:

The patent replaces the mechanical approach of comprehensive data recording with an intelligent system based on trained models. These models substitute the brute-force method of capturing all data with a selective, prediction-based approach that identifies and records only the most valuable information, reducing resource consumption while maintaining reliability.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If conventional systems purchase additional upgrades for new applications, then the spectrum of application can be expanded, but high costs are incurred

Engineering Contradiction:
Improvespectrum of applicationVSAvoidcost of upgrades
Core Design Contradiction:
Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

The patent implements a universal platform where a single trained model system can handle multiple different applications and specimen types. By training models on diverse datasets and using transfer learning, the system achieves multi-functionality without requiring separate hardware upgrades for each new application, significantly reducing costs while expanding the spectrum of usable applications.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system expands its application spectrum by changing parameters in the trained models rather than through hardware modifications. Different applications are handled by adjusting model parameters, training data, and processing conditions, allowing the same physical system to adapt to new applications cost-effectively through software-based parameter changes.

Inventive Principle:
Principle #35Parameter changes

4Measurement precision

If feedback methods are used to capture image recording conditions, then the accuracy can be improved, but reliable feedback is only possible after a sufficiently large number of recorded images

Engineering Contradiction:
Improveaccuracy of image processingVSAvoidtime to accumulate sufficient data
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-training models on large datasets before actual experimentation. This preliminary training phase allows the models to learn from extensive data in advance, so that during actual experiments, reliable feedback can be provided much faster without requiring accumulation of large numbers of images during the measurement process itself.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system provides beforehand cushioning by using pre-trained models that have already learned from extensive training data. This cushioning effect allows the system to maintain high measurement precision from the beginning of experiments without needing to accumulate sufficient data during the actual measurement, as the models are already prepared with prior knowledge.

Inventive Principle:
Principle #11Beforehand cushioning (Prior cushioning)

5Measurement precision

If postprocessing of images is used to optimize results, then the accuracy can be improved, but information from the recorded original images may be lost

Engineering Contradiction:
Improveaccuracy of measurement resultsVSAvoidinformation from original images
Core Design Contradiction:
Measurement precisionVSLoss of information

Solution Approach 1:

The patent inverts the conventional approach by applying trained models to the original recorded images before any postprocessing losses occur. Instead of recording images and then losing information during postprocessing, the system uses AI models to extract and preserve critical information directly from the original images, maintaining measurement precision while preventing information loss by reversing the traditional workflow sequence.

Inventive Principle:
Principle #13The other way round (Inversion)

Data Source

PatentUS12002273B2Inference microscopy
Publication Date: 2024.06.04 LEICA MICROSYSTEMS CMS GMBH
  • US12002273B2 patent drawing
  • US12002273B2 patent drawing
  • US12002273B2 patent drawing

AI summary

An apparatus for optimizing workflows of one or more microscopes and/or microscope systems includes one or more processors and one or more computer-readable storage media. The one or more computer-readable storage media have stored therein computer-executable instructions, which, when executed by the one or more processors cause execution of the following steps: implementing, by one or more components of the one or more microscopes and/or microscope systems, a workflow comprising a capture of first data; applying one or more trained models to the captured first data; and making at least one decision in relation to the workflow based on the application of the one or more trained models to the captured first data.