Microscope full-automatic intelligent parameter setting method and system

Abstract

The invention discloses a full-automatic intelligent parameter setting method and system for a microscope, and belongs to the field of microscopic imaging, and the method comprises the steps: carrying out the field adaptive fine tuning of a basic image encoder of an SAM2 model, carrying out the instance segmentation of a potential interested target through the optimized SAM2 model, and marking the boundary of each target; analyzing the segmented image by using a deep learning model, and predicting initial acquisition parameters; performing image acquisition according to the initial acquisition parameters, and calculating the score of the quality index of the acquired image; when the score of the quality index reaches a preset optimal threshold value or converges, taking the acquisition parameter at the moment as a final setting parameter; otherwise, the initial acquisition parameters are finely adjusted until the score of the quality index of the acquired image reaches a preset optimal threshold value or converges, through the steps, artificial intelligence and an automatic control technology are deeply fused, the use threshold and the professional requirement of equipment are reduced, and the efficiency, the standardization degree and the repeatability of data acquisition are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of microscopic imaging, and in particular to a fully automated intelligent parameter setting method and system for microscopes. Background Technology

[0002] Laser confocal microscopy, as the standard method for high-resolution 3D imaging, effectively suppresses defocus light and provides excellent axial resolution and image contrast, making it widely used in cell biology and medical research. However, confocal systems are complex in structure, image quality is greatly affected by parameters, and scanning speed is relatively slow. Beginners often find it difficult to quickly set the parameters corresponding to the sample, locate the target area, and perform scanning. Currently, most laser confocal microscopes on the market have increasingly complex parameter settings to cope with various usage scenarios, requiring users to understand and become familiar with various technical concepts related to laser confocal microscopy, resulting in a high learning cost. Summary of the Invention

[0003] In order to overcome the shortcomings of the prior art, one of the objectives of this invention is to provide a fully automatic intelligent parameter setting method for microscopes that can intelligently generate parameters.

[0004] In order to overcome the shortcomings of the prior art, the second objective of this invention is to provide a fully automatic intelligent parameter setting system for microscopes that can intelligently generate parameters.

[0005] One of the objectives of this invention is achieved through the following technical solution: A fully automated intelligent parameter setting method for a microscope includes the following steps: Sample Acquisition: Acquire a global preview image of the sample; Target segmentation: Using a large dataset of fluorescence and bright-field microscopic images containing various cell and tissue structures, the basic image encoder of the SAM2 model is fine-tuned for domain adaptation, making it more capable of extracting typical features from microscopic images. The optimized SAM2 model is then used to segment multiple potential targets of interest in the global preview image and identify the boundaries of each target. Intelligent analysis: Utilizes a pre-trained deep learning model to perform intelligent analysis on the segmented image, and predicts a set of near-optimal initial acquisition parameters based on the analysis results; Score Calculation: Based on the initial acquisition parameters, the quality index score of the acquired image is calculated. Parameter setting: When the quality index score reaches the preset optimal threshold or converges, the acquisition parameters at this time are used as the final setting parameters; when the quality index score does not reach the preset optimal threshold and does not converge, multi-dimensional, small-step parameter fine-tuning is performed within a limited range near the initial acquisition parameters until the quality index score of the acquired image reaches the preset optimal threshold or converges.

[0006] Furthermore, the fully automated intelligent parameter setting method for the microscope also includes a multiple scanning step, which is located after the parameter formation step, and the multiple scanning step includes: Retrieve historical reference images of the target area and their corresponding displacement stage coordinates; The control stage is moved to the vicinity of the coordinates, and a current preview image containing the expected target area and its surrounding environment is acquired; The registration parameters between the current preview image and the historical reference image are obtained through an image registration algorithm; Calculate the area to be collected based on the registration parameters.

[0007] Furthermore, the specific steps of obtaining the registration parameters between the current preview image and the historical reference image through the image registration algorithm are as follows: The transformation parameters are predicted by a deep learning model. The encoder part of the deep learning model is used to extract the depth features of the latest preview image and the historical reference image, and calculate the similarity between the features through the relevant layer. Finally, the decoder outputs the accurate affine transformation matrix. The affine transformation matrix is ​​used to transform the current preview image, and the matching degree between the transformed image and the historical reference image is evaluated. When the matching degree meets the requirements, the transformation parameters are output; when the matching degree does not meet the requirements, the scale-invariant feature transformation algorithm is used for feature matching. This algorithm detects extreme points in the scale space as feature points and generates their high-dimensional description vectors. Then, through nearest neighbor matching between feature vectors, the affine transformation matrix between the two images is calculated.

[0008] Furthermore, in the sample acquisition step, a wide-field camera channel is used to acquire a global preview image of the sample by utilizing the camera's high-speed imaging characteristics.

[0009] Furthermore, in the sample acquisition step, composite dark-field illumination is used to enhance image contrast.

[0010] Furthermore, in the target segmentation step, typical features in the microscopic image include at least one of cell boundaries, cell nuclei, and neural synapses.

[0011] Furthermore, in the target segmentation step, the step of using the optimized SAM2 model to perform instance segmentation of multiple potential targets of interest in the global preview image specifically involves: automatically generating a series of initial cue points or boxes using a target detection algorithm or saliency analysis, and using the series of initial cue points or boxes as input to the SAM2 model.

[0012] Furthermore, in the parameter setting step, the final setting parameters include at least one of laser power, PMT gain, pinhole diameter, and scanning speed.

[0013] Furthermore, in the score calculation step, the image quality index includes at least one of signal-to-noise ratio, dynamic range, and background uniformity.

[0014] The second objective of this invention is achieved by the following technical solution: A fully automated intelligent parameter setting system for microscopes is provided for implementing any of the aforementioned fully automated intelligent parameter setting methods for microscopes. The fully automated intelligent parameter setting system for microscopes includes an acquisition module, an identification module, an analysis module, and a judgment module. The acquisition module is used to acquire sample images. The identification module performs instance segmentation on the target of interest and identifies its boundaries. The analysis module predicts initial acquisition parameters and calculates the scores of quality indicators. The judgment module determines whether the acquisition parameters need to be optimized based on the scores of the quality indicators.

[0015] Compared to existing technologies, this invention's fully automated intelligent parameter setting method for microscopes utilizes a large dataset of fluorescence and bright-field microscopic images containing various cell and tissue structures. It performs domain-adaptive fine-tuning of the SAM2 model's basic image encoder, enhancing its feature extraction capabilities for typical features in microscopic images. The optimized SAM2 model is then used to segment multiple potential targets of interest in the global preview image, identifying the boundaries of each target. A pre-trained deep learning model intelligently analyzes the segmented images, predicting a near-optimal set of initial acquisition parameters based on the analysis results. Images are then acquired using these initial acquisition parameters, and the quality index of the acquired images is calculated. The target score is determined by the acquisition parameters. When the score reaches the preset optimal threshold or converges, the acquisition parameters at that time are used as the final set parameters. When the score does not reach the preset optimal threshold and does not converge, multi-dimensional, small-step parameter fine-tuning is performed within a limited range near the initial acquisition parameters until the quality index score of the acquired image reaches the preset optimal threshold or converges. Through the above steps, artificial intelligence and automatic control technology are deeply integrated, which greatly reduces the threshold for using the equipment and the professional requirements, and significantly improves the efficiency, standardization and repeatability of data acquisition, providing strong technical support for cutting-edge applications such as high-throughput screening and long-term live cell imaging. Attached Figure Description

[0016] Figure 1 This is a flowchart of the fully automated intelligent parameter setting method for microscopes according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or fixed through another intermediate component. When a component is said to be "connected to" another component, it can be directly connected to the other component or may also have another intermediate component present. When a component is said to be "set to" another component, it can be directly set on the other component or may also have another intermediate component present.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0020] Please see Figure 1 This application discloses a fully automated intelligent parameter setting method for a microscope, comprising the following steps: Sample Acquisition: Acquire a global preview image of the sample; Target segmentation: Using a large dataset of fluorescence and bright-field microscopic images containing various cell and tissue structures, the basic image encoder of the SAM2 model is fine-tuned for domain adaptation, making it more capable of extracting typical features from microscopic images. The optimized SAM2 model is then used to segment multiple potential targets of interest in the global preview image and identify the boundaries of each target. Intelligent analysis: Utilizes a pre-trained deep learning model to perform intelligent analysis on the segmented image, and predicts a set of near-optimal initial acquisition parameters based on the analysis results; Score Calculation: Based on the initial acquisition parameters, the quality index score of the acquired image is calculated. Parameter setting: When the quality index score reaches the preset optimal threshold or converges, the acquisition parameters at this time are used as the final setting parameters; when the quality index score does not reach the preset optimal threshold and does not converge, multi-dimensional, small-step parameter fine-tuning is performed within a limited range near the initial acquisition parameters until the quality index score of the acquired image reaches the preset optimal threshold or converges.

[0021] In the sample acquisition step, a wide-field camera channel is added to utilize the camera's high-speed imaging capabilities to acquire a global preview image of the sample. To address the issue of difficulty in obtaining clear images of samples under certain lighting intensities, composite dark-field illumination is employed to enhance image contrast, thus obtaining a clear global preview image. Based on this image, the region of interest is automatically segmented and located.

[0022] The specific steps for target segmentation are as follows: Domain-adaptive training: Using a large dataset of fluorescence and bright-field microscopic images containing various cell and tissue structures, the model's basic image encoder is fine-tuned for domain adaptation, giving it a stronger ability to extract typical features in microscopic images (such as cell boundaries, cell nuclei, and neural synapses), significantly improving the robustness of target recognition in complex biological contexts.

[0023] Optimized Prompt Strategy: To address the need for "no manual prompting" in automated processes, an automatic prompt generation mechanism was developed. This mechanism, based on low-magnification preview images, automatically generates a series of initial prompt points or bounding boxes using object detection algorithms or saliency analysis. These initial prompt points or bounding boxes are then used as input to the SAM2 model, thus achieving a functional leap from "interactive segmentation" to "fully automated segmentation."

[0024] Lightweighting and efficiency improvement: To meet the needs of real-time processing, selective pruning and quantization were performed on the model structure. While ensuring segmentation accuracy, the inference speed of the model on conventional computing devices was greatly improved, ensuring the smoothness of the entire automated acquisition process.

[0025] This application first obtains low-magnification preview images of the samples. After optimization and retraining, the SAM 2 model automatically performs instance segmentation on multiple potential targets of interest in the image, accurately identifying the pixel-level boundaries of each target. This method can efficiently handle complex scenes such as overlapping cells and irregular morphological structures, and its segmentation accuracy and generalization ability are far superior to traditional image processing methods and untrained basic models.

[0026] The intelligent analysis steps are as follows: A pre-trained deep learning model is used to intelligently analyze the initial preview images. The deep learning model employs a reinforcement learning framework, treating the microscope environment as an "environment," parameter adjustments as "actions," and improvements in image quality scores as "rewards." Through continuous interaction with the environment, it learns the optimal parameter adjustment strategy. This deep learning model is trained on massive amounts of expert experience data and can quickly predict and recommend a set of near-optimal initial acquisition parameters (including laser power, PMT gain, pinhole diameter, and scanning speed) based on characteristics such as the sample's fluorescent dye type, tissue density, and signal intensity distribution, providing a high-level optimization benchmark for subsequent fine-tuning.

[0027] The specific steps for calculating fractions are as follows: Image acquisition is performed based on initial acquisition parameters. The real-time image quality assessment algorithm performs quantitative analysis on the acquired image and calculates the scores of key quality indicators such as signal-to-noise ratio, dynamic range, and background uniformity.

[0028] The specific steps for forming parameters are as follows: When the quality metric score reaches the preset optimal threshold or converges, the acquired parameters at that time are used as the final set parameters. When the quality metric score does not reach the preset optimal threshold and does not converge, multi-dimensional, small-step parameter fine-tuning is performed within a limited range near the initial acquisition parameters until the quality metric score of the acquired image reaches the preset optimal threshold or converges, forming an efficient closed-loop feedback control mechanism. These quality scores serve as feedback signals, and the Bayesian optimization algorithm dynamically determines the next set of better parameter combinations, and acquisition and evaluation are performed again. This iterative process continues until the image quality assessment score reaches the preset optimal threshold or converges, thereby accurately and automatically locking the optimal parameter set for the current specific sample and field of view.

[0029] The intelligent analysis and parameter formation steps combine the macroscopic predictive capabilities of artificial intelligence with the microscopic precision adjustment capabilities of closed-loop control. This not only completely eliminates the reliance on human experience and significantly improves the speed and accuracy of parameter setting, but also effectively avoids phototoxic damage and signal overexposure, ensuring that high-quality, high-fidelity, and repeatable microscopic images can be obtained under most conditions. This provides strong technical support for quantitative research and automated high-throughput screening in the life sciences.

[0030] The microscope described in this application is a laser confocal microscope. In scientific research and testing applications of laser confocal microscopes, it is often necessary to scan the same batch of samples multiple times (e.g., to observe time-series changes, acquire additional high-resolution images, or perform multi-region analysis). Currently, rescanning operations heavily rely on the coordinate memory function of a high-precision displacement stage for position recovery. However, due to factors such as mechanical errors in the sample stage, secondary placement deviations of the sample, and possible minor deformations of the sample itself, sub-pixel-level precise repositioning cannot be achieved solely based on the displacement stage coordinates. This results in misalignment of image areas between consecutive scans, severely affecting the comparability of data and the accuracy of analysis.

[0031] To address the aforementioned technical issues, the fully automated intelligent parameter setting method for microscopes in this application also includes a multiple scanning step. This multiple scanning step abandons the traditional positioning approach that relies solely on physical coordinates and adopts a visual positioning strategy based on image content. This enables intelligent and precise repositioning that ignores mechanical and placement errors, ensuring that the rescanned image can be transformed in post-processing to obtain image boundaries consistent with those of historical acquisitions.

[0032] The multiple scan step follows the parameter formation step, and the multiple scan step is as follows: First, retrieve historical reference images of the target area and their corresponding displacement stage coordinates, which serve as a rough center for initial localization. Then, move the displacement stage to near these coordinates and acquire a current preview image with a wider field of view, encompassing the expected target area and its surrounding environment. Next, obtain image registration parameters using a self-developed image registration algorithm. This algorithm is tailored to the application scenario and maximizes its effectiveness across various conditions. The specific steps are as follows: Obtain historical reference images; The transformation parameters are predicted using a self-developed deep learning model. This model is an encoder-decoder structure. The encoder part is used to extract the depth features of the latest preview image and the historical reference image, and calculates the similarity between features through the relevant layer. Finally, the decoder outputs the accurate affine transformation matrix. The output affine transformation matrix is ​​applied to transform the current preview image, and the matching degree between the transformed image and the historical reference image is evaluated. When the matching degree meets the requirements, the transformation parameters are output; when the matching degree does not meet the requirements, the Scale Invariant Feature Transform (SIFT) algorithm is used for feature matching. This algorithm detects extreme points in the scale space as feature points and generates their high-dimensional descriptive vectors. Then, through nearest neighbor matching between feature vectors, the affine transformation matrix between the two images is calculated to achieve accurate alignment. In specific implementations, improved algorithms, such as accelerated robust feature algorithms, can also be tried, both of which can achieve good results.

[0033] Once the image registration parameters are obtained, the area to be acquired can be calculated to ensure that the re-acquired image can cover the entire area of ​​the historical reference image.

[0034] This application also discloses a fully automated intelligent parameter setting system for microscopes, used to implement the above-mentioned fully automated intelligent parameter setting method for microscopes. The fully automated intelligent parameter setting system for microscopes includes an acquisition module, an identification module, an analysis module, and a judgment module. The acquisition module is used to acquire sample images. The identification module performs instance segmentation on the target of interest and identifies the boundaries. The analysis module predicts the initial acquisition parameters and calculates the score of the quality index. The judgment module determines whether the acquisition parameters need to be optimized based on the score of the quality index.

[0035] Compared to existing technologies, this invention's fully automated intelligent parameter setting method for microscopes utilizes a large dataset of fluorescence and bright-field microscopic images containing various cell and tissue structures. It performs domain-adaptive fine-tuning of the SAM2 model's basic image encoder, enhancing its feature extraction capabilities for typical features in microscopic images. The optimized SAM2 model is then used to segment multiple potential targets of interest in the global preview image, identifying the boundaries of each target. A pre-trained deep learning model intelligently analyzes the segmented images, predicting a near-optimal set of initial acquisition parameters based on the analysis results. Images are then acquired using these initial acquisition parameters, and the quality index of the acquired images is calculated. The target score is determined by the acquisition parameters. When the score reaches the preset optimal threshold or converges, the acquisition parameters at that time are used as the final set parameters. When the score does not reach the preset optimal threshold and does not converge, multi-dimensional, small-step parameter fine-tuning is performed within a limited range near the initial acquisition parameters until the quality index score of the acquired image reaches the preset optimal threshold or converges. Through the above steps, artificial intelligence and automatic control technology are deeply integrated, which greatly reduces the threshold for using the equipment and the professional requirements, and significantly improves the efficiency, standardization and repeatability of data acquisition, providing strong technical support for cutting-edge applications such as high-throughput screening and long-term live cell imaging.

[0036] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These are all equivalent modifications and improvements made to the above embodiments based on the essential technology of the present invention, and all of these fall within the protection scope of the present invention.

Claims

1. A method for full-automatic intelligent parameter setting of a microscope, characterized in that, The method comprises the following steps: sample collection: collecting a global preview image of the sample; target segmentation: fine-tuning the base image encoder of the SAM2 model using a fluorescence and bright-field microscopic image dataset containing a large number of various cells and tissue structures, so that the model has stronger feature extraction capability for typical features in the microscopic image; performing instance segmentation on multiple potential targets in the global preview image using the optimized SAM2 model to identify the boundaries of each target; intelligent analysis: using a pre-trained deep learning model to perform intelligent analysis on the segmented image, and predicting a set of near-optimal initial acquisition parameters based on the analysis results; score calculation: acquiring an image using the initial acquisition parameters and calculating the score of the quality index of the acquired image; parameter formation: when the score of the quality index reaches a preset optimal threshold or converges, the acquisition parameters at this time are used as the final setting parameters; when the score of the quality index does not reach the preset optimal threshold and does not converge, multi-dimensional and small-step parameter fine-tuning is performed within a limited range around the initial acquisition parameters until the score of the quality index of the acquired image reaches the preset optimal threshold or converges.

2. The method of claim 1, wherein: The microscope fully automatic intelligent parameter setting method further comprises a multiple scanning step, which is located after the parameter formation step, and the multiple scanning step comprises: retrieve the historical reference image of the target area and its corresponding displacement table coordinates; control the displacement table to move to the vicinity of the coordinates and acquire a current preview image containing the expected target area and its surrounding environment; obtain the registration parameters of the current preview image and the historical reference image through an image registration algorithm; calculate the area to be acquired according to the registration parameters.

3. The method of claim 2, wherein: The registration parameters of the current preview image and the historical reference image obtained through the image registration algorithm are specifically: performing prediction of the transformation parameters through a deep learning model, wherein the encoder part of the deep learning model is used to extract the deep features of the latest preview image and the historical reference image, and the similarity between the features is calculated through related layers, and finally an accurate affine transformation matrix is output by the decoder; using the affine transformation matrix to perform image transformation on the current preview image and evaluating the matching degree of the transformed image and the historical reference image; when the matching degree meets the requirements, the transformation parameters are output; when the matching degree does not meet the requirements, a scale-invariant feature transformation algorithm is used for feature matching, which detects extreme points in the scale space as feature points and generates high-dimensional description vectors, and then calculates the affine transformation matrix between the two images through the nearest neighbor matching between the feature vectors.

4. The method for microscope full-automatic intelligent parameter setting according to claim 1, characterized in that: In the sample collection step, a wide-field camera channel is used to collect the global preview image of the sample by utilizing the high-speed imaging characteristics of the camera.

5. The method for microscope full-automatic intelligent parameter setting according to claim 1, characterized in that: In the sample collection step, composite dark-field illumination is used to enhance the contrast of the image.

6. The method for microscope full-automatic intelligent parameter setting according to claim 1, characterized in that: In the target segmentation step, the typical features in the microscopic image include at least one of cell boundaries, cell nuclei, and neural synapses.

7. The method for microscope full-automatic intelligent parameter setting according to claim 1, characterized in that: In the target segmentation step, the instance segmentation of the plurality of potential targets in the global preview image by using the optimized SAM2 model specifically comprises: automatically generating a series of initial prompt points or frames by using a target detection algorithm or saliency analysis, and taking the series of initial prompt points or frames as inputs of the SAM2 model.

8. The method for microscope full-automatic intelligent parameter setting according to claim 1, characterized in that: In the parameter forming step, the final setting parameters include at least one of laser power, PMT gain, pinhole diameter and scanning speed.

9. The method for microscope full-automatic intelligent parameter setting of claim 1, wherein: In the score calculation step, the quality indicators of the image include at least one of signal-to-noise ratio, dynamic range and background uniformity.

10. A microscope fully automatic intelligent parameter setting system for implementing the microscope fully automatic intelligent parameter setting method according to any one of claims 1-9, characterized in that: The microscope full-automatic intelligent parameter setting system comprises a collection module, an identification module, an analysis module and a judgment module, the collection module is used for collecting sample images, the identification module performs instance segmentation on the targets of interest and identifies boundaries; the analysis module predicts initial collection parameters and calculates scores of quality indicators; The judgment module judges whether the collection parameters need to be optimized according to the scores of the quality indicators.

Images