Microscope automatic focusing method and system for self-supervising image quality evaluation in combination with physical prior soft label
By combining a self-supervised image quality assessment method with physical prior soft labels, the problems of noise interference and manual labeling dependence in microscope autofocus under low signal-to-noise ratio environments are solved, and efficient and accurate autofocus is achieved under different optical systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTH CHINA NORMAL UNIV
- Filing Date
- 2026-02-02
- Publication Date
- 2026-05-15
AI Technical Summary
Existing microscope autofocus technology is susceptible to noise interference in low signal-to-noise ratio environments. Traditional methods have poor noise resistance, while deep learning methods rely on manual annotation, have weak generalization ability, and are difficult to adapt to different optical system parameters.
A self-supervised image quality assessment method based on physical prior soft labels is adopted. A smooth single-peak evaluation curve is generated through ROI perception and multi-scale sampling. Combined with a dual-branch feature encoding network and a lightweight regression network, training labels are automatically generated using optical system parameters. The LOSO cross-validation strategy is adopted to achieve self-supervised training without manual annotation.
It effectively suppresses background noise in low signal-to-noise ratio environments, improves focusing accuracy, reduces implementation costs, adapts to different optical system parameters, and enhances model generalization ability and focusing robustness.
Smart Images
Figure CN122043717A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of microscope focusing technology, specifically referring to a microscope autofocusing method and system that combines self-supervised image quality assessment with physical prior soft tags. Background Technology
[0002] Fluorescence microscopy is an indispensable imaging tool in biomedical research, widely used in cell biology, drug screening, pathological diagnosis, and other fields. Precise focusing is a crucial prerequisite for obtaining high-quality images during fluorescence microscopy imaging. Currently, automatic focusing in microscopes mainly relies on traditional sharpness evaluation functions and deep learning-based methods.
[0003] Fluorescence microscopy is a key tool in biomedical research. During the imaging process, the accuracy and speed of autofocus directly affect experimental efficiency and sample viability.
[0004] Existing microscope autofocusing techniques are mainly divided into two categories: one is the traditional sharpness evaluation function method, such as Brenner and Tenengrad, which finds the focal plane by calculating the statistical value of the image gradient. Although the calculation is simple, in fluorescence imaging, due to the low signal-to-noise ratio (Low SNR), dark background, and uneven texture distribution, the gradient function is easily affected by background noise, producing "false peaks" that lead to misjudgment or failure of the search algorithm.
[0005] Another type is the regression method based on supervised deep learning, which directly predicts focal length by training a neural network. While this type of method improves noise resistance to some extent, it has two fatal flaws: First, it heavily relies on a large number of manually labeled focus tags, making data collection and labeling extremely expensive and prone to subjective errors; second, the model has poor generalization ability and is usually strongly tied to specific microscope hardware parameters, such as numerical aperture and magnification. Once the equipment or objective lens is changed, the model often cannot be directly applied and requires retraining with newly collected data, which greatly limits its industrial application.
[0006] To address the aforementioned issues, the existing technology lacks a universal autofocus solution that can resist low signal-to-noise ratio interference, eliminate the need for manual labeling, and adapt to different optical system parameters. Summary of the Invention
[0007] To address the problems of poor noise resistance of traditional evaluation functions and the reliance on expensive manual annotation and weak generalization ability of deep learning-based methods in existing technologies, one of the objectives of this invention is to provide a self-supervised image quality assessment method for microscope autofocusing that combines physical prior soft labels, which can improve focusing accuracy.
[0008] This invention aims to solve the following technical problems:
[0009] Addressing the issue of unstable focusing in low signal-to-noise ratio (SNR) environments: In low SNR scenarios such as fluorescence imaging, traditional gradient functions are susceptible to noise and background interference, resulting in spurious peaks. This invention aims to provide a focusing method that effectively suppresses background noise and generates a smooth, single-peak evaluation curve. Solving the problem of deep learning model training relying on manual annotation: Addressing the high cost of existing data-driven methods requiring extensive manual annotation of "image-focal length" labels, this invention aims to achieve a self-supervised / weakly supervised training mechanism that eliminates the need for manual annotation, thus lowering the implementation threshold.
[0010] Addressing the issue that evaluation functions are easily affected by sample content: To address the problem that existing models tend to misjudge "cell texture richness" as "image sharpness," this invention aims to use a decoupling mechanism to ensure that the evaluation results only reflect the optical defocus state and are not affected by differences in sample morphology.
[0011] Solving the problem of difficult model migration across devices: In view of the problem that existing deep learning models are strongly coupled with specific hardware parameters and require a large amount of data to be collected and retrained when changing devices, this invention aims to provide a universal focusing solution that can quickly adapt to different optical parameters such as numerical aperture and magnification.
[0012] Specifically, all images belonging to the same Z-stack scan sequence are grouped together. When dividing the training and validation sets, it is mandatory that all images from the same Z-stack are either entirely in the training set or entirely in the validation set. This strategy simulates the model's operation under completely new fields of view, ensuring the algorithm's reliability in real-world microscopy applications.
[0013] The core algorithm principle and model construction of the autofocus method of this invention
[0014] The model of this invention employs a focus quality scoring model based on physical perception and self-supervised dual-branch coding, which serves as the core discriminator in the subsequent autofocus process. The model construction and training process is as follows:
[0015] ROI-aware foreground detection and multi-scale sampling mechanism
[0016] To address the challenges of large background areas and strong noise interference in fluorescence microscopy images, this invention employs a morphology-based region of interest (ROI) selection and multi-scale sampling strategy before inputting the images into the network. This ensures that the scoring model focuses only on valid biological structures.
[0017] (1) Generation of the foreground region, i.e., the ROI mask:
[0018] First, the original image is normalized by quantile normalization, for example, linearly mapping the 1% to 99.8% quantile range to [0,1] to eliminate the influence of extreme noise. Then, the Otsu adaptive thresholding algorithm is used for binary segmentation to generate an initial foreground mask, and morphological opening and closing operations are performed to optimize the selection. Structural elements and connected component analysis are used to filter out components with areas smaller than a preset threshold, for example... The final ROI mask is obtained by identifying small noise areas within the pixels. If there is no effective foreground in the entire image, the process reverts to full-image mode. The ROI mask generation method is not limited to Otsu adaptive thresholding and morphological opening / closing operations. Depending on the imaging conditions, it can be replaced with global intensity thresholding, local adaptive thresholding, threshold segmentation based on background estimation, connected component / edge constraint segmentation, or a lightweight segmentation model to obtain the foreground region. When no effective foreground is detected, the process can revert to the entire image or a preset center region as the ROI input.
[0019] Multi-scale patch random sampling:
[0020] Multi-scale sampling is performed within the ROI region to capture features under different receptive fields:
[0021] Scale definition: A multi-scale sampling strategy is adopted to crop and input the image at both small and large scales; for example, the small scale resolution is 256×256 and the large scale resolution is 512×512.
[0022] Validity screening: Random sampling The network generates image patches and calculates the percentage of foreground pixels within each patch. Only patches with a percentage higher than a preset coverage rate (e.g., 40%) are retained to ensure that the data input to the network contains sufficient texture information.
[0023] Texture guidance: Calculate the Laplacian variance of candidate patches and prioritize the patches with high texture intensity for subsequent calculations (Top-k strategy).
[0024] Patch validity screening and texture guidance metrics are not limited to foreground pixel ratio and Laplacian variance. They can be replaced by metrics such as gradient energy, frequency domain energy, local contrast, edge density, variance / entropy, etc.; the Top-k selection strategy can also be replaced by threshold screening, weighted sampling, or hierarchical random sampling.
[0025] Dual-branch feature encoding network architecture
[0026] To address the interference of sample content differences in fluorescence images on sharpness evaluation, this invention constructs a feature extractor comprising two independent branches, such as... Figure 2 As shown:
[0027] Content Branch: An encoder pre-trained on a biological / microscopic image dataset is used to extract high-level semantic features such as cell morphology and texture distribution. The pre-training data for the content branch encoder is not limited to a specific microscopic image dataset; it can be pre-trained using biological microscopic images, natural images, or a mixture of both.
[0028] Quality Branch: Employs an encoder pre-trained on a dataset containing various distortion types such as Gaussian blur and noise to extract low-level quality features such as the degree of blur and noise levels of images.
[0029] Feature fusion: The output features of the two branches are concatenated along the channel dimension and then input into a lightweight regression network to output the final focus quality score.
[0030] Configurable lightweight regression network architecture
[0031] The fused feature vector from the dual-branch output is input into a lightweight nonlinear regression network to establish a nonlinear mapping from a high-dimensional feature space to a one-dimensional focus quality score. To achieve the optimal balance between inference speed, parameter efficiency, and fitting accuracy, the lightweight regression head is not limited to MLP or KAN structures; any network capable of performing nonlinear regression on the dual-branch fused features to output a one-dimensional focus score is acceptable. Optionally, the regression head can employ a multilayer perceptron, spline / kernel regression structure, or other lightweight nonlinear regression networks with controlled parameter counts. Preferably, the regression head has fewer parameters than the encoder to facilitate rapid deployment by "freezing the encoder and fine-tuning the head."
[0032] Physical perception labels and regularized loss function
[0033] To address the issues of high cost and strong subjectivity associated with manual labeling, this invention utilizes the physical prior of an optical imaging system to automatically generate training labels and obtain the numerical aperture of a microscope objective. ) and imaging wavelength ( ).
[0034] Depth of field calculation: Calculate the theoretical depth of field of the optical system based on the Rayleigh criterion. ):
[0035] Equation (1);
[0036]
[0037] Tag generation: Generating soft tags that vary with defocus distance based on the Lorentzian distribution. :
[0038] Equation (2);
[0039] in, Sampling location, The focal plane position, The distribution width parameter is dynamically calculated from the numerical aperture and imaging wavelength, for example... , This label gives the focus quality model a clear optical physical meaning.
[0040] Hybrid Loss Function: For training the regression network, the following regularized total loss function is constructed:
[0041] Equation (3);
[0042] in, Predict the output for the scoring model; For physical perception soft tags; For predicting scores With physical labels The mean square error; ; This is the regularization weight coefficient.
[0043] Leave-one-out stack (LOSO) cross-validation strategy
[0044] To rigorously verify the model's generalization ability and prevent data leakage, a cross-validation strategy based on Z-stack grouping is employed during the training process.
[0045] The objective of this invention is achieved through the following technical solution: a microscopy autofocus method combining self-supervised image quality assessment with physical prior soft tags, characterized in that the method includes the following steps:
[0046] Step S1: Training and Deploying the Scoring Model
[0047] S1-1 Data Acquisition: Acquire at least one set of Z-stack image sequences obtained by scanning along the Z-axis, wherein the Z-stack image sequences cover the defocusing process from out of focus to focus and back to out of focus.
[0048] S1-2 Optical Parameter Reading and Depth of Field Calculation: Read the numerical aperture NA and imaging wavelength λ of the current objective lens, and calculate the theoretical depth of field DOF according to equation (1).
[0049] S1-3 ROI Mask Generation and Foreground Selection: The original image is normalized to suppress the influence of extreme noise, and an initial foreground mask is obtained through threshold segmentation. The original image is a single-frame fluorescence micrograph acquired at each z-axis position in the Z-stack image sequence. Then, morphological operations and connected component analysis are performed to remove small noise areas and obtain the ROI mask. When no effective foreground is detected, the entire image or a preset center region is used as the ROI input.
[0050] S1-4 Multi-scale image patch sampling and validity screening: Multi-scale sampling is performed within the area covered by the ROI mask obtained in step S1-3 to obtain different receptive field features; foreground coverage screening is performed on the sampled image patches, and texture-guided selection is performed in combination with texture intensity index, that is, texture intensity index T is calculated for each candidate patch. i Laplace variance is preferred.
[0051] Equation (4);
[0052] Among them, P i For the i-th patch, For the Laplace operator, then press T. i Sort the samples from largest to smallest and select the top-k patches as input samples;
[0053] S1-5 Physically Sensing Soft Tag Generation: Based on the theoretical depth of field (DOF) calculated in step S1-2, a soft tag Score(z) that varies with the defocus distance is constructed. Score(z) adopts the Lorentzian distribution shown in equation (2), where z is the sampling position. The focal plane position is used to give the training labels a clear optical and physical meaning.
[0054] S1-6 Model Training: The input samples obtained in step S1-4 are input into the focus quality scoring model for training. The focus quality scoring model includes a content branch encoder, a quality branch encoder, and a lightweight regression head for training. During the training process, the mean square error between the predicted score and the physical perception soft label is used as the basic loss, and the total loss function shown in equation (3) is superimposed. This is used to update network parameters.
[0055] S1-7 Cross-validation and generalization constraints: Images within the same Z-stack scan sequence are grouped together, and when dividing the training and validation sets, images from the same Z-stack are guaranteed to appear only in the training or validation sets to reduce the risk of data leakage and verify the model's generalization ability.
[0056] S1-8 Model Consolidation and Deployment: Save the trained focus quality scoring model parameters as a model to be deployed and configure it as an inference model that can be called during the focus execution phase.
[0057] Step S2: System Initialization and Parameter Adaptation
[0058] S2-1 Optical Parameter Reading: When autofocus starts, read the numerical aperture NA and imaging wavelength λ of the current objective lens.
[0059] S2-2 step size automatic calculation:
[0060] The theoretical depth of field (DOF) is calculated according to equation (1), and the following three key step size parameters required for subsequent searches are automatically generated based on the calculated theoretical depth of field (DOF):
[0061] Coarse search with large step size: k1 is preferably 10-20;
[0062] Fine-grained search with small steps: k2 is preferably 5-10;
[0063] Verification step size: k3 is preferably 3 to 6;
[0064] S2-3 Model Loading: Load the focus quality scoring model parameters saved in steps S1-8 as the inference model, freeze the encoder weights, and prepare for real-time inference of image focus quality score.
[0065] Step S3: Coarse Search Phase
[0066] S3-1: Control the stage with large steps Move within the preset scanning range.
[0067] S3-2: Real-time Acquisition and Calculation: During the focus search process, whenever the Z-axis actuator moves to a new axial position z, a single-frame fluorescence microscopic image at that position is acquired. After preprocessing, the image is used as input to the focus quality scoring model trained in steps S1-6 and loaded in step S2-3 for inference calculation. The model input is either a single-frame fluorescence image at the current Z position or a region of interest (ROI) image patch extracted from that image. The model output is the focus quality score D(z) corresponding to the current Z position. The focus quality score D(z) reflects the scalar value of the current focus quality scoring model's assessment of image sharpness. Simultaneously, a traditional sharpness score C(z) is calculated for the same frame image. The traditional sharpness score C(z) is preferably calculated using a gradient-type or high-frequency response sharpness operator and is used to characterize the local texture and edge intensity of the image. The focus quality score D(z) and the traditional sharpness score C(z) are then normalized and jointly fused as follows:
[0068] (a) Normalization: Focus quality score within the current search phase. Compared to traditional resolution rating Min-Max normalization is performed using the following formulas:
[0069] Equation (5);
[0070] Equation (6);
[0071] in, This refers to the focus quality score, specifically the focus quality scoring model at different positions. The output; For traditional sharpness scoring, that is, the traditional sharpness evaluation function at location The output; , Each in the current search phase The minimum and maximum values, , Each in the current search phase The minimum and maximum values; Focus quality rating Compared to traditional resolution rating Normalized score.
[0072] (b) Joint Integration: Defining the Joint Evaluation Function :
[0073] Equation (7);
[0074] in, For the joint evaluation function, through the joint evaluation function Calculate the joint score. Traditional evaluation weighting coefficients, This is the weighting coefficient for the focus quality score, and Preferably, .
[0075] S3-3: Dynamic Monitoring: Real-time monitoring of the changing trend of the joint score. The score sequence is smoothed, and the score difference between two adjacent samples is calculated in real time. When the score difference changes from positive to negative, and in subsequent consecutive... The sampling point remains negative, while the decrease reaches a preset threshold. When the peak value has been exceeded, the current scanning phase is stopped; among which, The number of sampling points for determining continuous decline is preferably 2 to 5; The threshold for the decrease is preferably set to 0.05–0.2 after the score is normalized to [0,1], or it can be set according to the dynamic range of the score in this stage. The scan is stopped when the score initially rises and then drops significantly, forming a peak shape.
[0076] S3-4: Range Locking: Determine a range centered on the detected approximate peak position. The region is used as the candidate interval.
[0077] Step S4: Fine-tuning search phase
[0078] S4-1: Control the stage to retract to the starting position of the candidate interval.
[0079] S4-2: Switch to small step size High-density scanning is performed within the candidate region.
[0080] S4-3: Acquire images for each scan point and calculate the joint score.
[0081] S4-4: The sliding window algorithm is used to smooth the scoring sequence obtained in step S4-3, and the local maximum position of the smoothing curve is located. .
[0082] Step S5: Local fine-tuning verification and spurious peak escape stage
[0083] To prevent the focus quality scoring model from generating false peaks (i.e., pseudo-peaks) at extremely low signal-to-noise ratios, this invention introduces a joint verification mechanism of three-point verification and dual-modal cross-validation. Only peaks that simultaneously satisfy both three-point verification and dual-modal cross-validation can be considered true peaks.
[0084] S5-1 Three-point sampling: Location of the local maximum value located in step S4-4 Three verification images were acquired at the tiny offset positions before and after the image, and the tiny offset positions were: and .
[0085] S5-2 three-point peak pattern confirmed:
[0086] Calculate the joint score of the three frames. If the peak value is high in the middle and low on both sides, then... and If the condition is met, it is considered a valid peak value. If the condition is not met, it means that the true focal plane may be located nearby, triggering the escape mechanism to continue fine-tuning the search along the upward direction of the joint score until the peak value condition is met.
[0087] S5-3 dual-modal cross-validation:
[0088] Calculate peak Traditional sharpness rating values for location images For example, the Brenner gradient value can be used as a traditional sharpness evaluation value and compared with the traditional evaluation value of the peak neighborhood. If the focus quality scoring model generates peak values... The deviation of the peak position obtained by the traditional method exceeds a preset threshold. If so, it is determined that the focus quality scoring model may be affected by texture interference. Preferably, Pick ,and Can be set to The value is a multiple of [a certain value]. At this point, automatic degradation occurs, prioritizing the peak position of the traditional evaluation function as the final focal plane position. .
[0089] Step S6: Focusing and Multi-field Scanning
[0090] S6-1 Execution Action:
[0091] Drive the electric stage precisely to the final focal plane position confirmed in step S5-3. Lock the microscope position and autofocus the current field of view.
[0092] S6-2 Status Update: The currently confirmed final focal plane position is updated. Stored as a state variable.
[0093] S6-3 Multi-Field Relay: During continuous scanning of multi-well plates or large tissue sections, the final focal plane position of the current field of view is recorded. As the initial search center for the next adjacent field of view, the coarse search range is narrowed from a global scan to a local scan; for example, the coarse search range is set to... This significantly reduces the focusing time for subsequent fields of view.
[0094] In low signal-to-noise ratio imaging scenarios with weak fluorescence and strong background noise, isolated noise points or background texture fluctuations often exist in the image. Traditional sharpness evaluation functions (such as evaluation based on gradient / frequency domain energy) are prone to misidentifying such noise fluctuations as sharpness peaks, resulting in spurious peaks during the Z-axis search process and causing incorrect convergence or repeated oscillations.
[0095] Unlike simply adding deep learning scores during the search phase, this invention introduces optical physics priors during the model training phase: calculating the depth of field based on the current objective lens numerical aperture and imaging wavelength, and constructing soft labels that vary with the defocus distance accordingly. This allows the learning objective of the scoring model to directly correspond to the "degree of defocus," resulting in a single-peak trend that better reflects the defocus process. Simultaneously, this invention employs a dual-branch structure with content and quality branches to extract sample morphological semantic features and quality features such as blur / noise, and then fuses and regresses them. This structurally suppresses content interference where "texture richness is misjudged as sharpness," making the score more stably reflect the optical defocus state.
[0096] Furthermore, this invention does not merely smooth the scoring sequence; instead, it constructs a closed-loop robust mechanism for low SNR scenarios within the execution process. After obtaining local extrema through fine-grained search, it uses three-point verification to determine whether the peak shape of "high in the middle and low on both sides" is satisfied. If not, an escape mechanism is triggered to continue searching along the upward direction of the score to avoid falling into spurious peaks. In addition, the recommended peak value for focus quality is cross-validated with the peak value of the traditional evaluation function. When the deviation between the two exceeds a depth-of-field threshold, the traditional peak value is automatically downgraded and adopted as a fallback result. The synergy of the physical prior constraints in the training phase, the decoupling of the model structure, and the verification / escape / downgrade in the execution phase enables the utilization of the noise resistance advantage of focus quality in low SNR scenarios while avoiding erroneous convergence caused by spurious peaks under extreme noise conditions, resulting in a stability improvement effect beyond that of simple combinations.
[0097] The second objective of this invention is to provide a microscope autofocus system that combines physical prior soft tags with self-supervised image quality assessment, which enables precise autofocusing of the microscope.
[0098] This objective of the present invention is achieved through the following technical solution: a microscope autofocus system combining self-supervised image quality assessment with physical prior soft tags, characterized in that the system comprises:
[0099] The host computer has an autofocus control unit for running the autofocus control program and performing data processing and decision-making.
[0100] A fluorescence microscope, used to create a fluorescence imaging optical path for a sample;
[0101] Z-axis actuator, which carries the fluorescence microscope, is capable of micron-level stepping and positioning in the Z-axis direction, thereby changing the relative axial distance between the sample and the objective lens to achieve focusing, so as to meet the sequence acquisition requirements under different defocusing degrees.
[0102] An electric objective lens switching device is used to switch objectives and to provide or provide feedback on the optical parameters related to the current objective lens;
[0103] The camera is connected to the fluorescence microscope in an optical path. The optical image output by the fluorescence microscope is coupled to the camera through the imaging port. The camera acquires the fluorescence image and outputs digital image data to the host computer after photoelectric conversion.
[0104] The host computer is connected to the Z-axis actuator, the electro-optical objective lens switching device, and the camera. It sends Z-axis motion control commands to the Z-axis actuator and receives motion status or position feedback through the autofocus control unit, sends objective lens switching control commands to the electro-optical objective lens switching device and receives objective lens parameters and status feedback, and receives image data streams output by the camera.
[0105] In this invention, the autofocus control unit includes:
[0106] Equipment control module: used to output displacement control commands to the Z-axis actuator and switching control commands to the electro-optical endoscope conversion device;
[0107] Image acquisition module: used to receive image data output by the camera and form a Z-stack image sequence or a frame-by-frame image sequence;
[0108] Parameter Adaptive Module: Used to read the objective lens numerical aperture NA and imaging wavelength λ at the start of microscope autofocus, and calculate the theoretical depth of field DOF and generate search step size parameters related to DOF.
[0109] Focus evaluation and inference module: used to calculate the focus quality score D(z) and the traditional sharpness score C(z) of the acquired image, and generate a joint evaluation F(z);
[0110] Focusing decision and process control module: Used to execute coarse search, fine search and verification processes based on the changing trend of the joint evaluation F(z), and output the final focal plane position. .
[0111] In this invention, the Z-axis actuator is an electric stage.
[0112] In this invention, the connection between the host computer and the Z-axis actuator, the electro-optical head conversion device, and the camera can be achieved through USB, GigE, serial port, network port, or other communication interfaces commonly used in the field.
[0113] Compared with the prior art, the present invention has the following beneficial effects:
[0114] 1. This invention utilizes the optical physical parameters of a microscope to calculate depth of field and combines them with the Lorentzian distribution to automatically generate soft labels for training. This eliminates the expensive and subjective manual labeling process, enabling self-supervised training (or weakly supervised / pseudo-label supervised training) of the model, allowing the algorithm to be deployed on a large scale at low cost.
[0115] 2. This invention decouples the "content features" and "quality features" of an image through a dual-branch architecture and introduces a "three-point verification" and "traditional method cross-validation" mechanism. Even under harsh imaging conditions with weak fluorescence and extremely high background noise, it can still accurately distinguish between "blurred cells" and "clear noise", effectively suppressing false peak interference and significantly improving the robustness of focusing.
[0116] 3. To address the poor generalization of existing deep learning models, this invention employs a "frozen backbone + fine-tuned regression head" strategy. When users change objectives or microscopes, there is no need to re-collect massive amounts of data to retrain the model; only a small amount of z-stack data is required for adaptation. This greatly improves the algorithm's versatility and engineering practicality.
[0117] 4. The focusing system of the present invention can achieve automated operation of the focusing process, integrate and optimize various complex operation steps, significantly reduce the time and manpower input consumed by manual operation, and thus effectively improve the working efficiency and reliability of experimental tools.
[0118] 5. This invention adopts a phased strategy of "coarse search-fine search" based on depth-of-field magnification, and combines it with a multi-view relay mechanism, which can effectively avoid the interference of local extreme values on focus determination and improve the accuracy of focusing. Attached Figure Description
[0119] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0120] Figure 1 This is a diagram showing the overall structural architecture of the autofocus system of the present invention;
[0121] Figure 2 This is a diagram of the dual-branch network model architecture of the present invention;
[0122] Figure 3 This is a flowchart of the label generation and training process based on physical priors of the present invention.
[0123] Figure 4 This is a flowchart of the autofocus method of the present invention;
[0124] Figure 5 This is a flowchart of the cross-scenario rapid transfer learning process of the present invention. Detailed Implementation
[0125] like Figure 1 The microscope autofocus system shown incorporates self-supervised image quality assessment based on physical prior soft tags, including:
[0126] The host computer, which can also be a computer, has an autofocus control unit used to run the autofocus control program and perform data processing and decision-making.
[0127] A fluorescence microscope, used to create a fluorescence imaging optical path for a sample;
[0128] The Z-axis actuator is an electric stage that carries the fluorescence microscope and is used to achieve micron-level stepping and positioning in the Z-axis direction to meet the needs of sequence acquisition under different defocus levels.
[0129] An electric objective lens switching device is used to switch objectives and to provide or provide feedback on the relevant optical parameters of the current objective lens;
[0130] The optical image output from the fluorescence microscope is coupled to the camera via the imaging port. The camera acquires the fluorescence image and outputs digital image data to the host computer after photoelectric conversion.
[0131] The autofocus control unit includes:
[0132] Equipment control module: used to output displacement control commands to the Z-axis actuator and switching control commands to the electro-optical endoscope conversion device;
[0133] Image acquisition module: used to receive image data output by the camera and form a Z-stack image sequence or a frame-by-frame image sequence;
[0134] Parameter Adaptive Module: Used to read the objective lens numerical aperture NA and imaging wavelength λ at the start of microscope autofocus, and calculate the theoretical depth of field DOF and generate search step size parameters related to DOF.
[0135] Focus evaluation and inference module: used to calculate the focus quality score D(z) and the traditional sharpness score C(z) of the acquired image, and generate a joint evaluation F(z);
[0136] Focusing decision and process control module: Used to execute coarse search, fine search and verification processes based on the changing trend of the joint evaluation F(z), and output the final focal plane position. .
[0137] The host computer is connected to the Z-axis actuator, the electro-optical lens conversion device, and the camera, respectively. It sends action commands to the Z-axis actuator and the electro-optical lens conversion device via the autofocus control unit and receives image data output from the camera.
[0138] The Z-axis actuator connection is used to send Z-axis motion control commands and receive motion status or position feedback.
[0139] An electric objective switching device is connected to send objective switching control commands and receive objective parameter / status feedback;
[0140] Camera connection, used to receive image data streams.
[0141] The connection between the host computer and the Z-axis actuator, the electro-optical head conversion device, and the camera is achieved through USB, GigE, serial port, network port, or other communication interfaces commonly used in this field.
[0142] The fluorescence microscope and the camera are connected via a conventional optical path in the art: the optical image output from the fluorescence microscope is coupled to the camera via the imaging port, and the camera performs photoelectric conversion and outputs digital image data.
[0143] The Z-axis actuator is mechanically connected to the microscope body to carry the sample and achieve Z-axis displacement, thereby changing the relative axial distance between the sample and the objective lens to achieve focusing.
[0144] The signal transmission relationships between the components in the system of this invention are as follows:
[0145] Control command signals: Output from the host computer to the Z-axis actuator, electro-optical mirror conversion device, and other actuator components.
[0146] Z-axis focusing control commands: Based on the output of the focusing decision and process control module, the host computer sends displacement commands, including step size, direction, and target position, to the Z-axis actuator to drive Z-axis scanning and positioning.
[0147] Data acquisition signal: Camera output to → host computer.
[0148] Digital image data signal: The camera acquires fluorescence images at each Z position and transmits the image data to the image acquisition module of the host computer for subsequent scoring calculation and focus decision.
[0149] Parameter / status feedback signals: The Z-axis actuator and the electro-optical endoscope conversion device output these actuators to the host computer.
[0150] Optical parameter feedback signal: The electro-optical objective conversion device feeds back the optical parameters such as NA corresponding to the current objective to the host computer. The host computer reads / determines the imaging wavelength λ and calculates the theoretical depth of field DOF, thereby generating the DOF-related search step size parameter.
[0151] Position / Status Feedback Signal: The Z-axis actuator feeds back its current position or motion completion status to the host computer for closed-loop control or process synchronization.
[0152] The microscopy autofocusing method of the present invention, which combines self-supervised image quality assessment with physical prior soft tags, is described in the above-described invention content. The autofocusing method of the present invention will be described and verified below through specific microscopy verification experiments.
[0153] Dataset Construction
[0154] To capture the complete process from "extreme blur" to "precise focus," and to verify the effectiveness of the autofocus method described above in this invention, the following dataset was collected:
[0155] Number of fields of view: 300 independent z-stacks; Total number of images: 9300.
[0156] Each z-stack consists of 31 images. The image sequence is symmetrically sampled along the positive and negative Z-axis in the interval [-15μm, 15μm], centered on the focal plane. The sampling stride of the training set is set to... It appears to be out of focus. Gradually becoming clearer Positive Focus The complete physical process of "gradually defocusing" is used to train the regression head of the dual-branch network. Optionally, only a small number of Z-stack sequences (small samples) are collected to validate or implement a fast transfer strategy of "freezing the backbone + fine-tuning the regression head" to reduce the retraining cost when hardware or samples change.
[0157] Experiment Example 1: Performance Verification of Focus Scoring Model
[0158] Experimental setup
[0159] Five-fold cross-validation was used to evaluate the performance of the focus scoring model. The dataset contains five independently acquired z-stack sequences, and the data is partitioned in units of z-stacks to ensure that different focal plane images of the same field of view do not appear in both the training and test sets simultaneously, thus avoiding overestimation of performance due to data leakage.
[0160] Evaluation indicators
[0161] SRCC (Spearman Rank Correlation Coefficient): Measures the monotonic relationship between the predicted score and the actual focal length position. A higher value indicates better consistency in the ranking of the focus curves.
[0162] PLCC (Pearson linear correlation coefficient): measures the linear correlation between predicted scores and physical perception labels;
[0163] Top-1 accuracy: The percentage of times the predicted peak position perfectly matches the actual focal plane position;
[0164] Standard deviation: reflects the stability of the model at different folds.
[0165] Experimental results
[0166] The autofocus method of the present invention was compared with six traditional sharpness evaluation functions, and the results are shown in Table 1.
[0167] Table 1: Performance Comparison of the Invention Method and Traditional Methods (5-fold Cross-validation)
[0168]
[0169] Results Analysis
[0170] Experimental results show that:
[0171] 1. The SRCC of the method of this invention is significantly better than that of the traditional method, and significantly improved compared with the suboptimal method Tenengrad;
[0172] 2. The PLCC produced by the method of this invention is significantly better than Tenengrad and SMD, with a marked improvement.
[0173] 3. The Top-1 accuracy of the method of this invention is tied for best with Laplacian and is superior to other traditional methods;
[0174] 4. The SRCC standard deviation of the method of this invention is much lower than that of traditional methods, indicating that the model has higher stability and generalization ability on different datasets.
[0175] Experiment Example 2: Simulation Verification of Actual Focusing Process
[0176] Experimental Objective
[0177] The effectiveness of the phased search strategy proposed in this invention in actual focusing tasks is verified, and the ability of the joint evaluation function to suppress spurious peaks in the focus quality score is also verified.
[0178] Experimental setup
[0179] The experimental microscope is equipped with a high-resolution 16-bit grayscale camera, an Olympus 60× / 1.4NA oil immersion lens, and a fully motorized closed-loop control Z-axis drive system (maximum control accuracy 50nm). The accompanying software supports stacked acquisition and imaging, and it features a bright-field illumination system compatible with slides for imaging tissue samples and stages for culture dishes. The lens depth of field (DOF) is approximately 0.1-0.2μm; DOF will be referred to as D thereafter.
[0180] A fluorescent section of bovine pulmonary artery endothelial cells (BPAE) was selected. Starting from a position far from the focal plane, z0 = -192D, a complete autofocus operation was performed. The optical and search parameters were set as follows, where D represents the current depth of field:
[0181] Coarse search with large step size: ∆zcoarse = k1 · D, k1=15, (k1 is preferably 10~20);
[0182] Fine-grained search with small step size: ∆zfine = k2 · D, k2=8, (k2 is preferably 5~10);
[0183] Verification step size: ∆zverify = k3 · D, k3=4, (k3 is preferably 3~6);
[0184] Joint evaluation function weights: traditional image evaluation function wc, focus quality scoring function wd;
[0185] Focusing process recording
[0186] The complete focusing process is shown in Table 2. The z-position in the table is expressed as a multiple of the relative depth of field D, which facilitates comparison between different optical systems.
[0187] Table 2: Focusing Process Search Record Table
[0188]
[0189] Results Analysis: Focusing Process Statistics
[0190] Coarse search phase: 26 steps (step size 15D), after full range scanning, D(z) is smoothed through a 3-point sliding window to lock in candidate intervals;
[0191] Fine-scale search phase: 6 steps (step size 8D), using a joint evaluation function. And perform 5-point sliding window smoothing to locate candidate peaks;
[0192] Verification phase: 3 steps, three-frame local verification detection Automatically adjusts to the correct focal plane;
[0193] Total sampling steps: 35 steps;
[0194] Final positioning result: z* = 0, which perfectly matches the actual focal plane, with a positioning error of 0.
[0195] The phased strategy of this invention completes focusing in just 35 steps. More importantly, the phased strategy effectively avoids erroneous convergence caused by spurious peaks in the focus quality score through a joint evaluation function and cross-validation mechanism, ensuring the accuracy and robustness of focusing.
[0196] Experiment Example 3: Migration and Deployment Verification
[0197] To verify the rapid transfer capability of the autofocus method of this invention, tests were conducted in the following scenarios:
[0198] Scenario A: Switching from BPAE cells to HeLa cells
[0199] Scene B: Switching from a 60× / 1.4NA oil immersion lens to a 40× / 0.95NA dry lens.
[0200] Transfer strategy: Keep the dual-branch encoder frozen, and retrain the regression head using only 15 z-stacks from the target scene. The training time is approximately 5 minutes. The results are shown in Table 3.
[0201] Table 3: Migration Deployment Performance Table
[0202]
[0203] The results show that:
[0204] Scenario A: After migration, SRCC recovered from 0.92 to 0.97 (close to the original performance of 0.985).
[0205] Scenario B: SRCC recovered from 0.88 to 0.96 after migration.
[0206] This verifies the ability of the method of the present invention to quickly adapt to new scenarios with a small amount of data.
Claims
1. A self-supervised image quality assessment method for microscope autofocusing incorporating physical prior soft tags, characterized in that, The method includes the following steps: Step S1: Training and Deploying the Scoring Model; Step S2: System initialization and parameter adaptation; Step S3: Coarse search phase; Step S4: Fine-grained search phase; Step S5: Local fine-tuning verification and pseudo-peak escape stage; Step S6: Focusing and multi-field scanning.
2. The microscope autofocusing method according to claim 1, characterized in that: Step S1 specifically includes the following steps: S1-1 Data Acquisition: Acquire at least one set of Z-stack image sequences obtained by scanning along the Z-axis, wherein the Z-stack image sequences cover the defocusing process from out of focus to focus and back to out of focus; S1-2 Optical Parameter Reading and Depth of Field Calculation: Read the numerical aperture NA and imaging wavelength λ of the current objective lens, and calculate the theoretical depth of field DOF according to equation (1); Equation (1); 3. S1-3 ROI Mask Generation and Foreground Selection: The original image is normalized to suppress the influence of extreme noise, and an initial foreground mask is obtained through threshold segmentation. The original image is a single-frame fluorescence micrograph acquired at each z-axis position in the Z-stack image sequence. Then, morphological operations and connected component analysis are performed to remove small noise areas, resulting in the ROI mask. When no effective foreground is detected, the entire image or a preset center region is used as the ROI input. S1-4 Multi-scale image patch sampling and validity screening: Multi-scale sampling is performed within the area covered by the ROI mask obtained in step S1-3 to obtain different receptive field features; foreground coverage screening is performed on the sampled image patches, and texture-guided selection is performed in combination with texture intensity index, that is, the texture intensity index T is calculated for each candidate patch using the Laplacian variance of equation (4). i : Equation (4); In the formula, P i For the i-th patch, For the Laplace operator, then press T. i Sort the samples from largest to smallest and select the top-k patches as input samples; S1-5 Physically Sensing Soft Tag Generation: Based on the theoretical depth of field (DOF) calculated in step S1-2, a soft tag Score(z) that varies with the defocus distance is constructed. The soft tag Score(z) adopts the Lorentzian distribution form shown in equation (2). Equation (2); in, Sampling location, The focal plane position, The distribution width parameter is dynamically calculated from the numerical aperture and imaging wavelength. , ; S1-6 Model Training: The input samples obtained in step S1-4 are input into the focus quality scoring model for training. The focus quality scoring model includes a content branch encoder, a quality branch encoder, and a lightweight regression head for training. During the training process, the mean square error between the predicted score and the physical perception soft label is used as the basic loss, and the total loss function shown in equation (3) is superimposed. ; Equation (3); In the formula, Predict the output for the scoring model; For physical perception soft tags; For predicting scores With physical labels The mean square error; ; These are the regularization weight coefficients; S1-7 Cross-validation and generalization constraints: Images within the same Z-stack scan sequence are grouped into the same set, and when dividing the training set and validation set, images from the same Z-stack appear only in the training set or the validation set; S1-8 Model Consolidation and Deployment: Save the trained focus quality scoring model parameters as a model to be deployed and configure it as an inference model that can be called during the focus execution phase.
4. The microscope autofocusing method according to claim 2, characterized in that: Step S2 specifically includes the following steps: S2-1 Optical Parameter Reading: When autofocus starts, read the numerical aperture NA and imaging wavelength λ of the current objective lens; S2-2 step size automatic calculation: The theoretical depth of field (DOF) is calculated according to equation (1), and the following three key step size parameters are generated based on the calculated theoretical depth of field (DOF): Coarse search with large step size: k1 is preferably 10-20; Fine-grained search with small steps: k2 is preferably 5-10; Verification step size: k3 is preferably 3 to 6; S2-3 Model Loading: Load the focus quality scoring model parameters saved in steps S1-8 as the inference model, freeze the encoder weights, and prepare for real-time inference of image focus quality score.
5. The microscope autofocusing method according to claim 3, characterized in that: Step S3 specifically includes the following steps: S3-1: Control the stage with large steps Move within the preset scanning range; S3-2: Real-time Acquisition and Calculation: During the focus search process, whenever the motorized stage moves to a new axial position z, a single-frame fluorescence microscopy image at that position is acquired. After preprocessing, the image is used as input to the focus quality scoring model trained in steps S1-6 and loaded in step S2-3 for inference calculation. The model input is a single-frame fluorescence image at the current Z position or a region of interest (ROI) image patch extracted from that image. The model output is the focus quality score D(z) corresponding to the current Z position. The focus quality score D(z) reflects the scalar value of the current focus quality scoring model's assessment of image sharpness. Simultaneously, a traditional sharpness score C(z) is calculated for the same frame image. The traditional sharpness score C(z) is calculated using a gradient-type or high-frequency response sharpness operator and is used to characterize the local texture and edge intensity of the image. The focus quality score D(z) and the traditional sharpness score C(z) are then normalized and jointly fused as follows: (a) Normalization: Focus quality score within the current search phase. Compared to traditional resolution rating Min-Max normalization is performed using the following formulas (5) and (6): Equation (5); Equation (6); in, This refers to the focus quality score, specifically the focus quality scoring model at different positions. The output; For traditional sharpness scoring, that is, the traditional sharpness evaluation function at location The output; , Each in the current search phase The minimum and maximum values, , Each in the current search phase The minimum and maximum values; Focus quality rating Compared to traditional resolution rating Normalized score; (b) Joint Integration: Defining the Joint Evaluation Function : Equation (7); in, For the joint evaluation function, through the joint evaluation function Calculate the joint score. Traditional evaluation weighting coefficients, This is the weighting coefficient for the focus quality score, and , ; S3-3: Dynamic Monitoring: Real-time monitoring of the changing trend of the joint score; smoothing the score sequence and calculating the score difference between two adjacent samples in real time; when the score difference changes from positive to negative, and in subsequent consecutive... The sampling point remains negative, while the decrease reaches a preset threshold. When the peak value has been exceeded, the current scanning phase is stopped; among which, The number of sampling points for determining continuous descent is selected as 2 to 5; The threshold for the decrease is set to 0.05–0.2 after the score is normalized to [0,1], or it can be set according to the dynamic range of the score in this stage. The scan is stopped when the score initially rises and then drops significantly, forming a peak shape. S3-4: Range Locking: Determine a range centered on the detected approximate peak position. The region is used as the candidate interval; Step S4 specifically includes the following steps: S4-1: Control the electric stage to retract to the starting position of the candidate area; S4-2: Switch to small step size Perform high-density scanning within the candidate region; S4-3: Acquire images for each scan point and calculate the joint score; S4-4: The sliding window algorithm is used to smooth the scoring sequence obtained in step S4-3, and the local maximum position of the smoothing curve is located. .
6. The microscope autofocusing method according to claim 4, characterized in that: Step S5 specifically includes the following steps: To prevent the focus quality scoring model from generating false peaks at extremely low signal-to-noise ratios, a joint verification mechanism of three-point verification and dual-modal cross-validation is introduced. Only peaks that simultaneously satisfy both three-point verification and dual-modal cross-validation can be considered true peaks. S5-1 Three-point sampling: Location of the local maximum value located in step S4-4 Three verification images were acquired at the tiny offset positions before and after the image, and the tiny offset positions were: and ; S5-2 three-point peak pattern confirmed: Calculate the joint score of the three frames; if the peak value is high in the middle and low on both sides, then... and If the condition is met, it is determined to be a valid peak value; if the condition is not met, it means that the true focal plane is located in a nearby position, triggering the escape mechanism to continue fine-tuning the search along the joint score upward direction until the peak value condition is met. S5-3 dual-modal cross-validation: Calculate peak Traditional sharpness rating values for location images The Brenner gradient value is used as a traditional sharpness evaluation value and compared with the traditional evaluation value of the peak neighborhood; if the focus quality scoring model generates a peak value... The deviation of the peak position obtained by the traditional method exceeds a preset threshold. If so, the focus quality scoring model is determined to be affected by texture interference; Pick At this point, the system automatically downgrades, using the peak position of the traditional evaluation function as the final focal plane position. .
7. The microscope autofocusing method according to claim 5, characterized in that: Step S6 specifically includes the following steps: S6-1 Execution Action: Drive the electric stage precisely to the final focal plane position confirmed in step S5-3. Lock the microscope position and complete the automatic focusing of the current field of view; S6-2 Status Update: The currently confirmed final focal plane position is updated. Stored as a state variable; S6-3 Multi-Field Relay: During continuous scanning of multi-well plates or large tissue sections, the final focal plane position of the current field of view is recorded. As the initial search center for the next adjacent field of view, and by narrowing the coarse search range from a global scan to a local scan, the coarse search range is set to... .
8. A microscope autofocusing system that combines self-supervised image quality assessment with physical prior soft tags, characterized in that, The system includes: The host computer has an autofocus control unit for running the autofocus control program and performing data processing and decision-making. A fluorescence microscope, used to create a fluorescence imaging optical path for a sample; Z-axis actuator, which carries the fluorescence microscope, is capable of micron-level stepping and positioning in the Z-axis direction, thereby changing the relative axial distance between the sample and the objective lens to achieve focusing, so as to meet the sequence acquisition requirements under different defocusing degrees. An electric objective lens switching device is used to switch objectives and to provide or provide feedback on the optical parameters related to the current objective lens; The camera is connected to the fluorescence microscope in an optical path. The optical image output by the fluorescence microscope is coupled to the camera through the imaging port. The camera acquires the fluorescence image and outputs digital image data to the host computer after photoelectric conversion. The host computer is connected to the Z-axis actuator, the electro-optical objective lens switching device, and the camera. It sends Z-axis motion control commands to the Z-axis actuator and receives motion status or position feedback through the autofocus control unit, sends objective lens switching control commands to the electro-optical objective lens switching device and receives objective lens parameters and status feedback, and receives image data streams output by the camera.
9. The microscope autofocus system according to claim 7, characterized in that: The autofocus control unit includes: Equipment control module: used to output displacement control commands to the Z-axis actuator and switching control commands to the electro-optical endoscope conversion device; Image acquisition module: used to receive image data output by the camera and form a Z-stack image sequence or a frame-by-frame image sequence; Parameter Adaptive Module: Used to read the objective lens numerical aperture NA and imaging wavelength λ at the start of microscope autofocus, and calculate the theoretical depth of field DOF and generate search step size parameters related to DOF. Focus evaluation and inference module: used to calculate the focus quality score D(z) and the traditional sharpness score C(z) of the acquired image, and generate a joint evaluation F(z); Focusing decision and process control module: Used to execute coarse search, fine search and verification processes based on the changing trend of the joint evaluation F(z), and output the final focal plane position. .
10. The microscope autofocus system according to claim 7, characterized in that: The Z-axis actuator is an electric stage.
11. The microscope autofocus system according to claim 7, characterized in that: The connection between the host computer and the Z-axis actuator, the electro-optical mirror conversion device, and the camera can be any one of USB, GigE, serial port, or Ethernet port.