Adaptive calibration method for confocal microscope based on LSTM temporal network

Abstract

The adaptive calibration method for confocal microscope based on LSTM temporal network, through the innovative combination of adaptive calibration algorithm and temporal neural network, enables the confocal microscope to have powerful learning ability, real-time distortion correction ability, image analysis and calibration ability and enhanced image resolution ability in metrological calibration. Through the learning and optimization of the temporal neural network, the microscope self-adjusts parameters and minimizes calibration errors to achieve optimal performance. At the same time, the temporal neural network can also help to identify and correct distortion in microscope images in real time and improve the accuracy of measurement. In addition, the use of temporal neural network for image analysis and calibration can realize automated detection and calibration, reduce human intervention and errors, and improve the efficiency and reliability of the calibration process. Finally, the super-resolution capability of the temporal neural network can also improve the resolution of microscope images and improve measurement accuracy and credibility.

Description

Technical Field

[0001] The present invention relates to the technical field of confocal microscope adaptive calibration, and in particular to a confocal microscope adaptive calibration method based on LSTM temporal network. Background Art

[0002] The confocal microscope is a high-precision microscope that can be used to observe the details of tiny objects such as cells and tissues. Due to its wide application in scientific research, medical treatment and other fields, accurate measurement of its observation results is very important to ensure the accuracy of the experiment and the reliability of the results. In the use of the confocal microscope, due to the influence of various factors, such as differences in instrument performance and the diversity of observation methods, errors in the observation results may occur. Therefore, it is very important to establish a complete confocal microscope measurement and calibration method to ensure its accuracy and reliability. However, my country currently lacks a complete nano-quantity transfer system, which is out of touch with the industry. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a confocal microscope adaptive calibration method based on LSTM temporal network, which can achieve high-resolution two-dimensional imaging under non-destructive conditions and has extremely high application potential.

[0004] To achieve the above object, the technical solution adopted by the present invention is:

[0005] The confocal microscope adaptive calibration method based on LSTM temporal network has the following specific steps:

[0006] 1) High-speed scanning self-correction:

[0007] The LSTM temporal network is used to improve the two-dimensional image reconstruction process of the confocal microscope. High-speed scanning self-correction will be implemented, and the electron microscope scanning will be modeled and simplified into a simple harmonic motion formula. The self-adjustment factor Ai will be introduced to solve the nonlinear distortion problem and improve the accuracy of the system scanning data. By using the temporal LSTM network to predict the self-adjustment factor Ai, the number of scanning data lines of the system can be predicted, thereby reducing distortion and better capturing the inertia of the galvanometer motor, which will provide a scientific and reasonable two-dimensional digital reconstruction image for digital image collection;

[0008] 2) Digital 2D image collection:

[0009] The point light source is driven to move on the sample plane by a galvanometer. After the laser is emitted from the light source, it will be divided into two beams by a beam splitter. One beam acts on the galvanometer, which is driven by the galvanometer source to move the point light source on the sample plane; the other beam will be directly received by the photomultiplier tube to excite fluorescence on the sample plane and collect the excited fluorescence signals. These light signals will then be numerically processed by a computer and converted into digital signals to form a digital two-dimensional image.

[0010] 3) High-resolution image restoration:

[0011] Further processing of digital two-dimensional images by combining Bayesian reasoning and convolution modules, i.e., temporal neural networks, aims to eliminate the effects of noise and other factors, including defocus and motion blur, on image quality, ensuring that the reconstructed image has resolution and clarity;

[0012] 4) Image analysis and calibration:

[0013] Continue to use temporal neural networks to conduct in-depth analysis and calibration of the restored images, use threshold segmentation methods to extract features from the restored images, measure the size and characteristics of cells, and clearly display the changes in biological slices, so as to better conduct scientific research.

[0014] As a further improvement of the present invention, the simple harmonic motion formula in step 1) is:

[0015]

[0016] Mx is the maximum deflection angle of the galvanometer, t is the sampling time, the distance between the objective lenses is f, the sampling frequency of the data acquisition card is Zs, and the frequency of the scanning galvanometer is Zx;

[0017] The scanning self-correction formula is:

[0018]

[0019] Mx is the maximum deflection angle of the galvanometer, t is the sampling time, the objective lens spacing is f, the sampling frequency of the data acquisition card is Zs, the frequency of the scanning galvanometer is Zx, Ai is the adjustment factor, which represents the position adjustment of the i-th sampling point. The adjustment factor Ai is predicted by the time series neural network LSTM to achieve compression of the middle sampling point and "stretching" of the sampling points on both sides to maintain the consistency of the distance between points.

[0020] As a further improvement of the present invention, the digital image acquisition formula for forming the digital two-dimensional image in step 2) is:

[0021]

[0022] Q xy is the coordinate of a point on the projected sample plane, θx is the X-axis galvanometer deflection angle, θ y is the deflection angle of the Y-axis galvanometer, e is the spatial distance between the galvanometers, and d is the distance from the Y-axis galvanometer to the sample plane.

[0023] As a further improvement of the present invention, the Bayesian inference formula in step 3) is:

[0024]

[0025] Z is the normalization factor, ||*|| is the vector norm, and the noisy image N g , original image sample N u , N g In N u and K, it obeys Gaussian distribution;

[0026] The convolution module formula is:

[0027] P(N u |N g ,K)=f(P(N g |N u ,K)*P(N u )) (5)

[0028] P(N u |N g ,K) is a given noisy image N g Under the condition of the fuzzy operator K, the original image sample N u The posterior probability, P(N g |N u ,K) is the given original image sample N u Under the condition of the fuzzy operator K, the N g The posterior probability, P(N u ) is the original image sample N u The prior probability of .

[0029] As a further improvement of the present invention, the image calibration formula in step 4) image analysis and calibration is:

[0030] F=g(LSTM(X,Y)) (6)

[0031] F represents the extracted image features, g represents the thresholding operation, and LSTM represents the reconstructed high-resolution image.

[0032] This application has the following benefits:

[0033] 1. Improve image quality:

[0034] Through high-speed scanning self-correction, nonlinear distortion problems can be effectively reduced, thereby improving the accuracy of reconstructed digital images. The introduction of self-adjustment factor Ai and LSTM network can more accurately predict the number of system scan data line points, thereby optimizing the reconstruction process of two-dimensional digital images.

[0035] 2. Image data collection:

[0036] Using a galvanometer to drive a point light source to move, combined with laser and fluorescence technology, can more accurately collect information on the sample plane. This method not only improves the efficiency of data collection, but also through photomultiplier tubes and computer processing, the optical signal can be converted into a digital signal in real time, which is convenient for subsequent analysis.

[0037] 3. Enhance image resolution:

[0038] Combining Bayesian P reasoning and convolution f module (temporal neural network) to process digital images can effectively eliminate influencing factors such as noise, defocus and motion blur, ensuring that the reconstructed image has higher resolution and clarity. This is especially important for fields that require precise observation and analysis of microstructures.

[0039] 4. Improve analysis accuracy:

[0040] By using a temporal neural network to conduct in-depth analysis and calibration of the restored images, features in the images, such as the size and characteristics of cells, can be extracted more accurately. The application of the threshold segmentation method also helps to more clearly show the changes in biological slices, thereby providing more reliable data support for scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A high-speed scanning self-correction flow chart of a confocal microscope adaptive calibration algorithm based on an LSTM temporal network provided in an embodiment of the present application;

[0042] Figure 2 A high-speed scanning image acquisition flow chart of a confocal microscope adaptive calibration algorithm based on an LSTM temporal network provided in an embodiment of the present application;

[0043] Figure 3 Schematic diagram of image denoising based on the confocal microscope adaptive calibration algorithm based on the LSTM temporal network provided in the embodiment of the present application

[0044] Figure 4 The overall image flow chart of the confocal microscope adaptive calibration algorithm based on the LSTM temporal network provided in the embodiment of the present application. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0046] like Figure 1 A high-speed scanning self-correction flow chart of a confocal microscope adaptive calibration algorithm based on an LSTM temporal network provided in an embodiment of the present application;

[0047] Step S1 High-speed scanning self-correction:

[0048] This step uses the LSTM temporal network to improve the two-dimensional image reconstruction process of the confocal microscope. This step implements high-speed scanning self-correction, models the electron microscope scanning, simplifies it into a simple harmonic motion formula, and introduces the self-adjustment factor Ai to solve the nonlinear distortion problem and improve the accuracy of the system scanning data. By using the temporal LSTM network, the self-adjustment factor Ai is predicted. This step can predict the number of scan data lines of the system, thereby reducing distortion and better capturing the inertia of the galvanometer motor. It will provide a scientific and reasonable two-dimensional digital reconstruction image for digital image collection.

[0049] The formula for simple harmonic motion is:

[0050]

[0051] Mx is the maximum deflection angle of the galvanometer, t is the sampling time, the distance between the objective lenses is f, the sampling frequency of the data acquisition card is Zs, and the frequency of the scanning galvanometer is Zx.

[0052] When the galvanometer scans the sample plane, in order to ensure the accuracy of imaging, it is necessary to ensure that the logical position of the sampling point is consistent with the actual spatial position. However, in practice, due to the nonlinear motion and inertia of the galvanometer, the sampling points of the galvanometer in the back-and-forth scanning stroke cannot match the actual spatial position, resulting in distortion of the image after two-dimensional rearrangement. In order to solve this problem, this step uses a temporal neural network to self-correct the sampling points.

[0053] The scanning self-correction formula is:

[0054]

[0055] Mx is the maximum deflection angle of the galvanometer, t is the sampling time, the objective lens spacing is f, the sampling frequency of the data acquisition card is Zs, the frequency of the scanning galvanometer is Zx, Ai is the adjustment factor, which represents the position adjustment of the i-th sampling point. The adjustment factor Ai is predicted by the time series neural network LSTM to achieve compression of the middle sampling point and "stretching" of the sampling points on both sides to maintain the consistency of the distance between points.

[0056] By introducing the self-adjusting factor Ai, this method can solve the problem of nonlinear distortion in the microscope scanning process. The self-adjusting factor can be adjusted in real time according to the characteristics of the system, thereby reducing image distortion. The LSTM network can predict the self-adjusting factor Ai, thereby improving the accuracy of the system scanning data. By predicting and adjusting various parameters during the scanning process, the system can better capture the inertia of the galvanometer motor and thus control the scanning process more accurately. By predicting the number of scan data lines of the system, this method can optimize the acquisition process of digital images. This optimization can reduce image distortion, making the image clearer and more accurate.

[0057] Step S2: Digital 2D image collection

[0058] In this step, a confocal galvanometer is used to drive a point light source to move on the sample plane. After the laser is emitted from the light source, it will be divided into two beams by a beam splitter. One beam acts on the galvanometer, which is driven by the point source to move the galvanometer, driving the point light source to move on the sample plane. Through the precise control of the galvanometer, high-precision point light source movement on the sample plane can be achieved. This precise control can ensure that the laser accurately irradiates the required sample position, thereby improving the accuracy and efficiency of fluorescence excitation. The other beam will be directly received by the photomultiplier tube. Due to the use of point light source and fluorescence excitation, this method can obtain high-resolution digital two-dimensional images. The small size of the point light source and the specificity of fluorescence make the image have higher contrast and detail expression, which is conducive to better observation and analysis of samples. In this step, fluorescence is excited on the sample plane and the excited fluorescence signal is collected. These light signals will then be numerically processed by a computer and converted into digital signals to form a digital two-dimensional image. Figure 2 The high-speed scanning image acquisition flow chart of the confocal microscope adaptive calibration algorithm based on LSTM temporal network

[0059] The digital image acquisition formula is:

[0060]

[0061] Q xy is the coordinate of a point on the projected sample plane, θ x is the X-axis galvanometer deflection angle, θ y is the Y-axis galvanometer deflection angle, e is the spatial distance between the galvanometers, and d is the distance from the Y-axis galvanometer to the sample plane. Since a photomultiplier tube is used to receive the light signal and a computer is used for numerical processing, this method can achieve real-time monitoring and rapid feedback. This allows researchers to observe the fluorescence changes and kinetics of the sample in real time, thereby better understanding the biological characteristics and behavior of the sample.

[0062] Step S3: High-resolution image restoration

[0063] This step combines Bayesian reasoning and convolution modules (combined with temporal neural networks) to further process the digital two-dimensional image, aiming to eliminate the impact of noise and other factors (such as defocus and motion blur) on image quality, ensuring that the reconstructed image has high resolution and high definition.

[0064] The Bayesian inference formula is:

[0065]

[0066] Z is the normalization factor, ||*|| is the vector norm, and the noisy image N g , original image sample N u . N g In N u and K, it obeys Gaussian distribution.

[0067] The convolution module formula is:

[0068] P(N u |N g ,K)=f(P(N g |N u ,K)*P(N u )) (5)

[0069] P(N u |N g ,K) is a given noisy image N g Under the condition of the fuzzy operator K, the original image sample N u The posterior probability of P(N g |N u ,K) is the given original image sample N u Under the condition of the fuzzy operator K, the N g The posterior probability of P(N u ) is the original image sample N u The prior probability of . Figure 3 Schematic diagram of high-speed scanning image denoising based on the confocal microscope adaptive calibration algorithm based on LSTM temporal network

[0070] Through Bayesian reasoning, this method can deduce the distribution of noisy images based on known noise models and image samples. Then, using this distribution, the impact of noise on image quality can be effectively eliminated, and the clarity and resolution of the image can be improved. Due to the introduction of convolutional modules and temporal neural networks, this method can better handle blur phenomena such as motion blur and defocus. By learning and simulating the behavior of blur operators, this method can effectively restore blurred images and improve the resolution and clarity of images.

[0071] Bayesian reasoning is used, and the method has a high tolerance to noise and blur. Even in the presence of large noise or severe blur, the method can still produce relatively accurate results. This makes the method highly robust and can run stably in various practical application scenarios.

[0072] Step S4: Image analysis and calibration

[0073] The image calibration formula is:

[0074] F=g(LSTM(X,Y)) (6)

[0075] F represents the extracted image features, g represents the thresholding operation, and LSTM represents the reconstructed high-resolution image.

[0076] By using a time series neural network to analyze the restored images, threshold segmentation can be used to extract key features from the images. This includes information such as cell size, shape, and color. This helps to gain a more comprehensive understanding of the structure and composition of biological samples. Not only can the restored images be analyzed in depth, the features in the images can be more accurately identified and extracted. The application of the threshold segmentation method can also further refine the feature extraction, ensure the accuracy of the extracted cell size and features, and thus improve the accuracy of image analysis. The application of time series neural networks can realize automated processing and analysis, reducing reliance on manual operations. This can greatly improve work efficiency, shorten processing time, and reduce the possibility of human error. Such as Figure 4 The image shows the overall flow chart of the confocal microscope adaptive calibration algorithm based on the LSTM temporal network.

[0077] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A confocal microscope adaptive calibration method based on LSTM temporal network, characterized in that: The specific steps are: 1) High-speed scanning self-correction: The LSTM temporal network is used to improve the two-dimensional image reconstruction process of the confocal microscope. High-speed scanning self-correction is implemented, the electron microscope scanning is modeled, simplified to a simple harmonic motion formula, and a self-adjustment factor A is introduced. i To solve the problem of nonlinear distortion and improve the accuracy of system scanning data, the self-adjustment factor A is predicted by using a time series LSTM network. i , can predict the number of scan data line points of the system, thereby reducing distortion and better capturing the inertia of the galvanometer motor, which will provide scientific and reasonable two-dimensional digital reconstruction images for digital image collection; The simple harmonic motion formula in step 1) is: Mx is the maximum deflection angle of the galvanometer, t is the sampling time, the distance between the objective lenses is f, and the sampling frequency of the data acquisition card is Z s , the scanning frequency of the galvanometer is Z x ; The scanning self-correction formula is: Mx is the maximum deflection angle of the galvanometer, t is the sampling time, the distance between the objective lenses is f, and the sampling frequency of the data acquisition card is Z s , the scanning frequency of the galvanometer is Z x ,A i is the self-adjustment factor, indicating the position adjustment of the i-th sampling point, and the adjustment factor A is predicted by the time series neural network LSTM i , realize the compression of the middle sampling point and the "stretching" processing of the sampling points on both sides, and maintain the consistency of the distance between points; 2) Digital 2D image collection: The point light source is driven to move on the sample plane by a galvanometer. After the laser is emitted from the light source, it will be divided into two beams by a beam splitter. One beam acts on the galvanometer, which is driven by the galvanometer source to move the point light source on the sample plane; the other beam will be directly received by the photomultiplier tube to excite fluorescence on the sample plane and collect the excited fluorescence signals. These light signals will then be numerically processed by a computer and converted into digital signals to form a digital two-dimensional image. The digital image acquisition formula for forming a digital two-dimensional image in step 2) is: Q xy is the coordinate of a point on the projected sample plane, θ x is the X-axis galvanometer deflection angle, θ y is the Y-axis galvanometer deflection angle, e is the spatial distance between the galvanometers, and d is the distance from the Y-axis galvanometer to the sample plane; 3) High-resolution image restoration: The digital 2D images are further processed by combining Bayesian reasoning and convolution modules to eliminate the effects of noise and other factors including defocus and motion blur on image quality, ensuring that the reconstructed images have resolution and clarity; The Bayesian inference formula in step 3) is: Z is the normalization factor, ||*|| is the vector norm, and the noisy image N g , original image sample N u , N g In N u and K, it obeys Gaussian distribution; The convolution module formula is: P(N u |N g ,K)=f(P(N g |N u ,K)*P(N u )) (5) P(N u |N g ,K) is a given noisy image N g Under the condition of the fuzzy operator K, the original image sample N u The posterior probability, P(N g |N u ,K) is a given original image sample N u Under the condition of the fuzzy operator K, the N g The posterior probability, P(N u ) is the original image sample N u The prior probability of 4) Image analysis and calibration: Continue to use the time series neural network to conduct in-depth analysis and calibration of the restored images, use the threshold segmentation method to extract features from the restored images, measure the size and characteristics of cells, and clearly show the changes in biological slices, so as to better conduct scientific research; The image calibration formula in the step 4) image analysis and calibration is: F=g(LSTM(X,Y)) (6) F represents the extracted image features, g represents the thresholding operation, and LSTM represents the reconstructed high-resolution image.

Images