High-precision microsphere positioning method based on deep learning deblurring

Abstract

The invention relates to a microsphere high-precision positioning method based on deep learning deblurring, and aims to solve the problem of defocusing blurring caused by movement of microspheres in the optical axis direction in the microscopic imaging process so as to effectively improve the three-dimensional positioning precision of the microspheres. According to the method, a deep learning image restoration network is introduced, real-time sharpening processing is carried out on a blurred image, and gray features used for microsphere center positioning are effectively reconstructed. In the network model training process, a structural similarity index is introduced as a performance evaluation standard, and the restoration quality of the image is improved by optimizing a loss function. The transverse positioning achieves nanoscale precision by adopting a gray centroid algorithm, and the axial positioning is realized by constructing a radial gray vector of an image and matching the radial gray vector with a calibration model. The method provided by the invention has the characteristics of high measurement precision, strong robustness and the like, and is suitable for particle tracking, micro-nano operation, robot micro-assembly and other precise positioning tasks with a relatively large space motion range.

Description

Technical Field

[0001] The present invention relates to a high-precision positioning method for microspheres based on deep learning deblurring, which aims to solve the problem of image defocus blur caused by the movement of microspheres along the optical axis during microscopic imaging, thereby significantly improving the accuracy and stability of three-dimensional positioning of microspheres. The method introduces a deep learning image restoration network to perform real-time clarity processing on blurred images, and effectively reconstructs the imaging details and grayscale distribution characteristics of the microspheres. The lateral (X, Y) positioning adopts a grayscale centroid weighted algorithm to achieve nanometer-level accuracy, and the axial (Z) positioning constructs a radial vector by changing the microsphere imaging characteristics with defocus, and matches it with a pre-established calibration model to achieve high-precision axial positioning. The present invention has the advantages of strong robustness and high real-time performance. It overcomes the problems of low restoration quality and susceptibility to background interference of traditional image restoration technology when processing blurred microsphere images, and effectively improves the measurement accuracy and reliability of the microscopic vision system in complex dynamic scenes. This method is mainly used for precision positioning tasks with a large spatial motion range, such as particle tracking, micro-nano operation, and robotic micro-assembly. Background Art

[0002] In microscopic vision systems, high-precision three-dimensional positioning of microscale targets, such as microspheres, is a key technology for achieving precision tasks such as particle tracking, cell manipulation, and micro-nanoassembly. Traditional three-dimensional positioning methods typically rely on clear images for feature extraction and position reconstruction. However, in practical applications, the slight movement of the microsphere along the optical axis (Z) during imaging can easily cause defocus blur, resulting in blurred edge information and loss of grayscale features, which seriously affects the stability of microsphere positioning and measurement accuracy.

[0003] In order to effectively solve the problems of position drift, image noise enhancement and imaging feature attenuation caused by microsphere defocus, and to improve the robustness and positioning accuracy of the microscopic vision measurement system in a dynamic environment, it is urgent to introduce an efficient restoration method with image deblurring capability to ensure the reliability and consistency of the microsphere three-dimensional positioning process.

[0004] In recent years, deep learning technology has made significant progress in the field of image deblurring. Its end-to-end modeling mechanism can simultaneously achieve global reconstruction and local detail compensation of degraded images, effectively improving the overall clarity and contrast of the image while preserving edge structure and imaging features. Compared with traditional image restoration technology that relies on the point spread function (PSF) model of the optical system, deep learning has stronger blur adaptation capabilities and is particularly suitable for dealing with complex non-uniform and nonlinear blur problems in microscopic images. By extracting multi-scale blur features and fusing spatial structure information with grayscale distribution relationships, deep learning models can achieve stable and accurate image restoration at different blur levels. When applied to the field of microsphere visual tracking, this technology can effectively restore the edge contours and grayscale details of blurred microsphere images, providing a clear and stable visual foundation for subsequent high-precision three-dimensional positioning. Summary of the Invention

[0005] The purpose of the present invention is to propose a three-dimensional high-precision positioning method for microspheres that integrates deep learning deblurring technology to address the image blur problem caused by the movement of microspheres along the optical axis during microscopic imaging.

[0006] To achieve the above objectives, the present invention provides a high-precision microsphere positioning method based on deep learning image restoration technology, which is characterized by the following steps:

[0007] Step 1: Place the glass slide with attached microspheres on a precision positioning platform to ensure that the microspheres to be tracked are within the camera's field of view. After imaging through the objective lens, the microsphere image is transmitted to the computer processing system.

[0008] Step 2: Adjust the coarse adjustment stage of the microscope so that the microsphere is accurately in the focal plane of the visual system to obtain a high-quality, clearly focused reference image I s .

[0009] Step 3: Drive the precision positioning platform to move gradually within the axial measurement range with a set step length, and collect a series of real blurred image datasets I generated by the defocus motion of the microspheres during the imaging process. d .

[0010] Step 4: Based on Streibl's partial coherence imaging theory, a point spread function model is established to obtain the reference image I s Perform defocus blur processing to generate multiple simulated image datasets at different axial positions I sim , and compared with the real dataset I d Combined, construct the fuzzy image training set I for training tra .

[0011] Step 5: Collect microsphere images with different defocus blur levels and lighting background conditions to construct a test set I for evaluating the robustness of the network model. tes.

[0012] Step 6: For training set I tra and test set I tes Normalization is performed, data enhancement operations are applied, and the preprocessed data is loaded into the deep learning network model for training and evaluation.

[0013] Step 7: To maintain the symmetry of the spatial structure of the restored microsphere image, a symmetry consistency error term is introduced into the loss function, and the central symmetry error threshold is set according to the required positioning accuracy.

[0014] Step 8: To ensure that the grayscale value distribution of the restored microsphere image is as close as possible to image I s , and to avoid the influence of background artifacts on the lateral measurement accuracy, the gray value cross entropy loss term is introduced into the loss function to enhance the consistency of grayscale information.

[0015] Step 9. Design a spatial weight matrix and perform adaptive weight allocation based on the distance from each pixel to the center of the microsphere image. This will increase the focus on the center area of the microsphere in the loss calculation, thereby improving the lateral positioning accuracy based on the grayscale centroid algorithm and further improving the axial measurement accuracy.

[0016] Step 10: Based on the constructed training dataset, the deep learning network model is trained to achieve the mapping of blurred images to clear images by optimizing the loss function. The trained network is then deployed into the microsphere three-dimensional positioning system to achieve high-precision lateral and axial positioning of the microspheres in the defocused state.

[0017] The simulated microsphere image training set is characterized by establishing a point spread function (PSF) model h(r, z) at different axial positions based on Streibl partial coherence imaging theory, and its expression is:

[0018]

[0019] Where, ρ S and ρ P are the numerical apertures of the condenser and objective lens, l is the radial transverse spatial frequency, s is the axial spatial frequency, and λ is the wavelength of the incident light. s Perform convolution operation with the PSF at different defocus depths to generate a set of simulated microsphere image datasets with different defocus distances I sim , thereby constructing a training sample dataset covering different defocus levels.

[0020] The symmetric consistency error term is characterized by introducing a symmetric consistency error term L in the loss function. sym As a structural constraint mechanism, it is used to measure the difference in pixel grayscale values between the restored image and its central symmetric image. Its expression is:

[0021]

[0022] Where, I res (x, y) is the restored microsphere image, W and H are the width and height of the microsphere image respectively. Set L according to the image restoration accuracy requirements. sym The threshold value can effectively suppress the grayscale information changes caused by background disturbance during the blur removal process, thereby improving the lateral positioning accuracy based on the grayscale centroid algorithm.

[0023] The grayscale cross entropy loss is characterized by introducing the grayscale cross entropy loss as the grayscale consistency constraint L in the loss function. ce , whose expression is:

[0024]

[0025] The loss term is obtained by considering the pixel gray value of the restored microsphere image as a probability distribution and comparing it with the clear image I s The grayscale distribution of the image is calculated pixel by pixel, which effectively suppresses the grayscale deviation and unnatural brightness artifacts in the local area, is conducive to optimizing the lateral positioning accuracy of the microsphere and further improving the matching accuracy of the axial calibration model.

[0026] The spatial weight matrix is characterized by adaptive weight allocation based on the geometric distance of each pixel to the image center. The closer the pixel is to the image center, the higher the corresponding weight value is, thereby giving a greater optimization proportion to the microsphere area during the loss calculation process. Its expression is:

[0027]

[0028] In the formula, (x c ,y c ) is the center of the microsphere image, and σ controls the size of the region of interest, which can be adjusted to accommodate microspheres of varying sizes. This loss term incentivizes the deep learning model to restore the grayscale distribution and structural features of the central region of the microsphere image. Combined with the grayscale centroid algorithm, this helps improve the lateral positioning accuracy of the microspheres.

[0029] The deep learning network model is characterized by introducing a structural similarity index (SSIM) to restore the image I during the training process. res With clear image I s The structural consistency between the two images is quantified and evaluated, which effectively enhances the network's ability to restore the grayscale information of defocused microsphere images, thereby significantly improving the robustness and accuracy of microspheres in three-dimensional positioning.

[0030] The high-precision lateral positioning of microspheres is characterized by using the grayscale values in the restored image as weights to calculate the grayscale centroid coordinates of the microsphere area and extract its lateral position (X, Y) in the image plane. This effectively avoids positioning deviations caused by boundary diffusion in the grayscale weighted centroid algorithm when the image is blurred, and improves the lateral measurement accuracy of microspheres under defocus conditions.

[0031] The high-precision axial positioning of the microspheres is characterized by constructing a series of concentric rings on the blurred microsphere image with the lateral position (X, Y) extracted from the restored image as the center of the circle, calculating the average grayscale value of the pixels in each ring one by one to form a one-dimensional radial grayscale vector, and matching the radial vector with a pre-established axial calibration model to achieve high-precision axial measurement of the microspheres.

[0032] By changing the number and diversity of samples in the training dataset, it is possible to restore microsphere images in different imaging environments and defocus blur levels, thereby improving the three-dimensional positioning accuracy of the microspheres within a larger axial range.

[0033] By varying the threshold of the symmetry consistency error term, we can select appropriate microsphere imaging feature restoration strategies for varying restoration accuracy requirements. A lower threshold enhances the restoration of the microsphere's central symmetry, ensuring high-precision 3D positioning of the microsphere.

[0034] By varying the control parameters in the spatial weight matrix, the degree of regional focus in the microsphere image restoration process can be adjusted. Smaller parameter values improve the restoration accuracy of the microsphere core region and enhance the lateral and axial measurement accuracy of the microsphere.

[0035] Compared with the previous methods, the present invention has the following advantages:

[0036] ① The use of a deep learning network can effectively preserve the high-frequency information and imaging details of the microsphere's central area, overcoming the microsphere center positioning deviation caused by over-smoothing in traditional deblurring algorithms, thereby ensuring high-precision three-dimensional positioning of the microsphere in a defocused state. ② By fusing real images and simulated images to construct a training dataset, the deep learning model can adapt to the various defocus blur characteristics of the microsphere, significantly improving the positioning accuracy and robustness of the microsphere within a large axial range. ③ Based on the microsphere imaging characteristics, a customized loss function term is designed and introduced to effectively suppress the overfitting of microsphere features and background artifacts that appear during the training process of the deep learning model, further improving the lateral measurement accuracy and axial matching accuracy of the microsphere image. ④ The deep learning network is used to restore the grayscale information and imaging details in the blurred microsphere image. Based on the restored image, the grayscale centroid algorithm is applied to achieve high-precision lateral positioning of the microsphere. With this centroid as the center of the circle, the radial grayscale vector is extracted from the original blurred microsphere image, and then matched with the pre-established defocus calibration model to obtain the axial position, achieving high-precision three-dimensional measurement of the microsphere in a defocused state. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the structure of the microsphere high-precision three-dimensional positioning system based on deep learning deblurring of the present invention;

[0038] Figure 2 Schematic diagram of the principle of three-dimensional positioning of defocused microsphere images based on deep learning deblurring of the present invention;

[0039] Figure 3 This is the implementation process of deblurring defocused microsphere images based on deep learning in the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose and technical solution of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.

[0041] Figure 1 This is a schematic diagram of the structure of the deep learning deblurring-based microsphere 3D positioning system provided by the present invention. The measurement system consists of an inverted microscope 1, a precision displacement platform 4, a glass slide 6, a high-speed camera 8, and microspheres 9. The coarse adjustment platform 2, microscope frame 3, light source 5, and objective lens 7 are part of the inverted microscope 1. The glass slide 6, with a small number of microspheres 9 attached, is attached to the precision displacement platform 4. The image of the microspheres is captured by the camera 8 after passing through the inverted microscope.

[0042] The light source 5 is mounted on the microscope frame 3, directly above the inverted microscope 1, providing uniform illumination for capturing microsphere images. The objective lens 7 is threadedly secured to the microscope frame 3 and is used to image the microspheres. The coarse adjustment stage 2 is used to obtain a clear image of the microspheres and determine their initial axial position. The precision displacement stage 4 drives the microspheres, achieving axial motion control with nanometer-level resolution.

[0043] Figure 2Schematic diagram of the principle of three-dimensional positioning of defocused microsphere images based on deep learning deblurring. Figure (a) shows the input of a preprocessed blurred microsphere image into a trained deep learning image restoration network. The network automatically extracts the blurred features in the image and restores and reconstructs the image in combination with multiple loss functions, outputting a clear restored microsphere image. Figure (b) shows the principle of lateral positioning of microspheres based on the grayscale centroid algorithm. By calculating the grayscale centroid coordinates of the microsphere image, the image is divided into four quadrants, and the grayscale value differences between the quadrants are analyzed to accurately determine the center position of the microsphere in the lateral (X, Y) direction. Figure (c) shows the microsphere image is divided into a series of concentric rings with the centroid coordinates as the center of the circle, and the average grayscale value of all pixels in each ring is calculated to construct the radial vector of the microsphere image. Figure (d) shows the acquisition of microsphere images at different defocus positions along the axial direction according to a set step size, the corresponding radial vector is extracted, and a calibration model between radial features and axial displacement is established to achieve high-precision Z-direction positioning.

[0044] Figure 3 This is a schematic diagram of the implementation process of defocused microsphere image restoration based on deep learning. The process consists of three parts: data preprocessing, model training and testing. Among them, the data preprocessing part refers to operations such as scaling, cropping, random flipping and grayscale normalization of the real collected microsphere images, and using the point spread function model based on Streibl's partial coherence imaging theory to generate simulated defocused microsphere images, and construct a training dataset with real structure and diverse blur distribution. The model training part includes the design of the deep learning network structure, the calculation of the loss function and the preservation of the model parameters. The model testing stage mainly includes the restoration of the defocused microsphere image to be located, and the quantitative evaluation of the restored image quality through the structural similarity index (SSIM), providing high-quality image input for the subsequent high-precision three-dimensional positioning of the microspheres.

Claims

1. A high-precision microsphere positioning method based on deep learning deblurring, characterized by: Place the glass slide with attached microspheres on a precision positioning platform to ensure that the microspheres to be measured are within the camera's field of view. The image is then transferred to the computer processor after being imaged by the objective lens. The microscope is then adjusted to ensure that the microspheres are in the focal plane to obtain a clearly focused reference image. s , and drives the platform to move step by step within the axial range to collect the blurred image sequence I caused by defocus d Based on Streibl partial coherence imaging theory, a point spread function model is constructed to s Perform simulated fuzzy processing to generate simulated image dataset I sim , and the real blurred image I d Combined to construct training set I tra . Further collect images with different defocus levels and lighting conditions to construct the test set I tes The above data are input into the deep learning network for training and evaluation. In order to improve the lateral positioning accuracy of the microspheres, multiple structure-aware constraints are introduced into the loss function, including a symmetry consistency error term used to maintain the central symmetry of the microsphere image; A grayscale cross-entropy loss term is used to control the consistency of grayscale distribution; and a spatial weighting matrix that adaptively assigns weights based on the distance between pixels and the image center improves the restoration quality of the central region of the microsphere used for centroid positioning. Based on the constructed dataset and optimized loss function, a deep learning network model is trained and integrated into the microsphere 3D positioning system, achieving high-precision lateral and axial positioning of the microsphere in defocused conditions.

2. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1 is characterized in that: The simulated microsphere image dataset is constructed by using the point spread function (PSF) model of different axial positions based on Streibl partial coherence imaging theory to convert the clear image I s Perform convolution operation with the PSF at different defocus depths to generate a set of simulated microsphere image datasets with different defocus distances I sim , construct a training sample dataset covering different defocus levels.

3. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1 is characterized in that: The symmetry consistency error term effectively suppresses the grayscale information change caused by background disturbance during the blur removal process by measuring the pixel grayscale value difference between the restored image and its central symmetric image, thereby improving the lateral positioning accuracy of the microsphere based on the grayscale centroid algorithm.

4. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1, characterized in that: The grayscale cross entropy loss is to regard the pixel grayscale value of the restored image as a probability distribution and compare it with the corresponding clear image I s The pixel-by-pixel cross entropy calculation of the grayscale distribution can effectively suppress the grayscale deviation and brightness artifacts in the local area, which is conducive to the optimization of the lateral positioning accuracy of the microsphere.

5. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1, characterized in that: The spatial weight matrix adaptively distributes weights according to the geometric distance between each pixel and the image center, so that the deep learning network model tends to accurately reconstruct the grayscale distribution and structural characteristics of the central area of the microsphere. Combined with the grayscale centroid algorithm, it helps to improve the lateral positioning accuracy of the microsphere.

6. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1, characterized in that: The deep learning network model introduces the structural similarity index (SSIM) to restore the image I during the training process. res With clear image I s The structural consistency between them is quantified and evaluated, which effectively enhances the network model's ability to restore the grayscale information of defocused microsphere images, thereby improving the robustness and accuracy of microspheres in three-dimensional positioning.

7. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1, characterized in that: The high-precision lateral positioning of microspheres uses the grayscale values in the restored image as weights, calculates the grayscale centroid coordinates of the microsphere area, and extracts its lateral position (X, Y) in the image plane. This effectively avoids the positioning deviation caused by boundary diffusion in the grayscale weighted centroid algorithm when the image is blurred, and improves the lateral measurement accuracy of microspheres under defocus conditions.

8. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1, characterized in that: The high-precision axial positioning of the microspheres is carried out by using the lateral coordinates (X, Y) extracted from the restored image as the center of the circle. A series of concentric rings are constructed on the blurred microsphere image, and the average grayscale value of the pixels in each ring is calculated one by one to form a one-dimensional radial grayscale vector. This radial vector is then matched with a pre-established axial calibration model to achieve high-precision axial measurement of the microspheres.

9. The microsphere high-precision positioning method based on deep learning deblurring according to claim 1, characterized in that: By changing the number and diversity of samples in the training dataset, microsphere images with different degrees of blur can be restored, thereby improving the three-dimensional positioning accuracy of the microspheres within a larger axial range. By changing the threshold of the symmetry consistency error term, the corresponding microsphere imaging feature restoration strategy can be selected for different positioning accuracy requirements. A lower threshold is conducive to strengthening the restoration of the central symmetry features of the microspheres, ensuring high-precision three-dimensional positioning performance of the microspheres. By changing the control parameters in the spatial weight matrix, the degree of attention paid to different areas in the microsphere image restoration process can be adjusted. Smaller parameter values help improve the restoration accuracy of the microsphere core area and improve the lateral and axial measurement accuracy of the microspheres.

Images