Fast z-stack image registration method based on relatively non-uniform Gaussian blur

Abstract

The present invention provides a z-stack image fast registration method based on relatively non-uniform Gaussian blur, comprising: inputting a z-stack image g1 and a z-stack image g2, selecting an image block Ω(u1) with rich edges and relatively Gaussian blur in the two images through relative quadratic gradient energy RGE, * ,v * ); Using the depth of field smoothness prior, for the image block area Ω(u * ,v * ) Relative non-uniform Gaussian blur and relative translation are solved by alternating optimization iterations. The present invention avoids estimating a fully focused clear image by using relative Gaussian blur, which reduces the number of unknown parameters and thus reduces the pathological nature of the translation estimation problem, and can greatly reduce the amount of calculation on the other hand; high-precision alignment of z-stack images is achieved by image registration technology (row / column registration error does not exceed 2 pixels), thereby not causing difficulties for fully focused imaging.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a z-stack image fast registration method based on relatively non-uniform Gaussian blur. Background Art

[0002] Existing image registration algorithms are mainly divided into two categories [R. Szeliski, "Image alignment and stitching: A tutorial," Foundations and Trends in Computer Graphics and Vision, vol. 2, no. 1, pp. 1–104, 2006.]: feature-based registration methods and pixel-based registration methods. Considering the impact of image blur on registration accuracy, this paper mainly introduces methods suitable for fuzzy image registration.

[0003] The registration algorithm based on image features first extracts image feature points, and then compares and calculates the relative translation and affine transformation. The most commonly used feature extraction algorithms are SIFT [Lowe DG. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 2004, 60(2):91-110] and SURF [H. Bay, T. Tuytelaars, and L. Van Gool, "Surf: Speeded up robust features," in Computer Vision - ECCV 2006, SpringerBerlin Heidelberg]. However, as a general feature extraction method, feature extraction is limited by the degree of image blur and the richness of image content; in other words, the blurrier the image or the less image content, the more difficult it is for this type of method to extract feature points. In addition, for cell images, due to the similarity of cell features, the extracted feature points have a higher probability of comparison errors. In recent years, Liang et al. [Y. Liang, Y. Mao, Z. Tang, M. Yan, Y. Zhao, and J. Liu, “Efficient misalignment-robust multi-focus microscopical images fusion,” SignalProcessing, vol. 161, pp. 111 – 123, 2019] proposed a method of simultaneous SURF feature extraction, registration, and fusion, which improved the computational efficiency compared to the two-step registration and fusion.

[0004] The pixel-based blur image registration algorithm mainly constructs a measure function to iteratively optimize and solve affine transformation, point spread function and clear image, etc. Commonly used measure functions include SSD (sum of squared differences), SAD (sum of absolute differences), phase correlation, etc. Using the idea of ​​blur invariant moments, Pedone et al. proposed a phase invariant moment based on the n-fold blur kernel. Experiments show that this method can also be applied to non-uniform out-of-focus blur, although Fourier transform is required to hold in theory. For moderate blur, the N-fold method is close to SURF in registration accuracy, but SURF is more than 10 times faster; for severe blur, the N-fold method is significantly better than SURF, but it is difficult to ensure that the registration error does not exceed 2 pixels.

[0005] The limited depth of field of bright field microscopes makes it impossible to achieve fully focused imaging of thick samples, and multi-layer scanning and fusion are required to obtain a fully focused image. However, during the multi-layer scanning process, the platform moves at high speed, and the platform movement accuracy is difficult to ensure that there is no offset between two adjacent layers of images (for a resolution of 0.25 microns / pixel, a repeatable motion accuracy of 0.2 microns is required to ensure no offset); in addition, the vibration of the motion platform itself and slight external vibrations (unstable airflow, indirect bumps, etc.) may cause z-stack offset;

[0006] 1) Spatially varying out-of-focus blur makes image registration more difficult, especially at high magnifications above 40x;

[0007] 2) It is difficult for existing technologies to simultaneously meet the requirements of high registration accuracy (error not exceeding 2 pixels) and fast (real-time) registration;

[0008] Therefore, the prior art has deficiencies and a new method needs to be provided to solve the above problems. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides a fast z-stack image registration method based on relatively non-uniform Gaussian blur.

[0010] To achieve the above object, the specific scheme of the present invention is as follows:

[0011] The present invention provides a z-stack image rapid registration method based on relatively non-uniform Gaussian blur, comprising the following steps:

[0012] S1, input z-stack image g1 and z-stack image g2, select the image blocks with rich edges and relative Gaussian blur in the two images through relative quadratic gradient energy RGE ;

[0013] S2, solve the translation estimation objective function J by alternating optimization

[0014] S201, for a given area The images g1 and g2 on the image are iteratively solved by the gradient descent line search method to solve the relative non-uniform Gaussian blur and construct the defocused image ;

[0015] S202, using an exhaustive method to calculate the translation value, and through the exhaustive translation value, the image g2 and the defocused image Minimum error;

[0016] S203, alternating steps S202 and S203 until a convergence condition is met.

[0017] Furthermore, in step S1, the selected image block is expressed by the following formula:

[0018] ;

[0019] in, Indicates the center point of the image block, ps indicates the set image block radius, which is within 50 pixels;

[0020] Further, in step S1, selecting an image block by relative quadratic gradient energy RGE comprises the following steps;

[0021] S101, the two input z-stack images are calculated by relative quadratic gradient energy RGE to find image blocks;

[0022] S102, traverse all pixels and find At maximum Image block at that time ;

[0023] Among them, the relative quadratic gradient energy RGE is used to find the area where the g1 gradient is rich and clear and the g2 edge is fuzzy, so as to ensure the existence of relative blur;

[0024] S103, if , then swap images g1 and g2 so as to keep in the image block The upper g2 is more blurred than g1;

[0025] S104, multiple images are registered one by one.

[0026] Furthermore, in step S1, the relative quadratic gradient energy RGE is calculated as follows:

[0027] ;in, and Represents the gradient operator in the horizontal and vertical directions.

[0028] Further, in step S2, for a given image g1 and relative blur parameter (Unknown hidden variables to be estimated), construct the defocused image through relatively non-uniform Gaussian blur ;

[0029] Out of focus image It is expressed as follows:

[0030] ;

[0031] Where k is the maximum radius of the blur kernel, which depends on the sampling interval and scanning resolution;

[0032] Relative non-uniform Gaussian blur, the formula is as follows:

[0033]

[0034] Step S2 includes the following steps:

[0035] S201, assuming relative fuzzy parameters In the image block Smoothness, the degree of smoothness is related to the depth of field, and the translation estimation objective function is constructed:

[0036] ;

[0037] in, represents the regularization parameter, which is used to control the smoothness of the fuzzy parameters;

[0038] S202, solving the translation and relative fuzzy parameters by alternating iterative variable optimization, given an estimated translation value , iteratively solved by gradient descent line search method, the specific formula is as follows:

[0039] ;

[0040] S203, solving the translation value by exhaustive method, the formula is as follows:

[0041] ;

[0042] Among them, S1 and S2 represent the set translation search range in the row direction and column direction, both of which depend on the accuracy of the motion platform and the resolution of the imaging system;

[0043] The technical solution of the present invention has the following beneficial effects:

[0044] 1. Avoid estimating the fully focused clear image by using relative Gaussian blur. On the one hand, it reduces the number of unknown parameters and thus reduces the pathological nature of the translation estimation problem. On the other hand, it can significantly reduce the amount of calculation.

[0045] 2. High-precision alignment of z-stack images is achieved through image registration technology (row / column registration error is small, for example, it can be no more than 2 pixels), so as not to cause difficulties in full-focus imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is image g1;

[0047] Figure 2 is image g2;

[0048] Figure 3 is the image block selected by RGE ;

[0049] Figure 4 is an image ;

[0050] Figure 5 is the final image obtained after translation matching;

[0051] Figure 6 It is an overall step diagram of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0053] In the description of the present invention, unless otherwise clearly specified and limited, the terms "connected", "connected", and "fixed" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0054] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0055] In the description of this embodiment, the terms "upper", "lower", "front", "back", "left", "right" and other directions or positional relationships are based on the directions or positional relationships shown in the drawings, and are only for the convenience of description and simplification of operation, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operate in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used to distinguish in the description and have no special meaning.

[0056] like Figure 6 As shown, the present invention provides a z-stack image fast registration method based on relatively non-uniform Gaussian blur, comprising the following steps:

[0057] S1, input z-stack image g1 and z-stack image g2, select the image blocks with rich edges and relative Gaussian blur in the two images through relative quadratic gradient energy RGE ;

[0058] S201, for a given area The image blocks g1 and g2 on the image are iteratively solved by the gradient descent line search method to obtain the relative non-uniform Gaussian blur and construct the defocused image. ;

[0059] S202, using an exhaustive method to calculate the translation value, through the exhaustive translation value, the image block g2 and the out-of-focus image Minimum error;

[0060] S203, alternating steps S202 and S203 until a convergence condition is met.

[0061] In step S1, the selected image block is expressed by the following formula:

[0062] ;

[0063] in, Indicates the center point of the image block, and ps indicates the set image block radius, which is within 50 pixels.

[0064] In step S1, an image block is selected by relative quadratic gradient energy RGE, including the following steps:

[0065] S101, the two input z-stack images are calculated by relative quadratic gradient energy RGE to find image blocks;

[0066] S102, traverse all pixels and find At maximum Image block at that time ;

[0067] Among them, the relative quadratic gradient energy RGE is used to find the area where the g1 gradient is rich and clear and the g2 edge is fuzzy, so as to ensure the existence of relative blur;

[0068] S103, if , then swap images g1 and g2 so as to keep in the image block The upper g2 is more blurred than g1;

[0069] S104, multiple images are registered one by one.

[0070] In step S1, the relative quadratic gradient energy RGE is calculated as follows:

[0071] ;in, and Represents the gradient operator in the horizontal and vertical directions.

[0072] In step S2, for given images g1 and g2, the relative blur parameter needs to be found (Unknown hidden variables to be estimated), construct the defocused image through relatively non-uniform Gaussian blur ;

[0073] Out of focus image It is expressed as follows:

[0074] ;

[0075] Where k is the maximum radius of the blur kernel, which depends on the sampling interval and scanning resolution;

[0076] Relative non-uniform Gaussian blur, the formula is as follows:

[0077] .

[0078] Step S2 specifically includes:

[0079] S201, assuming relative fuzzy parameters In the image block Smoothness, the degree of smoothness is related to the depth of field, and the translation estimation objective function is constructed:

[0080] ;

[0081] in, represents the regularization parameter, which is used to control the smoothness of the fuzzy parameters;

[0082] S202, solving the translation and relative fuzzy parameters by alternating iterative variable optimization, given an estimated translation value , iteratively solved by gradient descent line search method, the specific formula is as follows:

[0083] ;

[0084] S203, solving the translation value by exhaustive method, the formula is as follows:

[0085] ;

[0086] Among them, S1 and S2 represent the set translation search range in the row direction and column direction, both of which depend on the accuracy of the motion platform and the resolution of the imaging system;

[0087] Alternately update the relative blur parameter and relative translation.

[0088] like Figure 1As shown, it is the image g1 of the present invention, objective lens 40x, 2464x2056 pixels;

[0089] like Figure 2 As shown, it is the image g2 of the present invention, objective lens 40x, 2464x2056 pixels;

[0090] like Figure 3 As shown, it is an image block selected from g1 by RGE , 99x99 pixels;

[0091] like Figure 4 As shown, it is an image block synthesized by the algorithm , 49x49 pixels;

[0092] like Figure 5 As shown in FIG. 1 , the image block obtained after the algorithm translates g2 by (5, -2) pixels is 49x49 pixels.

[0093] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the protection scope of the present invention.

Claims

1. A fast z-stack image registration method based on relatively non-uniform Gaussian blur, characterized in that: The steps include: S1, input z-stack image g1 and z-stack image g2, select the image blocks with rich edges and relative Gaussian blur in the two images through relative quadratic gradient energy RGE ; S2, solving the translation estimation objective function J by alternating optimization method; S201, for a given area The images g1 and g2 on the image are iteratively solved by the gradient descent line search method to solve the relative non-uniform Gaussian blur and construct the defocused image ; S202, using an exhaustive method to calculate the translation value, and through the exhaustive translation value, the image g2 and the defocused image Minimum error; S203, alternating steps S202 and S203 until a convergence condition is met; In step S1, the selected image block is expressed by the following formula: ； in, Indicates the center point of the image block, ps indicates the set image block radius, which is within 50 pixels; In step S1, an image block is selected by relative quadratic gradient energy RGE, including the following steps: S101, the two input z-stack images are calculated by relative quadratic gradient energy RGE to find image blocks; S102, traverse all pixels and find At maximum Image block at that time ; Among them, the relative quadratic gradient energy RGE is used to find the area where the g1 gradient is rich and clear and the g2 edge is fuzzy, so as to ensure the existence of relative blur; S103, if , then swap images g1 and g2 so as to keep in the image block The upper g2 is more blurred than g1; S104, multiple images are registered one by one; In step S1, the relative quadratic gradient energy RGE is calculated as follows: ;in, and Represents the gradient operator in the horizontal and vertical directions; In step S2, for a given image g1 and relative blur parameter , construct the defocused image through relatively non-uniform Gaussian blur ; Out of focus image It is expressed as follows: ； Where k is the maximum radius of the blur kernel, which depends on the sampling interval and scanning resolution; Relative non-uniform Gaussian blur, the formula is as follows: ； Step S2 includes the following steps: S201, assuming relative fuzzy parameters In the image block Smoothness, the degree of smoothness is related to the depth of field, and the translation estimation objective function is constructed: ； in, represents the regularization parameter, which is used to control the smoothness of the fuzzy parameters; S202, solving the translation and relative fuzzy parameters by alternating iterative variable optimization, given the tth (t=0,1,2,...,10) alternating iterative estimated translation value , iteratively solved by gradient descent line search method, the specific formula is as follows: ； S203, solving the translation value by exhaustive method, the formula is as follows: ； Among them, S1 and S2 represent the set translation search ranges in the row direction and column direction, both of which depend on the accuracy of the motion platform and the resolution of the imaging system.

Images