Microscopic image integer pixel alignment method based on sub-pixel edge offset detection

Abstract

The invention discloses a microscopic image integer pixel alignment method based on sub-pixel edge offset detection, and relates to the field of microscopic image processing. The problems that an existing sub-pixel registration technology depends on iterative optimization or a complex deformation model, and real-time processing is difficult; sensor sampling errors and mechanical translation errors need to be compensated through a high-resolution camera or a precise displacement table, and the system cost is high. According to the invention, through sub-pixel-level offset compensation, sub-pixel alignment can be realized without depending on a high-precision displacement table, and the hardware cost is remarkably reduced. The method comprises the following steps: extracting image edge feature points by using an edge detection algorithm, fitting a gray centroid to calculate a sub-pixel offset, and counting an average offset; generating an affine transformation matrix according to the average offset, resampling the image, and aligning pixel values to an integer coordinate grid; and finally, cutting the image edge according to the interpolation algorithm type to eliminate the ghost. The method is also suitable for the application fields of fluorescence microscope multi-view splicing, super-resolution reconstruction and the like.

Description

Technical Field

[0001] This invention relates to the field of microscopic image processing technology, and specifically to a method for aligning whole pixels of microscopic images based on sub-pixel edge offset detection. Background Technology

[0002] In the field of microscopic imaging, techniques such as multi-field image stitching and super-resolution reconstruction rely on high-precision image alignment. Traditional methods typically achieve subpixel-level displacement compensation through hardware (such as piezoelectric ceramic stages), but high-precision hardware is expensive and complex to maintain. While software-based methods can partially replace hardware, existing subpixel registration techniques (such as elastic registration and optical flow methods) rely on iterative optimization or complex deformation models, making real-time processing difficult; sensor sampling errors and mechanical translation errors need to be compensated for through high-resolution cameras or precision displacement stages, resulting in high system costs. Summary of the Invention

[0003] This invention addresses the problems of existing subpixel registration techniques that rely on iterative optimization or complex deformation models, making real-time processing difficult; and the need for compensation of sensor sampling errors and mechanical translation errors through high-resolution cameras or precision displacement stages, resulting in high system costs.

[0004] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: Solution 1: This invention proposes a method for integer pixel alignment of microscopic images based on sub-pixel edge offset detection. The method includes the following steps: Step 1: Obtain the original microscopic image and preprocess the original microscopic image; Step 2: Extract edge feature points from the original microscopic image after preprocessing in Step 1, calculate the sub-pixel offset of the feature points by fitting the gray-level centroid, and calculate the statistical average offset. ; Step 3: Based on the average offset calculated in Step 2 Generate an affine transformation matrix and resample the original microscopic image, aligning pixel values ​​to an integer coordinate grid. Step 4: Crop the image edges after resampling in Step 3 to eliminate ghosting introduced by interpolation. This is used to eliminate subpixel edge offset of the original microscopic image and complete the whole-pixel alignment of the microscopic image based on subpixel edge offset detection.

[0005] Furthermore, a preferred embodiment is provided, in which a feature extraction algorithm is used to extract edge feature points from the preprocessed original microscopic image in step 2, and the extracted feature points are denoted as... Each feature point Including integer coordinates The average offset is calculated based on the distribution of the feature points. ; The feature extraction algorithm is any one of the following: Canny operator or other edge detection algorithm, scale-invariant feature transformation algorithm, accelerated robust feature algorithm, or directional gradient histogram algorithm.

[0006] Furthermore, a preferred embodiment is provided in which, in step 2, the sub-pixel offset of the feature point is calculated by fitting the gray-level centroid, and the statistical average offset is calculated. The method is as follows: For each feature point, extract 5 segments centered on the feature point. For a 5-pixel neighborhood, a coordinate network is created and the intensity value of the neighborhood corresponding to the feature point is read. The gray centroid of the edge point is calculated by binary quadratic Gaussian fitting, and the sub-pixel offset of the feature point is calculated. The sub-pixel offset of each edge feature point is: ,in The initial pixel coordinates of the feature points; To fit the grayscale centroid coordinates; average offset The arithmetic mean of the offsets of all feature points; The method for calculating the gray-level centroid of edge points using binary quadratic Gaussian fitting is as follows:

[0007] Where A is the maximum gray value of the neighboring pixels of the feature point; These are the sub-pixel level coordinates of the feature points; B is the background grayscale baseline. Let be the standard deviation of the Gaussian distribution, representing the degree of edge blurring; the solution is obtained using the least squares method.

[0008] Furthermore, in a preferred embodiment, the affine transformation matrix in step 3 contains only translation operations, and the translation amount is the average offset. .

[0009] Furthermore, a preferred embodiment is provided, wherein the specific method for resampling the original microscopic image in step 3 is as follows: The target pixel is mapped to a non-integer position in the original image using an affine matrix M, and the target pixel value is generated using an interpolation algorithm based on the non-integer coordinates of the mapped position. Specifically: Based on average offset Generate the affine transformation matrix, in the form of:

[0010] The original microscopic image is resampled, and the pixel values ​​are aligned to an integer coordinate grid. Matrix transformation is performed when resampling the original microscopic image. ; Where the target image coordinates are (u,v), the original image coordinates are (x,y), and M is the affine transformation matrix. It is its inverse matrix.

[0011] Furthermore, a preferred embodiment is provided, wherein the interpolation algorithm is either bilinear interpolation or bicubic interpolation.

[0012] Furthermore, a preferred embodiment is provided, wherein when the interpolation algorithm is bilinear interpolation, it is based on the four adjacent pixels of the target pixel in the original image. The interpolated grayscale value is calculated using linear weighting, and the specific formula is as follows:

[0013] in, The fractional part of the subpixel offset; When the interpolation algorithm is bicubic interpolation, it is based on the 16 adjacent pixels of the target pixel in the original image. to The interpolated grayscale value is calculated using a cubic polynomial kernel function with weighted average. The kernel function is:

[0014] Where t is the normalized offset from the target point, and the value range of parameter a is... .

[0015] Furthermore, a preferred embodiment is provided, wherein the cropping of the image edge after resampling in step 3 in step 4 is specifically as follows: the cropping width is 1.5 pixels when using bilinear interpolation, and the cropping width is 3 pixels when using bicubic interpolation.

[0016] Option 2: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in Option 1.

[0017] Option 3: A computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in Option 1.

[0018] The advantages of this invention are: The present invention discloses a microscopic image integer pixel alignment method based on subpixel edge offset detection. This method achieves subpixel alignment without relying on a high-precision displacement stage through subpixel-level offset compensation, significantly reducing hardware costs. The method utilizes an edge detection algorithm to extract image edge feature points, fits the gray-level centroid to calculate the subpixel offset, and calculates the average offset. An affine transformation matrix is ​​generated based on the average offset, and the image is resampled to align pixel values ​​to an integer coordinate grid. Finally, the image edges are cropped according to the interpolation algorithm type to eliminate ghosting.

[0019] This invention achieves subpixel-level alignment accuracy through low-cost subpixel correction, without relying on a high-precision displacement stage. It compensates for sensor sampling errors through algorithms, reducing reliance on high-resolution cameras and significantly lowering hardware costs. Moreover, the method is simple and easy to implement.

[0020] This invention is also applicable to applications such as multi-field stitching and super-resolution reconstruction of fluorescence microscopes. Attached Figure Description

[0021] Figure 1 This is a flowchart of a microscopic image whole-pixel alignment method based on sub-pixel edge offset detection, as described in Embodiment 1. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0023] Implementation Method 1: This implementation method proposes a microscopic image integer pixel alignment method based on sub-pixel edge offset detection. The method includes the following steps: Step 1: Obtain the original microscopic image and preprocess the original microscopic image; Step 2: Extract edge feature points from the original microscopic image after preprocessing in Step 1, calculate the sub-pixel offset of the feature points by fitting the gray-level centroid, and calculate the statistical average offset. ; Step 3: Based on the average offset calculated in Step 2 Generate an affine transformation matrix and resample the original microscopic image, aligning pixel values ​​to an integer coordinate grid. Step 4: Crop the image edges after resampling in Step 3 to eliminate ghosting introduced by interpolation. This is used to eliminate subpixel edge offset of the original microscopic image and complete the whole-pixel alignment of the microscopic image based on subpixel edge offset detection.

[0024] The preprocessing described in this embodiment includes grayscale transformation, noise reduction, and edge enhancement of the original microscopic image.

[0025] Implementation Method Two: This implementation method further defines the microscopic image integer pixel alignment method based on sub-pixel edge offset detection described in Implementation Method One. It uses a feature extraction algorithm to extract edge feature points from the preprocessed original microscopic image in step 2. The extracted feature points are denoted as... Each feature point Including integer coordinates The average offset is calculated based on the distribution of the feature points. ; The feature extraction algorithm is any one of the following: Canny operator or other edge detection algorithm, scale-invariant feature transformation algorithm, accelerated robust feature algorithm, or directional gradient histogram algorithm.

[0026] In this embodiment, the feature point extraction algorithm is any one of the following: Canny operator or other edge detection algorithm; Scale Invariant Feature Transform (SIFT) algorithm; Accelerated Robust Feature Transform (SURF) algorithm; Oriented Gradient Histogram (ORB) algorithm; or deep learning feature extraction model based on convolutional neural network.

[0027] Implementation Method 3: This implementation method further defines the microscopic image integer pixel alignment method based on sub-pixel edge offset detection described in Implementation Method 1. In step 2, the sub-pixel offset of feature points is calculated by fitting the gray-level centroid, and the statistical average offset is calculated. The method is as follows: For each feature point, extract 5 segments centered on the feature point. For a 5-pixel neighborhood, a coordinate network is created and the intensity value of the neighborhood corresponding to the feature point is read. The gray centroid of the edge point is calculated by binary quadratic Gaussian fitting, and the sub-pixel offset of the feature point is calculated. The sub-pixel offset of each edge feature point is: ,in The initial pixel coordinates of the feature points; To fit the grayscale centroid coordinates; average offset The arithmetic mean of the offsets of all feature points; The method for calculating the gray-level centroid of edge points using binary quadratic Gaussian fitting is as follows:

[0028] Where A is the maximum gray value of the neighboring pixels of the feature point; These are the sub-pixel level coordinates of the feature points; B is the background grayscale baseline. Let be the standard deviation of the Gaussian distribution, representing the degree of edge blurring; the solution is obtained using the least squares method.

[0029] In this embodiment, the method for fitting the gray-scale centroid is a binary quadratic Gaussian fitting.

[0030] Implementation Method 4: This implementation method further defines the microscopic image integer pixel alignment method based on sub-pixel edge offset detection described in Implementation Method 1. The affine transformation matrix in step 3 only contains translation operations, and its translation amount is the average offset. .

[0031] Implementation Method 5: This implementation method further defines the microscopic image integer pixel alignment method based on sub-pixel edge offset detection described in Implementation Method 3. The specific method for resampling the original microscopic image in step 3 is as follows: The target pixel is mapped to a non-integer position in the original image using an affine matrix M, and the target pixel value is generated using an interpolation algorithm based on the non-integer coordinates of the mapped position. Specifically: Based on average offset Generate the affine transformation matrix, in the form of:

[0032] The original microscopic image is resampled, and the pixel values ​​are aligned to an integer coordinate grid. Matrix transformation is performed when resampling the original microscopic image. ; Where the target image coordinates are (u,v), the original image coordinates are (x,y), and M is the affine transformation matrix. It is its inverse matrix.

[0033] In this embodiment, after performing matrix transformation, an interpolation algorithm is used to interpolate the image.

[0034] Implementation Method Six: This implementation method further defines the microscopic image whole pixel alignment method based on sub-pixel edge offset detection described in Implementation Method Five. The interpolation algorithm is either bilinear interpolation or bicubic interpolation.

[0035] Implementation Method Seven: This implementation method further defines the microscopic image integer pixel alignment method based on sub-pixel edge offset detection described in Implementation Method Six. When the interpolation algorithm is bilinear interpolation, it is based on the four adjacent pixels of the target pixel in the original image. The interpolated grayscale value is calculated using linear weighting, and the specific formula is as follows:

[0036] in, The fractional part of the subpixel offset; When the interpolation algorithm is bicubic interpolation, it is based on the 16 adjacent pixels of the target pixel in the original image. to The interpolated grayscale value is calculated using a cubic polynomial kernel function with weighted average. The kernel function is:

[0037] Where t is the normalized offset from the target point, and the value range of parameter a is... .

[0038] Implementation Method 8: This implementation method further defines the microscopic image whole-pixel alignment method based on sub-pixel edge offset detection described in Implementation Method 7. Specifically, in step 4, the cropping width of the image edge after resampling in step 3 is 1.5 pixels when using bilinear interpolation and 3 pixels when using bicubic interpolation.

[0039] Implementation Method Nine: This implementation method provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any one of Implementation Methods One to Seven. Implementation Method 10: This implementation method provides a computer device, including a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the method described in any one of Implementation Methods 1 to 7.

[0040] Implementation Method Eleven: This implementation method provides an example, which is used to explain the above-described implementation methods one through ten. The specific example is as follows: See Figure 1 This implementation method is described with reference to Figure 1 As shown, in step 1, a raw microscopic image is acquired using microscopic imaging technology, and the raw microscopic image is preprocessed. Specifically, an optical microscope imaging system equipped with a stepper motor stage (translation accuracy ±1μm) and a CMOS sensor can be used to capture a sequence of live cell microscopic images to obtain raw microscopic images; then, the raw microscopic images are converted to grayscale; the images are denoised using non-local means; and the images are equalized using an adaptive histogram to enhance image edges.

[0041] In step 2, edge feature points are extracted from the preprocessed original microscopic image, and sub-pixel offsets are calculated by fitting the gray-level centroids, and the average offset is statistically analyzed. ; In this study, the Canny operator was selected to perform edge detection and extract edge feature points. The set of extracted feature points is denoted as Each feature point Including integer coordinates The coordinate precision is integer pixels; each feature point is cropped to... Within a 5x5 pixel neighborhood centered on the target, the grayscale values ​​of the neighborhood are read, and a binary quadratic Gaussian fit is performed to calculate the grayscale centroid. Calculate the pixel offset of each feature point. Then, the average offset of all feature points is calculated. .

[0042] In step 3, based on the average offset Generate the affine transformation matrix, in the form of: The image is then resampled to align pixel values ​​to an integer coordinate grid. An affine matrix M is used to map the target pixels to non-integer locations in the original image. Matrix transformations are performed during image resampling. The target image coordinates are (u, v), the original image coordinates are (x, y), and M is the affine transformation matrix. It is the inverse matrix, and then the target pixel value is generated using the bilinear interpolation algorithm.

[0043] In step 4, the edges of the resampled image are cropped to eliminate ghosting introduced by interpolation, thereby eliminating subpixel edge offset of the initial image.

[0044] The width of the cropped pixel value is 1.5 pixels.

[0045] The edge feature point extraction method is SURF (SpeededUpRobustFeatures) feature extraction, and the other steps are consistent with the specific implementation method one.

[0046] The resampling process uses a bicubic interpolation algorithm to generate the target pixel value. Correspondingly, the cropping width after interpolation is 3 pixels to match the spatial influence range of the bicubic interpolation algorithm. Other steps and parameters are the same as in specific implementation method one or two.

[0047] The edge feature point extraction employs a pre-trained lightweight convolutional neural network (MobileNetV3) to reduce manual intervention. This implementation is applicable to microscopic image alignment across devices and samples, enhancing the algorithm's generalization ability.

[0048] Specifically, the pre-trained MobileNetV3 model is loaded using the `mobilenet_v3_small` function; a lightweight feature decoding layer is constructed using the `Conv2d` function, and the channels are compressed to 1 dimension; the initial image is standardized using the `preprocess_image` function to achieve size adaptation; the `preprocess_image` function is called to preprocess the input image `image` and `unsqueeze(0)` is used to add a dimension to the first dimension of the tensor to adapt to the input requirements of the deep learning model.

[0049] When extracting the edge feature points, the feature points of the standardized image are extracted using the extract_keypoints function; non-maximum suppression is applied using maximum_filter, and the neighborhood window is set to neighborhood_size=5 and k=1024. Filter local extrema points and find the flattened index of the top k maximum values ​​in the response graph, where k is the number of feature points extracted; use np.unravel_index to convert the one-dimensional index into a two-dimensional index; use np.column_stack to combine the two-dimensional indexes into an array, where each element represents the coordinates of a feature point. The edge feature point extraction employs the ORB (Oriented Fast and Rotated BRIEF) algorithm. In step 2, rotation-invariant feature points are extracted using the ORB algorithm to adapt to scenarios with rapid stage movement. The number of feature points is dynamically adjusted based on image complexity. After extraction, a binary quadratic Gaussian fit is performed based on a 5×5 neighborhood to calculate the average offset. In step 3, bilinear interpolation is used for rapid resampling, and in step 4, the cropping width is 1.5 pixels. This implementation is suitable for dynamic observation of live cells, balancing processing speed and alignment accuracy.

[0050] This embodiment adds low-light enhancement preprocessing to the first specific embodiment. In step 1, gamma correction is performed on the original fluorescence image (…). =1.8) and dark channel dehazing improve the visibility of weak signals; Top-Hat filtering is used to enhance fluorescence spots. Step 2, feature point extraction, uses the SURF algorithm (Hessian threshold = 500) to adapt to high-noise scenes. Step 3 uses bicubic interpolation to preserve details, and step 4 crops the width by 3 pixels. This implementation is optimized for low signal-to-noise ratio (SNR≥6dB) fluorescence images to ensure the stability of subpixel offset detection.

[0051] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or technical solutions of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0052] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended technical solutions are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the present invention. Clearly, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for integer pixel alignment of microscopic images based on sub-pixel edge offset detection, characterized in that, The method includes the following steps: Step 1: Obtain the original microscopic image and preprocess the original microscopic image; Step 2: Extract edge feature points from the original microscopic image after preprocessing in Step 1, calculate the sub-pixel offset of the feature points by fitting the gray-level centroid, and calculate the statistical average offset. ; Step 3: Based on the average offset calculated in Step 2 Generate an affine transformation matrix and resample the original microscopic image, aligning pixel values ​​to an integer coordinate grid. Step 4: Crop the image edges after resampling in Step 3 to eliminate ghosting introduced by interpolation. This is used to eliminate subpixel edge offset of the original microscopic image and complete the whole-pixel alignment of the microscopic image based on subpixel edge offset detection.

2. The microscopic image integer pixel alignment method based on sub-pixel edge offset detection according to claim 1, characterized in that, Edge feature points are extracted from the preprocessed original microscopic image in step 2 using a feature extraction algorithm. The extracted feature points are denoted as... Each feature point Including integer coordinates The average offset is calculated based on the distribution of the feature points. ; The feature extraction algorithm is any one of the following: Canny operator or other edge detection algorithm, scale-invariant feature transformation algorithm, accelerated robust feature algorithm, or directional gradient histogram algorithm.

3. The microscopic image whole-pixel alignment method based on sub-pixel edge offset detection according to claim 1, characterized in that, In step 2, the sub-pixel offset of feature points is calculated by fitting the gray-level centroid, and the statistical average offset is calculated. The method is as follows: For each feature point, extract 5 segments centered on the feature point. For a 5-pixel neighborhood, a coordinate network is created and the intensity value of the neighborhood corresponding to the feature point is read. The gray centroid of the edge point is calculated by binary quadratic Gaussian fitting, and the sub-pixel offset of the feature point is calculated. The sub-pixel offset of each edge feature point is: ,in The initial pixel coordinates of the feature points; To fit the grayscale centroid coordinates; average offset The arithmetic mean of the offsets of all feature points; The method for calculating the gray-level centroid of edge points using binary quadratic Gaussian fitting is as follows: Where A is the maximum gray value of the neighboring pixels of the feature point; These are the sub-pixel level coordinates of the feature points; B is the background grayscale baseline. Let be the standard deviation of the Gaussian distribution, representing the degree of edge blurring; the solution is obtained using the least squares method.

4. The microscopic image whole-pixel alignment method based on sub-pixel edge offset detection according to claim 1, characterized in that, The affine transformation matrix in step 3 contains only translation operations, and its translation amount is the average offset. .

5. The microscopic image integer pixel alignment method based on sub-pixel edge offset detection according to claim 1, characterized in that, The specific method for resampling the original microscopic image in step 3 is as follows: The target pixel is mapped to a non-integer position in the original image using an affine matrix M, and the target pixel value is generated using an interpolation algorithm based on the non-integer coordinates of the mapped position. Specifically: Based on average offset Generate the affine transformation matrix, in the form of: The original microscopic image is resampled, and the pixel values ​​are aligned to an integer coordinate grid. Matrix transformation is performed when resampling the original microscopic image. ; Where the target image coordinates are (u,v), the original image coordinates are (x,y), and M is the affine transformation matrix. It is its inverse matrix.

6. The microscopic image whole-pixel alignment method based on sub-pixel edge offset detection according to claim 5, characterized in that, The interpolation algorithm is either bilinear interpolation or bicubic interpolation.

7. The microscopic image integer pixel alignment method based on sub-pixel edge offset detection according to claim 6, characterized in that, When the interpolation algorithm is bilinear interpolation, it is based on the four adjacent pixels of the target pixel in the original image. The interpolated grayscale value is calculated using linear weighting, and the specific formula is as follows: in, The fractional part of the subpixel offset; When the interpolation algorithm is bicubic interpolation, it is based on the 16 adjacent pixels of the target pixel in the original image. to The interpolated grayscale value is calculated using a cubic polynomial kernel function with weighted average. The kernel function is: Where t is the normalized offset from the target point, and the value range of parameter a is... .

8. The microscopic image integer pixel alignment method based on sub-pixel edge offset detection according to claim 7, characterized in that, In step 4, the cropping of the image edges after resampling in step 3 is specifically as follows: the cropping width is 1.5 pixels when using bilinear interpolation, and 3 pixels when using bicubic interpolation.

9. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

10. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-7.

Images