Image rigid registration method and device

Abstract

The invention discloses an image rigid registration method and device, relates to the technical field of image processing, and provides an image rigid registration method and device, so as to solve the problem of calculation bottleneck of an existing image processing method during preprocessing, feature extraction and CPU-based registration of a high-resolution image. Graying, threshold processing, gradient operation, morphological operation, Gaussian blur and down-sampling are carried out on a template image and a target image, a mask image is generated based on a template image workpiece contour to limit a registration area, difference calculation and normalization processing are carried out on a gradient image in combination with a projective transformation matrix, and an optimal registration result is obtained through parameter optimization. Therefore, image rigid registration with both speed and precision is realized. The method is suitable for the fields with efficient requirements on image scanning and comparison, such as medical image analysis, remote sensing image processing, industrial detection and the like.

Description

Technical Field

[0001] It relates to the field of image processing technology, and specifically relates to an image rigid registration method and system based on CUDA acceleration. Background Art

[0002] In the field of image processing, image registration technology, as a fundamental and crucial technology, is widely applied in multiple fields such as medical image analysis, remote sensing image processing, and computer vision. The purpose of image registration is to align two or more different images spatially for subsequent analysis or processing. In these applications, image registration technology can effectively improve the accuracy and reliability of data analysis, so it has important significance in practical applications.

[0003] Existing image registration methods can be divided into feature-based methods, intensity-based methods, and transformation model-based methods. Feature-based methods achieve registration by extracting feature points (such as corner points, edges, etc.) in the image and matching the feature points. Intensity-based methods directly utilize the gray information of the image and perform registration by optimizing similarity measurement functions (such as mutual information, correlation, etc.). And transformation model-based methods achieve image registration by defining geometric transformation models (such as rigid, affine, projective transformations, etc.) and optimizing the model parameters.

[0004] However, existing CPU-based image registration methods face significant computational bottlenecks when processing high-resolution images. Although some methods accelerate the calculation by using parallel computing frameworks (such as CUDA), there are still certain challenges in terms of efficiency and accuracy. For example, the accuracy of traditional image registration methods is often affected when facing complex backgrounds or noises; at the same time, CPU computing cannot meet the requirements in large-scale image processing or real-time applications, and it is difficult to achieve efficient real-time processing.

[0005] In addition, existing methods also have certain limitations in image preprocessing and feature extraction. Especially in special scenarios such as industrial inspection or high specular reflection, the noise and background complexity of the image have a great impact on the registration result, and stronger robustness is required to ensure the registration accuracy. Summary of the Invention

[0006] To solve the technical problems existing in the prior art, that is, existing image processing methods have certain limitations in image preprocessing and feature extraction, and CPU-based image registration methods face significant computational bottlenecks when processing high-resolution images, the technical solution provided by the present invention is as follows:

[0007] An image rigid registration method, comprising:

[0008] Preprocess the template image and the target image to be registered, and perform steps of converting to grayscale images, threshold processing, Sobel operator gradient calculation, gradient map threshold processing, morphological dilation, Gaussian blurring of the image, and downsampling;

[0009] Generate a mask image based on the workpiece contour of the template image, and match the mask image with the gradient map;

[0010] Construct a projective transformation matrix according to the image projective transformation parameters, and transform the gradient maps of the template image and the target image;

[0011] Normalize the error;

[0012] Optimize the registration parameters until a predetermined number of iterations is reached or a set stop condition is satisfied;

[0013] Output the optimal projective transformation matrix, and register the template image or the target image with the projective transformation matrix;

[0014] Furthermore, a preferred embodiment is provided, wherein the loss function calculation step includes taking the absolute value of the horizontal and vertical gradient differences respectively and summing them to obtain the total error;

[0015] Furthermore, a preferred embodiment is provided, using the projective transformation matrix to transform the gradient maps of the template image and the target image;

[0016] Furthermore, a preferred embodiment is provided, calculating the difference between the transformed template image gradient map and the target image gradient map, restricting the calculation area to the pixels within the mask range, and normalizing the error;

[0017] Furthermore, a preferred embodiment is provided, using CUDA to accelerate the calculation of the loss function, and optimizing the registration parameters through the Jacobian matrix and the quasi-Newton method;

[0018] Furthermore, a preferred embodiment is provided, wherein the preprocessing includes threshold processing of the image, calculating the horizontal and vertical gradients of the image using the Sobel operator, morphological dilation operations, and Gaussian blurring to improve the processing speed and registration accuracy;

[0019] Based on the same inventive concept, the present invention also provides an image rigid registration device, including:

[0020] A module for preprocessing the template image and the target image to be registered, converting them to grayscale images and performing threshold processing, Sobel operator gradient calculation, gradient map threshold processing, morphological dilation, Gaussian blurring of the image, and downsampling;

[0021] A module for generating a mask image based on the workpiece contour of the template image and matching the mask image with the gradient map;

[0022] A module that constructs a projective transformation matrix based on image projective transformation parameters and transforms the gradient maps of the template image and the target image;

[0023] A module that normalizes the error;

[0024] A module that optimizes the registration parameters until a predetermined number of iterations is reached or a set stopping condition is satisfied;

[0025] A module that outputs the optimal projective transformation matrix and registers the template image or the target image with the projective transformation matrix.

[0026] Based on the same inventive concept, the present invention also provides a computer storage medium for storing a computing program, and when the computer program is read by a computer, the computer executes the method described above.

[0027] Based on the same inventive concept, the present invention also provides a computer, including a processor and a storage medium, and when the processor reads the computer program stored in the storage medium, the computer executes the method described above.

[0028] Based on the same inventive concept, the present invention also provides a computer program product, which, as a computer program, when executed, implements the method described above.

[0029] Compared with the prior art, the beneficial effects of the technical solution provided by the present invention are as follows:

[0030] The solution of the present invention effectively solves the problems in the prior art in multiple aspects and brings remarkable effects by introducing an image rigid registration method based on CUDA acceleration. First, CUDA parallel computing significantly improves the processing speed of image registration. Compared with traditional CPU-based image registration methods, GPU parallel computing can process a large amount of data simultaneously, thus significantly reducing the computing time during high-resolution image registration. This makes the method particularly suitable for scenarios that require real-time processing, such as industrial inspection and medical image analysis, which have high requirements for processing speed.

[0031] Secondly, the optimized loss function design improves the accuracy of image registration. By processing the image gradient map and using a mask to limit the calculation area during the calculation process, the present invention effectively reduces the interference of noise and makes the registration result more accurate. Compared with traditional corner or shape matching-based methods, the present invention uses a method of calculating the difference in gradient maps to improve the matching accuracy of images, especially in complex backgrounds, avoiding the accuracy loss caused by noise or unclear features in traditional methods.

[0032] In addition, the preprocessing of images and the ability to resist noise interference have also been significantly improved. This solution uses techniques such as multiple downsampling and Gaussian blur to effectively smooth the images, reducing the impact of noise on the registration results. Existing image registration methods often rely on complex noise removal algorithms when dealing with complex or highly reflective backgrounds. However, this invention greatly enhances the robustness through strengthening edge information and gradient map processing, especially suitable for highly reflective scenarios in industrial inspection.

[0033] Finally, the optimized local region calculation method further enhances the adaptability of this solution in complex backgrounds. By precisely controlling the registration region, it avoids the calculation errors caused by complex backgrounds or irregular shapes in traditional methods. Compared with other image registration methods based on global matching, this invention ensures high-precision calculation in key regions through the limitation of the mask region, avoiding the influence of irrelevant regions and improving the overall registration effect.

[0034] Through these improvements in technical means, the solution of this invention surpasses existing image registration technologies in terms of processing speed, accuracy, robustness, and adaptability, especially performing excellently in the real-time processing of high-resolution images and image registration under complex backgrounds.

[0035] It is applicable to tasks that require image scanning and comparison, such as medical image analysis, remote sensing image processing, and industrial inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the workpiece to be measured;

[0037] Figure 2 It is the registration effect diagram of the workpiece to be measured. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] To make the advantages and benefits of the technical solution provided by this invention more clearly manifested, the technical solution provided by this invention will be further described in detail with reference to the accompanying drawings. Specifically:

[0039] Embodiment 1. This embodiment provides an image rigid registration method, including:

[0040] Preprocess the template image and the target image to be registered, and perform steps such as converting to grayscale images, threshold processing, Sobel operator gradient calculation, gradient map threshold processing, morphological dilation, image Gaussian blur, and downsampling;

[0041] Generate a mask image based on the workpiece contour of the template image, and match the mask image with the gradient map;

[0042] Construct a projective transformation matrix according to the image projective transformation parameters, and transform the gradient maps of the template image and the target image;

[0043] Steps for normalizing errors;

[0044] Steps for optimizing the registration parameters until a predetermined number of iterations is reached or a set stopping condition is met;

[0045] Steps for outputting the optimal projective transformation matrix and registering the template image or the target image with the projective transformation matrix.

[0046] Specifically, this embodiment provides an image rigid registration method and system based on CUDA acceleration, mainly targeting the real-time processing scenario of high-resolution images. It utilizes CUDA parallel computing to accelerate the registration process and improves the accuracy and robustness of image registration through optimizing the loss function and noise processing techniques. It includes multiple steps, from image preprocessing, mask generation to image registration, and the whole process is highly optimized to ensure the application effect in fields such as industrial inspection and medical image analysis.

[0047] Step 1: Image preprocessing

[0048] First, preprocess the images to be registered (including the template image and the target image). The specific steps are as follows:

[0049] Convert to grayscale image: Convert the color image to a grayscale image to simplify subsequent processing.

[0050] Threshold processing: Perform threshold processing on the image, set the pixel values greater than the threshold to the threshold value to remove noise and enhance image edges. In particular, in the high specular reflection problem in stamping part detection, the part with pixel values greater than 72 is set to 72 to avoid interference from the highlight area.

[0051] Calculate gradients using the Sobel operator: Use the Sobel operator to calculate the horizontal and vertical gradient maps of the image to obtain the edge information of the image.

[0052] Threshold processing of the gradient map: Perform threshold processing on the gradient map to remove pixel values less than the set threshold, further reducing the influence of noise.

[0053] Morphological dilation: Perform morphological dilation on the gradient map to strengthen the edge information in the image for better feature localization in the subsequent registration process.

[0054] Gaussian blur: Apply Gaussian blur to the dilated gradient map to further smooth the image and reduce the influence of noise on subsequent calculations.

[0055] Downsampling processing: Perform downsampling on the processed image to reduce the amount of calculation and improve the processing speed of subsequent calculations.

[0056] This step provides an optimized image input for subsequent mask generation and loss function calculation.

[0057] Step 2: Mask Generation

[0058] The generation of the mask is a crucial step in the image registration process, and the specific process is as follows:

[0059] Draw the workpiece contour: Use the labelme tool to draw the workpiece contour of the template image to generate a JSON file. This file contains information about the workpiece area and serves as the basis for subsequent mask generation.

[0060] Generate the mask image: Generate the mask image based on the JSON file to define the registration area. The mask image indicates which areas will participate in the subsequent registration calculation and excludes the interference of background noise.

[0061] Downsample the mask image: Downsample the mask image to make it the same scale as the gradient map. In this way, the mask image can accurately match the preprocessed image and ensure the regional consistency of subsequent calculations.

[0062] The mask image provides a local area for the calculation of the loss function, thus ensuring that only the workpiece area is calculated during the registration process and avoiding the interference of irrelevant areas.

[0063] Step 3: Loss Function Calculation

[0064] The loss function is the core in image registration, which is used to measure the registration effect and optimize the registration parameters. The steps are as follows:

[0065] Construct the projective transformation matrix: According to the projective transformation parameters (h11, h12, h13, h21, h22, h23, h31, h32) of the image, construct a 3x3 projective transformation matrix. This matrix represents the spatial transformation from the template image to the target image.

[0066] Gradient map transformation: Use the projective transformation matrix to transform the horizontal and vertical gradient maps of the template image and the target image. In this way, the gradients of the template image are aligned with those of the target image.

[0067] Calculate the gradient difference: Calculate the difference between the transformed template gradient map and the target gradient map, and limit the calculation area to the pixels within the mask image. This operation ensures the accuracy of the registration result by restricting the calculation area and avoids background interference.

[0068] Error calculation: Sum the absolute values of the horizontal and vertical gradient differences respectively to obtain the total error, and normalize the error by the image area. This step provides the calculation basis for subsequent optimization.

[0069] The calculation of the loss function provides a quantitative basis for parameter optimization and is an essential step in the registration process.

[0070] Step 4: Image Registration Optimization

[0071] Optimize the registration parameters according to the calculation result of the loss function. The steps are as follows:

[0072] Initialize parameters: Initially set the projective transformation parameters (h11, h12, h13, h21, h22, h23, h31, h32) to (1, 0, 0, 0, 1, 0, 0, 0).

[0073] Loss value calculation and update: Calculate the loss value under the current parameters. If this loss value is less than the set maximum loss value (such as 1e8), then update the maximum loss value and the optimal parameters.

[0074] Temporary increment adjustment of parameters: Make a temporary increment adjustment to the current parameters, and calculate the change in the loss value caused by the temporary increment of each parameter. Calculate the Jacobian matrix by taking the difference between this change and the current loss value.

[0075] Solve the Jacobian matrix: Solve the first-order derivative of the parameter change through the Jacobian matrix to guide the optimization of the parameters.

[0076] Quasi-Newton method optimization: Use the Quasi-Newton method to calculate the Hessian matrix and adjust the step size to update the translation parameters to gradually optimize the registration result.

[0077] Iterative update: Repeat the above steps until the maximum number of iterations is reached or the norm of the parameter update is less than the set threshold. Finally, output the optimal projective transformation parameters.

[0078] The optimization process ensures continuous improvement of the image matching accuracy in multiple registration steps, and finally obtains the best registration result.

[0079] Step 5: Result Output

[0080] Perform a projective transformation on the image according to the optimal parameters finally optimized. The steps are as follows:

[0081] Construct the transformation matrix: Construct the projective transformation matrix according to the optimal parameters. When registering the template image to the target image, use this matrix for projective transformation; if it is necessary to register the target image to the template image, use the inverse of the projective matrix for transformation.

[0082] Result output: Output the registered image and provide the final matching result.

[0083] This step ensures the accuracy of the registration result and is applicable to various scenarios requiring image alignment, such as industrial inspection, medical image analysis, etc.

[0084] Embodiment 2. This embodiment further limits the image rigid registration method provided in Embodiment 1. The loss function calculation step includes taking the absolute value of the horizontal and vertical gradient differences respectively and summing them to obtain the total error.

[0085] Embodiment 3. This embodiment further limits the image rigid registration method provided in Embodiment 1. A projective transformation matrix is used to transform the gradient maps of the template image and the target image.

[0086] Embodiment 4. This embodiment further limits the image rigid registration method provided in Embodiment 1. Calculate the difference between the transformed template image gradient map and the target image gradient map, limit the calculation area to the pixels within the mask range, and normalize the error.

[0087] Embodiment 5. This embodiment further limits the image rigid registration method provided in Embodiment 1. CUDA is used to accelerate the calculation of the loss function, and the registration parameters are optimized through the Jacobian matrix and the quasi-Newton method.

[0088] Embodiment 6. This embodiment further limits the image rigid registration method provided in Embodiment 1. The preprocessing includes thresholding the image, calculating the horizontal and vertical gradients of the image using the Sobel operator, morphological dilation operation, and Gaussian blur to improve the processing speed and registration accuracy.

[0089] Embodiment 7. This embodiment provides an image rigid registration device, including:

[0090] A module for preprocessing the template image and the target image to be registered, converting them into grayscale images and performing thresholding, Sobel operator gradient calculation, gradient map thresholding, morphological dilation, image Gaussian blur, and downsampling;

[0091] A module for generating a mask image based on the workpiece contour of the template image and matching the mask image with the gradient map;

[0092] A module for constructing a projective transformation matrix according to the image projective transformation parameters and transforming the gradient maps of the template image and the target image;

[0093] A module for normalizing the error;

[0094] A module for optimizing the registration parameters until a predetermined number of iterations is reached or a set stop condition is satisfied;

[0095] A module for outputting the optimal projective transformation matrix and registering the template image or the target image with the projective transformation matrix.

[0096] Embodiment 8. This embodiment provides a computer storage medium for storing a computing program. When the computer program is read by a computer, the computer executes the method provided in Embodiment 1.

[0097] Embodiment 9. This embodiment provides a computer, including a processor and a storage medium. When the processor reads the computer program stored in the storage medium, the computer executes the method provided in Embodiment 1.

[0098] Embodiment 10. This embodiment provides a computer program product. As a computer program, when the computer program is executed, it implements the method provided in Embodiment 1.

[0099] Embodiment 11. In combination with Figure 1-2 describing this embodiment, this embodiment further describes the above-provided technical solution in detail through specific examples. Specifically:

[0100] 1. Image preprocessing (applicable to template images and target images):

[0101] Convert the image to grayscale.

[0102] Perform threshold processing, setting pixel values greater than the threshold to the threshold to remove noise and enhance edges.

[0103] Preferred implementation: For the high-reflection problem in stamping part detection, pixel values greater than 72 are set to 72.

[0104] Use the Sobel operator to calculate the horizontal and vertical gradients of the image.

[0105] Perform threshold processing on the gradient image to remove pixels less than the threshold and further reduce noise.

[0106] Use morphological dilation operation to strengthen edge information.

[0107] Perform Gaussian blur on the dilated image to smooth the gradient image.

[0108] Perform 2 times of downsampling on the gradient image to improve the processing speed.

[0109] 2. Mask generation:

[0110] Use the labelme tool to draw the workpiece contour of the template image and generate a JSON file.

[0111] Generate a mask image based on the JSON file to define the registration area.

[0112] Perform downsampling on the mask image to generate a mask with the same scale as the gradient image.

[0113] 3. Loss function:

[0114] Construct a 3x3 projective transformation matrix according to the image projective transformation parameters (h11, h12, h13, h21, h22, h23, h31, h32).

[0115]

[0116] Use the projective transformation matrix to transform the horizontal and vertical gradient maps of the mask and the template.

[0117] Calculate the difference between the template gradient map after projective transformation and the target gradient map, and limit the calculation area to the pixels within the mask range.

[0118] Take the absolute value of the horizontal and vertical gradient differences respectively and sum them to obtain the total error.

[0119] Normalize the error and standardize it according to the image area.

[0120] Image registration:

[0121] S1: Optimize implementation parameters: Set the maximum number of iterations to 40, the maximum loss value to 1e8, the step size coefficient to 0.1, and the step size decay rate to 0.95.

[0122] S2: Initialize the parameters (h11, h12, h13, h21, h22, h23, h31, h32) to (1, 0, 0, 0, 1, 0, 0, 0).

[0123] S3: Calculate the loss value under the current parameters. If it is less than the maximum loss value, update the maximum loss value and the optimal parameters.

[0124] S4: Make temporary incremental adjustments to the current parameters respectively. Calculate the loss value of each parameter's temporary increment using the loss function, and divide the difference between the loss value obtained by each parameter's temporary increment and the current loss by the temporary increment of the parameter to obtain the Jacobian matrix. The temporary increment of each parameter is (assuming that the resolution after downsampling of the preprocessed image is (w*h), where w>h. Then use the largest w for calculation):

[0125] The minimum change amounts of h11, h12, h21, h22 parameters: The minimum change amount of rotation is The minimum change amounts of scaling and shearing are Take the minimum of the two. Then the temporary increments of h11, h12, h21, h22 are

[0126] The minimum change amounts of h31, h32 parameters:

[0127] The minimum change amount of h13, h23 parameters: 1.

[0128] S5: Solve the first-order derivative of the parameter change through the Jacobian matrix.

[0129] S6: Use the quasi-Newton method to calculate the Hessian matrix and adjust the step size to update the translation parameters.

[0130] S7: Repeat the above steps S3 - S6 until the predetermined number of iterations is reached or the norm of the parameter update is less than the set threshold. Output the optimal parameters.

[0131] Result output:

[0132] Multiply the translation amounts of h13 and h23 in the optimal parameters by 4 to construct a 3x3 projective transformation matrix. (The reason for multiplying by 4 is that downsampling is performed twice in the preprocessing).

[0133] If it is the registration of the template image to the target image, perform projective transformation using the above transformation matrix. If it is the registration of the target image to the template image, perform projective transformation using the inverse of the above matrix.

[0134] The above further describes the technical solutions provided by the present invention through several specific embodiments to highlight the advantages and beneficial effects of the technical solutions provided by the present invention. However, the above several specific embodiments are not used as limitations to the present invention. Any reasonable modifications and improvements to the present invention, combinations of implementation manners, and equivalent replacements within the spirit and principle of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for rigid image registration, characterized in that: include: Preprocess the template image and target image to be registered, convert them into grayscale images and perform threshold processing, Sobel operator gradient calculation, gradient image threshold processing, morphological expansion, image Gaussian blur and downsampling steps; A step of generating a mask image based on the workpiece contour of the template image, and matching the mask image with the gradient image; The step of constructing a projection transformation matrix according to the image projection transformation parameters, and transforming the gradient images of the template image and the target image; Steps for normalizing the error; The registration parameters are optimized until a predetermined number of iterations is reached or a set stop condition is met; The step of outputting the optimal projective transformation matrix and aligning the template image or the target image with the projective transformation matrix.

2. The image rigid registration method according to claim 1, characterized in that: The loss function calculation step includes taking the absolute values ​​of the horizontal and vertical gradient differences and summing them up to obtain a total error.

3. The image rigid registration method according to claim 1, characterized in that: The gradient images of the template image and the target image are transformed using the projection transformation matrix.

4. The image rigid registration method according to claim 1, characterized in that: Calculate the difference between the transformed template image gradient map and the target image gradient map, limit the calculation area to the pixels within the mask range, and normalize the error.

5. The image rigid registration method according to claim 1, characterized in that: CUDA is used to accelerate the calculation of the loss function, and the registration parameters are optimized using the Jacobian matrix and quasi-Newton method.

6. The image rigid registration method according to claim 1, characterized in that: The preprocessing includes threshold processing of the image, calculation of the horizontal and vertical gradients of the image by Sobel operator, morphological dilation operation and Gaussian blurring, so as to improve the processing speed and registration accuracy.

7. An image rigid registration device, characterized in that: include: A module that preprocesses the template image and target image to be registered, converts them into grayscale images, performs threshold processing, Sobel operator gradient calculation, gradient image threshold processing, morphological expansion, image Gaussian blur, and downsampling; A module for generating a mask image based on the workpiece contour of the template image and matching the mask image with the gradient image; A module for constructing a projection transformation matrix according to image projection transformation parameters and transforming the gradient images of the template image and the target image; A module for normalizing errors; A module that optimizes the registration parameters until a predetermined number of iterations is reached or a set stop condition is met; A module that outputs the optimal projective transformation matrix and aligns the template image or target image with the projective transformation matrix.

8. A computer storage medium for storing a computing program, characterized in that: When the computer program is read by a computer, the computer executes the method of claim 1 .

9. A computer, comprising a processor and a storage medium, characterized in that: When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 1 .

10. A computer program product, being a computer program, characterized in that When the computer program is executed, the method of claim 1 is implemented.

Images