Image rotation and translation registration method and device
By using CUDA acceleration and optimized loss functions in image registration, combined with preprocessing technologies such as morphological expansion and Gaussian blur, the problems of slow processing speed, low accuracy and insufficient robustness in the prior art are solved, and efficient and accurate image registration is achieved.
Patent Information
- Application Number
- CN202510250104.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-04
- Publication Date
- 2025-06-27
AI Technical Summary
The prior art has slow calculation speed, low accuracy when processing high-resolution images, and is not robust in the case of noise and complex backgrounds.
Using the image rotation translation registration method based on CUDA acceleration, masked images are generated to define the registration area through pre-processing techniques such as grayscale conversion, morphological expansion and Gaussian blur, and optimized loss functions are constructed to calculate the translation and rotation parameters.
This significantly improves the processing speed of image registration, enhances accuracy and robustness, especially in scenarios where high-resolution images and strong noise interference are processed.
Smart Images
Figure CN120219447A_ABST
Abstract
Description
Technical Field
[0001] It relates to the field of image processing technology, especially high-precision image registration methods. Background Art
[0002] In the field of image processing, image registration technology, as a basic technology, is widely used in many fields such as medical image analysis, remote sensing image processing, computer vision, and industrial inspection. The goal of image registration is to align image data obtained under different perspectives, different times, or different imaging conditions for better analysis, recognition, and comparison.
[0003] Research Status of the Existing Technology
[0004] Image registration methods can be divided into two categories: feature-based registration methods and pixel-based registration methods. Feature-based registration methods match by extracting feature points or feature regions in the image and then calculate the transformation matrix through these feature points. Common methods include SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF), etc. These methods have good robustness and accuracy in static images.
[0005] Pixel-based registration methods directly use the pixel information of the image for registration without relying on explicitly extracted feature points. Common techniques include gray-level mutual information method, cross-correlation method, and mean square error method, etc. These methods usually perform excellently when the image has continuity and less noise, but may have problems of low registration accuracy and poor robustness when dealing with images with complex backgrounds, noise, or less texture.
[0006] In recent years, with the development of computer hardware, especially the emergence of graphics processing units (GPUs) and CUDA (Compute Unified Device Architecture), the computing speed of image registration technology has been significantly improved. GPU-accelerated image registration methods can effectively meet the real-time processing requirements of high-resolution images. Through CUDA technology, a large number of parallel computing tasks can be assigned to the GPU, thus significantly improving the processing speed and shortening the time required for image registration.
[0007] Problems in the Existing Technology
[0008] Although the existing technology has made significant progress in many applications, in some actual application scenarios, the existing methods still face the following technical problems:
[0009] Processing speed issue: Traditional CPU-based image registration methods often take a long time to process high-resolution images, making it difficult to meet real-time requirements. Especially in scenarios such as industrial inspection that require quick responses, the speed of existing methods often fails to meet the needs.
[0010] Accuracy issue: Although feature-based image registration methods can provide good matching accuracy in some cases, when the image quality is poor or the background is complex, the registration accuracy of traditional methods drops significantly, resulting in inaccurate registration results and affecting subsequent analysis and decision-making.
[0011] Noise and background complexity issue: Traditional image registration methods are prone to being interfered by noise when dealing with images with a lot of noise or complex backgrounds, leading to unstable or invalid registration results. Especially in some industrial applications, due to problems such as noise on the workpiece surface, light changes, or reflections, the robustness of existing technologies is insufficient. Summary of the Invention
[0012] To solve the technical defects of slow processing speed, low accuracy, and noise and background complexity existing in the prior art, the technical solution provided by the present invention is as follows:
[0013] An image rotation and translation registration method, comprising:
[0014] Steps of preprocessing the template image and the target image to be registered;
[0015] Steps of generating a mask image according to the workpiece contour of the template image, defining the registration area, and downsampling the mask image;
[0016] Steps of constructing a loss function, calculating the error between the template image and the target image through an affine transformation matrix, calculating the total error according to the pixels within the mask area, and normalizing the error;
[0017] Steps of initializing the translation and rotation parameters and calculating the optimal translation and rotation parameters;
[0018] Steps of using the optimal translation and rotation parameters to construct an affine transformation matrix and outputting the registered image.
[0019] Furthermore, a preferred embodiment is provided, where the preprocessing includes converting the image to a grayscale image, performing threshold processing, calculating the horizontal and vertical gradients of the image, performing morphological dilation, Gaussian blur, and downsampling operations.
[0020] Furthermore, a preferred embodiment is provided, where the optimal translation and rotation parameters are calculated based on the loss function through incremental adjustment, Jacobian matrix solution, and quasi-Newton method optimization.
[0021] Furthermore, a preferred embodiment is provided, in which a registered image is obtained by performing an affine transformation on the image.
[0022] Furthermore, a preferred embodiment is provided, in which the affine transformation matrix is a 2×3 affine transformation matrix.
[0023] Furthermore, a preferred embodiment is provided, in which the translation and rotation parameters are incrementally adjusted, and the Hessian matrix is calculated using the Jacobian matrix and the quasi-Newton method to obtain the optimal translation and rotation parameters.
[0024] Based on the same inventive concept, the present invention also provides an image rotation and translation registration device, including:
[0025] a module for preprocessing the template image and the target image to be registered;
[0026] a module for generating a mask image according to the workpiece contour of the template image, defining the registration area, and downsampling the mask image;
[0027] a module for constructing a loss function, calculating the error between the template image and the target image through the affine transformation matrix, calculating the total error according to the pixels within the mask area, and normalizing the error;
[0028] a module for initializing the translation and rotation parameters and calculating the optimal translation and rotation parameters;
[0029] a module for constructing an affine transformation matrix using the optimal translation and rotation parameters and outputting the registered image.
[0030] Based on the same inventive concept, the present invention also provides a computer storage medium for storing a calculation program, and when the computer program is read by a computer, the computer executes the method.
[0031] Based on the same inventive concept, the present invention also provides a computer, including a processor and a storage medium, and when the processor reads the computer program stored in the storage medium, the computer executes the method.
[0032] Based on the same inventive concept, the present invention also provides a computer program product, which, as a computer program, implements the method when the computer program is executed.
[0033] Compared with the prior art, the beneficial effects of the technical solution provided by the present invention are as follows:
[0034] The image translation + rotation registration method based on CUDA acceleration proposed by the present invention significantly improves the processing speed by adopting CUDA parallel computing. In traditional CPU-based image registration methods, due to the large amount of image data, especially in the case of high-resolution images, the computing speed often becomes a limiting factor. However, this solution optimizes parallel computing by using CUDA streams, allocates a large number of computing tasks to the GPU, and thus greatly improves the speed of image registration. This improvement makes the method particularly suitable for application scenarios that require real-time processing, such as industrial inspection and medical image analysis, can significantly shorten the image registration time, and has significant advantages compared with traditional CPU-based registration methods.
[0035] Through the optimized loss function and preprocessing anti-noise technology, the present invention performs excellently in terms of accuracy. In traditional image registration methods, when dealing with complex background and noisy images, the registration accuracy is often low, especially in the case of unclear edges or texture interference. To solve this problem, this solution calculates the loss using the gradient map and combines the masking technology to limit the calculation area to the workpiece area, thus effectively reducing the interference of background noise. In addition, for problems such as reflection, noise, and illumination changes in images, this solution further enhances the edge information and improves the registration accuracy through preprocessing operations such as morphological dilation and Gaussian blur. Compared with traditional corner matching and shape matching methods, the present invention has higher accuracy in dealing with complex environments.
[0036] The robustness of this solution is significantly improved, especially when dealing with images with noise or complex backgrounds. Traditional image registration methods are prone to failure in the case of more noise or complex backgrounds, while this solution calculates using the gradient map and combines the masking technology to effectively limit the calculation range, minimizing the interference of noise and complex backgrounds. Compared with feature point matching-based technologies, the gradient-based registration method is more stable, can better handle complex image backgrounds, and improves the stability and reliability of the registration results.
[0037] Generally speaking, by introducing CUDA acceleration, optimizing the loss function, and robust preprocessing technologies, this solution shows significant advantages over traditional methods in terms of speed, accuracy, and robustness, and is particularly suitable for processing high-resolution images and scenarios with strong noise interference.
[0038] It is applicable to medical image analysis, remote sensing image processing, industrial inspection, and other work that requires real-time registration of high-definition images. Brief Description of the Drawings
[0039] Figure 1 It is a schematic diagram of the workpiece to be measured.
[0040] Figure 2 It is a schematic diagram of the registration effect of the workpiece to be measured. Detailed implementation manners
[0041] To make the advantages and beneficial effects of the technical solution provided by the present invention more clearly manifested, the technical solution provided by the present invention will be further described in detail below with reference to the accompanying drawings. Specifically:
[0042] Embodiment 1. This embodiment provides an image rotation and translation registration method, including:
[0043] A step of preprocessing the template image and the target image to be registered;
[0044] A step of generating a mask image according to the workpiece contour of the template image, defining the registration area, and downsampling the mask image;
[0045] A step of constructing a loss function, calculating the error between the template image and the target image through an affine transformation matrix, calculating the total error according to the pixels within the mask area, and normalizing the error;
[0046] A step of initializing the translation and rotation parameters and calculating the optimal translation and rotation parameters;
[0047] A step of constructing an affine transformation matrix using the optimal translation and rotation parameters and outputting the registered image.
[0048] Specifically, an image translation + rotation registration method and system based on CUDA acceleration mainly implement the image registration process through the following steps. Each step is based on CUDA parallel computing to improve the processing speed and accuracy, and enhance the robustness through an optimized loss function and anti-noise technology. It includes:
[0049] 1. Image preprocessing
[0050] In this solution, first, the template image and the target image are preprocessed to ensure that the image quality is suitable for subsequent registration operations.
[0051] The specific steps include:
[0052] Conversion to grayscale image: Convert the color image to a grayscale image to reduce the interference of color information and retain the structural information of the image.
[0053] Threshold processing: Perform threshold processing on the grayscale image, set the pixel values greater than the threshold to the threshold to remove the noise in the image and enhance the edges. For the possible high-reflection problem in stamping part detection, the part with pixel values greater than 72 is uniformly set to 72.
[0054] Gradient calculation: Use the Sobel operator to calculate the horizontal and vertical gradients of the image to capture the edge information of the image.
[0055] Gradient map thresholding: Perform thresholding on the calculated gradient map to remove pixels smaller than the threshold, further reducing noise.
[0056] Morphological dilation: Perform morphological dilation on the gradient map to enhance the edge information of the image and strengthen the image features.
[0057] Gaussian blur: Apply Gaussian blur to the dilated gradient map to smooth the gradient map and reduce the impact of noise.
[0058] Image downsampling: Perform downsampling on the image to improve the subsequent processing speed, while reducing the detailed information in the image and focusing on the important features.
[0059] Through these preprocessing steps, the quality of the image can be greatly improved, and a more accurate input can be provided for the subsequent registration steps.
[0060] 2. Mask generation
[0061] When performing image registration, the mask technique is used to define the calculation area, ensuring that the registration algorithm only focuses on the key information areas in the image. The mask generation process is as follows:
[0062] Workpiece contour drawing: Manually draw the workpiece contour of the template image using the labelme tool and generate a JSON file. This file contains the geometric information of the workpiece and can define the registration area.
[0063] Mask image generation: According to the generated JSON file, use a specific algorithm to generate a mask image to define the registration area. The mask image is used to limit the calculation area, only considering the workpiece area and excluding irrelevant parts.
[0064] Mask image downsampling: Downsample the mask image to make it consistent with the gradient map in scale to ensure the correct application of the mask image in subsequent calculations.
[0065] This step ensures that the registration algorithm can only perform calculations within the workpiece area, avoiding interference from background noise and enhancing the calculation efficiency and accuracy.
[0066] 3. Loss function construction
[0067] To perform translation and rotation registration on the image, a loss function needs to be constructed to measure the registration error between the template image and the target image. The construction process of the loss function includes the following steps:
[0068] Affine transformation matrix calculation: According to the image translation and rotation parameters (such as translation amounts x and y, rotation angle θ), construct a 2x3 affine transformation matrix. This matrix is used to perform translation and rotation transformations on the image.
[0069] Gradient map transformation: Using the above affine transformation matrix, transform the horizontal and vertical gradient maps of the mask and the template image to obtain the transformed gradient maps.
[0070] Error calculation: Calculate the difference between the transformed template gradient map and the target gradient map. The error calculation is limited to the pixels within the mask area to improve the calculation efficiency and reduce interference. Take the absolute value of the horizontal and vertical gradient differences and sum them to obtain the total error.
[0071] Normalization processing: Normalize the error and standardize it according to the image area to eliminate the influence of the image size on the error and ensure the accuracy of registration.
[0072] The optimization objective of the loss function is to improve the registration accuracy by reducing the error value and ensure the best alignment between the template image and the target image.
[0073] 4. Image registration
[0074] Based on the loss function, the present invention realizes the translation and rotation registration of the image through the following steps:
[0075] Initial parameter setting: Set the initial translation and rotation parameters (for example, the translation amounts are x = 0, y = 0, and the rotation angle θ = 0), and set the maximum number of iterations to 40, the maximum loss value to 1e8, the minimum threshold norm length to 1, the step coefficient to 0.1, and the step decay rate to 0.95.
[0076] Loss calculation and update: Calculate the loss value according to the current translation and rotation parameters. If the current loss value is less than the maximum loss value, update the maximum loss value and the optimal parameters.
[0077] Incremental adjustment: Make a temporary incremental adjustment to the current translation and rotation parameters, and calculate the loss values of the horizontal and vertical displacements and the rotation respectively. The incremental value is set to 1 pixel, and the rotation parameter increment depends on the image resolution, usually 2 / w pixels, where w is the width of the image.
[0078] Jacobian matrix and quasi-Newton method: Solve the parameter changes through the Jacobian matrix, calculate the Hessian matrix using the quasi-Newton method, adjust the step size, and update the parameters.
[0079] Iterative optimization: Repeat the above steps until the predetermined number of iterations is reached or the norm length of the parameter update is less than the set threshold, and finally output the optimal translation and rotation parameters.
[0080] Through these steps, the template image can be accurately aligned with the target image to ensure the accuracy and efficiency of the registration result.
[0081] 5. Result output
[0082] Finally, according to the optimal translation and rotation parameters, output the registration result:
[0083] Affine transformation matrix construction: According to the optimal translation and rotation parameters, construct the final 2x3 affine transformation matrix. For registering the template image to the target image, use this matrix for affine transformation; for registering the target image to the template image, use the inverse of the matrix for affine transformation.
[0084] Image output: Output the registered image to complete the final registration result.
[0085] Embodiment 2: This embodiment further limits an image rotation and translation registration method provided in Embodiment 1. The preprocessing includes converting the image to a grayscale image, performing threshold processing, calculating the horizontal and vertical gradients of the image, performing morphological dilation, Gaussian blur, and downsampling operations.
[0086] Embodiment 3: This embodiment further limits an image rotation and translation registration method provided in Embodiment 1. Based on the loss function, through incremental adjustment, Jacobian matrix solution, and quasi-Newton method optimization, calculate the optimal translation and rotation parameters.
[0087] Embodiment 4: This embodiment further limits an image rotation and translation registration method provided in Embodiment 1. By performing an affine transformation on the image, obtain the registered image.
[0088] Embodiment 5: This embodiment further limits an image rotation and translation registration method provided in Embodiment 1. The affine transformation matrix is a 2*3 affine transformation matrix.
[0089] Embodiment 6: This embodiment further limits an image rotation and translation registration method provided in Embodiment 1. Perform incremental adjustment on the translation and rotation parameters, and use the Jacobian matrix and quasi-Newton method to calculate the Hessian matrix to obtain the optimal translation and rotation parameters.
[0090] Embodiment 7: This embodiment provides an image rotation and translation registration device, including:
[0091] A module for preprocessing the template image and the target image to be registered;
[0092] A module for generating a mask image according to the workpiece contour of the template image, defining the registration area, and performing downsampling on the mask image;
[0093] A module for constructing a loss function, calculating the error between the template image and the target image through the affine transformation matrix, calculating the total error according to the pixels within the mask area, and normalizing the error;
[0094] A module for initializing the translation and rotation parameters and calculating the optimal translation and rotation parameters;
[0095] A module that constructs an affine transformation matrix using the optimal translation and rotation parameters and outputs the registered image.
[0096] Embodiment 8. This embodiment provides a computer storage medium for storing a computing program. When the computer program is read by a computer, the computer executes the method provided in Embodiment 1.
[0097] Embodiment 9. This embodiment provides a computer, including a processor and a storage medium. When the processor reads the computer program stored in the storage medium, the computer executes the method provided in Embodiment 1.
[0098] Embodiment 10. This embodiment provides a computer program product. As a computer program, when the computer program is executed, it implements the method provided in Embodiment 1.
[0099] Embodiment 11. In combination with Figure 1-2 Describing this embodiment, this embodiment further describes the above-provided technical solution in detail through specific examples. Specifically:
[0100] This embodiment provides an image translation + rotation registration method based on CUDA acceleration. All calculations are implemented through the CUDA version of OpenCV and optimized through CUDA streams. The following are the specific steps of this method, assuming that all calculations use CUDA:
[0101] Image preprocessing (applicable to the template image and the target image):
[0102] Convert the image to a grayscale image.
[0103] Perform threshold processing, setting pixel values greater than the threshold to the threshold to remove noise and enhance edges.
[0104] Preferred implementation: For the high-reflectivity problem in stamping part detection, pixel values greater than 72 are set to 72.
[0105] Use the Sobel operator to calculate the horizontal and vertical gradients of the image.
[0106] Perform threshold processing on the gradient image to remove pixels less than the threshold and further reduce noise.
[0107] Use morphological dilation operations to strengthen edge information.
[0108] Perform Gaussian blur on the dilated image to smooth the gradient image.
[0109] Downsample the gradient image to improve processing speed.
[0110] Mask generation:
[0111] Use the labelme tool to draw the workpiece contour of the template image and generate a JSON file.
[0112] Generate a mask image based on the JSON file to define the registration area.
[0113] Downsample the mask image to generate a mask with the same scale as the gradient image.
[0114] Loss function:
[0115] Construct a 2x3 affine transformation matrix according to the image translation + rotation parameters (x, y, θ).
[0116]
[0117] Use the affine transformation matrix to transform the horizontal and vertical gradient images of the mask and the template.
[0118] Calculate the difference between the template gradient image after affine transformation and the target gradient image, and limit the calculation area to the pixels within the mask range.
[0119] Take the absolute value of the horizontal and vertical gradient differences respectively and sum them to obtain the total error.
[0120] Normalize the error and standardize it according to the image area.
[0121] Image registration:
[0122] Preferred implementation parameters: Set the maximum number of iterations to 40, the maximum loss value to 1e8, the minimum threshold norm length to 1, the step coefficient to 0.1, and the step decay rate to 0.95.
[0123] Initialize the parameters (x, y, θ) to (0, 0, 0).
[0124] Calculate the loss value under the current translation + rotation parameters. If it is less than the maximum loss value, update the maximum loss value and the optimal parameters.
[0125] Make a temporary incremental adjustment to the current translation + rotation parameters, and calculate the horizontal, vertical displacement and rotation loss respectively. The horizontal and vertical increments are 1 pixel, and the rotation parameter increment depends on the image resolution. Assume that the resolution of the image after preprocessing downsampling is (w*h), where w>h. Then use the maximum w to participate in the calculation. The increment of the rotation parameter θ is 2 / w pixels.
[0126] Solve the parameter change through the Jacobian matrix.
[0127] Use the quasi-Newton method to calculate the Hessian matrix and adjust the step size to update the parameters.
[0128] Repeat the above steps until the predetermined number of iterations is reached or the norm of the parameter update is less than the set threshold. Output the optimal parameters.
[0129] In the result output section:
[0130] Construct a 2x3 affine transformation matrix by multiplying the x and y translation amounts in the optimal parameters by 4. (The reason for multiplying by 4 is that two downsamplings were performed in the preprocessing).
[0131] If it is the registration of the template image to the target image, perform an affine transformation using the above transformation matrix. If it is the registration of the target image to the template image, perform an affine transformation using the inverse of the above matrix.
[0132] This embodiment combines CUDA parallel computing and an optimized image registration algorithm to achieve efficient and accurate image translation + rotation registration. Compared with traditional CPU-based computing methods, CUDA significantly improves the processing speed, especially suitable for real-time processing of high-resolution images. In addition, by using an optimized loss function and anti-noise processing, this embodiment performs excellently in terms of accuracy and has higher accuracy compared to traditional corner matching and shape matching methods. The main advantages include:
[0133] Fast calculation speed: With the help of CUDA parallel computing, the image registration speed is improved, suitable for real-time processing of high-resolution images.
[0134] High accuracy: By optimizing the loss function and preprocessing anti-noise technology, the registration accuracy is significantly improved.
[0135] Strong robustness: Use the gradient map for loss calculation and use a mask to limit the calculation to the workpiece area, thus effectively handling noise and complex backgrounds.
[0136] The above further describes the technical solutions provided by the present invention through several specific embodiments to highlight the advantages and beneficial effects of the technical solutions provided by the present invention. However, the above several specific embodiments are not used as limitations on the present invention. Any reasonable modifications and improvements, combinations of embodiments, and equivalent replacements based on the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An image rotation and translation registration method, characterized in that: include: A step of preprocessing the template image and the target image to be registered; The steps of generating a mask image according to the workpiece contour of the template image, limiting the registration area, and downsampling the mask image; The steps of constructing a loss function, calculating the error between the template image and the target image through an affine transformation matrix, calculating the total error according to the pixels in the mask area, and normalizing the error; Initialize the translation and rotation parameters and calculate the optimal translation and rotation parameters; The step of constructing an affine transformation matrix using the optimal translation and rotation parameters and outputting the aligned image.
2. The image rotation and translation registration method according to claim 1, characterized in that: Preprocessing includes converting the image into grayscale, performing threshold processing, calculating the horizontal and vertical gradients of the image, and performing morphological dilation, Gaussian blur and downsampling operations.
3. The image rotation and translation registration method according to claim 1, characterized in that: Based on the loss function, the optimal translation and rotation parameters are calculated through incremental adjustment, Jacobian matrix solution and quasi-Newton method optimization.
4. The image rotation and translation registration method according to claim 1, characterized in that: By performing affine transformation on the image, the registered image is obtained.
5. The image rotation and translation registration method according to claim 1, characterized in that: The affine transformation matrix is a 2*3 affine transformation matrix.
6. The image rotation and translation registration method according to claim 1, characterized in that: The translation and rotation parameters are incrementally adjusted, and the Hessian matrix is calculated using the Jacobian matrix and the quasi-Newton method to obtain the optimal translation and rotation parameters.
7. An image rotation and translation registration device, characterized in that: include: A module for preprocessing the template image and target image to be registered; A module for generating a mask image according to the workpiece contour of the template image, defining the registration area, and downsampling the mask image; A module that constructs a loss function, calculates the error between the template image and the target image through the affine transformation matrix, calculates the total error based on the pixels in the mask area, and normalizes the error; Initialize the translation and rotation parameters and calculate the module with the best translation and rotation parameters; A module that constructs an affine transformation matrix using the optimal translation and rotation parameters and outputs the registered image.
8. A computer storage medium for storing a computing program, characterized in that: When the computer program is read by a computer, the computer executes the method of claim 1 .
9. A computer, comprising a processor and a storage medium, characterized in that: When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 1 .
10. A computer program product, being a computer program, characterized in that When the computer program is executed, the method of claim 1 is implemented.