A CAD image and real image registration method based on point affine transformation

Abstract

This invention relates to the fields of image processing and computer vision, specifically to a method for registering CAD drawings and real-world images based on point affine transformation. The method includes image preprocessing, edge detection and contour extraction, point cloud registration, affine transformation, result fusion and display, and automatic correction feedback. By fully utilizing the edge information and contour features of the image, this invention can more accurately match CAD drawings and real-world images, effectively solving the discrepancy problem between them and improving the accuracy of operations such as cutting. The registration method based on contour point sets, compared to traditional global image registration algorithms, has lower computational load and faster speed, meeting the real-time requirements of actual production. The entire registration process is highly automated; users only need to input the CAD drawing and the real-world image to automatically complete the registration and display the results, without complex parameter adjustments or manual intervention. This method has good adaptability to different types of CAD drawings and real-world images.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a method for registering CAD drawings and real-world images based on point affine transformation. Background Technology

[0002] In industrial manufacturing and processing, deviations in scale, angle, and position between CAD drawings and actual images can occur during photography or scanning. Existing technologies mainly rely on global feature matching or template matching, which presents the following problems: 1. Insufficient registration accuracy: Traditional methods do not make full use of local edge information, resulting in large matching errors and affecting the accuracy of subsequent processing.

[0003] 2. Poor robustness: The matching effect is significantly reduced under conditions of noise, changes in lighting, or image blur.

[0004] 3. High computational complexity: Global optimization methods often require a large amount of computational resources, making it difficult to meet real-time requirements.

[0005] Therefore, a solution is needed that can achieve fast and high-precision registration by extracting fine-grained edge information and then using local point cloud registration and feedback optimization. Summary of the Invention

[0006] To achieve the above objectives, this invention provides a method for registering CAD drawings and real drawings based on point affine transformation; Includes the following steps: S1. Image preprocessing: Mean offset filtering, grayscale conversion, Gaussian blur and adaptive threshold segmentation are applied to the CAD drawing and the real image respectively to enhance edge contrast. A multi-scale preprocessing strategy is introduced to ensure that detailed edges are captured at different resolutions. S2. Edge detection and contour extraction: The maximum contour point set of the target region is extracted using the improved Canny edge detection algorithm, morphological operations, and region growing algorithm, and key points are selected based on curvature information. S3. Point cloud registration: The contour point sets extracted from the CAD drawing and the real drawing are used to form point cloud data. The rigid registration algorithm is used for preliminary matching to obtain the scaling factor, rotation matrix and translation vector. The point cloud is then weighted and optimized by calculating the overlap of local regions. S4. Affine Transformation: Construct an affine transformation matrix based on the parameters obtained from point cloud registration, and perform affine transformation on the CAD drawing to make it accurately correspond to the real drawing in terms of scale, rotation and translation. S5. Result Fusion and Display: The optimized CAD drawing and the actual drawing are weighted and fused, and the registration result is displayed using multi-channel fusion technology; S6. Automatic correction feedback: Analyzes registration error. If it exceeds the set threshold, it automatically adjusts the preprocessing parameters and edge detection threshold, and re-executes the registration process to achieve closed-loop feedback control.

[0007] Furthermore, the mean-shift filtering parameters in image preprocessing are spatial_window_radius=21 and color_window_radius=51; Gaussian blur uses a kernel size of (7,7) and a standard deviation of 9, and an adaptive kernel is used for large images; adaptive thresholding is implemented based on the Otsu method and local histogram equalization.

[0008] Furthermore, the improved Canny edge detection algorithm employs a high and low dual threshold strategy and local nonmaximum suppression to remove noisy edges; morphological operations include performing an "open" operation with a kernel size of 3×3 on the binary image to remove isolated noise points and a "close" operation to fill edge breaks.

[0009] Furthermore, the rigid registration algorithm in point cloud registration solves for the transformation parameters by minimizing the distance between point clouds, as shown in the formula: Where s is the scaling factor, R is the rotation matrix, and t is the translation vector; Weighted optimization involves calculating the overlap rate of the point cloud by region, introducing a weight factor for each region to adjust the matching parameters, and reducing the impact of noise.

[0010] Furthermore, adaptive iterative optimization calculates the mean square error of the overlapping region and uses feedback iteration to adjust the affine matrix parameters until the mean square error is less than 0.5 pixels.

[0011] Furthermore, the weighted fusion in the result fusion and display adopts a 0.5:0.5 weighting for the registered CAD image and the real image, with dynamic weighting for local areas; and displays the preprocessing, edge detection, point cloud distribution, preliminary matching and optimization results in real time through a graphical interface.

[0012] Furthermore, the automatic correction feedback module uses the error detection module to automatically determine the overall matching accuracy and adjusts the preprocessing and matching parameters based on the determination result.

[0013] Furthermore, the multi-scale preprocessing strategy includes preprocessing images of different sizes separately to ensure that detailed edges can be captured at different resolutions.

[0014] The beneficial effects of this invention are: 1. By fully utilizing the edge information and contour features of the image, it is possible to more accurately match CAD drawings with real images, effectively solve the deviation problem between the two, and improve the accuracy of operations such as cutting.

[0015] 2. The registration method based on contour point sets is adopted. Compared with the traditional global image registration algorithm, it has less computation and faster speed, which can meet the real-time requirements in actual production.

[0016] 3. The entire registration process is highly automated. Users only need to input CAD drawings and real drawings to automatically complete the registration and display the results, without the need for complex parameter adjustments or manual intervention.

[0017] 4. This method is well adaptable to different types of CAD drawings and real-world drawings, and can achieve effective matching regardless of changes in image size, shape, angle, etc. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the input CAD image in an embodiment of the present invention; Figure 2 This is a schematic diagram of the output CAD image according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the input real image in an embodiment of the present invention; Figure 4 This is a schematic diagram of the output real image of an embodiment of the present invention; Figure 5 This is a schematic diagram of the pre-registration distribution (point cloud distribution of CAD drawing and real drawing) in an embodiment of the present invention; Figure 6 This is a schematic diagram of the registration distribution according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the registration result image in an embodiment of the present invention. Detailed Implementation

[0020] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0021] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0022] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0023] Example 1 1. Image preprocessing Mean shift filtering: Using parameters spatial_window_radius=21 and color_window_radius=51, the formula is described as follows: Where W(x,y) is the window centered at (x,y), and K(i,j) is the kernel function.

[0024] Multi-scale Gaussian blur: In addition to a fixed kernel (7,7) and σ=9, an adaptive kernel is used for large images to ensure the integrity of edge information after blurring. Adaptive threshold segmentation: A more balanced binary image is obtained by combining global Otsu thresholding with local histogram equalization.

[0025] 2. Edge detection and contour extraction Improved Canny algorithm: Set high and low thresholds (e.g., low threshold 150, high threshold 200), and add local curvature judgment in edge connectivity to filter noisy edges; Morphological treatment: Perform an "open" operation (kernel size 3×3) on the binary image to remove isolated noise points, and then perform a "close" operation to fill edge breaks; Key point extraction: By using region growing and curvature analysis, corner points and high curvature points are selected from continuous contours to form a high-quality point cloud dataset.

[0026] like Figure 1 As shown: Input CAD image (demonstrating the processing results of the CAD image through steps such as mean shift filtering, grayscale conversion, Gaussian blur, and threshold segmentation). like Figure 2 As shown: Output CAD image (red dots are edge key points, blue bars are cutting lines) like Figure 3 As shown: Input a real image: like Figure 4 As shown: Output the real image (red dots in the image represent extracted edge points). 3. Point cloud registration The maximum contour point sets of the real map and the CAD map are used as point cloud data X and Y, respectively.

[0027] The RigidRegistration algorithm in the pycpd library is used to perform rigid registration of point cloud X and Y, and the transformed point cloud data and transformation parameters s (scaling factor), r (rotation matrix) and t (translation vector) are obtained.

[0028] The rigid registration algorithm solves for the transformation parameters by minimizing the distance between point clouds, as shown in the formula: Where s is the scaling factor, R is the rotation matrix, and t is the translation vector.

[0029] Preliminary rigid registration: The point cloud data of CAD and real-world maps are registered using RigidRegistration in pycpd. The calculation formula is as follows: Regional weighted optimization: The overlap rate of the point cloud is calculated by region, and a weight factor is introduced for each region to adjust the matching parameters and reduce the impact of noise. Adaptive Iterative Optimization: Based on the initial registration results, the mean square error (MSE) of the overlapping area is calculated, and the affine matrix parameters are adjusted iteratively using feedback until the MSE meets the preset conditions (e.g., MSE < 0.5 pixels).

[0030] like Figure 5 As shown: Distribution before registration (point cloud distribution of CAD drawing and real drawing); like Figure 6 As shown: Distribution after registration; 4. Affine Transformation Based on the transformation parameters s, r, and t obtained from point cloud registration, construct the affine transformation matrix M.

[0031] The OpenCV warpAffine function is used to perform an affine transformation on the CAD drawing, and the transformed image is the same size as the real one. Figure 1 To.

[0032] 5. Result fusion and automatic correction feedback Weighted fusion: A multi-channel fusion method is adopted to synthesize the registered CAD drawing and the actual drawing with a weight of 0.5:0.5. Dynamic weighting is used in local areas to ensure that the fused drawing clearly shows the registration boundary. Automatic feedback module: The error detection module automatically determines the overall matching accuracy. If a local error exceeds the set threshold, the preprocessing and matching parameters are adjusted, and the above steps are repeated until the accuracy requirements are met. Real-time display: such as Figure 7 As shown The graphical interface displays the preprocessing, edge detection, point cloud distribution, preliminary matching, and optimization results in real time, allowing operators to intuitively evaluate the entire process.

[0033] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0034] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for registering CAD drawings and real-world drawings based on point affine transformation, characterized in that, Includes the following steps: S1. Image preprocessing: Mean offset filtering, grayscale conversion, Gaussian blur and adaptive threshold segmentation are applied to the CAD drawing and the real image respectively to enhance edge contrast. A multi-scale preprocessing strategy is introduced to ensure that detailed edges are captured at different resolutions. S2. Edge detection and contour extraction: The maximum contour point set of the target region is extracted using the improved Canny edge detection algorithm, morphological operations, and region growing algorithm, and key points are selected based on curvature information. S3. Point cloud registration: The contour point sets extracted from the CAD drawing and the real drawing are used to form point cloud data. The rigid registration algorithm is used for preliminary matching to obtain the scaling factor, rotation matrix and translation vector. The point cloud is then weighted and optimized by calculating the overlap of local regions. S4. Affine Transformation: Construct an affine transformation matrix based on the parameters obtained from point cloud registration, and perform affine transformation on the CAD drawing to make it accurately correspond to the real drawing in terms of scale, rotation and translation. S5. Result Fusion and Display: The optimized CAD drawing and the actual drawing are weighted and fused, and the registration result is displayed using multi-channel fusion technology; S6. Automatic correction feedback: Analyzes registration error. If it exceeds the set threshold, it automatically adjusts the preprocessing parameters and edge detection threshold, and re-executes the registration process to achieve closed-loop feedback control.

2. The method according to claim 1, characterized in that, The mean offset filtering parameters in the image preprocessing are spatial_window_radius=21 and color_window_radius=51; the Gaussian blur uses a kernel size of (7,7) and a standard deviation of 9, and an adaptive kernel is used for large images; the adaptive threshold segmentation is implemented based on the Otsu method and local histogram equalization.

3. The method according to claim 1, characterized in that, The improved Canny edge detection algorithm employs a high and low dual threshold strategy and local nonmaximum suppression to remove noisy edges; the morphological operations include performing an "open" operation with a kernel size of 3×3 on the binary image to remove isolated noise points and a "close" operation to fill edge breaks.

4. The method according to claim 1, characterized in that, The rigid registration algorithm in the point cloud registration solves for the transformation parameters by minimizing the distance between point clouds, as shown in the formula: ； Where s is the scaling factor, R is the rotation matrix, and t is the translation vector; The weighted optimization includes calculating the overlap rate of the point cloud by region, introducing a weight factor to adjust the matching parameters for each region, and reducing the impact of noise.

5. The method according to claim 1, characterized in that, The adaptive iterative optimization calculates the mean square error of the overlapping region and uses feedback iteration to adjust the affine matrix parameters until the mean square error is less than 0.5 pixels.

6. The method according to claim 1, characterized in that, The weighted fusion in the result fusion and display adopts the method of synthesizing the registered CAD map and the real map with a weight of 0.5:0.5, and dynamic weighting is used in local areas; and the preprocessing, edge detection, point cloud distribution, preliminary matching and optimization results are displayed in real time through a graphical interface.

7. The method according to claim 1, characterized in that, The automatic correction feedback module uses the error detection module to automatically determine the overall matching accuracy and adjusts the preprocessing and matching parameters based on the determination result.

8. The method according to claim 1, characterized in that, The multi-scale preprocessing strategy includes preprocessing images of different sizes separately to ensure that detailed edges can be captured at different resolutions.

Images