Circuit board infrared image and visible light image registration method based on rectangular and circular characteristics
By extracting the rectangular and circular features of the circuit board, a unique descriptor is constructed, and combining bilateral matching and global iterative optimization algorithms, the error matching problem in the registration of the circuit board infrared image and visible light image is solved, and the accuracy of fault detection is improved Sex and robustness.
Patent Information
- Application Number
- CN202411004844.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-25
AI Technical Summary
The existing infrared image and visible image registration methods have mismatch problems in circuit board failure detection, especially because the characteristics of rectangular components and circular holes on the circuit board are similar, resulting in traditional point-based methods in the registration process. Poor robustness.
Using the infrared image and visible light image registration method based on rectangular and circular features, edge detection is performed through the Canny operator, rectangular and circular features are extracted, and unique descriptors are constructed, and bilateral matching, rotation angle estimation, and RANSAC algorithm are used. and global iterative optimization to reduce mismatch and improve registration accuracy.
It effectively avoids mismatch caused by similar feature point descriptors, improves the accuracy and reliability of circuit board failure detection, and enhances the robustness and accuracy of registration.
Smart Images

Figure CN119068027A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a circuit board infrared image and visible light image registration method based on rectangular and circular features, belonging to the technical field of infrared image and visible light image heterogeneous registration. Background Art
[0002] Circuit boards play a vital role in modern electronic devices. However, fault detection on circuit boards has always been a challenging task due to their complex structure and high-density component layout. In order to ensure the normal operation and reliability of circuit boards, it is very important to detect and diagnose potential faults.
[0003] In recent years, infrared thermal imaging technology has been widely used in the field of circuit board fault detection. Infrared thermal imaging technology can measure the heat distribution on the surface of an object and generate corresponding infrared images. By analyzing these infrared images, thermal abnormal areas on the circuit board, such as overheating, short circuit or other faults, can be detected. Therefore, infrared thermal imaging technology has become a very effective method for circuit board fault detection.
[0004] However, infrared images only provide thermal information on the circuit board and lack the detailed structural information that visible light images have. Visible light images can provide details on the circuit board, such as components, connections, and logos, which are critical for accurate fault location and further analysis.
[0005] Therefore, it is very important to register the infrared image with the visible light image. The registration of infrared image and visible light image refers to aligning the images of two different modalities in space so that their corresponding parts are in corresponding positions on the image. By realizing the registration of infrared image and visible light image, the advantages of infrared thermal imaging technology and visible light image can be fully utilized to improve the accuracy and reliability of circuit board fault detection.
[0006] At present, the research on the registration of infrared images and visible light images has made certain progress. The methods suitable for the registration of infrared images and visible light images of circuit boards mainly include feature-based methods and region-based methods. Region-based methods mainly include mutual information and cross-correlation methods. Due to the different imaging principles of visible light images and infrared images, infrared images contain far less information than visible light images. Such methods have poor robustness in the spatial domain and poor timeliness in the frequency domain. In contrast, feature-based methods are usually more stable. The features used in these methods include points, line segments, and contours. Feature-based methods can generally be described as the process of finding matching points and obtaining transformation matrices. Point features are widely distributed and easy to detect. Therefore, point feature-based methods are widely used, among which the most representative algorithms include SIFT, SURF, and ORB.
[0007] However, there are a large number of similar structures in circuit boards, and the point features obtained from the infrared image and visible light image of the circuit board have similar descriptors, which will lead to a large number of mismatches in the registration process. Taking rectangular components as an example, when extracting point features, its four corner points are usually extracted. Due to the requirement of rotation without deformation, the four corner points have the same angle and main direction. On the other hand, there are a large number of circular holes in the circuit board. Since the holes have no heat, the infrared image will show a black circular area. However, the circle is smooth and has no corners, and the traditional point feature-based method cannot extract valid feature points. Summary of the invention
[0008] In order to solve the shortcomings of the existing registration algorithms, the purpose of the present invention is to propose a registration method for infrared images and visible light images of circuit boards based on rectangular and circular features. By extracting the rectangular and circular features of the infrared and visible light images of the circuit board, the problem of no matching due to feature similarity in the traditional point or line feature-based registration algorithm is avoided.
[0009] In order to achieve the above object, the present invention adopts the following technical solution:
[0010] A registration method for infrared image and visible light image of a circuit board based on rectangular and circular features, which is special in that it includes the following steps:
[0011] Step 1: Obtain the infrared image and visible light image of the circuit board taken at the same time, use the Canny operator to perform edge detection on the infrared and visible light images respectively, and obtain the infrared and visible light binary edge images of the circuit board, in which the edge pixels are marked as white and the non-edge pixels are marked as black;
[0012] Step 2: Perform rectangle detection on the infrared and visible light binary edge images of the circuit board obtained in step 1, and extract rectangle features, including the four corner points, side lengths and angles of the rectangle;
[0013] Step 3: Perform circle detection on the infrared and visible light binary edge images of the circuit board obtained in step 1, and extract circle features, including the center and radius of the circle;
[0014] Step 4: Use the rectangle and circle information obtained in steps 2 and 3 to construct rectangle feature and circle feature descriptors;
[0015] Step 5: Use bilateral matching to preliminarily match the infrared image and the visible light image to obtain the initial matching feature point pairs;
[0016] Step 6: Apply rotation angle estimation and connection line consistency theory to remove obvious mismatched feature point pairs, and select hundreds of iterations of the RANSAC algorithm to remove the remaining small number of mismatched feature point pairs;
[0017] Step 7: Estimate the projection transformation matrix between infrared and visible light images using roughly matched feature point pairs;
[0018] Step 8: Use fine matching based on global iterative optimization to reduce the position deviation of feature points.
[0019] The above step 2 specifically includes the following steps:
[0020] Step 2.1, initialize an empty contour list based on the edge image;
[0021] Step 2.2, traverse each pixel of the edge image, when a white pixel is found, mark it as the current pixel, and start the contour tracing process. During the contour tracing process, determine how to move to the next pixel based on the connectivity of adjacent pixels and the boundary conditions in the edge image. This process will continue until it returns to the starting pixel and forms a closed contour. The formed contour is added to the contour list;
[0022] Step 2.3, repeat step 2.2 until the entire edge image is traversed, so that all closed contours in the image can be found and extracted;
[0023] Step 2.4, filter the detected contours, select the contours that meet the rectangular features, filter them according to the area attribute of the contours, and use the Shoelace formula to calculate the polygon area. The Shoelace formula is based on the vertex coordinates of the polygon, decomposes the polygon into triangles, calculates the area of the triangles, and then adds these areas. The formula is as follows:
[0024]
[0025] Where n represents the number of sides of the polygon, (x i ,y i ) represents the coordinates of the i-th vertex of the polygon;
[0026] Set the coefficient λ = 0.1. If the polygon contour area A>S, delete the contour, where S is the total area of the image;
[0027] Step 2.5, for the filtered contour, use the polygon approximation algorithm to approximate the contour into a polygon, and use the Douglas-Peucker algorithm to simplify the contour by retaining the key points that are far away in the contour. Select the point on the contour that is farthest from the original curve as the key point, and then split the curve into two sub-curves, recursively process them respectively, and define a function d(P i ,P j ) represents point P i To line segment P1P n The vertical distance is:
[0028] d(Pi ,P j )=|(P j -P1)×(P n -P1)| / |P n -P1|
[0029] Among them, P1 is the starting point, P j , P n is the endpoint of the resulting curve.
[0030] Select the starting point P1 and the ending point P on the curve n As the endpoint of the resulting curve, find the point P with the maximum distance by iteratively calculating the vertical distance between each point to the starting point and the end point. k ; If the maximum distance is greater than the given threshold, the current curve is P k Cut to get two sub-curves; recursively call the algorithm for the two sub-curves respectively until the maximum distance is less than the threshold, or no more cutting is possible; finally, connect the key points of each sub-curve to get the final approximate curve;
[0031] Step 2.6, for the polygon obtained by approximation, determine whether it is a rectangle by calculating the number of sides and angles of the polygon; first, obtain the number of sides of the polygon according to the number of contour corner points, and delete the polygons with sides not equal to 4; then calculate the inner angle of the polygon, for the current contour corner point, take the previous contour corner point and the next contour corner point, form two vectors, and calculate the inner angle;
[0032] Step 2.7, use the gradient descent method to fit the detected rectangle to optimize the corner points, side length, and angle of the rectangle; use the minimization of the square distance between the rectangle and the contour as the optimization objective function to initialize the parameters, calculate the gradient, and update the parameters;
[0033] The above step 3 specifically includes:
[0034] Step 3.1, applying Hough circle transform to the edge image obtained by rectangle detection, and setting the x-coordinate representing the center of the circle, the y-coordinate of the center of the circle, and the radius of the circle as a parameter space;
[0035] Step 3.2, traverse each edge point in the edge image, and perform the following operations on each edge point: for each possible combination of circle center and radius, calculate the circle center and radius according to the edge point, and perform accumulation operation at the corresponding position in the parameter space to increase the cumulative value of the combination;
[0036] Step 3.3, after the accumulation is completed, the detected circles are determined according to the set threshold; the threshold can control the detection sensitivity of the circle, a higher threshold can filter out weaker circle candidates, and a lower threshold can allow more circles to be detected;
[0037] In step 3.4, for each circle that meets the threshold, information such as the coordinates of the center and the radius can be extracted from the parameter space.
[0038] The above step 4 specifically includes:
[0039] Step 4.1, for each corner point of the rectangle, use its coordinates as part of the descriptor. In addition, for each corner point, find the two edges connected to it and calculate the angle and length of these two edges;
[0040] Step 4.2, normalize the descriptor, normalize the side length to a specific range [0,1], and map the angle to a specific range of 0 to π;
[0041] Step 4.3, combine the coordinates of the corner points, the angles of the connecting edges, and the edge length information into a fixed-dimensional descriptor vector; the coordinates, angles, and edge lengths are connected together to form a longer descriptor vector;
[0042] The above step 6 specifically includes:
[0043] Calculate the scaling factor ξ between the infrared and visible light images, ξ is equal to the ratio of the pixel areas of the same region in the infrared and visible light images adjusted to the same vertical resolution, and convert P2 to the same scale and orientation using the following expression:
[0044]
[0045]
[0046] Among them, φ p is the rotation angle calculated in the rotation angle estimation, R is the rotation orthogonal matrix of the infrared and visible light images, its purpose is to rotate the visible light image to the same viewpoint of the infrared image, P′ 2i is P′ 2i Feature points after rotation and scaling transformation.
[0047] The above step 8 specifically includes:
[0048] The projection transformation matrix M is calculated by the least squares method, and the formula is as follows:
[0049] minf=||P c1 -P c2 ||2
[0050] Among them, P c1 , P c2 The matching points obtained by rough matching in step 7;
[0051] The least squares solution of M is called:
[0052]
[0053] The fitting point corresponding to is expressed as:
[0054]
[0055] The residual of the sum is expressed as:
[0056] ε(i)=||P 2fit (i)-P c2 (i)||2,i=1,2,...
[0057] P c1 P c2 For the matching point pairs obtained by rough matching in step 7, the residual is iteratively optimized until it is smaller than the set threshold.
[0058] The present invention is beneficial in that:
[0059] Compared with the existing infrared image and visible light image registration based on point or straight line features, the present invention proposes a circuit board infrared image and visible light image registration method based on rectangular and circular features for a large number of rectangular heat-generating components and circular heat-free holes in the infrared image of the circuit board. The method extracts edge contours, closed contours, approximate polygons, and screens rectangles to obtain rectangular features, and constructs a unique descriptor for rectangular features; at the same time, it extracts edge contours, accumulates the center and radius, selects circular features, and constructs a unique descriptor for circular features. Compared with traditional point feature descriptors, our descriptors add information such as side length, center, radius, etc., which is more unique and avoids mismatching caused by similar feature point descriptors. In addition, in addition to using conventional matching algorithms such as rotation angle estimation, connection line consistency, and RANSAC algorithm in the stage of matching feature point pairs, our algorithm also adds a global optimization algorithm for adjusting the position of feature points, which further improves the registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 It is a flow chart of the method for registering infrared image and visible light image of a circuit board based on rectangular and circular features of the present invention;
[0061] Figure 2 Flowchart for extracting rectangular features from infrared and visible light images of circuit boards;
[0062] Figure 3 Flowchart for extracting circular features from infrared and visible light images of circuit boards. DETAILED DESCRIPTION
[0063] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.
[0064] like Figure 1As shown, embodiment 1 of the present invention provides a circuit board infrared image and visible light image registration method based on rectangular and circular features, comprising the following steps:
[0065] Step 1: Obtain the infrared image and visible light image of the circuit board taken at the same time, and use the Canny operator to perform edge detection on the infrared and visible light images to obtain a binary edge image. The edge pixels are marked as white, and the non-edge pixels are marked as black;
[0066] Step 2: Perform rectangle detection on the infrared and visible light binary edge images of the circuit board obtained in step 1, and extract rectangle features, including the four corner points, side lengths and angles of the rectangle;
[0067] like Figure 2 As shown, in a further preferred but non-limiting embodiment of the present invention, step 2 specifically includes:
[0068] Step 2.1, initialize an empty contour list based on the edge image;
[0069] Step 2.2, traverse each pixel of the edge image. When a white pixel is found, mark it as the current pixel and start the contour tracing process. During the contour tracing process, determine how to move to the next pixel based on the connectivity of adjacent pixels and the boundary conditions in the edge image. This process continues until it returns to the starting pixel and forms a closed contour. Add the formed contour to the contour list.
[0070] Step 2.3, repeat step 2.2 until the entire edge image is traversed. In this way, all closed contours in the image can be found and extracted.
[0071] Step 2.4, filter the detected contours and select the contours that meet the rectangular feature. Here, filter according to the area attribute of the contour. Here, the Shoelace formula is selected to calculate the area of the polygon. The Shoelace formula is based on the vertex coordinates of the polygon, decomposes the polygon into triangles, calculates the area of the triangles, and then adds these areas. The formula is as follows:
[0072]
[0073] Where n represents the number of sides of the polygon, (x i ,y i ) represents the coordinates of the th vertex of the polygon.
[0074] Set the coefficient λ=0.1. If the polygon contour area A>S, delete the contour, where S is the total area of the image.
[0075] Step 2.5, for the filtered contour, use the polygon approximation algorithm to approximate the contour into a polygon. We use the Douglas-Peucker algorithm to simplify the contour by retaining the key points that are far away in the contour. Select the point on the contour that is farthest from the original curve as the key point, and then split the curve into two sub-curves, which are recursively processed separately. Define a function d(P i ,P j ) represents point P i To line segment P1P n The vertical distance is:
[0076] d(P i ,P j )=|(P j -P1)×(P n -P1)| / |P n -P1|
[0077] Among them, P1 is the starting point, P j , P n is the endpoint of the resulting curve.
[0078] Select the starting point P1 and the ending point P on the curve n As the endpoint of the resulting curve. By iteratively calculating the vertical distance between each point and the starting point and the end point, find the point P with the maximum distance k If the maximum distance is greater than the given threshold, the current curve is converted to P k Cut and get two sub-curves. Recursively call the algorithm for the two sub-curves respectively until the maximum distance is less than the threshold or it is no longer possible to cut. Finally, connect the key points of each sub-curve to get the final approximate curve.
[0079] Step 2.6, for the polygon obtained by approximation, determine whether it is a rectangle by calculating the number of sides and angles of the polygon. First, the number of polygon sides is obtained according to the number of contour corner points, and polygons with sides not equal to 4 are deleted; then the inner angle of the polygon is calculated, and for the current contour corner point, the previous contour corner point and the next contour corner point are taken to form two vectors to calculate the inner angle.
[0080] Step 2.7, use the gradient descent method to fit the detected rectangle to optimize the corner points, side length, and angle of the rectangle. Use the minimization of the square distance between the rectangle and the contour as the optimization objective function to initialize the parameters, calculate the gradient, and update the parameters. Objective function:
[0081] ∑((x i -x c ) 2 +(y i -y c ) 2 )
[0082] Where (x i ,y i ) represents the coordinates of the th vertex of the polygon.
[0083] Step 3: Perform circle detection on the infrared and visible light binary edge images of the circuit board obtained in step 1, and extract circle features, including the center and radius of the circle;
[0084] like Figure 3 As shown, in a further preferred but non-limiting embodiment of the present invention, step 3 specifically includes:
[0085] Step 3.1, apply Hough circle transform to the edge image obtained by rectangle detection, and set the x coordinate representing the center of the circle, the y coordinate of the center of the circle, and the radius of the circle as a parameter space.
[0086] Step 3.2, traverse each edge point in the edge image, and perform the following operations on each edge point: for each possible combination of circle center and radius, calculate the circle center and radius according to the edge point. Perform accumulation operation at the corresponding position in the parameter space to increase the cumulative value of the combination.
[0087] Step 3.3, after the accumulation is completed, the detected circles are determined according to the set threshold. The threshold can control the detection sensitivity of the circle. A higher threshold can filter out weaker circle candidates, and a lower threshold can allow more circles to be detected.
[0088] In step 3.4, for each circle that meets the threshold, information such as the coordinates of the center and the radius can be extracted from the parameter space.
[0089] Step 4: Use the rectangle and circle information obtained in steps 2 and 3 to construct rectangle feature and circle feature descriptors;
[0090] In a further preferred but non-limiting embodiment of the present invention, step 4 specifically comprises:
[0091] Step 4.1, for each corner point of the rectangle, use its coordinates as part of the descriptor. In addition, for each corner point, find the two edges connected to it and calculate the angle and length of these two edges.
[0092] In step 4.2, the descriptor is normalized, the side length is normalized to a specific range [0, 1], and the angle is mapped to a specific range of 0 to π.
[0093] Step 4.3, combine the coordinates of the corner points, the angles of the connecting edges, and the length of the edges into a fixed-dimensional descriptor vector. The coordinates, angles, and lengths are connected together to form a longer descriptor vector
[0094] Step 5: Use bilateral matching to preliminarily match the infrared image and the visible light image to obtain the initial matching feature point pairs;
[0095] Step 6: Apply rotation angle estimation and connection line consistency theory to remove obvious mismatched feature point pairs, and select hundreds of iterations of the RANSAC algorithm to remove the remaining small number of mismatched feature point pairs;
[0096] In a further preferred but non-limiting embodiment of the present invention, step 6 specifically comprises:
[0097] Calculate the scale factor ξ between the infrared and visible images, which is equal to the ratio of the pixel areas of the same region in the infrared and visible images adjusted to the same vertical resolution. Convert P2 to the same scale and orientation using the following expression:
[0098]
[0099]
[0100] φ p is the rotation angle calculated in the rotation angle estimation, R is the rotation orthogonal matrix of the infrared and visible light images, its purpose is to rotate the visible light image to the same viewpoint of the infrared image, P2′ i is P2′ i Feature points after rotation and scaling transformation.
[0101] Step 7: Estimate the projection transformation matrix between infrared and visible light images using roughly matched feature point pairs;
[0102] Step 8: Use fine matching based on global iterative optimization to reduce the position deviation of feature points.
[0103] In a further preferred but non-limiting embodiment of the present invention, step 8 specifically comprises:
[0104] The projection transformation matrix M is calculated by the least squares method, and the formula is as follows:
[0105] minf=||P c1 -P c2 ||2
[0106] P c1 P c2 The matching points obtained by rough matching in step 7
[0107] The least squares solution of M is called:
[0108]
[0109] The fitting point corresponding to is expressed as:
[0110]
[0111] The residual of the sum is expressed as:
[0112] ε(i)=||P 2fit (i)-P c2 (i)||2,i=1,2,...
[0113] P c1 P c2 For the matching point pairs obtained by rough matching in step 7, the residual is iteratively optimized until it is smaller than the set threshold.
[0114] The present invention is beneficial in that:
[0115] Compared with the existing infrared image and visible light image registration based on point or straight line features, the present invention proposes a circuit board infrared image and visible light image registration method based on rectangular and circular features for a large number of rectangular heat-generating components and circular heat-free holes in the circuit board infrared image. The method extracts edge contours, closed contours, approximate polygons, and screens rectangles to obtain rectangular features, and constructs a unique descriptor for rectangular features; at the same time, it extracts edge contours, accumulates the center and radius, selects circular features, and constructs a unique descriptor for circular features. Compared with traditional point feature descriptors, our descriptors add information such as side length, center, radius, etc., which is more unique and avoids mismatching caused by similar feature point descriptors. In addition, in addition to using conventional matching algorithms such as rotation angle estimation, connection line consistency, and RANSAC algorithm in the stage of matching feature point pairs, our algorithm also adds a global iterative optimization algorithm for adjusting the position of feature points, which further improves the registration accuracy.
[0116] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.
Claims
1. A circuit board infrared image and visible light image registration method based on rectangular and circular features, characterized in that: The steps include: Step 1: Obtain the infrared image and visible light image of the circuit board taken at the same time, use the Canny operator to perform edge detection on the infrared and visible light images respectively, and obtain the infrared and visible light binary edge images of the circuit board, in which the edge pixels are marked as white and the non-edge pixels are marked as black; Step 2: Perform rectangle detection on the infrared and visible light binary edge images of the circuit board obtained in step 1, and extract rectangle features, including the four corner points, side lengths and angles of the rectangle; Step 3: Perform circle detection on the infrared and visible light binary edge images of the circuit board obtained in step 1, and extract circle features, including the center and radius of the circle; Step 4: Use the rectangle and circle information obtained in steps 2 and 3 to construct rectangle feature and circle feature descriptors; Step 5: Use bilateral matching to preliminarily match the infrared image and the visible light image to obtain the initial matching feature point pairs; Step 6: Apply rotation angle estimation and connection line consistency theory to remove obvious mismatched feature point pairs, and select hundreds of iterations of the RANSAC algorithm to remove the remaining small number of mismatched feature point pairs; Step 7: Estimate the projection transformation matrix between infrared and visible light images using roughly matched feature point pairs; Step 8: Use fine matching based on global iterative optimization to reduce the position deviation of feature points.
2. The circuit board infrared image and visible light image registration method based on rectangular and circular features according to claim 1 is characterized in that: The step 2 specifically includes: Step 2.1, initialize an empty contour list based on the edge image; Step 2.2, traverse each pixel of the edge image, when a white pixel is found, mark it as the current pixel, and start the contour tracing process. During the contour tracing process, determine how to move to the next pixel based on the connectivity of adjacent pixels and the boundary conditions in the edge image. This process will continue until it returns to the starting pixel and forms a closed contour. The formed contour is added to the contour list; Step 2.3, repeat step 2.2 until the entire edge image is traversed and all closed contours in the image are found and extracted; Step 2.4, filter the detected contours, select the contours that meet the rectangular features, filter according to the area attribute of the contours, and select the Shoelace formula to calculate the polygon area. The Shoelace formula is based on the vertex coordinates of the polygon, decomposes the polygon into triangles, calculates the area of the triangles, and then adds these areas. The formula is as follows: Where n represents the number of sides of the polygon, (x i ,y i ) represents the coordinates of the i-th vertex of the polygon; Set the coefficient λ = 0.
1. If the polygon contour area A>S, delete the contour, where S is the total area of the image; Step 2.5, for the filtered contour, use the polygon approximation algorithm to approximate the contour into a polygon, and use the Douglas-Peucker algorithm to simplify the contour by retaining the key points that are far away in the contour. Select the point on the contour that is farthest from the original curve as the key point, and then split the curve into two sub-curves, recursively process them respectively, and define a function d(P i ,P j ) represents point P i To line segment P1P n The vertical distance is: d(P i ,P j )=|(P j -P1)×(P n -P1)| / |P n -P1|; Select the starting point P1 and the ending point P on the curve n As the endpoint of the resulting curve, find the point P with the maximum distance by iteratively calculating the vertical distance between each point to the starting point and the end point. k If the maximum distance is greater than the given threshold, the current curve is converted to P k Cut to get two sub-curves, and recursively call the algorithm for the two sub-curves respectively until the maximum distance is less than the threshold, or it is no longer possible to cut. Finally, connect the key points of each sub-curve to get the final approximate curve; Step 2.6, for the polygon obtained by approximation, determine whether it is a rectangle by calculating the number of sides and angles of the polygon. First, the number of sides of the polygon is obtained according to the number of contour corner points, and the polygons with sides not equal to 4 are deleted; then the inner angle of the polygon is calculated. For the current contour corner point, the previous contour corner point and the next contour corner point are taken to form two vectors to calculate the inner angle; Step 2.7, use the gradient descent method to fit the detected rectangle to optimize the corner points, side length, and angle of the rectangle; use the minimization of the square distance between the rectangle and the contour as the optimization objective function to initialize the parameters, calculate the gradient, and update the parameters.
3. The circuit board infrared image and visible light image registration method based on rectangular and circular features according to claim 1 is characterized in that: The step 3 specifically includes: Step 3.1, applying Hough circle transform to the edge image obtained by rectangle detection, and setting the x coordinate representing the center of the circle, the y coordinate of the center of the circle, and the radius of the circle as a parameter space; Step 3.2, traverse each edge point in the edge image, and perform the following operations on each edge point: for each possible combination of circle center and radius, calculate the circle center and radius according to the edge point, and perform accumulation operation at the corresponding position in the parameter space to increase the cumulative value of the combination; Step 3.3, after the accumulation is completed, the detected circles are determined according to the set threshold. The threshold can control the detection sensitivity of the circle. A higher threshold can filter out weaker circle candidates, and a lower threshold can allow more circles to be detected; Step 3.4, for each circle that meets the threshold, extract information such as the center coordinates and radius from the parameter space.
4. The circuit board infrared image and visible light image registration method based on rectangular and circular features according to claim 1 is characterized in that: The step 4 specifically includes: Step 4.1, for each corner point of the rectangle, use its coordinates as part of the descriptor. In addition, for each corner point, find the two edges connected to it and calculate the angle and length of these two edges; Step 4.2, normalize the descriptor, normalize the side length to a specific range [0,1], and map the angle to a specific range of 0 to π; In step 4.3, the coordinates of the corner points, the angles of the connecting edges, and the edge length information are combined into a fixed-dimensional descriptor vector. The coordinates, angles, and edge lengths are connected together to form a longer descriptor vector.
5. The circuit board infrared image and visible light image registration method based on rectangular and circular features according to claim 1 is characterized in that: The step 6 specifically includes: Calculate the scale factor ξ between the infrared and visible light images, ξ is equal to the ratio of the pixel areas of the same region in the infrared and visible light images adjusted to the same vertical resolution, and convert P2 to the same scale and orientation by the following expression: φ p is the rotation angle calculated in the rotation angle estimation, R is the rotation orthogonal matrix of the infrared and visible light images, its purpose is to rotate the visible light image to the same viewpoint of the infrared image, P′ 2i is P′ 2i Feature points after rotation and scaling transformation.
6. The circuit board infrared image and visible light image registration method based on rectangular and circular features according to claim 1 is characterized in that: The step 8 specifically includes: The projection transformation matrix M is calculated by the least squares method, and the formula is as follows: minf=||P c1 -P c2 ||2 The least squares solution of M is called: Corresponding to P c2 The fitting points are expressed as: P 2fit and P c2 The residual is expressed as: ε(i) = ||P 2fit (i)-P c2 (i)||2,i=1,2,... P c1 P c2 For the matching point pairs obtained by rough matching in step 7, the residual is iteratively optimized until it is smaller than the set threshold.
Citation Information
Patent Citations
RGB-D registration precision test method and equipment
CN113256611A
Artificial neural network building texture mapping method and system based on sliding edge detection
CN114241024A
Heterogeneous Image Registration Method and System
US20220319011A1
