Method for detecting head height of bolt

Abstract

The invention discloses a method for detecting the head height of a bolt, and the method comprises the following steps: carrying out the projection of the head of the bolt through employing a structured light 3D camera, and obtaining the three-dimensional point cloud data of the head of the bolt; after abnormal points are processed, a bolt head area template is constructed; bolt head positioning is carried out on the bolt head area template and the actual detection area; selecting a plurality of local point cloud data in the bolt head area, and setting the local point cloud data as bolt head positions; selecting a plurality of local point cloud data on the outer side of the bolt head area, and setting the local point cloud data as bolt reference surface positions; and calculating the distance between the bolt head position and the bolt reference surface position by using a point-to-plane algorithm. The bolt head height detection method provided by the invention has the advantages of one-key operation, simple operation, fast measurement speed, automatic data and picture storage, and solving of pain points which cannot be traced by conventional contact measurement.

Description

Technical Field

[0001] The invention relates to the field of bolt installation detection, in particular to a method for detecting the head height of a bolt. Background Art

[0002] High-height locking bolts are critical connecting components in aerospace, railways, ships, automobiles, and other mechanical equipment. Accurately measuring the head height of these bolts, particularly in new energy copper busbar locks, is crucial for the safe operation of these equipment. Traditional mechanical contact detection methods require contact between the nut and the locked part to measure the gap. This is cumbersome and requires a certain level of operator skill. Each measurement is slow, taking tens of seconds, resulting in low detection efficiency, poor structural stability, and a lack of traceability. Furthermore, in practice, this method can damage the bolt surface and limits detection accuracy, making it difficult to meet the requirements of industrial automation and quality control.

[0003] With the development of computer vision technology, non-contact inspection methods based on image processing have become a research hotspot. Due to the reflective properties of the head surface of high-lock bolts, as well as the nonlinear coupling effects of the bolt's tilt angle during installation and the lighting environment, current visual inspection methods often suffer from large detection errors and poor robustness when handling these complex scenarios. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for detecting the head height of a bolt with one-button operation, easy operation, fast measurement speed, automatic data and image saving, and solving the pain point of the previous contact measurement that cannot be traced.

[0005] The technical solution adopted by the present invention to solve the above problem is a method for detecting the head height of a bolt, the steps of which are as follows: S1: Use a structured light 3D camera to project the bolt head to obtain the 3D point cloud data of the bolt head; S2: After processing abnormal points based on the 3D point cloud data of the bolt head, a template of the bolt head area is constructed; S3: Use image template matching method to locate the bolt head by comparing the bolt head area template with the actual detection area; S4: After the bolt head is positioned, multiple local point cloud data are selected in the bolt head area and set as the bolt head position; S5: Select multiple local point cloud data outside the bolt head area and set them as the bolt reference plane position; S6: Calculates the distance between the bolt head position and the bolt reference plane position using the point-to-plane algorithm.

[0006] Compared with the existing technology, the advantages of the present invention are: first, the present invention uses a structured light 3D camera to project the bolt head to obtain three-dimensional point cloud data of the bolt head, thereby avoiding the problems of low efficiency, poor stability, high operation requirements, and damage to the bolt in contact detection; second, in step S2, abnormal point data is screened out, and the three-dimensional reconstruction data of the bolt head is regenerated, and the data extracted in step S1 is not directly used, thereby avoiding the problems of large detection errors and poor robustness caused by abnormal point cloud data in traditional visual technology; and then in step S3, the deviation between the three-dimensional reconstruction data of the bolt head and the actual detection area is re-corrected, thereby eliminating The positioning error problem caused by abnormal points in the actual detection area can be solved, and point cloud data can be selected from the 3D reconstruction data of any bolt head for calculation. There is no need to worry about selecting abnormal point data from the 3D point cloud data of the bolt head, which will cause serious deviation in the calculation results; finally, the distance between the bolt head position and the bolt reference plane position is calculated through steps S4-S6, and the installation status of the bolt is judged according to the actual height of the bolt head to ensure the stability and firmness of the bolt fixed connection; during the entire detection process, all data and pictures are saved, so that they can be traced back in the subsequent verification process to ensure the safety of the bolt installation.

[0007] As an improvement of the present invention, step S2 includes step S2.1: using a flying point removal algorithm to screen out flying points, the flying point removal algorithm selects any point in the off-center area in the three-dimensional point cloud data of the bolt head as a reference point, and then selects a screening range based on the reference point, and judges that the points that are not close to the reference point are flying points based on the judgment standard. Through the improvement, in the three-dimensional point cloud data of the bolt head, a flying point group will be formed due to the reflective characteristics of the bolt head surface and the nonlinear coupling influence of the inclination angle during bolt installation and the lighting environment. The flying point group will cause serious judgment errors of the computer, so it is necessary to screen out the flying point group to ensure the judgment accuracy of the computer, and then ensure the accuracy of the detection.

[0008] As an improvement of the present invention, the reference parameters of the flying point removal algorithm include a proximity threshold, a proximity number, and a slope. The proximity threshold is the distance between the flying point to be detected and the reference point, the proximity number is the number of flying points to be detected, and the slope is the slope between the flying point to be detected and the reference point. Through the improvement, the reference parameters of the proximity threshold, the proximity number, and the slope are used to screen out obviously unreasonable flying point groups for elimination.

[0009] As an improvement of the present invention, the parameters of the flying point removal algorithm also include suppressing narrow edges, and the narrow edge suppression is to reduce or remove the points on the narrow edges or edge parts in the three-dimensional point cloud data of the bolt head. Through the improvement, the points on the narrow edges or edge parts may be inaccurate or unstable due to the edge effect of the sensor, incomplete scanning or other reasons. These points may introduce noise or affect the geometric shape analysis of the point cloud. Therefore, by removing the points on the narrow edges or edge parts, the instability and noise in the point cloud data can be reduced, and the stability and accuracy of the point cloud data can be improved.

[0010] As an improvement of the present invention, a smoothing algorithm is used to remove noise points. The smoothing algorithm uses point cloud data within a selected range to average to smooth the influence of noise points. Through the improvement, the influence of noise points on detection is eliminated, and it is avoided that noise points are selected when selecting the bolt head position, thereby affecting the accuracy of the detection results. By placing the smoothing algorithm after the flying point removal method, the flying point removal method can be used first to screen out noise points with obvious errors in the neighboring threshold, neighboring number and slope, thereby reducing the smoothing operation on obviously unreasonable noise points, improving the smoothing efficiency, and retaining the real height jump area.

[0011] As an improvement of the present invention, step S2.3 is also included: using a filling method to process the voids, the filling method includes an interpolation method and an expansion method, the interpolation method is to estimate the value through the valid points around the invalid points of the void to fill the invalid data of the void, the expansion method is to expand the boundary of the three-dimensional point cloud data of the bolt head outward to fill the void, the interpolation method is applicable to the horizontal area, and the expansion method is applicable to the vertical edge area. Through the improvement, the voids are filled, the influence of the voids on the detection is eliminated, and the voids are avoided from being selected when selecting the bolt head position, thereby affecting the accuracy of the detection results; wherein different filling methods are used for different areas of the three-dimensional point cloud data of the bolt head. If the expansion method is used for the horizontal area, the expanded cloud point data is likely to cause errors in the bolt head height detection. At the same time, the vertical edge area of ​​the bolt head is arc-shaped. If the interpolation method is used, the reference accuracy of the interpolation is low, and it cannot meet the accuracy after filling.

[0012] As an improvement of the present invention, it also includes step S2.4: after completing the processing of the bolt head data, the three-dimensional reconstruction data of the bolt head is obtained, and by constructing a bolt head area template, when the bolt head has a position offset during the actual detection process, the image template matching method can be used to locate the bolt head. Through the improvement, the positioning of the bolt head is corrected. When obtaining the three-dimensional point cloud data of the bolt head, there are many abnormal points, so the original positioning deviation is large. After eliminating various abnormal points, repositioning is performed to obtain the three-dimensional reconstruction data of the bolt head in the exact position, thereby ensuring the accuracy of the subsequent bolt head height calculation.

[0013] As an improvement of the present invention, the image template matching method is based on the NCC (Normalized Cross-Correlation) normalized cross-correlation method, and its formula is as follows: ; Where: T(x,y) is the pixel value of the template image; I(x+u,y+v) is the pixel value of the target image at position (u,v); is the template image mean, ; is the mean of the window corresponding to the target image at position (u, v), Through the improvement, the correction between the three-dimensional reconstruction data of the bolt head and the actual detection position is achieved.

[0014] As an improvement of the present invention, in step S4, after locating the bolt head area image, four rectangular frames are used to select local area point cloud data of four uniform arrays on the circumference of the bolt head, which are set as the bolt head position A. The bolt head position A selects the maximum value, minimum value or mean value of the point cloud data. Through the improvement, the point cloud data of the bolt head position A is selected.

[0015] As an improvement of the present invention, in step S5, four more rectangular boxes are used to select point cloud data of four uniformly arrayed local areas on the outer circumference of the bolt head area, which are set as the bolt reference plane B. The bolt reference plane B is fitted with a multi-point plane using the least squares method to obtain a plane. The distance from the bolt head position A to the bolt reference plane B is calculated using the point-to-plane algorithm to obtain the minimum, maximum, and mean values ​​of the heights of the four areas. The height means of the four areas are then averaged to obtain the bolt head height data. The distance formula between the bolt head position A and the bolt reference surface B is: ; The plane equation in the formula is Ax+By+Cz+D=0; The coordinates of the bolt head position A (x0, y0, z0); It is the distance from the bolt head position A to the bolt reference plane B. Through the improvement, the four area distances from the bolt head position A to the bolt reference plane B are used to judge the installation status of the bolt. If the four distances are equal or the deviation is within the allowable error range, and the distance is equal to the thickness of the bolt head, it proves that the bolt is installed in place; if the four distances are equal or the deviation is within the allowable error range, and is greater than the thickness of the bolt head, it proves that the bolt is not installed in place; if there is a significant difference between the four distances, it proves that the bolt is tilted during the installation process and the installation quality is poor. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a process flow chart of the present invention.

[0017] Figure 2 It is a three-dimensional selected image of the bolt head position and the bolt reference surface of the present invention.

[0018] 1. Bolt head position A, 2. Bolt reference surface B. DETAILED DESCRIPTION

[0019] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0020] like Figure 1 As shown, a method for detecting the head height of a bolt, the steps are as follows: S1: Use a structured light 3D camera to project the bolt head to obtain the 3D point cloud data of the bolt head; S2: After processing abnormal points based on the 3D point cloud data of the bolt head, a template of the bolt head area is constructed; S2.1: Use the flying spot removal algorithm to filter out flying spots; S2.2: Use smoothing algorithm to remove noise; S2.3: Use the filling method to deal with the voids; S2.4: After the bolt head data is processed, the three-dimensional reconstruction data of the bolt head is obtained and a bolt head area template is constructed; S3: Use image template matching method to locate the bolt head by comparing the bolt head area template with the actual detection area; S4: After locating the bolt head area image, use four rectangular boxes to select the local area point cloud data of four uniform arrays on the circumference of the bolt head, and set it as the bolt head position A1. The bolt head position A1 selects the maximum value, minimum value or mean value of the point cloud data; S5: Use another four rectangular boxes to select the point cloud data of four uniform arrays of local areas on the outer circumference of the bolt head area, and set them as the bolt reference plane B2. The bolt reference plane B2 is obtained by multi-point fitting using the least squares method; S6: Use the point-to-plane algorithm to calculate the distance from the bolt head position A1 to the bolt reference plane B2. The minimum, maximum, and mean heights of the four regions can be obtained. The mean heights of the four regions are then averaged to obtain the bolt head height data.

[0021] The distance formula between the bolt head position A1 and the bolt reference surface B2 is: ; The plane equation in the formula is Ax+By+Cz+D=0; The coordinates of the bolt head position A1 (x0, y0, z0); It is the distance from the bolt head position A1 to the bolt reference surface B2.

[0022] like Figure 2 As shown, the distances between the bolt head position A1 and the bolt reference surface B2 are used to determine the bolt's installation status. The red area represents the point cloud data of the local area of ​​the bolt head position A1, and the average of the four local area point cloud data of the bolt head position A1 is calculated. The blue area represents the point cloud data of the local area of ​​the bolt reference surface B2, and the average of the four local area point cloud data of the bolt reference surface B2 is calculated. One calculation method is to average the four local area point cloud data of the bolt head position A1, and then calculate the distance between the final average value of the bolt head position A1 and the average value of the four local area point cloud data of the bolt reference surface B2. Another calculation method is to calculate the distance between the average value of the four local area point cloud data of the bolt head position A1 and the average value of the four local area point cloud data of the bolt reference surface B2. If the four distances are equal or the deviation is within the allowable error range, and the distance is equal to the thickness of the bolt head, the bolt is properly installed. If the four distances are equal or the deviation is within the allowable error range and is greater than the thickness of the bolt head, the bolt is not properly installed. If there is a significant difference between the four distances, it indicates that the bolt has tilted during installation and the installation quality is poor.

[0023] Among them, the least squares method is a conventional algorithm.

[0024] The flying point removal algorithm selects any point in the off-center area of ​​the bolt head three-dimensional point cloud data as a reference point, and then selects a screening range based on the reference point, and judges the points that are not close to the reference point as flying points based on the judgment criteria. The reference parameters of the flying point removal algorithm include a proximity threshold, a proximity number, and a slope. The proximity threshold is the distance between the flying point to be detected and the reference point, the proximity number is the number of flying points to be detected, and the slope is the slope between the flying point to be detected and the reference point. The three reference parameters of the proximity threshold, the proximity number, and the slope are used to filter out flying points by detecting unreasonable data of flying points; the parameters of the flying point removal algorithm also include narrow edge suppression, which is to reduce or remove points on the narrow edge or edge part of the three-dimensional point cloud data of the bolt head. Suppressing narrow edges is to improve the accuracy of detection by reducing or removing flying points that are prone to cause detection errors.

[0025] The smoothing algorithm uses the point cloud data within the selected range to average to smooth the influence of noise points. The larger the selection range window, the larger the range of points involved in the average, and the smaller the influence of the smoothed noise points. The influence of noise points on the detection is eliminated to avoid selecting noise points when selecting the bolt head position, thereby affecting the accuracy of the detection results. If the smoothing algorithm is placed after the flying point removal method, the flying point removal method can be used first to screen out noise points with obvious errors in the neighboring threshold, neighboring number and slope, thereby reducing the smoothing operation on obviously unreasonable noise points, improving the smoothing efficiency, and retaining the real height jump area. The height jump area refers to the area with a significant difference in height.

[0026] The filling method includes interpolation and expansion. The interpolation method is to estimate the invalid data of the hole through the valid points around the invalid points of the hole. The expansion method is to expand the boundary of the three-dimensional point cloud data of the bolt head outward to fill the hole. The interpolation method is applicable to the horizontal area, and the expansion method is applicable to the vertical edge area. The hole is filled to eliminate the influence of the hole on the detection, and the hole is avoided from being selected when the bolt head position is selected, thereby affecting the accuracy of the detection result. Different filling methods are used for different areas of the three-dimensional point cloud data of the bolt head. If the expansion method is used in the horizontal area, the expanded cloud point The data is prone to errors in bolt head height detection. Furthermore, the vertical edge of the bolt head is curved. If interpolation is used, the interpolation reference accuracy is low and the accuracy of the infilled data cannot be met, making it difficult to ensure that the infilled point cloud data meets the arc standard. Placing the infilling method after the flying point removal method not only prevents the infilling method from processing obviously unreasonable flying point clusters, which affects subsequent inspection quality and work efficiency, but also reduces or removes errors caused by narrow edge suppression. The two complement each other and verify each other, thus ensuring the integrity of the subsequent 3D reconstruction data of the bolt head. Because the bolt edge is annular, the 3D point cloud data of the bolt head appears jagged after magnification. When narrow edge suppression is performed, the jagged point cloud data will be reduced or removed. Using the edge line as the boundary, if the proportion of jagged point cloud data within the edge line is much greater than that outside the edge line, this jagged point cloud data will cause edge holes in the 3D reconstruction data of the bolt head. In this case, the infilling method is used to fill these edge holes to ensure the integrity of the 3D reconstruction data of the bolt head.

[0027] The interpolation methods include linear interpolation, Lagrange interpolation and Newton interpolation. Linear interpolation, Lagrange interpolation and Newton interpolation are all conventional interpolation methods.

[0028] The image template matching method is based on the NCC (Normalized Cross-Correlation) normalized cross-correlation method, and its formula is as follows: ; Where: T(x,y) is the pixel value of the template image; I(x+u,y+v) is the pixel value of the target image at position (u,v); is the template image mean, ; is the mean of the window corresponding to the target image at position (u, v), , so that the 3D reconstruction data of the bolt head can be located with the actual position during the inspection process.

[0029] The design of a bolt head height detection method has the following advantages: 1. It avoids the problems of low efficiency, poor stability, high operating requirements and damage to bolts caused by contact detection; 2. During the entire testing process, all data and pictures are saved, so that they can be traced during the subsequent verification process to ensure the firmness, integrity and safety of the bolt installation; 3. Greatly improved the accuracy of detection; 4. Simple operation and low requirements on operators; 5. High efficiency.

[0030] The above description is merely a description of the preferred embodiment of the present invention and is not to be construed as limiting the scope of the claims. The present invention is not limited to the above embodiment, and variations in the specific structure are permitted. Any variations within the scope of the independent claims of the present invention are also within the scope of protection of the present invention.

Claims

1. A method for detecting the head height of a bolt, characterized in that: The steps are as follows: S1: Use a structured light 3D camera to project the bolt head to obtain the 3D point cloud data of the bolt head; S2: After processing abnormal points based on the 3D point cloud data of the bolt head, a template of the bolt head area is constructed; S3: Use image template matching method to locate the bolt head by comparing the bolt head area template with the actual detection area; S4: After the bolt head is positioned, multiple local point cloud data are selected in the bolt head area and set as the bolt head position; S5: Select multiple local point cloud data outside the bolt head area and set them as the bolt reference plane position; S6: Calculates the distance between the bolt head position and the bolt reference plane position using the point-to-plane algorithm.

2. A method for detecting the head height of a bolt according to claim 1, characterized in that: Step S2 includes step S2.1: using a flying point removal algorithm to filter out flying points. The flying point removal algorithm selects any point in the off-center area in the three-dimensional point cloud data of the bolt head as a reference point, and then selects a screening range based on the reference point, and judges that points that are not close to the reference point are flying points based on the judgment criteria.

3. The method for detecting the head height of a bolt according to claim 2, wherein: The reference parameters of the flying spot removal algorithm include a proximity threshold, a proximity number, and a slope. The proximity threshold is the distance between the flying spot to be detected and the reference point. The proximity number is the number of flying spots to be detected. The slope is the slope between the flying spot to be detected and the reference point.

4. The method for detecting the head height of a bolt according to claim 2, wherein: The parameters of the flying point removal algorithm also include suppressing narrow edges, which is to reduce or remove points on narrow edges or edge parts in the three-dimensional point cloud data of the bolt head.

5. The method for detecting the head height of a bolt according to claim 2, wherein: The method further includes step S2.2: using a smoothing algorithm to remove noise, wherein the smoothing algorithm uses point cloud data within a selected range to average to smooth the influence of noise.

6. A method for detecting the head height of a bolt according to claim 5, characterized in that: It also includes step S2.3: using a filling method to process the void, the filling method includes an interpolation method and an expansion method, the interpolation method is to estimate the valid points around the invalid points of the void to fill the invalid data of the void, the expansion method is to expand the boundary of the three-dimensional point cloud data of the bolt head outward to fill the void, the interpolation method is applicable to horizontal areas, and the expansion method is applicable to vertical edge areas.

7. A method for detecting the head height of a bolt according to claim 6, characterized in that: The method also includes step S2.4: after completing the processing of the bolt head data, obtaining the three-dimensional reconstruction data of the bolt head, and constructing a bolt head area template. When the bolt head has a position offset during the actual detection process, the image template matching method can be used to locate the bolt head.

8. A method for detecting the head height of a bolt according to claim 7, characterized in that: The image template matching method is based on the NCC (Normalized Cross-Correlation) normalized cross-correlation method, and its formula is as follows: ； Where: T(x,y) is the pixel value of the template image; I(x+u,y+v) is the pixel value of the target image at position (u,v); is the template image mean, ; is the mean of the window corresponding to the target image at position (u, v), .

9. The method for detecting the head height of a bolt according to claim 1, wherein: In step S4, after locating the bolt head area image, four rectangular frames are used to select the local area point cloud data of four uniform arrays on the circumference of the bolt head, which are set as the bolt head position A (1). The bolt head position A (1) selects the maximum value, minimum value or mean value of the point cloud data.

10. A method for detecting the head height of a bolt according to claim 9, characterized in that: In step S5, four other rectangular frames are used to select the local area point cloud data of four uniform arrays on the outer circumference of the bolt head area, which are set as the bolt reference plane B (2). The bolt reference plane B (2) is fitted with multiple points using the least squares method to obtain a plane; in step S6, the distance from the bolt head position A (1) to the position of the bolt reference plane B (2) is calculated using the point-to-plane algorithm, and the minimum, maximum and average values ​​of the height of the four areas can be obtained. Then, the average values ​​of the height of the four areas are taken to obtain the bolt head height data; The distance formula between the bolt head position A (1) and the bolt reference surface B (2) is: ; The plane equation in the formula is Ax+By+Cz+D=0; The coordinates of the bolt head position A (1) (x0, y0, z0); The distance from the bolt head position A (1) to the bolt reference surface B (2).

Images