Part defect monitoring method based on three-dimensional point cloud data

By constructing a 3D structured light vision system and a three-dimensional reconstruction algorithm, automatic non-contact monitoring of key components of large mechanical equipment is achieved, safety hazards and real-time problems of manual inspection are solved, and the safety and reliability of equipment are improved .

CN118883554APending Publication Date: 2024-11-01NAVAL UNIV OF ENG PLA

Patent Information

Application Number
CN202410774056.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

In the prior art, monitoring of key components (such as sliders and bead blockers) in large mechanical equipment mainly relies on manual inspection, which poses safety hazards, cannot be tested in real time, is expensive and affects the normal operation of the equipment.

Method used

The part defect monitoring method based on three-dimensional point cloud data is adopted, and the 3D structured light vision system is constructed, the point cloud data is obtained using the three-dimensional reconstruction algorithm, and abnormal defects are detected through the point cloud defect detection algorithm to realize automatic contactless monitoring.

Benefits of technology

It realizes high-precision detection and positioning of key components, improves the safety and reliability of equipment operation, can monitor weak defects during equipment operation in real time, reduces hardware costs, and improves the degree of automation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118883554A_ABST
    Figure CN118883554A_ABST
Patent Text Reader

Abstract

The invention discloses a part defect monitoring method based on three-dimensional point cloud data. The part defect monitoring method comprises the steps of constructing a 3D structured light vision system, performing three-dimensional reconstruction on a to-be-detected area of a part by using a three-dimensional reconstruction algorithm to obtain to-be-detected point cloud data, and detecting whether abnormal defects exist in the to-be-detected area of the part or not through a point cloud defect detection algorithm. The part defect detection and positioning precision is high; the non-contact measurement method does not influence the normal operation state of the equipment; the real-time performance is high, the sensitivity is high, key vulnerable parts can be monitored in the equipment operation process, and weak defects of the parts in the equipment operation process can be automatically detected in time; the method is high in automation degree, low in hardware cost and high in titer ratio, and has a wide application prospect in the field of industrial system maintenance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of artificial intelligence and equipment maintenance, and specifically relates to a part defect monitoring method based on three-dimensional point cloud data. Background Art

[0002] Key components such as sliders and bead retainers in large-scale mechanical equipment are subject to long-term high-load operation, strong vibration and impact, and are prone to deformation and damage. Once key components are deformed and damaged, it may lead to mission failure at the least, equipment damage or even casualties at the worst. Therefore, it is very necessary to monitor these vulnerable parts, detect and handle faults in a timely manner, and improve the safety and reliability of equipment operation.

[0003] Currently, the detection of whether key components have failed mainly relies on manual inspections, which have great limitations and defects: inspections require shutdown to ensure personnel safety, and it is impossible to monitor running equipment; manual inspections are usually regular inspections, which make it difficult to detect faults and issue alarms in a timely manner; manual inspections rely on the professional quality and physical and mental state of the staff, the working environment is relatively poor, which increases the labor intensity of the staff and poses many safety hazards. Summary of the invention

[0004] In order to overcome the defects of the above-mentioned background technology, the present invention provides a part defect monitoring method based on three-dimensional point cloud data, aiming to realize automatic non-contact monitoring of vulnerable parts such as sliders and bead stoppers in large-scale mechanical equipment, timely detect failure anomalies of key components and issue alarms, thereby improving the safety and reliability of equipment operation.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for monitoring part defects based on three-dimensional point cloud data, comprising:

[0007] Step 1: Construct a 3D structured light vision system;

[0008] Step 2: Use a 3D reconstruction algorithm to perform 3D reconstruction on the area to be inspected to obtain point cloud data to be inspected;

[0009] Step 3: Use the point cloud defect detection algorithm to detect whether there are abnormal defects in the area to be inspected.

[0010] Preferably, the method of constructing a 3D structured light vision system in step 1 includes:

[0011] Step 11, using a structured light source to project RGB color Gray code structured light onto the surface of the part to be inspected;

[0012] Step 12, obtaining a coded image through a camera;

[0013] Step 13, complete the system calibration, and complete the three-dimensional reconstruction of the surface to be inspected of the component according to the system calibration result.

[0014] Preferably, in step 13, the system calibration is completed. According to the system calibration result, the method for completing the three-dimensional reconstruction of the surface to be inspected of the component includes: using a camera calibration method based on a 2D plane checkerboard, using a Matlab camera calibration tool to complete the calibration of camera parameters,

[0015] First, the laser projects a structured light plane onto a flat chessboard to form laser stripes, and the laser stripe image is captured by a CCD camera;

[0016] Then, the image is processed, and four arbitrary laser stripes on the laser stripe image are selected, and four characteristic points are obtained respectively according to the four laser stripes;

[0017] Finally, the four feature points on the structured light plane are used to determine the parameters of the structured light plane equation.

[0018] Preferably, the method for selecting any four laser stripes on the laser stripe image includes: controlling the laser to move from two calibrated positions to another two calibrated positions to obtain two laser stripes, and repeating the above steps to obtain another two laser stripes.

[0019] Preferably, the method of determining the parameters of the structured light plane equation using four feature points on the structured light plane includes: converting the image coordinates of the four laser stripe feature points into the coordinates of the four laser stripe feature points in the camera coordinate system, and obtaining the structured light plane equation using the least squares method.

[0020] Preferably, in step 2, the method of using a 3D reconstruction algorithm to perform 3D reconstruction on the area to be inspected of the component to obtain the point cloud data to be inspected comprises:

[0021] Step 21, the structured light source projects the coded pattern onto the surface of the part to be reconstructed, and solves the equation of the straight line corresponding to each control point at the intersection of the projection light and the measured object;

[0022] Step 22, after the calibration in step 13 is completed, the conversion relationship between the reference coordinate systems of the camera and the projector and the defined world coordinate system is obtained, and the three-dimensional coordinates of the control point M in the world coordinate system are calculated by coordinate conversion, so as to obtain the three-dimensional point cloud data of the object to be inspected;

[0023] Preferably, the specific method for solving the equation of the straight line corresponding to each control point at the intersection of the projection light and the measured object in step 21 includes:

[0024] Obtain the pixel coordinates of each control point, and solve the equation of the line connecting the coordinates of the projection point of each control point on the normalized imaging plane of the camera in the camera coordinate system and the optical center of the camera.

[0025] P={pln'(pq,)=0}

[0026]

[0027] Among them, n T is the normal vector of the light plane, q p For M p The coordinates of q L M c The coordinates of the camera ray L c The direction vector of the ray is κ, and the coefficient of the ray is in the real number range. The projector light plane P and the corresponding camera ray L c The intersection point solution is shown as follows:

[0028]

[0029] Preferably, the specific method of step 3 detecting whether there are abnormal defects in the area to be inspected of the component by using a point cloud defect detection algorithm includes:

[0030] Step 31, read the point cloud data, use the filtering method based on voxel downsampling to pre-process the point cloud, voxelize the three-dimensional space, and then calculate the centroid of the non-empty voxel to replace all the points in the voxel. The specific method includes: creating voxels: calculating the bounding box of the point cloud, and then discretizing the bounding box into small voxels; the length, width and height of the voxel can be set manually, or by setting the number of grid points in the three directions of the bounding box. Each small voxel contains a number of points, and the center point or the point closest to the center point is taken as the sampling point;

[0031] Step 32: Calculate the point histogram feature PFH of the point cloud, that is, set a fixed neighborhood in the point cloud based on experience or experiments, then calculate several eigenvalues ​​in the neighborhood, and establish a histogram for the eigenvalues ​​to obtain statistical features;

[0032] Step 33: Point cloud registration;

[0033] Step 34: Calculate the distance between the current point cloud data and the template point cloud data. The specific method includes:

[0034] In the choice of distance formula, the most common actual distance from point to surface is selected, which measures the absolute distance from a point to a surface in multidimensional space. This distance can directly reflect the distance from the point to the surface. The calculation formula is as follows:

[0035]

[0036] The equation of the plane is ax+by+cz+d=0, and the coordinates of the point in space are (x 0 ,y 0 , z0 ), a, b and c are measured by initial calibration.

[0037] Preferably, step 32: calculating the point histogram feature (PFH) of the point cloud, that is, setting a fixed neighborhood in the point cloud based on experience or experiment, and then calculating several eigenvalues ​​in the neighborhood, establishing a histogram for the eigenvalues, and obtaining the statistical features includes:

[0038] First, define a local coordinate system, let D psi Point is the point where PFH needs to be calculated. pk1 The point is a point in its neighborhood. Based on the normal vector between the two lines, the local coordinate system is defined. The calculation method of the three axes u, v and w of the local coordinate is as follows:

[0039] u=n s

[0040]

[0041] w=u·v

[0042] n s D pk1 The normal of the point is obtained based on the calculation formula of the three axes of the local coordinates. pk1 The local coordinate system of D is calculated based on the local coordinate system. psi Point and D pk1 The triplet feature representation between points is:

[0043] α=y·n t

[0044]

[0045] θ=arctan(w·n t ,u·n t )

[0046] Got D psi is the source point, for D pk1 The triple feature representation of α, φ and θ, PFH calculates the triple features of all point pairs in the neighborhood of any point in the point cloud data, and based on these features, a histogram representation is established to obtain the PFH feature representation;

[0047] Preferably, step 33 uses a double iterative closest point algorithm to perform point cloud precise registration, and the specific method includes:

[0048] The first iteration is to first obtain the coarse registration source point cloud P ci And the coarse registration target point cloud Q cj Perform ICP precise registration and record the calculated rotation matrix as Rc , the calculated translation matrix is ​​recorded as T c , the specific steps are:

[0049] (1) Set the termination condition as:

[0050] ① The transformed coarse registration source point cloud P ci Roughly align the target point cloud Q cj The average distance is less than the adaptive convergence threshold e thr As shown in the formula, N is the overlap rate, that is, the ratio of the number of overlapping points in the point cloud to the total number of point clouds; L r is the horizontal resolution, R e is the systematic measurement error;

[0051]

[0052] ②The number of registration iterations has reached the set maximum value N ε Otherwise, the transformed P ci Continue to iterate as a new coarsely registered source point cloud;

[0053] (2) Search for the coarse registration target point cloud Q cj Each point in the coarsely registered source point cloud P ci The nearest point in the , set the maximum threshold N of the number of iterations ε , and at the same time calculate the overlap rate N;

[0054] (3) Calculate the rigid body transformation (R) corresponding to the minimum average distance between the nearest points c , T c );

[0055] (4) Use the parameters obtained in step (2) to calculate P ci Perform rotation and translation to obtain a new set of transformed points;

[0056] (5) Repeat steps (2) to (4) until the termination condition is reached. Otherwise, the transformed P ci Continue to iterate as a new coarsely registered source point cloud;

[0057] The first iteration is to precisely align the feature points of the source point cloud and the target point cloud. The source point cloud can still be aligned with the target point cloud after being transformed by the calculated rotation and translation matrix, so the second iteration is performed. The rotation and translation parameters after the alignment are substituted into the original source point cloud for calculation to obtain the new source point cloud data. Finally, the same steps as the first iteration are used again to perform ICP precise alignment between it and the target point cloud.

[0058] The beneficial effects of the present invention are: high accuracy in part defect detection and positioning; the non-contact measurement method does not affect the normal operation of the equipment; strong real-time performance and high sensitivity, capable of monitoring key vulnerable parts during equipment operation, and capable of timely and autonomously detecting slight defects in parts during equipment operation; high degree of automation, low hardware cost, and high cost-effectiveness, and has broad application prospects in the field of industrial system maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a schematic diagram of the method of an embodiment of the present invention.

[0060] Figure 2 This is a flow chart of target deformation detection based on structured light 3D reconstruction according to an embodiment of the present invention.

[0061] Figure 3 This is a flow chart of a three-dimensional damage and deformation detection algorithm according to an embodiment of the present invention.

[0062] Figure 4 This is a graph showing the results of a three-dimensional damage detection experiment according to an embodiment of the present invention.

[0063] Figure 5 This is a diagram showing the experimental results of three-dimensional deformation detection according to an embodiment of the present invention.

[0064] Figure 6 This is a diagram of the three-dimensional deformation detection experimental results of an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0066] A method for monitoring part defects based on three-dimensional point cloud data, comprising:

[0067] Step 1: constructing a 3D structured light vision system. The method of constructing a 3D structured light vision system includes:

[0068] Step 11, using a structured light source to project RGB color Gray code structured light onto the surface of the part to be inspected;

[0069] Step 12, obtaining a coded image through a camera;

[0070] Step 13, complete the system calibration, and complete the three-dimensional reconstruction of the surface to be inspected of the component according to the system calibration result.

[0071] Step 13, completing the system calibration. According to the system calibration result, the method for completing the 3D reconstruction of the surface to be inspected of the component includes: adopting a camera calibration method based on a 2D plane checkerboard, using a Matlab camera calibration tool to complete the calibration of camera parameters,

[0072] First, the laser projects a structured light plane onto a flat chessboard to form laser stripes, and the laser stripe image is captured by a CCD camera;

[0073] Then, the image is processed, and any four laser stripes on the laser stripe image are selected, and four feature points are obtained respectively based on the four laser stripes; the method for selecting any four laser stripes on the laser stripe image includes: controlling the laser to move from two calibration positions to another two calibration positions to obtain two laser stripes, and repeating the above steps to obtain another two laser stripes.

[0074] Finally, the parameters of the structured light plane equation are determined using the four feature points on the structured light plane. The method for determining the parameters of the structured light plane equation using the four feature points on the structured light plane includes: converting the image coordinates of the four laser stripe feature points into the coordinates of the four laser stripe feature points in the camera coordinate system, and obtaining the structured light plane equation using the least square method.

[0075] This method can complete the calibration of the structured light plane equation parameters during the camera parameter calibration process. At the same time, this method does not require the processing of high-precision three-dimensional calibration targets or manual measurement, and the calibration process is simple and practical.

[0076] Step 2: Use a 3D reconstruction algorithm to perform 3D reconstruction on the area to be inspected to obtain the point cloud data to be inspected, which specifically includes the following steps:

[0077] Step 21, the structured light source projects the coded pattern onto the surface of the part to be reconstructed, and solves the equation of the straight line corresponding to each control point at the intersection of the projection light and the object to be measured; the specific method includes:

[0078] Obtain the pixel coordinates of each control point, and solve the equation of the line connecting the coordinates of the projection point of each control point on the normalized imaging plane of the camera in the camera coordinate system and the optical center of the camera.

[0079] P={p|n T (pq p )=0}

[0080]

[0081] Among them, n T is the normal vector of the light plane, q p For M p The coordinates of q L M c The coordinates of the camera ray L c The direction vector of the ray is κ, and the coefficient of the ray is in the real number range. The projector light plane P and the corresponding camera ray L c The intersection point solution is shown as follows:

[0082]

[0083] Step 22, after the calibration in step 13 is completed, the conversion relationship between the reference coordinate systems of the camera and the projector and the defined world coordinate system is obtained, and the three-dimensional coordinates of the control point M in the world coordinate system are calculated by coordinate conversion, so as to obtain the three-dimensional point cloud data of the object to be inspected;

[0084] By intersecting the structured light plane P with the image plane on the surface of the object to be measured and the corresponding camera ray L c Intersect at point M. In the structured light system, the origin of the projector's reference coordinate system is O p , the origin of the camera's reference coordinate system is O c In an ideal state, it is assumed that a point M on the intersection of the light plane and the projector image plane p , click M c For ray L c The projection on the camera image plane is M, O p 、M p In the same optical plane, M, O c 、M c On the same ray.

[0085] Step 3: Use the point cloud defect detection algorithm to detect whether there are abnormal defects in the area to be inspected. The specific method includes:

[0086] Step 31, read the point cloud data, and use a filtering method based on voxel downsampling to preprocess the point cloud. Voxel downsampling is a kind of filtering of the point cloud, the purpose of which is to reduce the number of point clouds by keeping the original state of the point cloud as much as possible, thereby reducing the amount of calculation. This method not only keeps the shape characteristics of the point cloud basically unchanged, but also basically retains the spatial structure information. It is very practical in point cloud registration, surface reconstruction, shape recognition and other algorithm speeds. Voxelize the three-dimensional space, and then calculate the center of mass of the non-empty voxel to replace all the points in the voxel. The specific method includes: creating voxels: calculating the bounding box of the point cloud, and then discretizing the bounding box into small voxels; the length, width and height of the voxel can be set manually, or by setting the number of grid points in the three directions of the bounding box. Each small voxel contains a number of points, and the center point or the point closest to the center point is taken as the sampling point;

[0087] Step 32: Calculate the point histogram feature PFH of the point cloud, that is, set a fixed neighborhood in the point cloud based on experience or experiments, then calculate several eigenvalues ​​in the neighborhood, and establish a histogram for the eigenvalues ​​to obtain statistical features;

[0088] Step 32: Calculate the point histogram feature (PFH) of the point cloud, that is, set a fixed neighborhood in the point cloud based on experience or experiments, then calculate several eigenvalues ​​in the neighborhood, and establish a histogram for the eigenvalues ​​to obtain the statistical features. The method includes:

[0089] First, define a local coordinate system, set ps point as the point where PFH needs to be calculated, and pt point as a point in its neighborhood. Define a local coordinate system based on the normal vector between the two lines. The calculation method of the three axes u, v and w of the local coordinates is as follows:

[0090] u=n s

[0091] v=(p t -p s )×u

[0092] w=u×v

[0093] n s For p s The normal direction of the point is obtained based on the calculation formula of the three axes of the local coordinates. s The local coordinate system of p is calculated based on the local coordinate system. t Point and p s The triplet feature representation between points is:

[0094] a=v·n i

[0095]

[0096] θ=arctan(w·n t , u·n t )

[0097] Got p s is the source point, for p t The triple feature representation of α, φ and θ, PFH calculates the triple features of all point pairs in the neighborhood of any point in the point cloud data, and based on these features, establishes a histogram representation, and then obtains the PFH feature descriptor;.

[0098] Step 33: Point cloud registration: Step 33 uses a double iterative closest point algorithm (Iterative Closest Point, ICP) method to perform point cloud precise registration. The specific method includes:

[0099] The first iteration is to first obtain the coarse registration source point cloud P ci And the coarse registration target point cloud Q cj Perform ICP precise registration and record the calculated rotation matrix as R c , the calculated translation matrix is ​​recorded as Tc , the specific steps are:

[0100] (1) Set the termination condition as:

[0101] ① The transformed coarse registration source point cloud P ci Roughly align the target point cloud Q cj The average distance is less than the adaptive convergence threshold e thr As shown in the formula, N is the overlap rate, that is, the ratio of the number of overlapping points in the point cloud to the total number of point clouds; L r is the horizontal resolution, R e is the systematic measurement error;

[0102]

[0103] ②The number of registration iterations has reached the set maximum value N ε Otherwise, the transformed P ci Continue to iterate as a new coarsely registered source point cloud;

[0104] (2) Search for the coarse registration target point cloud Q cj Each point in the coarsely registered source point cloud P ci The nearest point in the , set the maximum threshold N of the number of iterations ε , and at the same time calculate the overlap rate N;

[0105] (3) Calculate the rigid body transformation (R) corresponding to the minimum average distance between the nearest points c , T c );

[0106] (4) Use the parameters obtained in step (2) to calculate P ci Perform rotation and translation to obtain a new set of transformed points;

[0107] (5) Repeat steps (2) to (4) until the termination condition is reached. Otherwise, the transformed P ci Continue to iterate as a new coarsely registered source point cloud;

[0108] The first iteration is to precisely align the feature points of the source point cloud and the target point cloud. The source point cloud can still be aligned with the target point cloud after being transformed by the calculated rotation and translation matrix, so the second iteration is performed. The rotation and translation parameters after the alignment are substituted into the original source point cloud for calculation to obtain the new source point cloud data. Finally, the same steps as the first iteration are used again to perform ICP precise alignment between it and the target point cloud.

[0109] Step 34: Calculate the distance between the current point cloud data and the template point cloud data. The specific method includes:

[0110] In the choice of distance formula, the most common actual distance from point to surface is selected, which measures the absolute distance from a point to a surface in multidimensional space. This distance can directly reflect the distance from the point to the surface. The calculation formula is as follows:

[0111]

[0112] The equation of the plane is ax+by+cz+d=0, and the coordinates of the point in space are (x 0 ,y 0 , z 0 ), a, b and c are measured by initial calibration.

[0113] The experimental results of the 3D detection algorithm for damage and deformation of some parts are as follows Figure 4 , Figure 5 and Figure 6 As shown, the experimental results prove that the detection algorithm proposed in the present invention can accurately and effectively detect and locate deformation and damage anomalies in three-dimensional point cloud data.

[0114] Figure 4 The results of damage detection, where (a) is the front visualization result of the damage detection point cloud. The red area in the figure is the sunken damaged area, and the blue area is the normal area. (b) is the visualization result observed from the back, and it can be seen that the red sunken area is convex on the back.

[0115] Figure 5 The deformation detection results based on 3D point cloud, where (a) is the visualization result of deformation surface point cloud and deformation area, the red area is the deformation area point cloud, and the blue area is the normal area point cloud. (b) is the visualization result of normal point cloud, deformation point cloud and deformation area point cloud, where the yellow area is the normal point cloud, the blue area is the deformation surface point cloud, and the red area is the deformation area point cloud. The detection accuracy is about 0.05mm at a distance of about 0.4mm.

[0116] Figure 6 It is the deformation result calculated after the binocular camera generates the 3D point cloud. Among them, the left side is the RGB image of the actual deformation, the red rectangular box marks the deformation area to be detected, and the actual measured maximum deformation distance is 1.25mm. The right side is the visualization result of the 3D point cloud, where the red area and the red bounding box are the detected deformation area point cloud, and the maximum deformation distance of the point cloud is 0.93mm. According to the maximum deformation actually measured and the maximum deformation based on the calculation of the 3D point cloud, it can be obtained that the measurement error of the binocular camera is 0.32mm. It should be understood that for ordinary technicians in this field, improvements or changes can be made according to the above description, and all these improvements and changes should belong to the scope of protection of the claims attached to the present invention.

Claims

1. A method for monitoring part defects based on three-dimensional point cloud data, characterized in that: include: Step 1: Construct a 3D structured light vision system; Step 2: Use a 3D reconstruction algorithm to perform 3D reconstruction on the area to be inspected to obtain point cloud data to be inspected; Step 3: Use the point cloud defect detection algorithm to detect whether there are abnormal defects in the area to be inspected.

2. A method for monitoring part defects based on three-dimensional point cloud data according to claim 1, characterized in that: The method of constructing a 3D structured light vision system in step 1 includes: Step 11, using a structured light source to project RGB color Gray code structured light onto the surface of the part to be inspected; Step 12, obtaining a coded image through a camera; Step 13, complete the system calibration, and complete the three-dimensional reconstruction of the surface to be inspected of the component according to the system calibration result.

3. The method for monitoring part defects based on three-dimensional point cloud data according to claim 2, characterized in that: The step 13 is to complete the system calibration. According to the system calibration result, the method for completing the three-dimensional reconstruction of the surface to be inspected of the component includes: using a camera calibration method based on a 2D plane checkerboard and using a Matlab camera calibration tool to complete the calibration of camera parameters. First, the laser projects a structured light plane onto a flat chessboard to form laser stripes, and the laser stripe image is captured by a CCD camera; Then, the image is processed, and any four laser stripes on the laser stripe image are selected, and four feature points are respectively obtained based on the four laser stripes; Finally, the four feature points on the structured light plane are used to determine the parameters of the structured light plane equation.

4. The method for monitoring part defects based on three-dimensional point cloud data according to claim 3, characterized in that: The method for selecting any four laser stripes on the laser stripe image includes: controlling the laser to move from two calibrated positions to another two calibrated positions to obtain two laser stripes, and repeating the above steps to obtain another two laser stripes.

5. The method for monitoring part defects based on three-dimensional point cloud data according to claim 3, characterized in that: The method for determining the parameters of the structured light plane equation using four feature points on the structured light plane includes: converting the image coordinates of the four laser stripe feature points into the coordinates of the four laser stripe feature points in the camera coordinate system, and obtaining the structured light plane equation using the least squares method.

6. The method for monitoring part defects based on three-dimensional point cloud data according to claim 3, characterized in that: The method of performing three-dimensional reconstruction of the area to be inspected of the component using a three-dimensional reconstruction algorithm to obtain the point cloud data to be inspected in step 2 includes: Step 21, the structured light source projects the coded pattern onto the surface of the part to be reconstructed, and solves the equation of the straight line corresponding to each control point at the intersection of the projection light and the measured object; Step 22, after the calibration in step 13 is completed, the conversion relationship between the reference coordinate systems of the camera and the projector and the defined world coordinate system is obtained, and the three-dimensional coordinates of the control point M in the world coordinate system are calculated through coordinate conversion, so as to obtain the three-dimensional point cloud data of the object to be inspected.

7. The method for monitoring part defects based on three-dimensional point cloud data according to claim 6, characterized in that: The specific method for solving the equation of the straight line corresponding to each control point at the intersection of the projection light and the measured object in step 21 includes: Obtain the pixel coordinates of each control point, and solve the equation of the line connecting the coordinates of the projection point of each control point on the normalized imaging plane of the camera in the camera coordinate system and the optical center of the camera. P={p|n T (p-q p )=0} Among them, n T is the normal vector of the light plane, q p For M p The coordinates of q L M c The coordinates of v are the camera ray L c The direction vector of the ray is κ, and the coefficient of the ray is in the real number range. The projector light plane P and the corresponding camera ray L c The intersection point solution is shown as follows:

8. The method for monitoring part defects based on three-dimensional point cloud data according to claim 2, characterized in that: The specific method of step 3 detecting whether there are abnormal defects in the area to be inspected of the component by using the point cloud defect detection algorithm includes: Step 31, read the point cloud data, use the filtering method based on voxel downsampling to pre-process the point cloud, voxelize the three-dimensional space, and then calculate the centroid of the non-empty voxel to replace all the points in the voxel. The specific method includes: creating voxels: calculating the bounding box of the point cloud, and then discretizing the bounding box into small voxels; the length, width and height of the voxel can be set manually, or by setting the number of grid points in the three directions of the bounding box. Each small voxel contains a number of points, and the center point or the point closest to the center point is taken as the sampling point; Step 32: Calculate the point histogram feature PFH of the point cloud, that is, set a fixed neighborhood in the point cloud based on experience or experiments, then calculate several eigenvalues ​​in the neighborhood, and establish a histogram for the eigenvalues ​​to obtain statistical features; Step 33: Point cloud registration; Step 34: Calculate the distance between the current point cloud data and the template point cloud data. The specific method includes: In the choice of distance formula, the most common actual distance from point to surface is selected, which measures the absolute distance from a point to a surface in multidimensional space. This distance can directly reflect the distance from the point to the surface. The calculation formula is as follows: The equation of the plane is ax+by+cz+d=0, the coordinates of the point in space are (x0, y0, z0), and a, b, and c are measured through initial calibration.

9. The method for monitoring part defects based on three-dimensional point cloud data according to claim 8, characterized in that: Step 32: Calculate the point histogram feature (PFH) of the point cloud, that is, set a fixed neighborhood in the point cloud based on experience or experiment, then calculate several eigenvalues ​​in the neighborhood, and establish a histogram for the eigenvalues ​​to obtain the statistical features. The method includes: First, define a local coordinate system, let D psi Point is the point where PFH needs to be calculated. pk1 The point is a point in its neighborhood. Based on the normal vector between the two lines, the local coordinate system is defined. The calculation method of the three axes u, v and w of the local coordinate is as follows: u=n s w=u·v n s D pk1 The normal of the point is obtained based on the calculation formula of the three axes of the local coordinates. pk1 The local coordinate system of D is calculated based on the local coordinate system. psi Point and D pk1 The triplet feature representation between points is: α=v·n t θ=arctan(w·n,u·n t ) Got D psi is the source point, for D pk1 The triple feature representation of α, φ and θ, PFH calculates the triple features of all point pairs in the neighborhood of any point in the point cloud data, and based on these features, establishes a histogram representation, and then obtains the PFH feature descriptor.

10. The method for monitoring part defects based on three-dimensional point cloud data according to claim 9, characterized in that: Step 33 uses a double iterative closest point algorithm to perform point cloud precise registration. The specific method includes: The first iteration is to first obtain the coarse registration source point cloud P ci And the coarse registration target point cloud Q cj Perform ICP precise registration and record the calculated rotation matrix as R c , the calculated translation matrix is ​​recorded as T c , the specific steps are: (1) Set the termination condition as: ① The transformed coarse registration source point cloud P ci Roughly align the target point cloud Q cj The average distance is less than the adaptive convergence threshold e thr As shown in the formula, N is the overlap rate, that is, the ratio of the number of overlapping points in the point cloud to the total number of point clouds; L r is the horizontal resolution, R e is the systematic measurement error; ②The number of registration iterations has reached the set maximum value N ε Otherwise, the transformed P ci Continue to iterate as a new coarsely registered source point cloud; (2) Search for the coarse registration target point cloud Q cj Each point in the coarsely registered source point cloud P ci The nearest point in the , set the maximum threshold N of the number of iterations ε , and at the same time calculate the overlap rate N; (3) Calculate the rigid body transformation (R) corresponding to the minimum average distance between the nearest points c , T c ); (4) Use the parameters obtained in step (2) to calculate P ci Perform rotation and translation to obtain a new set of transformed points; (5) Repeat steps (2) to (4) until the termination condition is reached. Otherwise, the transformed P ci Continue to iterate as a new coarsely registered source point cloud; The first iteration is to precisely align the feature points of the source point cloud and the target point cloud. The source point cloud can still be aligned with the target point cloud after being transformed by the calculated rotation and translation matrix, so the second iteration is performed. The rotation and translation parameters after the alignment are substituted into the original source point cloud for calculation to obtain the new source point cloud data. Finally, the same steps as the first iteration are used again to perform ICP precise alignment between it and the target point cloud.

Citation Information

Patent Citations

  • Industrial defect detection system based on part point cloud registration detection

    CN114494134A

  • Multi-view vision balance ring part surface three-dimensional reconstruction system and method based on linear structured light projection

    CN115082538A

  • Multi-view three-dimensional point cloud reconstruction method

    CN115731348A

Cited By

  • Power transmission image defect detection and defect duplicate removal method and system based on deep learning image segmentation algorithm

    CN120411120A

  • Rail train part anomaly detection method based on point-surface distance

    CN120580233A

  • A track train part abnormality detection method based on point-to-face distance

    CN120580233B

  • PCB defect detection method based on surface structured light three-dimensional reconstruction

    CN120997196A

  • PCB defect detection method based on surface structure light three-dimensional reconstruction

    CN120997196B