A method for detecting gap and flushness of workpieces based on point cloud

Abstract

The present invention discloses a method for detecting workpiece gap and flushness based on point cloud, including: 1) constructing a template point cloud of a reference part and a matching part in a sensor camera coordinate system using a triangular mesh model of the reference part and the matching part; 2) collecting the measured point cloud of the reference part and the matching part using a sensor, and aligning the measured point cloud of the reference part with its template point cloud; then moving the template point cloud of the matching part to align with the measured point cloud; using the aligned template point cloud of the reference part and the matching part as the calculated point cloud; 3) projecting all point clouds within a preset distance from a preset measurement plane on the calculated point cloud to the measurement plane, and obtaining the contour coordinate information of the reference part and the matching part corresponding to the measuring point; 4) calculating the gap value or flushness value at the measuring point based on the contour coordinate information of the reference part and the matching part. This method can avoid the influence of noise through point cloud processing, and can complete the acquisition of multiple measuring point information of the measured position in one measurement, effectively improving the detection efficiency of gap and flushness.

Description

Technical Field

[0001] The present invention relates to the technical field of visual inspection, and in particular to a workpiece gap and flushness detection method based on point cloud. Background Art

[0002] In the process of assembling parts in industrial manufacturing, the installation accuracy of two assembly parts needs to be detected and controlled; and this process is usually reflected by measuring the installation gap between the two assembly parts or the height difference between the two surfaces, that is, the installation accuracy is reflected by measuring the gap surface difference between the two assembly parts.

[0003] The traditional measurement method is to measure manually using a vernier caliper or feeler gauge. This measurement method is limited by human subjective factors, so the measurement accuracy is limited and the efficiency is low. With the development of visual inspection technology, some scenarios have begun to introduce line laser sensors to measure gap face difference values. Its working principle is to collect images of the object with laser lines projected on the surface, and obtain the contour information of the object at the projection position of the laser line through image processing, and then obtain the gap face difference value. However, since the gap face difference values ​​of two assembly parts at different measuring points are not the same, when using line laser sensors for gap face difference measurement, it is often necessary to adjust the posture of the line laser sensor multiple times, and measure multiple measuring points separately to obtain measurement values. Although this method has improved the detection accuracy to a certain extent, it requires multiple posture adjustments to complete the gap face difference measurement of an assembly position, resulting in low measurement efficiency. In addition, the sensor is easily affected by environmental noise or posture deviation during the measurement process, resulting in the loss of laser lines on the collected image of the object, which makes it impossible to carry out subsequent image processing and solution processes. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a workpiece gap and flushness detection method based on point cloud. Through point cloud processing, the influence of noise can be avoided, and the information of multiple measuring points of the measured position can be acquired in one measurement, thereby effectively improving the detection efficiency of gap and flushness.

[0005] To this end, the technical solution is as follows:

[0006] A workpiece gap and flushness detection method based on point cloud is used to detect the gap value or flushness value between a reference part and a matching part assembled together. The workpiece gap and flushness detection method comprises the following steps:

[0007] 1) Using the triangular mesh model of the assembled reference part and the matching part, obtain its two-dimensional projection on the camera image plane through camera calibration parameters, and construct the template point cloud of the reference part and the matching part in the camera coordinate system of the detection sensor according to the three-dimensional reconstruction method;

[0008] 2) Using the detection sensor to collect the measured point clouds of the reference part and the matching part, and aligning the measured point cloud of the reference part with its template point cloud so that the two overlap; then moving the template point cloud of the matching part to align with the measured point cloud; using the aligned template point clouds of the reference part and the matching part as the calculated point clouds for calculating the gap and flushness;

[0009] 3) constructing a measurement plane according to preset measuring points and measuring direction vectors, and projecting all point clouds within a preset distance from the measuring plane on the calculated point cloud onto the measuring plane, and obtaining contour coordinate information of the reference part and the matching part corresponding to the measuring points;

[0010] 4) Calculating the gap value or the face difference value at the measuring point according to the contour coordinate information of the reference part and the matching part.

[0011] Furthermore, the method for obtaining the template point cloud in step 1) includes the following steps:

[0012] S1, obtaining the position and posture transformation relationship P between the camera coordinate system of the detection sensor and the world coordinate system where the reference part to be measured and the matching part are located through calibration;

[0013] S2. By means of the posture conversion relationship P, each triangular mesh vertex in the triangular mesh model of the reference part and the matching part is converted to the detection sensor camera coordinate system, and the spatial plane corresponding to the triangular mesh is obtained according to the coordinates of the three vertices of the single triangular mesh in the detection sensor camera coordinate system;

[0014] S3, based on the camera intrinsic parameters of the detection sensor, convert each triangular mesh vertex of the triangular mesh model in the detection sensor camera coordinate system to the detection sensor camera image plane, and obtain its two-dimensional projection point in the detection sensor camera image plane; take the camera field of view as the interval, and obtain the two-dimensional projection point falling within the camera field of view in the camera image plane;

[0015] S4, a plane triangle composed of two-dimensional projection points whose three vertices of any triangular mesh simultaneously fall within the camera's field of view is regarded as a two-dimensional point set, and a spatial straight line is formed by a line connecting the camera's optical center and any point in the two-dimensional point set, and its intersection with the spatial plane corresponding to the triangular mesh is recorded as a single point A;

[0016] S5. Repeat S4 to traverse all triangular meshes within the camera field of view to obtain multiple single points A, which are recorded as template point clouds of the reference part and the matching part in the sensor camera coordinate system.

[0017] Furthermore, in order to balance the accuracy and efficiency of registration, the method for registering the measurement point cloud of the reference part with its template point cloud is: registration based on FPFH descriptor, registration based on PFH descriptor or registration based on PPF descriptor.

[0018] Furthermore, in order to maintain good integrity of the profile information of the reference part and the matching part obtained on the measuring plane and filter out unnecessary noise, the preset distance in step 3) is 1.5 to 2 times the point pitch. This method uses a point cloud sensor to measure the assembly workpiece to be measured, and can obtain all the surface profile information of the assembly workpiece to be measured at one time, avoiding the problem that the traditional line laser sensor needs to perform multiple posture adjustments to complete the acquisition of measurement information of a measurement assembly position;

[0019] In addition, after the acquired measured point cloud and the template point cloud are registered with the reference parts, the template point cloud of the matching part is used to align with its corresponding measured point cloud to complete the supplement and optimization of the measured point cloud, which effectively solves the problem that traditional line laser sensors are easily affected by environmental noise or posture deviation, resulting in missing information collection; and after acquiring the measured point cloud once, this method can quickly obtain the measurement values ​​of multiple measuring points only through point cloud processing, effectively improving the measurement efficiency of gap and flushness while ensuring accuracy. DETAILED DESCRIPTION

[0020] The technical solution of the present invention is described in detail below in conjunction with embodiments.

[0021] A workpiece gap and flushness detection method based on point cloud is used to detect the gap value or flushness value between a reference part and a matching part assembled together. The workpiece gap and flushness detection method comprises the following steps:

[0022] 1) Using the triangular mesh model of the assembled reference part and the matching part, obtain its two-dimensional projection on the camera image plane through camera calibration parameters, and construct the template point cloud of the reference part and the matching part in the camera coordinate system of the detection sensor according to the three-dimensional reconstruction method;

[0023] 2) Using the detection sensor to collect the measured point clouds of the reference part and the matching part, and aligning the measured point cloud of the reference part with its template point cloud so that the two overlap; then moving the template point cloud of the matching part to align with the measured point cloud; using the aligned template point clouds of the reference part and the matching part as the calculated point clouds for calculating the gap and flushness;

[0024] 3) constructing a measurement plane according to preset measuring points and measuring direction vectors, and projecting all point clouds within a preset distance from the measuring plane on the calculated point cloud onto the measuring plane, and obtaining contour coordinate information of the reference part and the matching part corresponding to the measuring points;

[0025] 4) Calculate the gap value or face difference value at the measuring point based on the contour coordinate information of the reference part and the matching part.

[0026] The method for obtaining the template point cloud in step 1) includes the following steps:

[0027] S1, obtaining the position and posture transformation relationship P between the camera coordinate system of the detection sensor and the world coordinate system where the reference part to be measured and the matching part are located through calibration;

[0028] S2. By means of the posture conversion relationship P, each triangular mesh vertex in the triangular mesh model of the reference part and the matching part is converted to the detection sensor camera coordinate system, and the spatial plane corresponding to the triangular mesh is obtained according to the coordinates of the three vertices of the single triangular mesh in the detection sensor camera coordinate system;

[0029] S3, based on the camera intrinsic parameters of the detection sensor, convert each triangular mesh vertex of the triangular mesh model in the detection sensor camera coordinate system to the detection sensor camera image plane, and obtain its two-dimensional projection point in the detection sensor camera image plane; take the camera field of view as the interval, and obtain the two-dimensional projection point falling within the camera field of view in the camera image plane;

[0030] S4, a plane triangle composed of two-dimensional projection points whose three vertices of any triangular mesh simultaneously fall within the camera's field of view is regarded as a two-dimensional point set, and a spatial straight line is formed by a line connecting the camera's optical center and any point in the two-dimensional point set, and its intersection with the spatial plane corresponding to the triangular mesh is recorded as a single point A;

[0031] S5. Repeat S4 to traverse all triangular meshes within the camera field of view to obtain multiple single points A, which are recorded as template point clouds of the reference part and the matching part in the sensor camera coordinate system.

[0032] The following is a specific method for obtaining a template point cloud. The steps are as follows:

[0033] S1. Obtaining the position transformation relationship P between the camera coordinate system of the detection sensor and the world coordinate system of the reference part and the matching part to be measured through calibration. The position transformation relationship P includes the rotation matrix R and the translation matrix T;

[0034] S2. Through the posture conversion relationship P, each triangular mesh vertex in the triangular mesh model of the reference part and the matching part is converted to the detection sensor camera coordinate system. The conversion process is realized by the following formula:

[0035] R×M i +T=C i

[0036] Among them, M i is the coordinate of the i-th vertex in the triangular mesh model; C i is the coordinate of the i-th vertex in the triangular mesh model in the detection sensor camera coordinate system;

[0037] According to the coordinates of the three vertices of a single triangular mesh in the detection sensor camera coordinate system, the spatial plane corresponding to the triangular mesh in the detection sensor camera coordinate system is obtained by the least square method, and the plane equation ax+by+cz+d=0 is obtained, where a, b, c, and d are plane parameters; those skilled in the art may also select other existing spatial plane acquisition methods according to the purpose of implementation;

[0038] S3, according to the camera intrinsic parameters of the detection sensor, transform each triangular mesh vertex of the triangular mesh model in the detection sensor camera coordinate system to the detection sensor camera image plane, and obtain its two-dimensional projection point in the detection sensor camera image plane; for any triangular mesh model vertex C i (x i ,y i , z i ), whose two-dimensional projection point coordinates in the sensor camera plane (u i , v i ) is calculated as follows:

[0039]

[0040] In the formula, f x 、f y , u0, v0 are all camera internal parameters; among them, f x =f / dx,f y =f / dy, f is the focal length, dx and dy represent the pixel size of a single pixel, and u0 and v0 represent the offset of the principal point position in the pixel coordinate system;

[0041] Taking the camera field of view (W×H) as the interval, obtain the two-dimensional projection points that fall within the camera field of view in the camera image plane; that is, the two-dimensional projection point coordinates (u i , v i ) satisfies, u i falls within the interval [0, W], v i Falls in the interval [0, H];

[0042] S4, a plane triangle composed of two-dimensional projection points whose three vertices of any triangular mesh simultaneously fall within the camera's field of view is taken as a two-dimensional point set, and a spatial straight line l is formed by connecting the camera's optical center and any point in the two-dimensional point set, and its intersection with the spatial plane corresponding to the triangular mesh is recorded as a single point A; that is, the single point A is obtained by combining the linear equation of the spatial straight line l and the spatial plane equation ax+by+cz+d=0;

[0043] S5. Repeat S4 to traverse all triangular meshes within the camera field of view to obtain multiple single points A, which are recorded as template point clouds of the reference part and the matching part in the sensor camera coordinate system.

[0044] Specifically, in order to balance the accuracy and efficiency of registration, the method for registering the measured point cloud of the reference part with its template point cloud is: registering based on FPFH descriptor, registering based on PFH descriptor or registering based on PPF descriptor. Those skilled in the art may also select other existing registration methods according to the purpose of implementation.

[0045] Among them, the preset measuring point position in step 3) is marked on the template point cloud in advance according to the detection needs, and the measuring direction is vertically pointed to the direction where the gap or flushness to be detected exists with the measuring point as the starting point; the measuring plane constructed by the measuring point and the measuring direction vector can display the gap or flushness to the greatest extent.

[0046] In order to maintain good integrity of the contour information of the reference part and the matching part obtained on the measuring plane and to filter out unnecessary noise, the preset distance in step 3) is preferably 1.5 to 2 times the point spacing.

[0047] This method uses a triangular mesh digital model and constructs a template point cloud of the assembly to be measured through three-dimensional reconstruction according to the calibration parameters and the posture transformation relationship, thereby realizing the acquisition of the feature point cloud of the assembly to be measured section; the template point cloud of the reference part in the assembly is aligned with the measured point cloud, and after the two are overlapped, the template point cloud of the matching part is adjusted to align with the measured point cloud, so that the template point cloud completes the optimization of the measured point cloud, and then the gap and face difference of the specified measuring point is calculated on the template point cloud that has been aligned with the measured point cloud, which effectively solves the problem that traditional line laser sensors are easily affected by environmental noise or posture deviation, resulting in missing information collection; and after acquiring the measured point cloud once, this method can quickly obtain the measurement values ​​of multiple measuring points only through point cloud processing, effectively improving the measurement efficiency while ensuring accuracy.

[0048] In addition, during the process of acquiring the template point cloud, the method has obtained the pose conversion relationship P between the camera coordinate system of the detection sensor and the world coordinate system of the reference part and the mating part to be measured. After using the method to obtain the gap / face difference measurement value, the detection result can be fed back to the automatic assembly system. The automatic assembly system can calculate the assembly compensation amount through the pose conversion relationship P and the gap / face difference measurement value to guide the adjustment of the assembly action; through the linkage between the method and the assembly system, the automatic detection and adjustment of the assembly process can be better realized.

[0049] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. A method for detecting gap and flushness of workpieces based on point cloud, which is used to detect the gap value or flushness value between a reference part and a matching part assembled together, characterized in that: The workpiece gap flushness detection method comprises the following steps: 1) Using the triangular mesh model of the assembled reference part and the matching part, obtain its two-dimensional projection on the camera image plane through camera calibration parameters, and construct the template point cloud of the reference part and the matching part in the camera coordinate system of the detection sensor according to the three-dimensional reconstruction method; 2) Using the detection sensor to collect the measured point clouds of the reference part and the matching part, and aligning the measured point cloud of the reference part with its template point cloud so that the two coincide; Then move the template point cloud of the mating part to align with the actual measured point cloud; The template point clouds of the aligned reference part and the matching part are used as the calculation point clouds for calculating the gap and flushness; 3) constructing a measurement plane according to preset measuring points and measuring direction vectors, and projecting all point clouds within a preset distance from the measuring plane on the calculated point cloud onto the measuring plane, and obtaining contour coordinate information of the reference part and the matching part corresponding to the measuring points; 4) Calculating the gap value or the face difference value at the measuring point according to the contour coordinate information of the reference part and the matching part.

2. The method for detecting workpiece gap and flushness based on point cloud as claimed in claim 1, characterized in that: The method for obtaining the template point cloud in step 1) includes the following steps: S1, obtaining the position and posture transformation relationship P between the camera coordinate system of the detection sensor and the world coordinate system where the reference part to be measured and the matching part are located through calibration; S2. By means of the posture conversion relationship P, each triangular mesh vertex in the triangular mesh model of the reference part and the matching part is converted to the detection sensor camera coordinate system, and the spatial plane corresponding to the triangular mesh is obtained according to the coordinates of the three vertices of the single triangular mesh in the detection sensor camera coordinate system; S3, based on the camera intrinsic parameters of the detection sensor, convert each triangular mesh vertex of the triangular mesh model in the detection sensor camera coordinate system to the detection sensor camera image plane, and obtain its two-dimensional projection point in the detection sensor camera image plane; take the camera field of view as the interval, and obtain the two-dimensional projection point falling within the camera field of view in the camera image plane; S4, a plane triangle composed of two-dimensional projection points whose three vertices of any triangular mesh simultaneously fall within the camera's field of view is regarded as a two-dimensional point set, and a spatial straight line is formed by a line connecting the camera's optical center and any point in the two-dimensional point set, and its intersection with the spatial plane corresponding to the triangular mesh is recorded as a single point A; S5. Repeat S4 to traverse all triangular meshes within the camera field of view to obtain multiple single points A, which are recorded as template point clouds of the reference part and the matching part in the sensor camera coordinate system.

3. The method for detecting workpiece gap and flushness based on point cloud as claimed in claim 1, characterized in that: The method for registering the measurement point cloud of the reference part with its template point cloud is: registering based on FPFH descriptor, registering based on PFH descriptor or registering based on PPF descriptor.

4. The method for detecting workpiece gap and flushness based on point cloud as claimed in claim 1, characterized in that: The preset distance in step 3) is 1.5 to 2 times the dot pitch.

Images