A point cloud depth acquisition method based on normal projection

Abstract

The present invention provides a point cloud depth acquisition method based on normal projection, which performs voxel processing on the digital model of the object to be measured, and stores it and the three-dimensional point cloud in a unified coordinate system; obtains the contour of the feature area and the center position coordinates; stores the point cloud in the feature area into a point set I; obtains the three-dimensional coordinates A of the center position coordinates; uses the three-dimensional coordinates and the camera optical center point to determine as the incident light; obtains the normal vector A at the intersection of the incident light and the digital model; projects the point set I onto the normal vector A, and records the maximum distance between the two projection points as the depth of the feature area; the method is not affected by the surface morphology of the object to be measured, has the characteristics of high accuracy, high efficiency and convenience, and can extract the surface depth information of the object to be measured with any morphology in real time.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional measurement, and in particular to a point cloud depth acquisition method based on normal projection. Background Art

[0002] In modern precision machining and manufacturing industries, such as automobile manufacturing, aerospace, and rail transportation, there are strict requirements for the machining quality of workpieces. In order to meet the functional and aesthetic requirements of the overall product, the workpiece surface is usually subjected to local grinding and polishing finishing, such as surface defect treatment of automotive parts. Usually, the machining equipment plans the grinding and polishing process according to the machining allowance and performs precise grinding and polishing. Therefore, the grinding and polishing quality depends on how to accurately determine the machining allowance, that is, the length, width, and depth of the range covered by the defect. Generally, rich point cloud information on the workpiece surface can be obtained through three-dimensional measurement methods, but how to accurately extract the depth information of the area covered by the defect (especially complex surfaces) from the point cloud is a difficult problem that needs to be solved. In addition to defect treatment, the actual contour of the complex surface of the workpiece does not match the design value due to the machining accuracy error, resulting in insufficient functionality or aesthetics. Such problems also require accurate acquisition of the depth information of the area to be processed and secondary finishing. For example, the stamping contour lines on the front cover of a car, the recessed contours at the door handles, the waistlines on both sides of the car body, the edges of the doors and other raised or recessed sheet metal crease contours on the workpiece surface require a processing tolerance of less than ±0.5mm, and usually require secondary finishing.

[0003] Regarding the depth information detection of contour areas and defect areas, existing methods usually use the surface point cloud of the object to be measured to fit the plane, calculate the distance between the points in the concave or convex area in the point cloud and the fitting plane, and use this distance as the depth value, such as CN109658398A, a part surface defect identification and evaluation method based on three-dimensional measurement point cloud; since such methods must perform plane fitting on the surface of the object to be measured, when the surface of the object to be measured is a plane in the strict sense, the depth information calculated by such methods is accurate and effective, but when the surface of the object to be measured is a curved surface or a special-shaped surface, such methods are affected by the surface morphology of the object to be measured, and the depth information obtained will no longer be reliable. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a point cloud depth acquisition method based on normal projection, which performs normal projection on each point in the defect area, and uses the maximum distance value of the projection point as the depth information. It is not affected by the surface morphology of the object to be measured, has the characteristics of high accuracy, high efficiency and convenience, and can extract the surface depth information of objects to be measured with arbitrary morphology in real time.

[0005] The technical solution is as follows:

[0006] A point cloud depth acquisition method based on normal projection, wherein the depth is the maximum value of the height difference between a characteristic region on the object to be measured and the surface of the object to be measured, and the characteristic region is a convex region or a concave region;

[0007] Use the two-dimensional image, three-dimensional point cloud and digital model of the surface of the object to be tested to perform the following steps:

[0008] voxelize the digital model of the object to be measured, and store it in the same coordinate system as the three-dimensional point cloud, which is recorded as a unified coordinate system;

[0009] Obtaining the contour of the feature area and the coordinates of the center position of the feature area from the two-dimensional image; corresponding the contour of the feature area to the three-dimensional point cloud, searching for point cloud information corresponding to each point in the contour of the feature area, and storing each point cloud information in point set I;

[0010] Using the camera intrinsic parameters, obtain the three-dimensional coordinates A of the center position coordinates in the camera coordinate system;

[0011] According to the conversion relationship between the pre-calibrated camera coordinate system and the unified coordinate system, the spatial coordinates of the camera optical center point and the three-dimensional coordinate A in the unified coordinate system are respectively solved, and the spatial straight line is determined by using the two obtained spatial coordinates, which is recorded as the incident light ray;

[0012] The surface patch in the digital model of the object to be measured that intersects with the incident light is recorded as a feature surface patch, the intersection point of the incident light and the feature surface patch is obtained, and the normal vector at the intersection point is recorded as a normal vector A;

[0013] Project each point in the point set I onto the normal vector A to obtain the coordinates of each projection point;

[0014] Traverse any two projection points and calculate the distance between the two points; record the maximum distance as the depth of the feature area.

[0015] Further, using the camera internal parameters, the three-dimensional coordinates A of the center position coordinates (u0, v0) in the camera coordinate system are obtained:

[0016] Where (x0, y0) is the calibrated camera principal point coordinates, and (fx, fy) is the equivalent focal length of the camera.

[0017] Further, the surface patch in the digital model of the object to be measured that intersects with the incident light is recorded as a feature surface patch, the intersection point of the incident light and the feature surface patch is obtained, and the normal vector at the intersection point is recorded as a normal vector A;

[0018] The specific steps are:

[0019] Select one of the facets from the digital model of the object to be measured; construct the plane equation of the facet according to the vertex coordinates on the facet, combine the plane equation with the incident light equation, and solve the intersection point;

[0020] Traverse each edge of the face patch separately, and each time an edge is traversed, determine whether other vertices and intersection points are located on the same side of the currently traversed edge; the other vertices are vertices on the current face patch that are not located on the traversed edge;

[0021] When all edges are traversed and the judgment results are all yes, the current face is recorded as a feature face, otherwise the face is removed and other faces are traversed continuously, and the same judgment is applied to other faces until a feature face is found.

[0022] Furthermore, the contour of the feature area and the center position coordinates of the feature area are obtained from the two-dimensional image. The specific method is as follows:

[0023] The two-dimensional image is processed by the local adaptive binarization method to obtain a binary image, and the connected domain is extracted and the morphological opening and closing operations are performed on the binary image. The processed connected domain contour is recorded as the feature area contour, the minimum bounding rectangle of the feature area contour is obtained, and the center coordinates of the minimum bounding rectangle are marked as the center position coordinates.

[0024] This method obtains the spatial straight line equation of the incident light, intersects it with the digital model of the workpiece to be measured, obtains the spatial normal, and then performs normal projection on each point in the defect area, and uses the maximum distance value of the projection point as the depth information. The acquisition process is not affected by the surface morphology of the object to be measured. Regardless of whether it is a curved surface, a flat surface or a special-shaped surface, this method can effectively detect the depth information of the feature area (decorative line area, defect area). BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the relationship between the digital model of the object to be measured, the point cloud, and the position of the incident light in a specific implementation manner. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0027] A point cloud depth acquisition method based on normal projection, wherein the depth is the maximum height difference between a characteristic area on the object to be measured and the surface of the object to be measured, and the characteristic area is a convex area or a concave area, such as a stamping contour line, a spatter defect, and a pit defect;

[0028] Use the two-dimensional image, three-dimensional point cloud and digital model of the surface of the object to be tested to perform the following steps:

[0029] The digital model of the object to be measured is voxelized and stored in the same coordinate system as the three-dimensional point cloud, which is recorded as the unified coordinate system;

[0030] Obtain the contour of the feature area and the coordinates of the center position of the feature area from the two-dimensional image; correspond the contour of the feature area to the three-dimensional point cloud, find the point cloud information corresponding to each point in the contour of the feature area, and store each point cloud information in point set I;

[0031] Using the camera internal parameters, obtain the three-dimensional coordinates A of the center position coordinates (u0, v0) in the camera coordinate system

[0032]

[0033] Where (x0, y0) is the calibrated camera principal point coordinates, and (fx, fy) is the equivalent focal length of the camera.

[0034] Then, according to the conversion relationship between the pre-calibrated camera coordinate system and the unified coordinate system, the spatial coordinates of the camera optical center point and the three-dimensional coordinate A in the unified coordinate system are solved respectively, and the spatial straight line is determined by using the two obtained spatial coordinates, which is recorded as the incident light ray;

[0035] The patch in the digital model of the object to be tested that intersects with the incident light is recorded as a feature patch, the intersection point of the incident light and the feature patch is obtained, and the normal vector at the intersection point is recorded as the normal vector A;

[0036] Project each point in the point set I onto the normal vector A to obtain the coordinates of each projection point;

[0037] Traverse any two projection points and calculate the distance between the two points; record the maximum distance as the depth of the feature area.

[0038] In this embodiment, the contour of the feature area and the center position coordinates of the feature area are obtained from the two-dimensional image. The specific method is:

[0039] The two-dimensional image is processed by the local adaptive binarization method to obtain a binary image, and the connected domain is extracted and the morphological opening and closing operations are performed on the binary image. The processed connected domain contour is recorded as the feature area contour, the minimum bounding rectangle of the feature area contour is obtained, and the center coordinates of the minimum bounding rectangle are marked as the center position coordinates.

[0040] The specific steps for finding feature patches are:

[0041] Select one of the facets from the digital model of the object to be measured; construct the plane equation of the facet according to the vertex coordinates on the facet, combine the plane equation with the incident light equation, and solve the intersection point;

[0042] Traverse each edge of the face separately. Each time you traverse an edge, determine whether other vertices and intersection points are on the same side of the currently traversed edge; other vertices are vertices on the current face that are not on the traversed edge;

[0043] When all edges are traversed and the judgment results are all yes, the current face is recorded as a feature face, otherwise the face is removed and other faces are traversed continuously, and the same judgment is applied to other faces until a feature face is found.

[0044] Let the three vertices of the triangle be ABC, the edge currently traversed be AB, and the intersection point be P;

[0045] like Then point P and point C are on the same side of side AB. Similarly, determine whether point P and point A are on the same side of BC, and whether point P and point B are on the same side of CA.

[0046] When the judgment results are all yes, the current patch is recorded as a feature patch.

[0047] This method uses the maximum distance value of the projection points as depth information, and the acquisition process is not affected by the surface morphology of the object to be measured. The depth information of the feature area (decoration line area, defect area) can be effectively detected.

[0048] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. A point cloud depth acquisition method based on normal projection, characterized in that: The depth is the maximum value of the height difference between a characteristic region on the object to be measured and the surface of the object to be measured, and the characteristic region is a convex region or a concave region; Use the two-dimensional image, three-dimensional point cloud and digital model of the surface of the object to be tested to perform the following steps: voxelize the digital model of the object to be measured, and store it in the same coordinate system as the three-dimensional point cloud, which is recorded as a unified coordinate system; Obtaining the contour of the feature area and the coordinates of the center position of the feature area from the two-dimensional image; corresponding the contour of the feature area to the three-dimensional point cloud, searching for point cloud information corresponding to each point in the contour of the feature area, and storing each point cloud information in point set I; Using the camera intrinsic parameters, obtain the three-dimensional coordinates A of the center position coordinates in the camera coordinate system; According to the conversion relationship between the pre-calibrated camera coordinate system and the unified coordinate system, the spatial coordinates of the camera optical center point and the three-dimensional coordinate A in the unified coordinate system are respectively solved, and the spatial straight line is determined by using the two obtained spatial coordinates, which is recorded as the incident light ray; The surface patch in the digital model of the object to be measured that intersects with the incident light is recorded as a feature surface patch, the intersection point of the incident light and the feature surface patch is obtained, and the normal vector at the intersection point is recorded as a normal vector A; The method of searching for the feature patch is as follows: traverse each edge of the patch respectively, and each time an edge is traversed, determine whether other vertices and intersections are located on the same side of the currently traversed edge; the other vertices are vertices on the current patch that are not located on the traversed edge; When all edges are traversed and the judgment results are all yes, the current face is recorded as a feature face. Otherwise, the face is removed and other faces are traversed again. The same judgment is applied to other faces until a feature face is found. Project each point in the point set I onto the normal vector A to obtain the coordinates of each projection point; Traverse any two projection points and calculate the distance between the two points; record the maximum distance as the depth of the feature area.

2. The method according to claim 1, characterized in that: Using the camera internal parameters, the center position coordinates are obtained ( u 0, v 0) The three-dimensional coordinate A in the camera coordinate system ( , , 1): in( x 0, y 0) is the calibrated camera principal point coordinate, (fx, fy) is the equivalent focal length of the camera.

3. The method according to claim 1, characterized in that: Get the contour of the feature area and the center position coordinates of the feature area from the two-dimensional image. The specific method is: The two-dimensional image is processed by the local adaptive binarization method to obtain a binary image, and the connected domain is extracted and the morphological opening and closing operations are performed on the binary image. The processed connected domain contour is recorded as the feature area contour, the minimum bounding rectangle of the feature area contour is obtained, and the center coordinates of the minimum bounding rectangle are marked as the center position coordinates.

Images