A method for unknown object surface measurement viewpoint planning based on boundary inspection

Abstract

The application discloses a viewpoint planning method for unknown object surface measurement based on boundary detection, belongs to the technical field of three-dimensional measurement, and comprises the following steps: manually guiding a 3D camera to an initial viewpoint; acquiring a point cloud according to viewpoint shooting; gridizing the point cloud; registering and adjusting the gridized model; extracting a boundary from the registered and gridized model; sorting the boundary edges; confirming the boundary to be verified; and ending the viewpoint planning method for unknown object surface measurement based on boundary detection. First, the initial object point cloud is acquired by manually guiding the 3D camera, serving as a starting point for subsequent viewpoint planning; the viewpoint iteration is completed through feedback shooting; and the boundary to be verified serves as a planning condition for viewpoint expansion, so that the viewpoint planning operation is automatically completed without sufficient prior information, and the automatic measurement of the unknown object surface is realized. In addition, the application is not limited to the specific structure of the object, and therefore has strong adaptability to the viewpoint planning of the complex object surface.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of three-dimensional measurement, and particularly relates to a viewpoint planning method based on boundary inspection. BACKGROUND

[0002] Three-dimensional measurement technology is widely used in the fields of aerospace, automobile and ship, and after an object is measured by three-dimensional measurement technology, the object is digitized and used in detection, identification, path planning and the like. A complex object surface exists in the above fields, and it is difficult to obtain complete information of the object surface without prior information due to the complex structure of the object surface.

[0003] Viewpoint planning is one of the key technologies in three-dimensional measurement technology, and at present, in order to obtain complete three-dimensional data for an object surface with complex diversity, viewpoint planning for the object surface is usually completed by manual teaching, which is tedious and consumes human resources, and it is difficult to obtain an optimal solution of the viewpoint.

[0004] Chinese Patent Publication No. CN109977466A, entitled "Three-dimensional scanning viewpoint planning method, device and computer readable storage medium", uses obtained low-precision digitized model point cloud data to perform viewpoint planning calculation, obtains a point cloud normal vector of the surface of the object to be scanned through surface normal estimation, and determines a viewpoint by using the normal vector and a focusing distance of a three-dimensional sensor; and the termination condition of the algorithm is related to the number of point clouds to be calculated at present.

[0005] The three-dimensional scanning viewpoint planning method proposed in the scheme has the following deficiencies: in order to complete three-dimensional scanning viewpoint planning for an unknown object, a low-precision digitized model of the unknown object needs to be obtained in advance, and the automation level of surface measurement for the unknown object is low. SUMMARY

[0006] The main purpose of the application is to provide a viewpoint planning method for surface measurement of an unknown object based on boundary inspection, which aims to solve the problem that in the prior art, three-dimensional scanning viewpoint planning for an unknown object needs to obtain a low-precision digitized model of the unknown object in advance, and the automation level of surface measurement for the unknown object is low.

[0007] The viewpoint planning method for surface measurement of an unknown object based on boundary inspection comprises the following steps:

[0008] Step 1: manually guide a 3D camera to an initial viewpoint;

[0009] Let the coordinate system of the unknown object F be {O}, and let the right-hand Cartesian coordinate system of the jth round of viewpoint planning of the 3D camera be {C (j，i)}, { i = 1, 2,..., N, j = 0, 1, 2,..., M}, { C(j，i) The direction vector is {c} x c y c z}, where vector c z Let c be the direction of the camera's principal optical axis. x For the horizontal direction of the camera, vector c y The vertical direction of the camera; the horizontal and vertical angles of the 3D camera's view are θ and θ', respectively. H and θ V Vector c z The direction measurement range is [L] min L max ], vector c z The tilt angle relative to the surface normal of the unknown object F ranges from [-β]. max ,β max ];

[0010] The 3D camera is manually guided to an initial viewpoint V0 to capture images of an unknown object F. At this point, the 3D camera's vector c... z The normal angle between the unknown object F and the surface is β, where β ∈ [-β]. max ,β max The shooting distance from object F is d, where d∈[L] min L max ];

[0011] Step 2: Obtain point clouds based on viewpoint shooting;

[0012] The point cloud of the unknown object F, captured from the initial viewpoint V0, is denoted as... Where C (0，1) Let be the coordinate system of the camera at the initial viewpoint; let be the set of viewpoints planned in the j′-th round for the object F captured by the 3D camera. in Let x be the viewpoint position, where x is the viewpoint position. i y i , z i for In coordinate system C (0，1) The coordinates below, Let A be the viewpoint pose, where A is the viewpoint pose. i B i C i They are respectively In coordinate system C (0，2) Euler angles of rotation about the x, y, and z axes, with V j′ Point cloud of object F acquired by photography

[0013] Connect V0 and V j′ The combination is denoted as V j From the perspective of the j-th round of planning V j The point cloud of the object F acquired by the photograph is denoted as

[0014] Step 3: Point cloud gridding;

[0015] The is preprocessed and the object point cloud model is gridded by the triangular mesh algorithm of region growing in formula (1) to obtain the gridded model in the 3D camera coordinate system {C (j，i)}

[0016]

[0017] Let the transformation matrix between {C (j，i)} and {C (0，1)} be According to formula (2), the gridded model of the object point cloud in the initial viewpoint coordinate system {C (0，1)} is obtained by converting

[0018]

[0019] The j-th round of viewpoint V j corresponding to the gridded model of the point cloud in the initial viewpoint coordinate system {C (0，1)} is obtained

[0020] Step 4: Registration and adjustment of the gridded model;

[0021] The gridded model of the object point cloud after registration and adjustment is represented by formula (3). The overlapping area of and is extracted by the iterative closest point algorithm (ICP) to obtain the gridded model of the point cloud after registration and adjustment

[0022]

[0023] Step 5: Extraction of the boundary of the gridded model after registration;

[0024] The triangular edge set of the gridded model of the point cloud after registration is searched Each edge in the triangular edge set is searched If only one triangle is associated with the edge, the edge is a boundary edge, and

[0025]

[0026] The boundary edge set is obtained by formula (4)​

[0027] Step 6: Boundary edge sorting;

[0028] Take the initial boundary edge, add the ordered boundary edge set Search the boundary edge connected to the initial boundary edge and add the adjacent boundary edge using the newly added boundary edge, until the found boundary edge is the initial boundary edge, i.e. the boundary is closed;

[0029] In this way, from the ordered boundary edge set and the corresponding boundary point set

[0030] Step 7: To-be-verified boundary confirmation;

[0031] Confirm the to-be-verified boundary point set in the model boundary point set after the jth round of viewpoint set shooting using formula (5)

[0032]

[0033] Step 8: End condition of unknown object surface measurement viewpoint planning method based on boundary detection;

[0034] If i.e. all points in the set are real boundary points, the viewpoint planning task is terminated, and the planned viewpoint set V = {V0, V1, V2,..., V j} is obtained;

[0035] If take P T as the to-be-verified boundary point, continue the j+1th round of viewpoint planning through steps 9 and 10, and repeat steps 2-6 to confirm the to-be-verified boundary of the j+1th round of viewpoint planning registered and adjusted gridding model;

[0036] Step 9: To-be-detected boundary segmentation;

[0037] Segment the boundary of P T , set k as the boundary point sampling interval, sample the boundary, and calculate the angle a between vectors and using formula (6)

[0038]

[0039] Set the threshold value as T α , and if a < T​α Then the edge With edge Belonging to the same boundary segment, record

[0040]

[0041] Using formula (7), P T The corresponding boundary S T Divide into N boundary segments, and obtain Each boundary

[0042] Step 10: Generate viewpoints based on the spatial features of each boundary segment;

[0043] First, determine the viewpoint attitude. The results were obtained using principal component analysis, with each boundary segment... midpoint p i The direction vector of the third principal component of the neighborhood of the circle with center and radius r, i.e., the midpoint p. i normal vector n i , {i=1,2,...,N}, the vector c of the 3D camera z With normal vector n i Parallel and opposite, as in equation (8);

[0044] c z =-n i , i = 0, 1, 2, ..., N (8)

[0045] When the vector c of the 3D camera z When the direction is known, the boundary can be obtained using principal component analysis. The maximum principal component direction vector ξ i , {i=1,2,...,N}.

[0046] From formula (9), we can obtain vector c. y c x ,

[0047]

[0048] From formula (10), we can obtain vector c. x c y c z In coordinate system C (0，1) Euler angles A of rotation about the x-axis, y-axis, and z-axis i B i C i ,

[0049]

[0050] Obtain viewpoint attitude

[0051] Re-calculate the viewpoint position By formula (11), get The length of

[0052]

[0053] The measured distance d of the 3D camera is calculated by formula (12);

[0054]

[0055] Let the vector c of the 3D camera be z Align the boundary The midpoint p of i The viewpoint position is obtained by formula (8), (13)

[0056]

[0057] Get the viewpoint set corresponding to each boundary segment in the j+1th round

[0058] The beneficial effects of the present application are:

[0059] The present application is a kind of unknown object surface measurement viewpoint planning method based on boundary inspection, first, artificial guide 3D camera obtains initial object point cloud, as the starting point of subsequent viewpoint planning, through feedback shooting, the viewpoint iteration is completed, and the verified boundary is used as the planning condition of viewpoint expansion, so as to realize the automatic completion of viewpoint planning operation in the absence of sufficient prior information, realize the automatic measurement of unknown object surface.In addition, the present application does not limit the specific structure of the object, so it has strong adaptability for viewpoint planning of complex object surface. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 It is a kind of unknown object surface measurement viewpoint planning method based on boundary inspection of the present application flow chart;

[0061] Figure 2 It is the viewpoint shooting schematic diagram of the present application;

[0062] Figure 3 It is the initial viewpoint shooting position schematic diagram of the present application;

[0063] Figure 4 It is the boundary edge schematic diagram of the present application;

[0064] Figure 5 It is the boundary comparison schematic diagram before and after model registration adjustment of the present application; ​

[0065] Figure 6 is a boundary edge segment diagram of the present application;

[0066] Figure 7 is an initial view point to be verified boundary segment diagram of the present application;

[0067] Figure 8 is a boundary view point pose diagram of the present application;

[0068] Figure 9 is a first round view point pose diagram of the present application;

[0069] Figure 10 is a second round view point pose diagram of the present application;

[0070] Figure 11 is a second round view point before and after shooting boundary comparison diagram of the present application;

[0071] Figure 12 is a set of all view points of the present application. DETAILED DESCRIPTION

[0072] The present application will be further described in detail below with reference to the accompanying drawings.

[0073] As shown in the figure, the view point planning method for unknown object surface measurement based on boundary inspection includes the following steps: Figure 1

[0074] Step 1: manually guide the 3D camera to the initial view point

[0075] Set the F coordinate system of the car rear bumper as {O}, and there is no prior information of the object F. Set the right-hand Cartesian coordinate system of the i-th view point of the j-th round view point planning of the Gionee FM830-GI active structured light binocular camera as {C (j，i)}, {i = 1, 2,..., N, j = 0, 1, 2,..., M}, the direction vector of {C (j，i)} is {c x , c y , c z}, wherein the vector c z is the direction of the main optical axis of the camera, the vector c x is the horizontal direction of the camera, and the vector c y is the vertical direction of the camera. The horizontal angle and the vertical angle of the camera view area are θ H = 56° and θ V = 46°, respectively, the measurement range of the vector c z is [0.7m, 1m], and the inclination range of the vector c z relative to the surface normal of the car rear bumper F is [-15°, 15°], and the view point shooting is as follows​Figure 2 as shown.

[0076] The camera is manually guided to the initial viewpoint V0 to take a photo of the rear bumper F, at which time the vector c z of the camera is β = 3.6°, the shooting distance from the rear bumper F of the car is d = 720 mm, and the initial viewpoint shooting position is as shown. Figure 3

[0077] Step 2: Obtain point cloud according to viewpoint shooting

[0078] The point cloud of the rear bumper F of the car taken at the initial viewpoint V0 is denoted as , wherein C (0，1) is the coordinate system of the camera at the initial viewpoint.

[0079] Step 3: Grid the point cloud

[0080] Preprocess , and perform triangular gridding through the formula (1) region growing triangular gridding algorithm to obtain the grid model under the camera coordinate system {C (0，1)}.

[0081]

[0082] According to formula (2), the conversion of obtains the point cloud gridding model of the rear bumper F under the initial viewpoint coordinate system {C (0，1)}.

[0083]

[0084] Thus, the corresponding point cloud gridding model of the initial viewpoint V0 under the initial viewpoint coordinate system {C (0，1)} is obtained.

[0085] Step 4: Register and adjust the gridding model

[0086] Since the point cloud gridding model of the initial viewpoint shooting is empty in the last round, formula (3) is used to extract the overlapping area of and for registration, and the registered and adjusted point cloud gridding model is obtained.

[0087]

[0088] Step 5: Extract the boundary of the registered gridding model

[0089] Search the registered gridding model the triangle edge set each edge in the triangle edge set if is associated with a triangle, is a boundary edge, then

[0090]

[0091] The boundary edge set is obtained from formula (4) The boundary edges are shown as Figure 4 .

[0092] Step 6: Boundary edge sorting

[0093] Take as the initial boundary edge, and add it to the ordered boundary edge set Search for the boundary edge connected to the initial boundary edge in and add it to Continue to iterate the search for the adjacent boundary edge using the newly added boundary edge until the searched boundary edge is the initial boundary edge, i.e., the boundary is closed.

[0094] In this way, the ordered boundary edge set and the corresponding boundary point set are obtained by searching .

[0095] Step 7: Verification of the to-be-verified boundary

[0096] The boundary comparison before and after model registration adjustment is shown as Figure 5 , and the to-be-verified boundary point set in the model boundary point set after the initial viewpoint shooting is confirmed using formula (5)

[0097]

[0098] Step 8: End condition of the unknown object surface measurement viewpoint planning method based on boundary detection

[0099] Take P T as the to-be-verified boundary point, continue the first round of viewpoint planning through steps 9 and 10, and repeat steps 2-6 to verify the to-be-verified boundary of the registered and adjusted mesh model in the first round of viewpoint planning.

[0100] Step 9: Segmentation of the to-be-detected boundary

[0101] Segment the P T boundary, k = 50, sample the boundary, calculate the angle α between the vectors and using formula (7), and the boundary segmentation is shown asFigure 6 As shown,

[0102]

[0103] Let the threshold be T α =70°, if α < T α Then the edge With edge Belonging to the same boundary segment, record

[0104]

[0105] Using formula (8), P T The corresponding boundary S T Divided into 6 boundary segments, resulting in Where the boundary boundary boundary boundary boundary boundary The boundary segments to be verified of the model after viewpoint shooting are as follows: Figure 7 As shown.

[0106] Step 10: Generate viewpoints based on the spatial features of each boundary segment.

[0107] Boundary viewpoint pose Figure 8 As shown, from the first round of viewpoints For example, first, we need to calculate the viewpoint pose. The boundary conditions were obtained using principal component analysis. The midpoint p1 = [2262.77, -740.15, 1545.18] T Let n1 be the direction vector of the third principal component of the neighborhood with center p1 and radius r = 15 mm, i.e., the normal vector of the midpoint p1, which is n1 = [0.0817244, 0.0133536, 0.996566]. T The camera's vector c z Parallel to and opposite to the normal vector n1, as shown in equation (11).

[0108]

[0109] When the camera's vector c z When the direction is known, the boundary can be obtained using principal component analysis. The maximum principal component direction vector ξ1 = [0.514976, -0.856653, -0.0307523] T From formula (12), we can obtain vector c. y c x ,

[0110]

[0111] From equation (10), vector c x , c y , c z In the coordinate system C (0，1) Euler angles A1, B1, C1,

[0112]

[0113] Get the viewpoint pose

[0114] Then get the viewpoint position Use equation (14) to get The length of

[0115]

[0116] Use equation (15) to calculate the camera's measured distance d.

[0117]

[0118] Let the vector c z Align the midpoint p1 of the boundary , get the viewpoint position

[0119]

[0120] Get the first round of viewpoint set corresponding to each boundary The viewpoint planning pose result is shown in Table 1, and the first round of viewpoint pose is shown in Figure 9 .

[0121] Table 1 First round of viewpoint pose

[0122]

[0123]

[0124] Get the second round of viewpoint set corresponding to each boundary The viewpoint planning pose result is shown in Table 2, and the first round of viewpoint pose is shown in Figure 10 .

[0125] Table 2 Second round of viewpoint pose

[0126]

[0127] Second round of viewpoint set after shooting model boundary​​ After the first round of viewpoint set shooting, the model boundary As shown in Figure 11 The boundary no longer changes.

[0128] That is All points in the middle are real boundary points, and the viewpoint planning task is terminated, obtaining the planned viewpoint set V = {V0, V1, V2}, as shown in Figure 12 .

[0129] So far, the embodiment based on the scheme of the application is completed, and the self-termination condition is completed by the boundary verification. Without prior information, the viewpoint planning of the unknown object can also be completed, and the complete point cloud model of the object is constructed.

Claims

1. A viewpoint planning method for surface measurement of unknown objects based on boundary checks, characterized in that... The method includes the following steps: Step 1: Manually guide the 3D camera to the initial viewpoint; Let the coordinate system of the unknown object F be {O}, and let the right-hand Cartesian coordinate system of the i-th viewpoint in the j-th round of 3D camera viewpoint planning be {C}. (j，i) },{i=1,2,...,N,j=0,1,2...,M},{C (j，i) The direction vector is {c} x c y c z }, where vector c z Let c be the direction of the camera's principal optical axis. x For the horizontal direction of the camera, vector c y The vertical direction of the camera; the horizontal and vertical angles of the 3D camera's view are θ and θ', respectively. H and θ V Vector c z The direction measurement range is [L] min L max ], vector c z The tilt angle relative to the surface normal of the unknown object F ranges from [-β]. max ,β max ]; The 3D camera is manually guided to an initial viewpoint V0 to capture images of an unknown object F. At this point, the 3D camera's vector c... z The normal angle between the unknown object F and the surface is β, where β ∈ [-β]. max ,β max The shooting distance from object F is d, where d∈[L] min L max ]; Step 2: Obtain point clouds based on viewpoint shooting; The point cloud of the unknown object F, captured from the initial viewpoint V0, is denoted as... Where C (0，1) Let be the coordinate system of the camera at the initial viewpoint; let be the set of viewpoints planned in the j′-th round for the object F captured by the 3D camera. in Let x be the viewpoint position, where x is the viewpoint position. i y i , z i for In coordinate system C (0，1) The coordinates below, Let A be the viewpoint pose, where A is the viewpoint pose. i B i C i They are respectively In coordinate system C (0，1) Euler angles of rotation about the x, y, and z axes, with V j′ Point cloud of object F acquired by photography Connect V0 and V j′ The combination is denoted as V j From the perspective of the j-th round of planning V j The point cloud of the object F acquired by the photograph is denoted as Step 3: Mesh the point cloud; Will Preprocessing is performed, and the object point cloud model is processed using the region growing triangular mesh algorithm according to formula (1). Triangulation is performed to obtain the 3D camera coordinate system {C}. (j，i) Mesh model under} Let {C} (j，i) } and {C (0，1) The transformation matrix between} is From formula (2), The transformation yields the coordinates in the initial viewpoint coordinate system {C} (0，1) Meshized model of object point cloud under} This yields the viewpoint V in the j-th round. j In the initial viewpoint coordinate system {C (0，1) The corresponding point cloud mesh model Step 4: Register and adjust the meshed model; This represents the registration and adjustment mesh model after j-1 rounds of viewpoint shooting; using formula (3), the iterative nearest point algorithm (ICP) is used to extract... and The overlapping regions are registered to obtain the registered and adjusted point cloud mesh model. Step 5: Extract boundaries from the registered meshed model; Retrieval and registration of the gridded model The set of triangle sides Each edge in like Only one triangle is associated. If it is a boundary edge, then The set of boundary edges is obtained from formula (4). Step 6: Sort the boundary edges; Pick As the initial boundary edges, add them to the ordered boundary edge set. search Add the boundary edge connected to the initial boundary edge. The newly added boundary edge is used to continue iteratively searching for its adjacent boundary edges until the found boundary edge is the initial boundary edge, that is, the boundary is closed; Therefore, from Retrieve the ordered set of boundary edges and the corresponding set of boundary points Step 7: Confirm the boundary to be verified; Use formula (5) to confirm the model boundary point set after the j-th round of viewpoint shooting. The set of boundary points to be verified in Step 8: Termination conditions for the viewpoint planning method for unknown object surface measurement based on boundary detection; like Right now All points are real boundary points. The viewpoint planning task terminates, and the planned viewpoint set V = {V0, V1, V2, ..., V...} is obtained. j }; like With P T For the boundary points to be verified, continue the viewpoint planning for the (j+1)th round through steps 9 and 10, and repeat steps 2 to 6 to confirm the boundary points to be verified of the gridded model after the (j+1)th round of viewpoint planning registration and adjustment. Step 9: Segment the boundary to be detected; For P T Boundary segmentation processing: Let k be the sampling interval of boundary points. Sample the boundary points and calculate the vector using formula (6). and The included angle α between them Let the threshold be T α If α < T α Then the edge With edge Belonging to the same boundary segment, record Using formula (7), P T The corresponding boundary S T Divide into N boundary segments, and obtain Each boundary Step 10: Generate viewpoints based on the spatial features of each boundary segment; First, determine the viewpoint attitude. The results were obtained using principal component analysis, with each boundary segment... midpoint p i The direction vector of the third principal component of the neighborhood of the circle with center and radius r, i.e., the midpoint p. i normal vector n i , {i=1,2,...,N}, the vector c of the 3D camera z With normal vector n i Parallel and opposite, as in equation (8); c z =-n i ,i=0,1,2...,N (8) When the vector c of the 3D camera z When the direction is known, the boundary can be obtained using principal component analysis. Maximum principal component direction vector ξ i , {i = 1, 2, ..., N}; From formula (9), we can obtain vector c. y c x , From formula (10), we can obtain vector c. x c y c z In coordinate system C (0，1) Euler angles A of rotation about the x-axis, y-axis, and z-axis i B i C i , Obtain viewpoint attitude Next, determine the viewpoint position. Using formula (11), we get length The measurement distance d of the 3D camera is calculated using formula (12); Let the vector c of the 3D camera be... z Align with boundary midpoint p i The viewpoint position can be obtained from equations (8) and (13). Obtain the boundary of each segment in the (j+1)th round viewpoint set

Images