Geometric parameter identification method and system based on three-dimensional reconstruction of space moving targets

Abstract

The present invention discloses a method and system for geometric parameter identification based on three-dimensional reconstruction of a space moving target. The method includes: acquiring monocular visual image data and sparse point cloud data; performing pixel-level segmentation and fitting on the monocular visual image data to construct an optical model of the space moving target; fusing the sparse point cloud data of the sparse radar with the optical model based on a visual mapping method to obtain a point cloud coarse extraction result; fusing the point cloud coarse extraction result based on an iterative matching multi-view point cloud fusion method; and performing three-dimensional reconstruction of the point cloud fusion result and autonomous identification of the geometric parameters of the space moving target based on a feature and learning hybrid method. By using the present invention, the accuracy, stability, and timeliness of the three-dimensional reconstruction and autonomous identification of geometric parameters of the space moving target can be improved. As a method and system for geometric parameter identification based on three-dimensional reconstruction of a space moving target, the present invention can be widely used in the field of computer vision.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for identifying geometric parameters based on three-dimensional reconstruction of a space moving target. Background Art

[0002] Robotic grasping tasks generally involve three processes: positioning, tracking, and grasping a moving target. If the moving target moves at a certain speed, the robotic grasping becomes significantly more difficult. Therefore, during the positioning process, a visual system is required to identify the geometric parameters and estimate the relative pose of the moving target. This presupposes that the initial relative pose is known. Three-dimensional reconstruction can obtain point cloud information of the moving target, thereby determining its initial relative pose and geometric parameters, providing important information for subsequent tracking and grasping tasks. Therefore, before estimating the relative pose, it is necessary to perform three-dimensional reconstruction of the moving target to obtain an accurate reference relative pose. Visual cameras contain rich edge information, while radar data better reflects scale information. However, traditional methods do not match visual and point cloud data and cannot fully utilize the rich edge features of the image. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for geometric parameter identification based on three-dimensional reconstruction of spatial moving targets. Through multi-source fusion, information complementarity of multi-perspective vision and point cloud can be achieved, thereby improving the accuracy, stability and timeliness of three-dimensional reconstruction of spatial moving targets and autonomous identification of geometric parameters.

[0004] The first technical solution adopted by the present invention is: a method for identifying geometric parameters based on three-dimensional reconstruction of a space moving target, comprising the following steps:

[0005] The visual image data and laser point cloud data of the moving target in space are collected by a monocular camera and a multi-line laser radar, and multiple acquisitions are performed based on different perspectives to obtain monocular vision image data and sparse point cloud data;

[0006] The autonomous visual region segmentation method is used to segment and fit the monocular vision image data at the pixel level to construct an optical model of the spatial moving target.

[0007] Based on the visual mapping method, the sparse point cloud data of the sparse radar is fused with the optical model of the space moving target to obtain the rough point cloud extraction result of the space moving target;

[0008] Based on the iterative matching method of multi-view point cloud fusion of spatial moving targets, the rough extraction results of the point cloud of the spatial moving target are fused to obtain the point cloud fusion results of the spatial moving target;

[0009] A feature-based and learning hybrid method is used to reconstruct the point cloud fusion results of the spatial moving target in three dimensions and autonomously identify the geometric parameters of the spatial moving target based on the reconstruction results to obtain the identification results.

[0010] Furthermore, the step of performing pixel-level segmentation and fitting on the monocular vision image data by the autonomous visual region segmentation method to construct an optical model of the spatial moving target specifically includes:

[0011] Perform spatial conversion on the monocular vision image data of the space moving target to obtain the HSV image of the space moving target;

[0012] Set the preset color segmentation threshold to segment the HSV image of the spatial moving target and obtain the preliminary segmentation result;

[0013] The incomplete parts of the spatial moving target in the preliminary segmentation result are filled by the image expansion technology to obtain the filled image of the spatial moving target;

[0014] Extract and filter the contour of the filled image of the space moving target to obtain the edge contour straight line of the space moving target;

[0015] Fit the edge contour line of the space moving target and extract the effective vertices of the edge line;

[0016] Based on the polar coordinate idea, the valid vertices of the edge line are sorted and connected end to end according to the sorting results to construct an optical model of the spatial moving target, that is, a mask image of the spatial moving target.

[0017] Furthermore, the step of fusing the sparse point cloud data of the sparse radar with the optical model of the space moving target based on the visual mapping method to obtain a rough point cloud extraction result of the space moving target specifically includes:

[0018] According to the optical model of the space moving target, the sparse point cloud data of the space moving target is mapped to obtain the corresponding pixel coordinates;

[0019] Matching pixel coordinates within the effective field of view of monocular vision with three-dimensional points to obtain the vertex of the spatial moving target in the point cloud space, where the three-dimensional points are three-dimensional points in the point cloud of the spatial moving target;

[0020] A constrained nonlinear optimization problem is introduced, and the nearest neighbor strategy is used to match the vertices of the spatial moving target in the point cloud space with the vertices in the visual space, i.e., the valid vertices, to obtain the point cloud rough extraction result of the spatial moving target.

[0021] Furthermore, the expression of the constrained nonlinear optimization problem is specifically as follows:

[0022]

[0023]

[0024] ( cam R lidar ) Tcam p lidar =I

[0025]

[0026] In the above formula, Optical model representing a moving target in space, A in Represents the intrinsic parameter matrix of monocular vision, A ex represents the external parameter matrix of monocular vision, lidar represents the sparse radar coordinate system, cam represents the camera coordinate system, s i represents the pixel coordinates of the point i projected on the image plane, cam R lidar Represents the rotation matrix of lidar relative to cam, ω i represents the field of view of monocular vision, Indicates the calculated value based on the optical model lidar p i The pixel coordinates of .

[0027] Furthermore, the step of fusing the rough extraction results of the point cloud of the spatial moving target based on the iterative matching multi-view point cloud fusion method to obtain the point cloud fusion result of the spatial moving target specifically includes:

[0028] Determine the reference point cloud and obtain the k-th frame of the point cloud coarse extraction result of the spatial moving target;

[0029] Iteratively solve the rotation matrix and translation vector of the k-th frame's neighboring point cloud aligned to the reference point cloud based on the iterative closest point algorithm;

[0030] A constrained nonlinear optimization equation is introduced to iteratively match the k-th frame's adjacent point cloud with the reference point cloud, and the matching results are judged.

[0031] If it is determined that the matching result is greater than the preset threshold, the rotation matrix and translation vector of the adjacent point cloud of the kth frame are updated to align with the reference point cloud and match again until the matching result is less than the preset threshold. The adjacent point cloud of the kth frame and the reference point cloud are unified and fused in turn to obtain the point cloud fusion result of the spatial moving target.

[0032] Furthermore, the expression of the constrained nonlinear optimization equation is specifically as follows:

[0033]

[0034]

[0035] In the above formula, ref R k Indicates the rotation matrix of the k-th frame adjacent point cloud aligned to the reference point cloud, ref t k Indicates the translation vector of the k-th frame adjacent point cloud aligned to the reference point cloud, p k,i Indicates the k-th frame of adjacent point cloud data, p ref,j Represents the reference point cloud data, N k Indicates the number of 3D points contained in the adjacent point cloud of the kth frame.

[0036] Furthermore, the feature-based and learning hybrid method performs three-dimensional reconstruction on the point cloud fusion result of the spatial moving target and autonomously identifies the geometric parameters of the spatial moving target based on the reconstruction result to obtain the identification result. This step specifically includes:

[0037] Based on the iterative 3D reconstruction method of spatial moving targets, the point cloud fusion results of spatial moving targets are decomposed and optimized to obtain the optimal parameters of the orthogonal plane.

[0038] Combining the optimal parameters of the orthogonal planes, the three-dimensional reconstruction of the space moving target is performed based on the optimized three-dimensional reconstruction method of the space moving target to obtain the three-dimensional reconstruction result of the space moving target;

[0039] Based on the three-dimensional reconstruction result of the spatial motion target, the geometric parameters of the spatial motion target are self-identified to obtain an identification result, which includes the vertex parameters of the spatial motion target, the size vector of the spatial motion target, the coordinate axis of the spatial motion target and the center of the spatial motion target.

[0040] Furthermore, the iterative method for 3D reconstruction of a moving object in space decomposes and optimizes the point cloud fusion result of the moving object in space to obtain the optimal parameters of the orthogonal plane, which specifically includes:

[0041] Decomposing the point cloud fusion result of the spatial moving target to obtain three single planes of the spatial moving target, wherein the three single planes of the spatial moving target are orthogonal to each other;

[0042] Based on a single plane of a spatial moving target, considering the constraint relationship of the plane normal vector, three non-collinear points are selected to obtain six spatial points that meet the preset spatial relationship conditions;

[0043] The preset spatial relationship condition is that the first spatial point, the second spatial point, and the third spatial point are located in the first plane and are not collinear, the fourth spatial point and the fifth spatial point are located in the second plane and the straight line formed by the fourth spatial point and the fifth spatial point is not perpendicular to the first plane, and the sixth spatial point is located in the third plane;

[0044] Based on the three single planes and six spatial points of the spatial motion target, mathematical models of three orthogonal planes are constructed, wherein the mathematical models of the three orthogonal planes include normal vectors and offsets of the corresponding planes;

[0045] The mathematical model of the orthogonal planes is estimated by a random sampling consensus algorithm to obtain the optimal parameters of the orthogonal planes.

[0046] Furthermore, the step of performing three-dimensional reconstruction processing on the spatial moving target based on the optimized spatial moving target three-dimensional reconstruction method in combination with the optimal parameters of the orthogonal planes to obtain a three-dimensional reconstruction result of the spatial moving target specifically includes:

[0047] The iterative single-plane fitting algorithm is used to segment the three planes of the spatial moving target and filter out outliers to obtain the coarse parameters of the corresponding planes.

[0048] The mathematical model of the plane is optimized according to the rough parameters of the plane to obtain the overdetermined equations of the corresponding plane;

[0049] Transform the overdetermined equations of the plane to obtain the least squares problem of the corresponding plane;

[0050] The corresponding plane is fitted according to the least square problem of the corresponding plane to obtain the fitting result of the corresponding plane;

[0051] The fitting results of each plane are combined to construct the three-dimensional reconstruction result of the spatial moving target.

[0052] The second technical solution adopted by the present invention is: a geometric parameter identification system based on three-dimensional reconstruction of a space moving target, comprising:

[0053] The multi-source data acquisition module is used to collect visual image data and laser point cloud data of spatial moving targets through a monocular camera and a multi-line laser radar, and to perform multiple acquisitions based on different perspectives to obtain monocular vision image data and sparse point cloud data;

[0054] The multi-source data segmentation module is used to perform pixel-level segmentation and fitting of monocular vision image data using the autonomous visual region segmentation method to construct an optical model of a spatial moving target;

[0055] The multi-source data fusion module fuses the sparse point cloud data of the sparse radar with the optical model of the space moving target based on the visual mapping method to obtain the point cloud rough extraction result of the space moving target;

[0056] The multi-source data extraction module fuses the rough extraction results of the point cloud of the spatial moving target based on the iterative matching multi-view point cloud fusion method to obtain the point cloud fusion result of the spatial moving target;

[0057] The multi-source data reconstruction module reconstructs the point cloud fusion results of the spatial moving target in three dimensions based on a feature and learning hybrid method, and autonomously identifies the geometric parameters of the spatial moving target based on the reconstruction results to obtain the identification results.

[0058] The beneficial effects of the method and system of the present invention are as follows: the present invention obtains monocular vision image data and sparse point cloud data, performs pixel-level segmentation and fitting on the monocular vision image data through an autonomous visual region segmentation method, reduces the influence of image noise and obstructions, and uses image expansion technology to fill in the incomplete parts of the spatial moving target, thereby realizing the "pixel-color-category" association. Further, based on the visual mapping method, the sparse point cloud data of the sparse radar is fused with the optical model of the spatial moving target to determine whether the three-dimensional point belongs to the spatial moving target, thereby realizing the autonomous point cloud region segmentation of the spatial moving target and the "pixel-color-category-depth" association. Furthermore, based on the iterative matching spatial moving target multi-view point cloud fusion method, the rough extraction results of the point cloud of the spatial moving target are fused, and adjacent multi-view point clouds can be aligned by coordinate transformation to achieve information complementarity between the multi-view point clouds, thereby improving the effect of three-dimensional reconstruction and the accuracy of information extraction. That is, through multi-source fusion, information complementarity between multi-view vision and point cloud can be achieved, thereby improving the accuracy, stability and timeliness of three-dimensional reconstruction and autonomous identification of geometric parameters of spatial moving targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of the steps of the geometric parameter identification method based on three-dimensional reconstruction of a space moving target of the present invention;

[0060] Figure 2 It is a structural block diagram of the geometric parameter identification system based on three-dimensional reconstruction of space moving targets of the present invention;

[0061] Figure 3 It is a coordinate relationship diagram of the monocular vision and sparse radar measurement system of the present invention;

[0062] Figure 4 It is a block diagram of the feature-based autonomous visual area segmentation of the present invention;

[0063] Figure 5It is a flow chart of the feature-based autonomous visual region segmentation of the present invention;

[0064] Figure 6 This is a schematic diagram of the autonomous region segmentation process based on vision and point cloud fusion of the present invention;

[0065] Figure 7 This is a flowchart of the multi-view point cloud fusion of spatial moving targets based on iterative matching in the present invention;

[0066] Figure 8 Schematic diagram of the fused point cloud result of the spatial motion target of the present invention;

[0067] Figure 9 It is a schematic diagram of a model of a space motion target of the present invention;

[0068] Figure 10 is a fitted dendrogram of the first plane of the present invention;

[0069] Figure 11 is a fitted dendrogram of the second plane of the present invention;

[0070] Figure 12 3D reconstruction result of a space moving target based on the iterative space moving target 3D reconstruction method of the present invention;

[0071] Figure 13 3D reconstruction of a moving object in space is performed based on the optimized 3D reconstruction method of the moving object in space according to the present invention;

[0072] Figure 14 It is a histogram of geometric parameter errors of a space moving target based on the iterative space moving target three-dimensional reconstruction method of the present invention;

[0073] Figure 15 It is a histogram of geometric parameter errors of a space moving target based on the optimized space moving target three-dimensional reconstruction method of the present invention. DETAILED DESCRIPTION

[0074] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0075] Reference Figure 1 The present invention provides a method for identifying geometric parameters based on three-dimensional reconstruction of a space moving target, the method comprising the following steps:

[0076] S1. Use a monocular camera and a multi-line laser radar to collect visual image data and laser point cloud data of a moving target in space, and perform multiple acquisitions based on different perspectives to obtain monocular vision image data and sparse point cloud data;

[0077] Specifically, a monocular camera and a multi-line laser radar are used to collect visual image data and laser point cloud data of a moving target in space, and multiple collections are performed based on different perspectives to obtain monocular vision image data and sparse point cloud data. The forward direction of the multi-line laser radar is the X-axis, the left direction is the Y-axis, and the vertical upward direction is the Z-axis. The right direction of the monocular camera is the X-axis, the vertical downward direction is the Y-axis, and the forward direction is the Z-axis.

[0078] Reference Figure 3 , the coordinate system {cam} represents the camera coordinate system, the coordinate system {lidar} represents the sparse radar coordinate system, and the coordinate system {img} represents the image coordinate system. cam T lidar 、 cam R lidar and cam t lidar Respectively represent the homogeneous coordinate transformation matrix, rotation matrix and translation vector of {lidar} relative to {cam}, cam p i =[ cam x i , cam y i , cam z i ] T and lidar p i =[ lidar x i , lidar y i , lidar z i ] T They represent the Cartesian coordinates of the i-th point on the spatial moving target relative to the coordinate systems {cam} and {lidar}, respectively, and s i =[u i ,v i ] T Represents the pixel coordinates of the point projected on the image plane.

[0079] S2. Perform pixel-level segmentation and fitting of monocular vision image data using autonomous visual region segmentation to construct an optical model of a moving target in space.

[0080] Specifically, refer to Figure 4 and Figure 5,This paper proposes a feature-based autonomous visual region segmentation method: first, the spatial moving target is preliminarily segmented according to the ,color features; second, the outer contour of the spatial moving target ,is fitted according to the shape characteristics.

[0081] S21, segmentation based on color features;

[0082] Specifically, compared to RGB space, HSV space better expresses image hue characteristics, making it easier to compare and extract the colors of moving objects. Therefore, we use hue thresholds to perform preliminary segmentation on the HSV image of moving objects. To reduce the effects of noise and occlusions, we use image dilation techniques to fill in missing parts of moving objects.

[0083] S22, fitting based on appearance features;

[0084] Specifically, first, to eliminate background interference on the moving object, the image's outline is initially extracted and filtered. Second, straight lines are fitted along the edge of the moving object. Third, valid intersection points of the edge lines are extracted. The extracted vertices are sorted using polar coordinates and then connected end-to-end to form a model of the moving object.

[0085] S3. Based on the visual mapping method, the sparse point cloud data of the sparse radar is integrated with the optical model of the space moving target to obtain the point cloud rough extraction result of the space moving target;

[0086] Specifically, refer to Figure 6 , according to the pinhole imaging principle, the optical model of monocular vision can be expressed as:

[0087]

[0088] in, and A ex =[ cam R lidar , cam t lidar ] represent the intrinsic parameter matrix and extrinsic parameter matrix of monocular vision respectively;

[0089] Based on the optical model of monocular vision, the point cloud of a moving target can be mapped to the image plane to obtain the corresponding pixel coordinates. If the pixel coordinates are within the effective field of view of the monocular vision, the 3D point can be matched with the pixel point to form a "pixel-color-depth" association. Since the vertices of the moving target in the image plane have been extracted, the nearest neighbor strategy can be used to match the vertices in the point cloud space and the visual space. Therefore, the constrained nonlinear optimization problem can be expressed as:

[0090]

[0091]

[0092] ( cam R lidar ) Tcam R lidar =I

[0093]

[0094] In the above formula, Optical model representing a moving target in space, A in Represents the intrinsic parameter matrix of monocular vision, A ex represents the external parameter matrix of monocular vision, lidar represents the sparse radar coordinate system, cam represents the camera coordinate system, s i represents the pixel coordinates of the point i projected on the image plane, cam R lidar Represents the rotation matrix of lidar relative to cam, ω i represents the field of view of monocular vision, Indicates the calculated value based on the optical model lidar p i The pixel coordinates of

[0095] Hypothesis I obj and I mask They represent the original image and mask image of the spatial moving target respectively, and the relationship between the two can be expressed as:

[0096]

[0097] Among them, I obj (s) and I mask (s) represent I obj and I mask The value at pixel coordinate s.

[0098] In addition, in autonomous visual region segmentation, the "pixel-color-category" association is realized. Ultimately, combining the two associations can obtain the "pixel-color-category-depth" association. Based on this association, it can be determined whether a 3D point belongs to a spatial moving target, thereby realizing autonomous point cloud region segmentation of spatial moving targets, that is:

[0099]

[0100] S4, based on the iterative matching of the multi-view point cloud fusion method of the spatial moving target, the point cloud rough extraction results of the spatial moving target are fused to obtain the point cloud fusion result of the spatial moving target;

[0101] Specifically, refer to Figure 7 and Figure 8 The single-frame point cloud of a moving target is relatively sparse, which results in poor 3D reconstruction and low accuracy in extracting key information. In fact, during the movement of a moving target, a sparse radar can collect point clouds of the moving target from different perspectives. If adjacent multi-perspective point clouds can be aligned through coordinate transformation, information complementarity between the multi-perspective point clouds can be achieved, thereby improving the 3D reconstruction effect and the accuracy of information extraction. Based on this, the present invention proposes a multi-point cloud fusion algorithm for moving targets based on iterative optimization.

[0102] Assume P ref ={p ref，j |j=1,2,…,N ref} represents the reference point cloud, P k ={p k，i |i=1,2,…,N k} represents the kth frame adjacent point cloud, where k = 1, 2, ... K and K ≥ 1. k Align to P ref As an example, we first use the iterative closest point algorithm to iteratively solve P k Align to P ref The rotation matrix ref R k and translation vectors ref t k , the constrained nonlinear optimization equation can be expressed as:

[0103]

[0104]

[0105] In the above formula, ref R k Indicates the rotation matrix of the k-th frame adjacent point cloud aligned to the reference point cloud, ref t k Indicates the translation vector of the k-th frame adjacent point cloud aligned to the reference point cloud, p k，i Indicates the k-th frame of adjacent point cloud data, p ref，j Represents the reference point cloud data, N k Indicates the number of 3D points contained in the adjacent point cloud of the kth frame;

[0106] Assume that p k，i and p ref，j If they match each other, the transformation relationship between the two point clouds can be expressed as:

[0107] p ref，j = ref R k p k，i + ref tk

[0108] Second, we use ref R k and ref t k P k Unified with P ref The corresponding reference coordinate system is used to obtain the aligned point cloud Q k Finally, all aligned point clouds are fused. Therefore, the fused point cloud P fusion (Its initialization value is P ref ) can be expressed as:

[0109]

[0110] S5. Based on the feature and learning hybrid method, the point cloud fusion results of the spatial moving target are three-dimensionally reconstructed and the geometric parameters of the spatial moving target are autonomously identified according to the reconstruction results to obtain the identification results.

[0111] Specifically, refer to Figure 9 , in terms of the orthogonal plane parameters of the space moving target, n1=[n 1x , n 1y , n 1z ] T and k1 represent the normal vector and offset of the first plane respectively; n2=[n 2x , n 2y , n 2z ] T and k2 represent the normal vector and offset of the second plane respectively; n3=[n 3x , n 3y , n 3z ] T and k3 represent the normal vector and offset of plane 3 respectively. In terms of the geometric parameters of the space moving target, a obj =[a1, a2, a3] T Represent the size vector, p v0 and p c Represent the vertex and center respectively, x obj 、y obj and z obj Represents three coordinate axes respectively, and R obj =[x obj ,y obj , z obj ] T In addition, let the three coordinate axes of the spatial moving target be parallel to the normal vectors of the three orthogonal planes, that is, x obj / / n1,y obj / / n2 and z obj / / n3.

[0112] S51. Three-dimensional reconstruction method of spatial moving objects based on iteration;

[0113] Specifically, the random sampling consensus algorithm can estimate the model parameters of the target in an iterative manner. Since a plane has three degrees of freedom, the idea of ​​fitting a single plane is: first, randomly sample three non-collinear points from the observation point cloud, then solve the plane parameters according to the theorem of "three points determine a plane", and finally use the solved parameters to calculate the number / proportion of internal points in the observation point cloud, so as to iteratively update the optimal parameters of the single plane model. Based on this idea, we decompose the spatial moving target into three single planes. In theory, at least nine points are required to fit these three single planes. However, since the three planes of the spatial moving target are orthogonal to each other, this provides three constraints on the normal vector. Therefore, at least six points are required to fit the three orthogonal planes of the spatial moving target.

[0114] The three orthogonal planes of the spatial moving target can be modeled using six points p1, p2, p3, p4, p5, and p6 in space, which need to meet the following conditions:

[0115] (1) Six points are distributed on three planes, that is, p1, p2, and p3 are on the first plane, p4 and p5 are on the second plane, and p6 is on plane 3;

[0116] (2) The three points on the first plane are not collinear, that is, p1, p2, and p3 are not collinear;

[0117] (3) The straight line formed by the two points on the second plane is not perpendicular to the first plane, that is, the vector v 45 =p4-p5 is not perpendicular to the second plane.

[0118] Furthermore, according to the plane equation n T With p+k=0 and the corresponding normal vector orthogonality constraints, we can establish mathematical models of three orthogonal planes:

[0119] (1) The normal vector n1 and offset k1 of the first plane can be expressed as:

[0120]

[0121] (2) According to the orthogonality between the first plane and the second plane, the normal vector n2 of the second plane is simultaneously with n1 and v 45 If requirement 3 is not met, then n1 and v 45 Parallel, missing a constraint on n2. Therefore, based on p4, p5 and n1, the normal vector n2 and offset k2 of the second plane can be expressed as:

[0122]

[0123] (3) Since the three planes are orthogonal, the normal vectors of the three planes are perpendicular to each other. Therefore, the normal vector n3 and offset k3 of plane 3 can be expressed as:

[0124]

[0125] Finally, the random sampling consensus algorithm is used to estimate the optimal parameters of the orthogonal planes. The iterative 3D reconstruction algorithm can also filter out outliers in the process of fitting the spatial moving object, and the single-step fitting procedure is simple.

[0126] S52. Three-dimensional reconstruction method of spatial moving objects based on optimization;

[0127] Specifically, iterative 3D reconstruction algorithms only use a few points to fit the moving object, losing information about most of the points. Optimization-based methods can address this issue by comprehensively considering information from all points, but they are susceptible to the influence of outliers. Therefore, before using optimization-based methods to reconstruct the 3D moving object, the point cloud needs to be divided into three parts corresponding to the three planes. Thanks to the simplicity of the single-step fitting procedure, an iterative single-plane fitting algorithm is used to segment the three planes of the moving object and filter out outliers. This method can also extract the coarse parameters of the planes.

[0128] According to the normal vector, the mathematical model of the plane can be divided into three cases, namely n 1x ≠0, n 1y ≠0 and n 1z ≠0. When choosing a reasonable mathematical model, use the plane rough parameters to make a judgment. x ≠0, the equation of the first plane is Can be simplified to:

[0129] m 11 y+m 12 z+m 13 =x

[0130] in,

[0131] Assume P1 = {p i =[x i ,y i , z i ] T |i=1,2,…,N1} represents the inner point cloud of the first plane. According to the formula, we can establish an overdetermined set of equations about m1:

[0132]

[0133] The formula can be transformed into a least squares problem:

[0134]

[0135] Finally, the closed-form solution is In addition, the singular value decomposition (SVD) method can also be used to solve the formula, such as Figure 10 shown.

[0136] Reference Figure 11 , with n 2x ≠0, the equation of the second plane is Can be simplified to:

[0137] m 12 y+m 22 z+m 23 =x

[0138] in,

[0139] Based on the orthogonality of the first plane and the second plane, the mathematical relationship between the normal vectors of the two planes is established:

[0140]

[0141] Similarly, according to the formula and, we can establish the least squares problem with linear constraints on m2:

[0142]

[0143]

[0144] Where A2 and b2 represent the coefficient matrix and constant vector of the overdetermined equations about m2, and k1=[m 11 m 12 0] T ;

[0145] The normal vector of plane 3 can be expressed as n3=n1×n2, and the least squares solution of k3 can be expressed as:

[0146]

[0147] S53, autonomous identification method of geometric parameters of space moving targets;

[0148] S531, the vertex of the spatial moving target;

[0149] Specifically, for the spatial moving target vertex p v0 To solve the problem, we only need to solve the general equations of three orthogonal planes n T p+k=0, and its solution is pv0 Therefore, p v0 It can be expressed as:

[0150]

[0151] Where N = [n x , n y , n z ],B1=[-k,n y , n z ],B2=[n x , -k, n z ],B3=[n x , n y ,-k],n x =[n 1x , n 2x , n 3x ] T , n y =[n 1y , n2 y , n 3y ] T , n z =[n 1z , n 2z , n 3z ] T , k=[k1,k2,k3] T .

[0152] S532, size vector of the spatial moving target;

[0153] Specifically, for the spatial motion target size vector a SNCT The solution is as follows: first, calculate the maximum distance from the inner point cloud to the i-th plane; second, select n far The farthest points corresponding to the i-th plane form the farthest point cloud Q i , and for Q i Perform linear fitting to obtain the straight line L i ; Finally, a i It can be expressed as the i-th plane and L i The distance between

[0154] S533, coordinate axis of the space moving target;

[0155] Specifically, for the spatial motion target coordinate axis x SNCT 、y SNCT and Z SNCT The solution is as follows: First, calculate the inner point cloud P of the i-th plane i The centroid projection point p c，i ; Secondly, calculate p c，iProjection point q to the jth (j≠i) plane i，j ; Finally, the coordinate axis v corresponding to the kth (k≠i and k≠j) plane can be expressed as:

[0156]

[0157] S534, the center of a moving target in space;

[0158] Specifically, the center of the spatial moving target coordinates can be expressed as:

[0159] p c =p v0 +R SNCT a SNCT

[0160] Reference Figure 2 , a geometric parameter identification system based on three-dimensional reconstruction of space moving targets, including:

[0161] The multi-source data acquisition module is used to collect visual image data and laser point cloud data of spatial moving targets through a monocular camera and a multi-line laser radar, and to perform multiple acquisitions based on different perspectives to obtain monocular vision image data and sparse point cloud data;

[0162] The multi-source data segmentation module is used to perform pixel-level segmentation and fitting of monocular vision image data using the autonomous visual region segmentation method to construct an optical model of a spatial moving target;

[0163] The multi-source data fusion module fuses the sparse point cloud data of the sparse radar with the optical model of the space moving target based on the visual mapping method to obtain the point cloud rough extraction result of the space moving target;

[0164] The multi-source data extraction module fuses the rough extraction results of the point cloud of the spatial moving target based on the iterative matching multi-view point cloud fusion method to obtain the point cloud fusion result of the spatial moving target;

[0165] The multi-source data reconstruction module reconstructs the point cloud fusion results of the spatial moving target in three dimensions based on a feature and learning hybrid method, and autonomously identifies the geometric parameters of the spatial moving target based on the reconstruction results to obtain the identification results.

[0166] Further numerical simulation is performed based on the method of the present invention. In the experimental platform, the space moving target is controlled to move along a certain trajectory. Figure 12 、 Figure 13 、 Figure 14 and Figure 15 As shown, where:

[0167] for Figure 12This is the 3D model reconstruction result based on the iterative 3D reconstruction method for spatial moving targets. The reconstructed model fits the point cloud of the spatial moving target well.

[0168] for Figure 13 The 3D model reconstruction result based on the optimized 3D reconstruction method of space moving targets is shown in the figure. The reconstructed model better fits the point cloud of the space moving target.

[0169] for Figure 14 The three-dimensional geometric parameter identification error histograms of the iterative 3D reconstruction method for a moving object in space are shown in the following figure. The three figures represent the geometric parameter errors on the X, Y, and Z axes, respectively. The horizontal axis represents ten frames of data collected during the motion of the moving object in space. It can be seen that the errors are small (the average values ​​of the X, Y, and Z axis errors are 5.5609mm, 9.5463mm, and 6.8343mm, respectively) and the robustness is high (the standard deviations of the X, Y, and Z axis errors are 3.9431mm, 7.8998mm, and 3.3252mm, respectively).

[0170] for Figure 15 The following are histograms of the 3D geometric parameter identification errors for the optimized 3D reconstruction method for a moving space target. The three figures represent the geometric parameter errors along the X, Y, and Z axes, respectively. The horizontal axis represents ten frames of data collected during the motion of the moving space target. The results show lower errors (the average values ​​of the X, Y, and Z axis errors are 0.9837mm, 4.8449mm, and 1.4981mm, respectively) and higher robustness (the standard deviations of the X, Y, and Z axis errors are 0.6638mm, 0.9897mm, and 0.9437mm, respectively).

[0171] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0172] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A geometric parameter identification method based on three-dimensional reconstruction of a moving target in space, characterized in that: The following steps are involved: The visual image data and laser point cloud data of the moving target in space are collected by a monocular camera and a multi-line laser radar, and multiple acquisitions are performed based on different perspectives to obtain monocular vision image data and sparse point cloud data; The autonomous visual region segmentation method is used to segment and fit the monocular vision image data at the pixel level to construct an optical model of the spatial moving target. Based on the visual mapping method, the sparse point cloud data of the sparse radar is fused with the optical model of the space moving target to obtain the rough point cloud extraction result of the space moving target; Based on the iterative matching method of multi-view point cloud fusion of spatial moving targets, the rough extraction results of the point cloud of the spatial moving target are fused to obtain the point cloud fusion results of the spatial moving target; Based on the feature and learning hybrid method, the point cloud fusion results of the spatial moving target are 3D reconstructed and the geometric parameters of the spatial moving target are autonomously identified according to the reconstruction results to obtain the identification results; The method based on feature and learning hybrid performs three-dimensional reconstruction on the point cloud fusion result of the spatial moving target and autonomously identifies the geometric parameters of the spatial moving target based on the reconstruction result to obtain the identification result. This step specifically includes: Based on the iterative 3D reconstruction method of spatial moving targets, the point cloud fusion results of spatial moving targets are decomposed and optimized to obtain the optimal parameters of the orthogonal plane. Combining the optimal parameters of the orthogonal planes, the three-dimensional reconstruction of the space moving target is performed based on the optimized three-dimensional reconstruction method of the space moving target to obtain the three-dimensional reconstruction result of the space moving target; Based on the three-dimensional reconstruction result of the spatial motion target, the geometric parameters of the spatial motion target are self-identified to obtain an identification result, which includes the vertex parameters of the spatial motion target, the size vector of the spatial motion target, the coordinate axis of the spatial motion target and the center of the spatial motion target.

2. The method for geometric parameter identification based on three-dimensional reconstruction of a moving object in space according to claim 1, characterized in that: The step of performing pixel-level segmentation and fitting on the monocular vision image data by the autonomous visual region segmentation method to construct an optical model of the spatial moving target specifically includes: Perform spatial conversion on the monocular vision image data of the space moving target to obtain the HSV image of the space moving target; Set the preset color segmentation threshold to segment the HSV image of the spatial moving target and obtain the preliminary segmentation result; The incomplete parts of the spatial moving target in the preliminary segmentation result are filled by the image expansion technology to obtain the filled image of the spatial moving target; Extract and filter the contour of the filled image of the space moving target to obtain the edge contour straight line of the space moving target; Fit the edge contour line of the space moving target and extract the effective vertices of the edge line; Based on the polar coordinate idea, the valid vertices of the edge line are sorted and connected end to end according to the sorting results to construct an optical model of the spatial moving target, that is, a mask image of the spatial moving target.

3. The method for geometric parameter identification based on three-dimensional reconstruction of a moving object in space according to claim 2, characterized in that: The step of fusing the sparse point cloud data of the sparse radar with the optical model of the space moving target based on the visual mapping method to obtain a rough point cloud extraction result of the space moving target specifically includes: According to the optical model of the space moving target, the sparse point cloud data of the space moving target is mapped to obtain the corresponding pixel coordinates; Matching pixel coordinates within the effective field of view of monocular vision with three-dimensional points to obtain the vertex of the spatial moving target in the point cloud space, where the three-dimensional points are three-dimensional points in the point cloud of the spatial moving target; A constrained nonlinear optimization problem is introduced, and the nearest neighbor strategy is used to match the vertices of the spatial moving target in the point cloud space with the vertices in the visual space, i.e., the valid vertices, to obtain the point cloud rough extraction result of the spatial moving target.

4. The method for geometric parameter identification based on three-dimensional reconstruction of a moving object in space according to claim 3, characterized in that: The expression of the constrained nonlinear optimization problem is specifically as follows: ( cam R lidar ) T ( cam R lidar )=I In the above formula, Optical model representing a moving target in space, A in Represents the intrinsic parameter matrix of monocular vision, A ex represents the external parameter matrix of monocular vision, lidar represents the sparse radar coordinate system, cam represents the camera coordinate system, s i represents the pixel coordinates of point i projected on the image plane, cam R lidar Represents the rotation matrix of lidar relative to cam, ω i represents the field of view of monocular vision, Indicates the calculated value based on the optical model lidar p i The pixel coordinates of .

5. The method for geometric parameter identification based on three-dimensional reconstruction of a moving object in space according to claim 4, characterized in that: The step of fusing the rough extraction results of the point cloud of the spatial moving target based on the iterative matching multi-view point cloud fusion method to obtain the point cloud fusion result of the spatial moving target specifically includes: Determine the reference point cloud and obtain the k-th frame of the point cloud coarse extraction result of the spatial moving target; Iteratively solve the rotation matrix and translation vector of the k-th frame's neighboring point cloud aligned to the reference point cloud based on the iterative closest point algorithm; A constrained nonlinear optimization equation is introduced to iteratively match the k-th frame's adjacent point cloud with the reference point cloud, and the matching results are judged. If it is determined that the matching result is greater than the preset threshold, the rotation matrix and translation vector of the adjacent point cloud of the kth frame are updated to align with the reference point cloud and match again until the matching result is less than the preset threshold. The adjacent point cloud of the kth frame and the reference point cloud are unified and fused in turn to obtain the point cloud fusion result of the spatial moving target.

6. The method for geometric parameter identification based on three-dimensional reconstruction of a moving object in space according to claim 5, characterized in that: The expression of the constrained nonlinear optimization equation is specifically as follows: s.t.( refR k ) T ( refR k )=I In the above formula, ref R k Indicates the rotation matrix of the k-th frame adjacent point cloud aligned to the reference point cloud, ref t k Indicates the translation vector of the k-th frame adjacent point cloud aligned to the reference point cloud, p k,i Indicates the k-th frame of adjacent point cloud data, p ref,j Represents the reference point cloud data, N k Indicates the number of 3D points contained in the adjacent point cloud of the kth frame.

7. The method for geometric parameter identification based on three-dimensional reconstruction of a moving object in space according to claim 6, characterized in that: The iterative three-dimensional reconstruction method for spatial moving objects decomposes and optimizes the point cloud fusion results of the spatial moving object to obtain the optimal parameters of the orthogonal plane, which specifically includes: Decomposing the point cloud fusion result of the spatial moving target to obtain three single planes of the spatial moving target, wherein the three single planes of the spatial moving target are orthogonal to each other; Based on a single plane of a spatial moving target, considering the constraint relationship of the plane normal vector, three non-collinear points are selected to obtain six spatial points that meet the preset spatial relationship conditions; The preset spatial relationship condition is that the first spatial point, the second spatial point, and the third spatial point are located in the first plane and are not collinear, the fourth spatial point and the fifth spatial point are located in the second plane and the straight line formed by the fourth spatial point and the fifth spatial point is not perpendicular to the first plane, and the sixth spatial point is located in the third plane; Based on the three single planes and six spatial points of the spatial motion target, mathematical models of three orthogonal planes are constructed, wherein the mathematical models of the three orthogonal planes include normal vectors and offsets of the corresponding planes; The mathematical model of the orthogonal planes is estimated by a random sampling consensus algorithm to obtain the optimal parameters of the orthogonal planes.

8. The method for geometric parameter identification based on three-dimensional reconstruction of a space moving object according to claim 7, characterized in that: The step of performing three-dimensional reconstruction processing on the spatial moving target based on the optimized spatial moving target three-dimensional reconstruction method in combination with the optimal parameters of the orthogonal planes to obtain the three-dimensional reconstruction result of the spatial moving target specifically includes: The iterative single-plane fitting algorithm is used to segment the three planes of the spatial moving target and filter out outliers to obtain the coarse parameters of the corresponding planes. The mathematical model of the plane is optimized according to the rough parameters of the plane to obtain the overdetermined equations of the corresponding plane; Transform the overdetermined equations of the plane to obtain the least squares problem of the corresponding plane; The corresponding plane is fitted according to the least square problem of the corresponding plane to obtain the fitting result of the corresponding plane; The fitting results of each plane are combined to construct the three-dimensional reconstruction result of the spatial moving target.

9. A geometric parameter identification system based on three-dimensional reconstruction of a moving target in space, characterized in that: Used to execute the geometric parameter identification method based on three-dimensional reconstruction of a space moving target as claimed in claim 1, Includes the following modules: The multi-source data acquisition module is used to collect visual image data and laser point cloud data of spatial moving targets through a monocular camera and a multi-line laser radar, and to perform multiple acquisitions based on different perspectives to obtain monocular vision image data and sparse point cloud data; The multi-source data segmentation module is used to perform pixel-level segmentation and fitting of monocular vision image data using the autonomous visual region segmentation method to construct an optical model of a spatial moving target; The multi-source data fusion module fuses the sparse point cloud data of the sparse radar with the optical model of the space moving target based on the visual mapping method to obtain the point cloud rough extraction result of the space moving target; The multi-source data extraction module fuses the rough extraction results of the point cloud of the spatial moving target based on the iterative matching multi-view point cloud fusion method to obtain the point cloud fusion result of the spatial moving target; The multi-source data reconstruction module reconstructs the point cloud fusion results of the spatial moving target in three dimensions based on a feature and learning hybrid method, and autonomously identifies the geometric parameters of the spatial moving target based on the reconstruction results to obtain the identification results.

Images