A method for three-dimensional reconstruction of a grid structure by fusing unmanned aerial vehicle video and point cloud data

Abstract

The application discloses a kind of net rack structure three-dimensional reconstruction methods of fusing unmanned aerial vehicle video and point cloud data.The method detects and cross-frame association in video ball node, obtains the observation sequence of ball node with continuous identity mark, and obtains the initial three-dimensional coordinates of ball node with continuous identity mark by space target ray intersection;It is used as video semantic anchor point to generate candidate bar connection relationship;Along candidate connection relationship, mid-point cloud of bar is extracted, and bar axis, bar radius and both end node identity mark are obtained by cylindrical model fitting and verification;Then, the reconstruction coordinates of ball node are obtained by using the intersection of multiple effective bar axes, and the radius of ball node is obtained by spherical fitting, finally, the parametric three-dimensional model containing node, bar and topological connection relationship is generated.The application can reduce the influence of dense node area point cloud aliasing and single frame image matching instability on reconstruction results, and improve the efficiency of net rack structure inspection and three-dimensional reconstruction.

Description

Technical Field

[0001] This invention relates to the field of civil engineering structure inspection technology, specifically to a method for three-dimensional reconstruction of a grid structure that integrates UAV video and point cloud data. Background Technology

[0002] Space frame structures are widely used in large-span spatial structures. These structures are typically composed of numerous spherical nodes and members, characterized by a large number of nodes, dense members, and complex spatial configurations. During the construction phase, it is necessary to inspect the node coordinates, member installation status, and overall geometry. During the service phase, it is also necessary to obtain the actual geometric state of the structure to provide basic data for structural safety assessment, deformation monitoring, and digital twin modeling.

[0003] While terrestrial 3D laser scanning can acquire high-precision point cloud data, it typically requires deploying multiple stations around or inside the structure for large-span space frame structures, followed by multi-station point cloud registration. For space frame structures with high roof heights, large spans, limited internal spaces, or complex construction site environments, terrestrial scanning involves a significant workload in terms of on-site organization, limiting acquisition efficiency and making it difficult to meet the needs of repeated inspections during construction and service phases. In contrast, UAV platforms can carry RGB cameras and LiDAR equipment to quickly acquire large-area video and point cloud data from above or to the side of the structure, reducing the need for manual access to high-altitude structures and the deployment of multiple stations. This makes them more suitable for low-cost, rapid, and large-area inspections of large-span space frame structures. However, point clouds acquired by UAVs often exhibit characteristics such as unstructured data, partial occlusion, and overlapping point clouds in nodal areas, requiring further component-level analysis.

[0004] Existing point cloud reconstruction methods typically extract spherical nodes and cylindrical members directly from within the point cloud, and then determine the connection relationships of the members based on their geometric positions. In densely populated areas, these methods are easily affected by the mixing of spherical node point clouds, member end point clouds, and point clouds of adjacent members, leading to misidentification, omission of members, or incorrect determination of topological relationships.

[0005] Other methods utilize images or videos to assist point cloud processing, but these often focus on image-point cloud projection matching, pose registration, or local target localization. For space frame structures, multiple sphere nodes and members are prone to overlap on the image plane, and the same physical sphere node may appear repeatedly from different viewpoints. If only single-frame image detection results are used to assist point cloud processing, it is difficult to stably maintain the consistency of the identity of the same sphere node in multi-view observations, and it is also difficult to form reliable member topological constraints.

[0006] Therefore, there is a need for a method that can leverage the advantages of continuous observation of large areas of UAV video to transform the continuous identity information of sphere nodes in the video into semantic anchors for point cloud processing, and further guide the identification of rods, topology inference, and node coordinate reconstruction in lidar point clouds, so as to achieve integrated three-dimensional reconstruction of nodes, rods, and topological relationships of grid structures. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for three-dimensional reconstruction of a grid structure that integrates UAV video and point cloud data.

[0008] To achieve the above objectives, this invention provides a method for three-dimensional reconstruction of a grid structure by fusing UAV video and point cloud data, comprising: Acquire RGB video data, LiDAR point cloud data, camera parameters, and UAV pose data of the structure under test; the structure under test includes ball nodes and rods connecting the ball nodes; Target detection and cross-frame association are performed on sphere nodes in RGB video data to obtain sphere node observation sequences with continuous identification; based on camera parameters and UAV pose data, multi-frame sphere node 2D observations with the same continuous identification are converted into spatial target rays, and the initial 3D coordinates of sphere nodes with continuous identification are obtained through spatial intersection. The initial 3D coordinates of the ball nodes with persistent identity are used as video semantic anchors to generate candidate rod connection relationships, retaining the persistent identity of the ball nodes at both ends; the mid-segment point cloud of the rod is extracted from the LiDAR point cloud data along the candidate rod connection relationship, and the rod axis, rod radius, and persistent identity of the ball nodes at both ends of the valid rod are obtained through cylindrical model fitting and validity verification. Based on the axis intersection positioning of multiple effective members connected to the same ball node, the reconstructed coordinates and radius of the ball node are obtained; Based on the reconstructed coordinates of the ball nodes, the radius of the ball nodes, the axis of the members, the radius of the members, and the continuous identity identifiers of the ball nodes at both ends of the effective members, a parametric 3D model of the space frame structure is generated.

[0009] Furthermore, the target detection and cross-frame association of sphere nodes in RGB video data includes: identifying sphere node detection boxes in video frames and using the center point of the sphere node detection boxes as the two-dimensional observation of the sphere nodes; matching sphere node detection results in different video frames based on at least one of motion prediction, detection box position, detection confidence, or appearance features; and assigning the same persistent identity identifier to multiple detection results belonging to the same physical sphere node.

[0010] Furthermore, the step of converting multi-frame sphere node two-dimensional observations of the same continuous identity into a spatial target ray includes: converting the sphere node two-dimensional observations into a target direction vector in the camera coordinate system according to camera parameters; converting the target direction vector to the world coordinate system according to UAV pose data; and constructing a spatial target ray with the camera optical center in the corresponding video frame as the ray starting point and the target direction vector in the world coordinate system as the ray direction.

[0011] Furthermore, the spatial intersection includes: calculating the point with the smallest sum of squared vertical distances among multiple spatial target rays corresponding to the same persistent identity, and using it as the initial three-dimensional coordinates of the sphere node corresponding to the persistent identity; based on the first vertical residual from the initial three-dimensional coordinates to each spatial target ray, when the first vertical residual is greater than a preset residual threshold or greater than a preset multiple of the median of the first vertical residual, the corresponding spatial target ray is discarded, and the initial three-dimensional coordinates of the sphere node are recalculated using the remaining spatial target rays.

[0012] Further, the generation of candidate link connection relationships includes: calculating the distance between the initial three-dimensional coordinates of the ball nodes corresponding to any two video semantic anchor points; setting a maximum connection distance threshold based on the prior link length or the statistical results of node spacing, and filtering node pairs that meet the distance condition; wherein, the distance condition is that the distance between the initial three-dimensional coordinates of the ball nodes corresponding to the two video semantic anchor points is less than or equal to the maximum connection distance threshold; for node pairs that meet the distance condition, if the absolute value of the difference between the broken line distance from the third ball node to the two endpoints of the node pair and the distance of the node pair is within the collinearity discrimination tolerance, then the node pair is removed; the remaining node pairs and their corresponding persistent identity identifiers are used as candidate link connection relationships.

[0013] Further, the step of extracting the mid-segment point cloud of the rod along the candidate rod connection relationship in the lidar point cloud data includes: constructing a bounding region centered on the midpoint of the initial three-dimensional coordinates of the two ball nodes corresponding to the candidate rod connection relationship, and extracting a first candidate point cloud from the lidar point cloud data; using the line connecting the initial three-dimensional coordinates of the two ball nodes as the axis, extracting a second candidate point cloud from the first candidate point cloud whose vertical distance to the axis is less than a preset distance threshold; and removing point clouds in the second candidate point cloud that meet the end truncation ratio, while retaining the mid-segment point cloud of the rod.

[0014] Furthermore, the cylindrical model fitting and validity verification includes: performing cylindrical model fitting on the point cloud of the middle section of the rod to obtain the candidate rod axis and candidate rod radius; when the direction of the line connecting the candidate rod axis and the two sphere nodes of the corresponding candidate rod connection relationship satisfies the direction consistency condition, and the candidate rod radius satisfies the rod radius rationality condition, the candidate rod connection relationship is confirmed as a valid rod.

[0015] Furthermore, the axis intersection positioning based on the axis lines of multiple valid members connected to the same ball node includes: extracting the axis lines of multiple valid members connected to the ball node corresponding to the same continuous identity identifier; calculating the point with the smallest sum of squared vertical distances between the axis lines of the multiple valid members as the reconstructed coordinates of the ball node; and, based on the second vertical residual from the reconstructed coordinates of the ball node to each axis line, when the second vertical residual is greater than a preset axis residual threshold, removing the corresponding axis line and recalculating the reconstructed coordinates of the ball node using the remaining axis lines.

[0016] Furthermore, the sphere node radius acquisition operation is as follows: taking the reconstructed coordinates of the sphere node as the center, extracting local sphere node point clouds from the lidar point cloud data; performing spherical fitting on the local sphere node point clouds to obtain the sphere node radius.

[0017] To achieve the above objectives, the present invention also provides a 3D reconstruction system for a grid structure that integrates UAV video and point cloud data, comprising: The data acquisition module is used to acquire RGB video data, LiDAR point cloud data, camera parameters, and UAV pose data of the grid structure under test. The video semantic anchor construction module is used to perform target detection and cross-frame association on sphere nodes in RGB video data to obtain the observation sequence of sphere nodes with continuous identity. Based on camera parameters and UAV pose data, the two-dimensional observations of sphere nodes with the same continuous identity in multiple frames are converted into spatial target rays, and the initial three-dimensional coordinates of sphere nodes with continuous identity are obtained through spatial intersection. The pole topology and geometry reconstruction module is used to take the initial 3D coordinates of the ball nodes with persistent identities as video semantic anchor points, generate candidate pole connection relationships, and retain the persistent identities of the ball nodes at both ends; extract the mid-segment point cloud of the pole along the candidate pole connection relationship in the LiDAR point cloud data, and obtain the pole axis, pole radius, and persistent identities of the ball nodes at both ends of the valid pole through cylindrical model fitting and validity verification; The node geometry reconstruction module is used to perform axis intersection positioning based on the axis of multiple effective members connected to the same ball node, and obtain the reconstructed coordinates and radius of the ball node; The parametric model generation module is used to generate a parametric 3D model of the space frame structure based on the reconstructed coordinates of the ball nodes, the radius of the ball nodes, the axis of the members, the radius of the members, and the continuous identity identifiers of the ball nodes at both ends of the effective members.

[0018] The core of this invention lies not in simply fusing drone video and point cloud data, but in: 1. Utilize the cross-frame association results of sphere nodes in the RGB video of the drone to obtain the continuous identity identifier of the same physical sphere node; 2. Convert multiple frames of two-dimensional observations corresponding to the same persistent identity into spatial target rays, and obtain the initial three-dimensional coordinates of the sphere nodes with persistent identity through spatial intersection; 3. Use the initial 3D coordinates of the ball nodes with persistent identity identifiers as video semantic anchor points to generate candidate rod connection relationships; 4. Using the connection relationship of candidate rods as topological constraints, locally extract the point cloud of the middle section of the rods from the lidar point cloud and fit it with a cylindrical model; 5. Obtain the reconstructed coordinates of the ball node by intersecting the axes of the effective members, and further obtain the radius of the ball node; 6. Based on the reconstructed coordinates of the ball nodes, the radius of the ball nodes, the axis of the members, the radius of the members, and the continuous identity identifiers of the ball nodes at both ends of the members, a parametric 3D model of the space frame structure is generated.

[0019] The above process elevates video information from general image features or projection regions into semantic anchors with persistent identity and topological guidance, thereby achieving the fusion of video data and LiDAR point cloud data in component-level topology reconstruction. Compared with existing technologies, this invention has the following advantages: 1. This invention utilizes a drone platform to simultaneously acquire RGB video data and lidar point cloud data, reducing the workload of multi-station deployment, complex registration, and manual close-range measurement in ground lidar measurement, and enabling rapid, large-scale inspection of large-span grid structures with lower on-site implementation costs; 2. This invention utilizes the cross-frame continuous identity identifiers of sphere nodes in UAV videos to construct video semantic anchors, enabling the same physical sphere node to maintain consistent identity in multi-view observations, thus solving the problem of the difficulty in stably establishing node correspondence relationships in single-frame image detection. 3. This invention uses video semantic anchor points to guide the generation of candidate link connection relationships, so that visual information is no longer just an image projection or local target localization result, but becomes a semantic constraint with topological guidance function; 4. This invention extracts the mid-segment point cloud of the rod along the connection relationship of the candidate rods in the lidar point cloud, avoiding blind clustering in the point cloud of the whole scene and reducing the impact of point cloud aliasing in dense node areas on rod identification. 5. This invention performs cylindrical model fitting under candidate topological relationship constraints, so that the geometric parameters of effective members and the continuous identity identification of the spherical nodes at both ends are obtained synchronously, which improves the consistency of member geometric reconstruction and topological relationship identification. 6. This invention utilizes the axes of multiple rods connected to the same sphere node to reconstruct the sphere node coordinates, reducing the impact of local point cloud occlusion or incompleteness on node coordinate positioning. 7. This invention can generate a parametric 3D model containing spherical node coordinates, spherical node dimensions, member axes, member radii, and member connection relationships, which can be used for construction quality verification, structural inspection, and digital twin modeling of space frame structures. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments are briefly described below; the following drawings are only used to illustrate typical embodiments of the present invention and do not constitute a limitation on the scope of protection of the present invention: Figure 1 This is a schematic diagram of the RGB video data and lidar point cloud data of the grid structure under test in this invention; Figure 2 This is a schematic diagram of ball node detection and cross-frame association in RGB video of a drone according to the present invention; Figure 3 This is a schematic diagram illustrating the initial three-dimensional coordinates of the sphere node obtained by the ray intersection of spatial targets in this invention. Figure 4 This is a schematic diagram of extracting the point cloud of the middle section of a member along the connection relationship of candidate members in this invention; Figure 5 This is a schematic diagram of the parametric three-dimensional model of the space frame structure of the present invention. Detailed Implementation

[0021] The implementation process of the present invention is illustrated below with reference to a specific embodiment. This embodiment is used to explain the present invention and is not intended to limit the scope of protection of the present invention.

[0022] This embodiment provides a method for 3D reconstruction of a space frame structure by integrating UAV video and point cloud data. The method mainly includes: acquiring UAV RGB video data and LiDAR point cloud data; detecting and cross-frame association of sphere nodes in the video; obtaining initial 3D coordinates of sphere nodes with continuous identity identifiers through multi-line-of-sight spatial intersection; using these initial 3D coordinates as video semantic anchors to generate candidate member connection relationships; extracting mid-segment point clouds of members from the LiDAR point cloud along the candidate member connection relationships; performing cylindrical model fitting and validity verification on the mid-segment point clouds of members; obtaining reconstructed coordinates of sphere nodes based on the intersection of multiple valid member axes, and obtaining the sphere node radius through spherical fitting; and finally generating a parametric 3D model of the space frame structure. This method utilizes a UAV platform to acquire RGB video data and LiDAR point cloud data of the space frame structure, reducing the workload of ground multi-station deployment, manual close-range measurement, and complex on-site registration; simultaneously, it guides the topological reconstruction of members in the point cloud through video semantic anchors, achieving component-level 3D reconstruction suitable for low-cost, rapid, and large-scale inspection of large-span space frame structures.

[0023] I. Data Acquisition.

[0024] In this embodiment, an unmanned aerial vehicle (UAV) platform equipped with an RGB camera and a LiDAR device is used to conduct aerial surveys of the grid structure under test, obtaining RGB video data and LiDAR point cloud data covering the grid structure, such as... Figure 1 As shown, camera parameters and UAV pose data are recorded simultaneously.

[0025] Let the set of RGB video frames of the drone be:

[0026] in, I k Indicates the first k Frame video images, n Indicates the number of video frames.

[0027] Let the lidar point cloud data be:

[0028] in, Represents the first point in the point cloud s A three-dimensional point, T Indicates the transpose operation; Represents the set of real numbers. N q This represents the number of points in a point cloud.

[0029] Camera parameters include the camera intrinsic matrix K, distortion parameters, and camera extrinsic parameters; UAV pose data includes the position C of the camera optical center in the world coordinate system. k and camera pose matrix R k RGB video data and LiDAR point cloud data are converted to the same world coordinate system or engineering local coordinate system.

[0030] II. Sphere node detection and cross-frame association.

[0031] like Figure 2 As shown, target detection is performed on sphere nodes in RGB video data to obtain sphere node detection boxes in video frames. For the ... k Frame Image k If the identity identifier is detected i For a sphere node, the center point of the detection box of that sphere node is used as its two-dimensional observation point in that video frame:

[0032] in, u i,k , v i,k These represent the points where the sphere node is at the 1st position. k Pixel coordinates in a frame.

[0033] Subsequently, based on at least one of motion prediction, detection box location, detection confidence, or appearance features, cross-frame association is performed on the ball node detection results in consecutive video frames. Multiple detection results belonging to the same physical ball node are assigned the same persistent identity, forming a ball node observation sequence:

[0034] in, Observe i Indicates identity identifier i The observation sequence of the sphere node, Ω i This represents the set of video frames observed for the sphere node. Through the above processing, the same physical sphere node maintains the same identity across different video frames and viewpoints. This persistent identity is not only used to distinguish different sphere nodes but also serves as a basis for semantic association in subsequent LiDAR point cloud processing.

[0035] III. Construction of spatial target rays and acquisition of initial three-dimensional coordinates of sphere nodes.

[0036] like Figure 3 As shown, based on camera parameters and UAV pose data, the two-dimensional observations of multiple sphere nodes corresponding to the same continuous identity are converted into spatial target rays, and the initial three-dimensional coordinates of the sphere nodes are obtained through multi-line spatial intersection.

[0037] First, observe the two-dimensional observation point p of the sphere node. i,k Written in homogeneous pixel coordinates:

[0038] Based on the camera intrinsic parameter matrix K, it is converted into a direction vector in the camera coordinate system:

[0039] And perform normalization:

[0040] in, This represents the unit target direction vector in camera coordinates. This represents the direction vector of the sphere node in the camera coordinate system. Represents the homogeneous pixel coordinates of the two-dimensional observation point of the sphere node.

[0041] According to the camera pose matrix R k Transform the direction vector to the world coordinate system:

[0042] in, This represents the unit target direction vector in the world coordinate system.

[0043] With the firstk Frame camera optical center C k As the starting point of the ray, with Construct a space target ray along the ray direction:

[0044] in, λ These are the ray parameters.

[0045] For the same persistent identity i For the corresponding multiple spatial target rays, the initial three-dimensional coordinates of the sphere node are solved by minimizing the sum of the squares of the perpendicular distances from the point to be determined to each target ray. :

[0046] Where I is the identity matrix, Indicates the initial three-dimensional coordinates of the ball node. This represents the three-dimensional coordinate variable to be determined.

[0047] To reduce the impact of anomaly detection, short-term occlusion, or attitude errors on spatial rendezvous results, the vertical residuals from the initial 3D coordinates to the rays of each spatial target are calculated:

[0048] When vertical residual r i,k If the residual value is greater than a preset residual threshold, or greater than a certain multiple of the median residual, the corresponding spatial target ray is discarded, and the initial three-dimensional coordinates of the sphere node are recalculated using the remaining spatial target rays. In this embodiment, r i,k The threshold is set to 0.02m based on the required accuracy of on-site measurements. When the relative distance between the drone and the drone is relatively short and the camera calibration and pose accuracy are high, a smaller threshold can be used; when the shooting distance is long, there are many obstructions, or the video detection error is large, a larger threshold can be used.

[0049] IV. Construction of video semantic anchor points and generation of candidate rod connection relationships.

[0050] The initial 3D coordinates of the sphere nodes with persistent identity identifiers are used as video semantic anchors. The set of video semantic anchors is defined as follows:

[0051] Among them, ID i Indicates the first i The persistent identity identifier of each ball node This represents the initial three-dimensional coordinates of the sphere node. N p This indicates the number of ball nodes identified.

[0052] For any two video semantic anchors a i and a j Calculate the distance between the initial three-dimensional coordinates of the corresponding sphere nodes. :

[0053] Set the maximum connection distance threshold based on prior knowledge of member length or statistical results of node spacing. Node pairs will be added when the following conditions are met ( i , j As initial candidate node pairs:

[0054] This is used to exclude long-distance node pairs that are clearly impossible to connect directly, reducing the number of candidate members to be searched. In this embodiment, Based on the estimated longest member length of the measured space frame, it is set to 4m. To avoid misclassifying multi-segment members containing intermediate nodes as single members, collinearity of intermediate nodes is eliminated for initial candidate node pairs. If a third ball node exists... m , so that:

[0055] in, Represents node pairs ( i , m The distance, Represents node pairs ( m , j The distance.

[0056] If the third ball node is located at node i With nodes j Near the line segment connecting them, remove the node pair. i , j ).in, ε c Tolerance for collinearity is established. A set of candidate member connection relationships is obtained after screening. :

[0057] Among them, the connection relationship of each candidate member e ij All retain the persistent identity ID of their two end sphere nodes. i ID j This set constitutes the topological prior for subsequent local extraction of LiDAR point clouds and rod fitting.

[0058] V. Extraction of local point clouds of rods and acquisition of point clouds of mid-section of rods.

[0059] like Figure 4 As shown, for each candidate member connection relationship, the local point cloud of the member is extracted from the lidar point cloud data along the line connecting the two sphere nodes corresponding to the candidate member connection relationship, and the point cloud of the middle section of the member is obtained from the local point cloud of the member.

[0060] Let the unit direction vector between the initial three-dimensional coordinates of the two sphere nodes be:

[0061] Let the midpoint of the line connecting the two nodes be:

[0062] First, with M ij Construct a surrounding region centered on the lidar point cloud CP and extract the first candidate point cloud. :

[0063] in, Represents any point in the first candidate point cloud; R s,ij The radius of the enclosed region can be determined based on the node spacing D. ij Positioning error determined.

[0064] Then, calculate any point in the first candidate point cloud. Vertical distance to the axis of the candidate member :

[0065] According to the preset distance threshold R c Extracting the second candidate point cloud :

[0066] in, R c The combined influence range is greater than that of the member radius, initial node positioning error, and construction deviation. In this embodiment, R c The value is set to 0.2m, which allows it to retain the point cloud of the real rod surface even when there is a certain deviation in the initial positioning coordinates of the ball node, while excluding the point clouds of adjacent rods and the background point cloud that are far from the candidate axis.

[0067] To remove the mixed point cloud near the sphere node, calculate any point in the second candidate point cloud. Normalized projection scale along the axis of the candidate member:

[0068] Set the end cutoff ratio β The point cloud satisfying the following formula is retained as the point cloud of the middle section of the rod. :

[0069] Through the above processing, the mid-section point cloud corresponding to the candidate rod can be extracted from the global LiDAR point cloud, reducing the influence of the sphere node point cloud and adjacent rod point cloud on the subsequent cylindrical model fitting.

[0070] VI. Fitting and validity verification of cylindrical rod model.

[0071] Point cloud of the middle section of the rod A cylindrical model is fitted to obtain the candidate member axes and radii. Let the first... i , j The axis of the candidate member between the two ball nodes is from point A on the axis. ij and unit direction vector v ij This indicates that the radius of the candidate member is... r ij The cylindrical model fitting can be expressed as:

[0072] in, This represents a point on the axis of the candidate cylinder. This represents the unit direction vector of the candidate cylinder axis. Indicates the radius of the candidate member. Point cloud representing the middle section of the rod Any point in the middle.

[0073] The effectiveness of the fitted candidate members is verified. The directional consistency verification is as follows:

[0074] in, γ This is the directional consistency threshold. In this embodiment, the directional consistency deviation angle is considered to be less than 3°, therefore, it is taken as... γ The cosine value of 3° is 0.984.

[0075] The radius's rationality is verified as follows:

[0076] in, Let R represent the radius of a standard member, and R be the set of standard member radii. ε r This is the radius tolerance. In this embodiment, Based on the design data of the space frame to be tested, ε r Set it to 0.015m.

[0077] Furthermore, verification was performed based on the number of points inside the cylinder model. (Number of points inside the cylinder) satisfy:

[0078] in, N min The minimum number of points.

[0079] Candidate member connections that simultaneously meet the requirements of directional consistency, reasonable radius, and number of interior points are confirmed as valid members.

[0080] The validity verification also includes at least one of the following conditions: the number of interior points of the cylindrical model is greater than a preset point number threshold (this threshold is determined based on the actual point cloud density collected; in this embodiment, the threshold is set to 1000); the residual from the point cloud of the middle section of the rod to the surface of the cylindrical model is less than a preset residual threshold.

[0081] The valid members are recorded as follows:

[0082] Among them, ID i ID j For the continuous identification of the ball nodes at both ends of the effective member.

[0083] VII. Reconstruction of Sphere Node Coordinates and Acquisition of Dimensional Parameters.

[0084] For any sphere node i Extract the set of valid members connected to it. : ={ With the ball node i connect} Each active member provides a member axis. For active members m ij The axis of its member can be represented as:

[0085] in, μ For axis parameters.

[0086] The reconstructed coordinates of the sphere node are obtained by finding the point that minimizes the sum of the squares of the perpendicular distances between the axes of multiple members. :

[0087] To eliminate abnormal member axes, calculate the residuals from the reconstructed coordinates of the ball nodes to the axes of each member:

[0088] when ρ ij When the residual value exceeds a preset axis residual threshold, the corresponding member axis is discarded, and the coordinates of the sphere nodes are reconstructed using the remaining member axes. In this embodiment, the axis residual threshold is set to 0.02m to exclude member axes caused by the mixing of point clouds from adjacent members or misjudgment of candidate connection relationships.

[0089] Furthermore, coordinates are reconstructed using sphere nodes. Centered on the local spherical node point cloud extracted from lidar point cloud data. :

[0090] in, R s Extracting the radius for local spherical node point cloud. Represents lidar point cloud Any point in the middle.

[0091] Perform spherical fitting on the local spherical point cloud and solve for the center of the sphere. and sphere node radius R i :

[0092] in, Let these represent the coordinates of the center of the sphere to be determined. Denotes the radius of the sphere node to be determined. Represents a local spherical point cloud Any point in the middle.

[0093] 8. Parametric 3D Model Generation.

[0094] like Figure 5 As shown, a parametric 3D model of the space frame structure is generated based on the reconstructed coordinates of the ball nodes, the radius of the ball nodes, the axis of the members, the radius of the members, and the continuous identity identifiers of the ball nodes at both ends of the effective members.

[0095] The final parametric 3D model of the space frame structure includes the reconstructed coordinates of the sphere nodes, the radius of the sphere nodes, the axis of the members, the radius of the members, and the connection relationships of the members.

[0096] This invention also provides a 3D reconstruction system for a grid structure that integrates UAV video and point cloud data to implement the above method, comprising: The data acquisition module is used to acquire RGB video data, LiDAR point cloud data, camera parameters, and UAV pose data of the grid structure under test. The video semantic anchor construction module is used to perform target detection and cross-frame association on sphere nodes in RGB video data to obtain the observation sequence of sphere nodes with continuous identity. Based on camera parameters and UAV pose data, the two-dimensional observations of sphere nodes with the same continuous identity in multiple frames are converted into spatial target rays, and the initial three-dimensional coordinates of sphere nodes with continuous identity are obtained through spatial intersection. The pole topology and geometry reconstruction module is used to take the initial 3D coordinates of the ball nodes with persistent identities as video semantic anchor points, generate candidate pole connection relationships, and retain the persistent identities of the ball nodes at both ends; extract the mid-segment point cloud of the pole along the candidate pole connection relationship in the LiDAR point cloud data, and obtain the pole axis, pole radius, and persistent identities of the ball nodes at both ends of the valid pole through cylindrical model fitting and validity verification; The node geometry reconstruction module is used to perform axis intersection positioning based on the axis of multiple effective members connected to the same ball node, and obtain the reconstructed coordinates and radius of the ball node; The parametric model generation module is used to generate a parametric 3D model of the space frame structure based on the reconstructed coordinates of the ball nodes, the radius of the ball nodes, the axis of the members, the radius of the members, and the continuous identity identifiers of the ball nodes at both ends of the effective members.

[0097] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

Claims

1. A method for three-dimensional reconstruction of a grid structure by integrating UAV video and point cloud data, characterized in that, include: Acquire RGB video data, LiDAR point cloud data, camera parameters, and UAV pose data of the structure under test; The space frame structure under test includes ball nodes and rods connecting the ball nodes; Target detection and cross-frame association are performed on sphere nodes in RGB video data to obtain observation sequences of sphere nodes with continuous identification. Based on camera parameters, the two-dimensional observation of the sphere node is converted into a target direction vector in the camera coordinate system; based on UAV pose data, the target direction vector is converted to the world coordinate system; using the camera optical center in the corresponding video frame as the ray origin and the target direction vector in the world coordinate system as the ray direction, a spatial target ray is constructed; the point with the smallest sum of squared vertical distances among multiple spatial target rays corresponding to the same persistent identity is calculated as the initial three-dimensional coordinates of the sphere node corresponding to that persistent identity; based on the first vertical residual from the initial three-dimensional coordinates to each spatial target ray, when the first vertical residual is greater than a preset residual threshold or greater than a preset multiple of the median of the first vertical residual, the corresponding spatial target ray is discarded, and the initial three-dimensional coordinates of the sphere node are recalculated using the remaining spatial target rays; The initial 3D coordinates of the ball nodes with persistent identity are used as video semantic anchors to generate candidate rod connection relationships, retaining the persistent identity of the ball nodes at both ends; the mid-segment point cloud of the rod is extracted from the LiDAR point cloud data along the candidate rod connection relationship, and the rod axis, rod radius, and persistent identity of the ball nodes at both ends of the valid rod are obtained through cylindrical model fitting and validity verification. Based on the axis intersection positioning of multiple effective members connected to the same ball node, the reconstructed coordinates and radius of the ball node are obtained; Based on the reconstructed coordinates of the ball nodes, the radius of the ball nodes, the axis of the members, the radius of the members, and the continuous identity identifiers of the ball nodes at both ends of the effective members, a parametric 3D model of the space frame structure is generated.

2. The method for three-dimensional reconstruction of a grid structure by fusing UAV video and point cloud data according to claim 1, characterized in that, The target detection and cross-frame association of sphere nodes in RGB video data includes: identifying sphere node detection boxes in video frames and using the center point of the sphere node detection boxes as the two-dimensional observation of the sphere nodes; matching sphere node detection results in different video frames based on at least one of motion prediction, detection box position, detection confidence, or appearance features; and assigning the same persistent identity to multiple detection results belonging to the same physical sphere node.

3. The method for three-dimensional reconstruction of a grid structure by fusing UAV video and point cloud data according to claim 1, characterized in that, The process of generating candidate link connection relationships includes: calculating the distance between the initial three-dimensional coordinates of the ball nodes corresponding to any two video semantic anchor points; setting a maximum connection distance threshold based on the prior link length or the statistical results of node spacing, and filtering node pairs that meet the distance condition; wherein, the distance condition is that the distance between the initial three-dimensional coordinates of the ball nodes corresponding to the two video semantic anchor points is less than or equal to the maximum connection distance threshold; for node pairs that meet the distance condition, if the absolute value of the difference between the broken line distance from the third ball node to the two endpoints of the node pair and the distance of the node pair is within the collinearity discrimination tolerance, then the node pair is removed; the remaining node pairs and their corresponding persistent identity identifiers are used as candidate link connection relationships.

4. The method for three-dimensional reconstruction of a grid structure by fusing UAV video and point cloud data according to claim 1, characterized in that, The step of extracting the mid-segment point cloud of a rod from the lidar point cloud data along the connection relationship of candidate rods includes: constructing a bounding region centered on the midpoint of the initial three-dimensional coordinates of the two sphere nodes corresponding to the connection relationship of the candidate rods; extracting a first candidate point cloud from the lidar point cloud data; extracting a second candidate point cloud from the first candidate point cloud whose vertical distance to the axis is less than a preset distance threshold, using the line connecting the initial three-dimensional coordinates of the two sphere nodes as the axis; and removing point clouds in the second candidate point cloud that meet the end truncation ratio, while retaining the mid-segment point cloud of the rod.

5. The method for three-dimensional reconstruction of a grid structure by fusing UAV video and point cloud data according to claim 1, characterized in that, The cylindrical model fitting and validity verification includes: fitting a cylindrical model to the point cloud of the middle section of the rod to obtain the candidate rod axis and candidate rod radius; when the line connecting the candidate rod axis and the two spherical nodes of the corresponding candidate rod connection relationship meets the direction consistency condition, and the candidate rod radius meets the rod radius rationality condition, the candidate rod connection relationship is confirmed as a valid rod.

6. The method for three-dimensional reconstruction of a grid structure by fusing UAV video and point cloud data according to claim 1, characterized in that, The method of performing axis intersection positioning based on the axis lines of multiple valid members connected to the same ball node includes: extracting the axis lines of multiple valid members connected to the ball node corresponding to the same continuous identity identifier; calculating the point with the smallest sum of squared vertical distances between the axis lines of the multiple valid members as the reconstructed coordinates of the ball node; and, based on the second vertical residual from the reconstructed coordinates of the ball node to each axis line, when the second vertical residual is greater than a preset axis residual threshold, removing the corresponding axis line and recalculating the reconstructed coordinates of the ball node using the remaining axis lines.

7. The method for three-dimensional reconstruction of a grid structure by fusing UAV video and point cloud data according to claim 1, characterized in that, The operation to obtain the spherical node radius is as follows: taking the reconstructed coordinates of the spherical node as the center, extracting local spherical node point clouds from the lidar point cloud data; performing spherical fitting on the local spherical node point clouds to obtain the spherical node radius.

8. A three-dimensional reconstruction system for a grid structure that integrates UAV video and point cloud data to implement the method described in claim 1, characterized in that, include: The data acquisition module is used to acquire RGB video data, LiDAR point cloud data, camera parameters, and UAV pose data of the grid structure under test. The video semantic anchor construction module is used to perform target detection and cross-frame association on sphere nodes in RGB video data to obtain the observation sequence of sphere nodes with continuous identity. Based on camera parameters and UAV pose data, the two-dimensional observations of sphere nodes with the same continuous identity in multiple frames are converted into spatial target rays, and the initial three-dimensional coordinates of sphere nodes with continuous identity are obtained through spatial intersection. The pole topology and geometry reconstruction module is used to take the initial 3D coordinates of the ball nodes with persistent identities as video semantic anchor points, generate candidate pole connection relationships, and retain the persistent identities of the ball nodes at both ends; extract the mid-segment point cloud of the pole along the candidate pole connection relationship in the LiDAR point cloud data, and obtain the pole axis, pole radius, and persistent identities of the ball nodes at both ends of the valid pole through cylindrical model fitting and validity verification; The node geometry reconstruction module is used to perform axis intersection positioning based on the axis of multiple effective members connected to the same ball node, and obtain the reconstructed coordinates and radius of the ball node; The parametric model generation module is used to generate a parametric 3D model of the space frame structure based on the reconstructed coordinates of the ball nodes, the radius of the ball nodes, the axis of the members, the radius of the members, and the continuous identity identifiers of the ball nodes at both ends of the effective members.

Images