Real-time three-dimensional modeling, earthwork quantity calculation and construction progress monitoring method for earthwork based on unmanned aerial vehicle measurement
By combining drones with oblique photogrammetry and airborne lidar technology, a high-precision 3D model is generated, which solves the problem of poor feature element identification in traditional methods and realizes real-time monitoring and intelligent management of earthwork projects.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING JIAOTONG UNIV
- Filing Date
- 2024-05-31
- Publication Date
- 2026-08-04
AI Technical Summary
Traditional earthwork engineering surveying methods suffer from poor identification of feature elements in 3D reconstruction models in complex terrain, slow modeling speed, difficulty in distinguishing feature information at different stages of earthwork engineering, and inability to accurately connect monitoring results with digital models, resulting in low monitoring efficiency.
By combining UAV surveying with oblique photogrammetry and airborne LiDAR, and through semantic segmentation and convolutional neural networks, high-precision terrain feature point information is obtained, a refined 3D model is generated and updated in real time, and the construction progress is monitored by combining time series multi-frame input method.
It enables efficient and accurate 3D modeling of complex terrain, real-time monitoring of earthwork volume and construction progress, reduction of human error, improvement of construction efficiency and resource utilization, and support for intelligent decision-making.
Smart Images
Figure CN118608709B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of civil engineering construction technology, and relates to a method for real-time three-dimensional modeling of earthwork projects, calculation of earthwork volume, and monitoring of construction progress based on UAV surveying. Background Technology
[0002] Traditional earthwork allocation methods rely primarily on manual surveying and experience-based judgment, which suffers from low accuracy, inefficiency, and a lack of automation. Manual surveying is extremely time-consuming and labor-intensive, especially for complex terrain and large-scale projects, limiting the accuracy and efficiency of earthwork allocation. Furthermore, traditional methods often depend on experience-based judgment, making them susceptible to subjective biases and resulting in inconsistent and unreliable results. Additionally, due to the complexity of the terrain and the obstruction of existing structures, traditional methods struggle to accurately predict earthwork volumes and achieve cut-fill balance, potentially leading to resource waste and environmental impact.
[0003] However, the rapid advancements in UAV surveying and computer vision technologies have brought new opportunities for digitalization and intelligentization in civil engineering construction. UAV surveying technology boasts advantages in high precision and rapid acquisition of terrain data. However, traditional UAV surveying methods still have some problems and limitations, affecting their efficiency and accuracy. For example, when dealing with complex terrain, the 3D reconstruction models established by traditional methods suffer from poor identification of different feature elements. They cannot distinguish the feature information of different stages of earthwork engineering, such as fill surfaces, excavation surfaces, construction personnel, and machinery. This makes it difficult to achieve automatic monitoring and management of feature elements in complex scenes, such as personnel and equipment. Furthermore, the monitoring results cannot be precisely integrated with the digital model, hindering subsequent applications and management decisions. Moreover, in complex terrain, the modeling speed is slow when UAVs perform high-precision scanning. The technology for processing point cloud data has not been updated accordingly, and excessive modeling time leads to a decrease in monitoring efficiency for large-scale earthwork projects, offering little reference value for subsequent earthwork project scheduling. Summary of the Invention
[0004] The purpose of this invention is to provide a method for real-time 3D modeling, earthwork volume calculation, and construction progress monitoring of earthwork engineering based on UAV surveying. This method addresses the shortcomings of existing methods in complex terrain, where the 3D reconstruction model established by traditional UAV surveying methods suffers from poor identification of different feature elements, slow modeling speed, inability to distinguish feature information at different stages of earthwork engineering, difficulty in automatically monitoring feature elements in complex scenes, inability to achieve precise integration of monitoring results with digital models, and low monitoring efficiency due to slow modeling speed.
[0005] The technical solution adopted in this embodiment of the invention is: a real-time 3D modeling method for earthwork engineering based on UAV measurement, comprising the following steps:
[0006] Step S1: Use UAVs and planned flight paths to acquire multi-view image data of the entire area, and combine oblique photogrammetry and 3D reconstruction algorithms to obtain a 3D terrain mesh of the entire area.
[0007] Step S2: Using multi-view image data and semantic segmentation model, mark the location and attribute information of complex terrain feature points in the earthwork project area, and use airborne lidar to scan the area where the marked complex terrain feature points are located to obtain ground point cloud data of complex terrain.
[0008] Step S3: Using ground point cloud data from airborne lidar and a convolutional neural network based on an attention mechanism, obtain the location and attribute information of the corrected complex terrain feature points, and calibrate the accuracy of the three-dimensional terrain grid of UAV oblique photogrammetry.
[0009] Step S4: Utilize the location and attribute information of complex terrain feature points and high-precision three-dimensional terrain mesh to generate a refined three-dimensional model of earthwork engineering, and update and visualize the three-dimensional model of earthwork engineering in real time.
[0010] Furthermore, the specific implementation process of step S1 is as follows:
[0011] Step S11: Before starting the UAV measurement, conduct test flights and preliminary surveys. The preliminary surveys include terrain analysis, map study and data collection of the measurement area to determine the measurement requirements and select appropriate UAVs, sensors and measurement parameters.
[0012] Step S12: Based on the terrain features and measurement requirements, formulate a flight path design scheme for the UAV, and determine the flight altitude, flight path spacing, and flight path direction;
[0013] Step S13: The UAV flies along a predetermined route and at a set shooting frequency. It takes ground images from different angles and heights using oblique photogrammetry to obtain ground multi-view image data, and synchronizes it with precise time and space to form a ground multi-view image sequence.
[0014] Step S14: Correct and process the acquired multi-view image data. Image processing includes image registration, geometric correction, color balance, and elimination of distortion and errors caused by flight motion and lighting changes.
[0015] Step S15: Combine the multi-view image data acquired by the UAV with the oblique photography method to perform data processing and point cloud generation. Use a 3D reconstruction algorithm to transform the point cloud into a visualized 3D terrain mesh. Data processing includes meshing, surface reconstruction, and texture mapping.
[0016] Furthermore, the specific implementation process of step S15 is as follows:
[0017] Step S151, Multi-view image processing: By using a feature matching algorithm and the nearest neighbor matching principle to efficiently match feature points, the oblique photogrammetry data is automatically matched with corresponding feature points to establish the topological relationship of the image space.
[0018] Step S152, Aerial Triangulation and Sparse Point Cloud Generation: Aerial triangulation is performed using bundle adjustment to pair multiple sets of ground feature points with corresponding image points. Combining the true 3D coordinates of the feature points and multiple matched images, multiple collinearity condition equations are established, as shown in equations (1) and (2). Using the least squares method and the known coordinates of the ground feature points, the imaging parameters, i.e., the interior and exterior orientation elements, are solved. Then, through the determined imaging parameters and the collinearity condition equations, the accurate coordinates of other undetermined ground points in the 3D coordinate system are derived, forming a sparse 3D point cloud model.
[0019]
[0020] Where x, y, and z are the coordinates of the ground point; x s y s z s Here, f represents the coordinates of the camera center in the ground-auxiliary coordinate system; f is the focal length; Δx and Δy are the system error correction values, which are generally obtained by repeatedly measuring the image control points and statistically analyzing the observation errors of each control point in the image coordinate system. Based on this, the system error correction values are estimated using methods such as least squares or Kalman filtering; x0 and y0 are interior orientation elements; a1, a2, a3, b1, b2, b3, c1, c2, and c3 form a 3×3 rotation matrix, where a1, a2, and a3 represent the rows of the rotation matrix, b1, b2, and b3 represent the columns of the rotation matrix, and c1, c2, and c3 represent the values of the rotation matrix.
[0021] Step S153, Multi-view Dense Matching and Ultra-High Density Point Cloud Generation: First, the original point cloud data is standardized and normalized to facilitate the training and prediction of the deep learning model. CNN is used to extract features from the point cloud. Using the extracted features, the number of points in the point cloud is increased through upsampling technology, thereby improving its resolution. Finally, the generated high-resolution point cloud is optimized and corrected to improve its quality and accuracy. The advantage of this method is that it can automatically learn the complex patterns of point cloud data and generate more refined and high-quality point clouds.
[0022] Furthermore, the specific implementation process of step S2 is as follows:
[0023] Step S21: Based on the preliminary investigation and earthwork requirements, identify areas with complex terrain and arrange terrain feature points within these areas as reference points within the measurement area.
[0024] Step S22: Collect and organize the multi-view image data acquired by the UAV, and use the semantic segmentation algorithm to identify and classify the terrain feature points in the complex terrain area, and label the location and attribute information of the terrain feature points, including height information and volume information.
[0025] Step S23: Based on the initial 3D terrain grid and terrain feature point data, re-optimize the UAV's flight path design and use the airborne lidar to obtain the accurate coordinates and high-precision point cloud data of the area where the marked complex terrain feature points are located.
[0026] Furthermore, the specific implementation process of step S3 is as follows:
[0027] Step S31, Convolutional Neural Network Design and Training: Design a convolutional neural network model based on an attention mechanism for the task of recognizing and calibrating the accuracy of terrain feature points;
[0028] Step S32, Terrain Feature Point Identification: The trained convolutional neural network model is used to identify the ground point cloud data of complex terrain acquired by the airborne lidar in step S2, and the location and attribute information of the corrected complex terrain feature points are obtained.
[0029] Step S33, Measurement accuracy calibration: Based on the position and attribute information of the corrected terrain feature points, the accuracy of the three-dimensional terrain grid measured by the UAV tilt measurement is calibrated.
[0030] Furthermore, the specific implementation process of step S33 is as follows:
[0031] The three-dimensional terrain point cloud data obtained by UAV multi-view oblique photogrammetry in step S1 is compared with the position and attribute information of the corrected terrain feature points. The error of the measurement result is quantified by the least squares method. Then, according to the error distribution law of different feature points, the overall measurement result is corrected to correct and rectify the systematic error of the UAV preliminary measurement data.
[0032] Furthermore, the specific implementation process of step S4 is as follows:
[0033] Step S41, Model Optimization and Improvement: Based on the location and attribute information of complex terrain feature points and high-precision 3D terrain mesh, the 3D model generated by UAV oblique photogrammetry is optimized and improved, including removing redundant information, filling holes, smoothing, and correcting model deviations.
[0034] Step S42, Real-time model generation and update:
[0035] According to step S2, based on the selection principle of complex terrain areas and UAV multi-view image data, the semantic segmentation algorithm is used to classify the ground point cloud data of complex terrain areas, divide the ground point cloud data into different semantic categories, and identify and classify the feature elements with obvious changes. The feature elements include terrain outline, excavation face, fill face, buildings, roads, rivers, and vegetation.
[0036] The real ground point cloud data corrected by the lidar and the semantic information output by the semantic segmentation algorithm are imported into the three-dimensional model of earthwork engineering generated in step S41 to correct the details and structure of the ground part.
[0037] Furthermore, for point clouds identified as ground-type objects, corresponding 3D ground models are generated to recreate the shape and details of the real ground.
[0038] For point clouds identified as ground structures, they are marked and located, corresponding 3D models and 3D entities corresponding to the point cloud data are established, and annotations and attribute associations are performed. Labels and descriptive information are added to each feature element. Type labels such as fill surface and excavation surface are added to different categories of ground point clouds, role labels are added to personnel point clouds, and equipment type labels are added to machinery point clouds.
[0039] Another technical solution adopted in this embodiment of the invention is: a method for calculating earthwork volume based on UAV measurement, the specific process of which is as follows:
[0040] Step 1: Construct a real-time 3D model of earthwork engineering according to the real-time 3D modeling method of earthwork engineering based on UAV measurement as described above;
[0041] Step 2: Using ground point cloud data from airborne lidar and a convolutional neural network based on an attention mechanism, obtain the location and attribute information of corrected complex terrain feature points. After calibrating the accuracy of the 3D terrain mesh obtained from UAV oblique photogrammetry, a density of 100–1000 points / m is obtained. 2 The point cloud data was imported into the established real-time 3D model of the earthwork project to obtain the first phase of earthwork data.
[0042] Step 3: Based on the site design elevation, i.e. the coordinate point data of the final finished surface required by the design documents, compare it with the current earthwork measurement data to determine the excavation and filling areas and the required excavation and filling earthwork volume;
[0043] Step 4: Then, using the ground point cloud data of the airborne lidar and the convolutional neural network based on the attention mechanism, update the point cloud data of the three-dimensional model of the earthwork project in real time. Repeat the method of determining the earthwork measurement data of the previous period to generate the second earthwork measurement data, that is, the updated current earthwork measurement data. By comparing with the site design elevation, determine the updated excavation and filling area and the remaining excavation and filling earthwork volume.
[0044] Repeat step 4 above at certain intervals, use the real-time data from the drone to update the current earthwork measurement data in real time, and reflect the remaining proportion of the earthwork project by determining the ratio of the remaining excavation and filling volume to the initial excavation and filling volume in real time.
[0045] Another technical solution adopted in this embodiment of the invention is: a method for monitoring the construction progress of earthwork projects based on UAV measurement, comprising the following steps:
[0046] Construct a real-time 3D model of earthwork engineering based on UAV measurement as described above.
[0047] Based on the real-time 3D model of the earthwork project and the 3D entities of personnel and machinery, extract the progress information of the earthwork project and provide real-time feedback on the construction site situation, including:
[0048] Measure and calculate the completion status of excavation and filling operations, and display the overall progress of earthwork construction;
[0049] Define the fill surface and the excavation surface, distinguish the excavation and fill areas, determine whether the current excavation and fill operation is within the planned excavation and fill area, and issue a warning if it is not within the area;
[0050] Based on the location of personnel and machinery within the excavation and filling construction area, determine whether the location of construction personnel complies with safety regulations;
[0051] Monitor whether construction machinery is operating within the correct construction area.
[0052] The beneficial effects of the embodiments of the present invention are:
[0053] (1) By using airborne lidar to acquire high-quality ground point cloud data in real time, richer and more complete three-dimensional information is provided. The ground point cloud can be clearly and accurately divided into different semantics, such as fill surface and excavation surface. Based on the design elevation data, the excavation and filling area and the excavation and filling volume can be determined. The target area of earthwork volume can be extracted efficiently and accurately, which improves the accuracy of measurement results and solves the shortcomings of existing three-dimensional reconstruction models, which have poor recognition of different feature elements and cannot distinguish the feature information of different stages of earthwork.
[0054] (2) This invention first utilizes a time-series multi-frame input method to track changes in feature targets in real time. This step helps capture the dynamic changes of moving or occluded targets in the scene. Then, deep learning semantic segmentation technology is used to automatically identify the semantic information of various engineering elements, including personnel, machinery, etc. Through high-precision identification, the construction scene can be understood more accurately. Then, the identified classification results are integrated with the 3D model. Finally, end-to-end management of earthwork target features is achieved, eliminating the need to repeatedly call 3D models at different times, thus improving work efficiency. This invention solves the problem that existing 3D reconstruction results cannot distinguish feature information of different stages of earthwork engineering, and realizes automatic monitoring of feature elements in complex scenes. At the same time, the monitoring results are finely integrated with the digital model. This invention can operate around the clock, unaffected by weather, lighting, and other factors, improving the stability of measurement results;
[0055] (3) By using the high-precision measurement data provided by UAVs and combining computer vision algorithms to identify and segment targets such as terrain surfaces, on-site personnel and engineering machinery, the spatial location and features of each target can be accurately extracted. Based on three-dimensional modeling, the earthwork volume can be predicted more accurately, the cut-fill balance can be achieved, the accuracy of allocation can be improved and the waste of resources can be reduced.
[0056] (4) Through real-time 3D modeling, it is possible to monitor and control the progress of earthwork projects in real time; by comparing the difference between the current status and the planned progress, deviations in project progress can be detected in a timely manner, and corresponding adjustment measures can be taken, providing closed-loop support for dynamic monitoring and management of construction accuracy and modification of earthwork construction plans; differentiating different construction period information, real-time and accurate calculation of changes in excavation and filling earthwork volume, providing reliable overall progress monitoring, and facilitating the calculation and management of construction costs; visualization technology can effectively monitor the safety of personnel and equipment operation, thereby improving the efficiency of project construction and avoiding the waste of time and resources;
[0057] (5) It realizes the automated processing and intelligent decision-making of earthwork engineering allocation. Through intelligent algorithms and real-time modeling, it can automatically identify targets, predict quantities, and analyze data, reducing the need for manual intervention and subjective judgment, reducing the impact of human error, and promoting the digital and intelligent development of earthwork engineering construction management. Attached Figure Description
[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0059] Figure 1 This is a flowchart of a real-time 3D modeling method for earthwork engineering based on UAV surveying.
[0060] Figure 2 This is a schematic diagram of the test area.
[0061] Figure 3 It is a comparison map of three-dimensional terrain data before and after earthwork construction. Detailed Implementation
[0062] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0063] Example 1
[0064] This embodiment provides a real-time 3D modeling method for earthwork engineering based on UAV surveying, such as... Figure 1 As shown, it includes the following steps:
[0065] Step S1: Use UAVs and planned flight paths to acquire multi-view image data of the entire area, and combine oblique photogrammetry and 3D reconstruction algorithms to obtain a 3D terrain mesh of the entire area.
[0066] Step S2: Using multi-view image data and semantic segmentation model, mark the location and attribute information of complex terrain feature points in the earthwork project area, and use airborne lidar to scan the area where the marked complex terrain feature points are located to obtain ground point cloud data of complex terrain.
[0067] Step S3: Using ground point cloud data from airborne LiDAR and a convolutional neural network based on an attention mechanism, the location and attribute information of the corrected complex terrain feature points are obtained, and the accuracy of the 3D terrain mesh obtained by UAV oblique photogrammetry is calibrated. Accuracy calibration is achieved by identifying complex terrain feature points and obtaining the true coordinates of the corresponding terrain feature points from the airborne LiDAR, comparing them with the terrain data obtained by UAV oblique photogrammetry, quantifying the error of the measurement results using the least squares method, and then correcting the overall measurement results according to the error distribution pattern of different feature points, so that the measurement results are closer to the real terrain in terms of location and scale, thereby improving the measurement accuracy and automation level of the 3D terrain mesh.
[0068] Step S4: Utilizing the location and attribute information of complex terrain feature points and a high-precision 3D terrain mesh, a refined 3D model of the earthwork project is generated. This model is then updated and visualized in real time, providing intuitive display and data analysis functions, including:
[0069] Analysis of cut and fill area distribution: Based on the design elevation and 3D model data, determine the earthwork cut and fill areas and the volume of earthwork cut and fill;
[0070] Spatial distribution analysis: Analyze the spatial distribution of construction areas, personnel, and machinery, including two-dimensional plan views and three-dimensional views;
[0071] Quality control analysis: Conduct quality control analysis on earthwork projects, including verification of earthwork volume, verification of construction area, and comparison of quality control standards;
[0072] Green construction analysis: This involves analyzing the green construction practices of earthwork projects, including the volume of excavated soil, the volume of excavated soil, and the backfill rate.
[0073] Cost analysis: Analyze the cost of earthwork projects, including total cost and unit cost;
[0074] Safety assessment and analysis: Analyze the safety management of construction personnel.
[0075] In some embodiments, the specific implementation process of step S1 is as follows:
[0076] Step S11: Conduct test flights and preliminary surveys before starting UAV measurements to ensure the normal operation of the UAV and the suitability of the measurement area; test flights are used to verify the flight performance and stability of the UAV and to verify the working status of the measurement equipment; preliminary surveys include terrain analysis, map study and relevant data collection of the measurement area to determine measurement requirements and select appropriate UAVs, sensors and measurement parameters.
[0077] Step S12: Based on the terrain features and measurement requirements, formulate a flight path design scheme for the UAV, and determine parameters such as flight altitude, flight path spacing, and flight path direction. The flight altitude should be determined according to the terrain complexity and the required image resolution. The flight path spacing and direction should be able to cover the measurement area and acquire sufficient overlapping multi-view image data. The flight path design scheme should ensure the safety of the flight path, smooth transitions between flight paths, and ensure that the spatial coverage can be fully utilized for effective measurement.
[0078] Step S13: The UAV takes ground images from different angles and heights using oblique photogrammetry to obtain ground multi-view image data, and synchronizes it with precise time and space to form a ground multi-view image sequence; the UAV should fly according to the predetermined route and set shooting frequency to ensure full coverage and acquire sufficient image data;
[0079] Step S14: Correct and process the acquired multi-view image data to eliminate distortions and errors caused by factors such as flight motion and changes in lighting. More specifically, the image processing includes image registration, geometric correction, color balance, etc., to ensure the accuracy and consistency of the image data.
[0080] Step S15: Combine the multi-view image data acquired by the UAV with the oblique photography method to perform data processing and point cloud generation. Use a 3D reconstruction algorithm to transform the point cloud into a visualized 3D terrain mesh, including meshing, surface reconstruction, texture mapping and other operations, to generate the surface structure and texture information of the initial 3D model.
[0081] In some embodiments, the specific implementation process of step S2 is as follows:
[0082] Step S21: Based on the preliminary investigation and earthwork requirements, identify areas with complex terrain, such as areas with significant terrain changes or key areas (including excavation faces, fill faces, buildings, roads, rivers, vegetation, etc.). Arrange terrain feature points within these complex terrain areas as reference points within the measurement area to calibrate the measurement data, helping the UAV obtain more accurate location information and measurement data, thereby improving measurement accuracy. Depending on the specific circumstances, arrange no fewer than 5 terrain feature points in flat terrain areas; for areas with significant terrain changes, set at least 10 terrain feature points as needed to ensure detailed and accurate terrain data. The arrangement of terrain feature points should cover the entire measurement area as much as possible, taking into account the importance and complexity of terrain changes.
[0083] Step S22: Collect and organize multi-view image data acquired by the UAV, and use semantic segmentation algorithms to identify and classify terrain feature points in complex terrain areas, labeling the location and attribute information of terrain feature points. Attribute information includes height information and volume information; height information is the elevation information of ground features, which can reflect the location information of ground features such as buildings, roads, rivers, etc.; height information can usually be represented by discrete height values; volume information is the volume information of ground features, which can usually be represented by continuous volume values, such as the volume of buildings, the length of roads, the area of rivers, etc.; terrain feature points can include significant geomorphic feature points such as hillside edges, terrain intersections, and building corners, realizing the conversion from point cloud to semantic information and providing a richer description of terrain structure;
[0084] Step S23: Based on the initial 3D terrain mesh and terrain feature point data, re-optimize the UAV's flight path design and use the airborne LiDAR to obtain the accurate coordinates and high-precision point cloud data of the area where the marked complex terrain feature points are located. These data and markings will serve as input and supervision signals for training the convolutional neural network.
[0085] In some embodiments, the specific implementation process of step S3 is as follows:
[0086] Step S31, Convolutional Neural Network Design and Training: For the task of terrain feature point recognition and measurement accuracy calibration, a convolutional neural network model based on an attention mechanism is designed. This embodiment uses FPN (Feature Point Network). Pyramid Network (FPN) is a convolutional neural network (CNN) architecture for semantic segmentation. It improves segmentation accuracy by learning features at different scales. FPN includes multiple convolutional layers, pooling layers, and upsampling layers. Input layer: The input image is divided into features at different scales, from low-level features to high-level features. Low-level features are representations of the original image, while high-level features are abstract representations of the low-level features. Convolutional layers: Convolutional layers are used to extract image features. FPN uses convolutional layers to extract features from each feature layer. Pooling layers: Pooling layers are used to reduce the size of feature maps. Pooling operations reduce computational load and allow feature maps to communicate between different levels. Upsampling layers: Upsampling layers are used to enlarge the size of feature maps. Upsampling operations improve segmentation accuracy because they allow more detail to be added to high-level features. Fusion layer: Fusion layers fuse features from different levels to produce the final segmentation result. Fusion operations improve segmentation accuracy by learning the relationships between features at different levels. This convolutional neural network model should possess sufficient perception capability and accuracy to identify terrain feature points and calibrate measurement results to improve measurement precision. Through an attention mechanism, the convolutional neural network model can automatically focus on important terrain feature points and reduce sensitivity to visual noise and invalid information, thereby improving the recognition accuracy and robustness of terrain feature points. The training of this convolutional neural network model requires semi-supervised learning using terrain point cloud data labeled with the location and attribute information of terrain feature points. An attention mechanism is then used to weight key areas (including excavation faces, fill faces, buildings, roads, rivers, vegetation, etc.). This weighting is performed through normalization or activation functions, with the aim of feature selection. Based on the weight of each element in the original feature vector, the elements in the original feature vector are weighted to extract key features, thus achieving feature selection.
[0087] Step S32, Terrain Feature Point Identification: The trained convolutional neural network model is used to identify the ground point cloud data of complex terrain acquired by the airborne lidar, and the location and attribute information of the corrected complex terrain feature points are obtained.
[0088] Step S33, Measurement Accuracy Calibration: Based on the corrected location and attribute information of the terrain feature points, the accuracy of the 3D terrain mesh obtained by UAV oblique measurement is calibrated. The preliminary 3D terrain point cloud data obtained by UAV multi-view oblique photogrammetry is compared with the corrected location and attribute information of the terrain feature points. The error of the measurement result is quantified by the least squares method. Then, according to the error distribution law of different feature points, the overall measurement result is corrected to correct and correct the systematic error of the preliminary UAV measurement data, improve the accuracy and realism of the 3D mesh, provide a more accurate terrain reconstruction for subsequent earthwork calculation and monitoring work, and facilitate the extraction of more detailed information hidden in the point cloud.
[0089] In some embodiments, the specific implementation process of step S4 is as follows:
[0090] Step S41, Model Optimization and Refinement: Based on the location and attribute information of complex terrain feature points and a high-precision 3D terrain mesh, the 3D model generated by UAV oblique photogrammetry is optimized and refined. This may include operations such as removing redundant information, filling holes, smoothing, and correcting model deviations to improve the quality and accuracy of the model, resulting in a more realistic and accurate 3D terrain model.
[0091] Point cloud filtering: Point clouds may contain some noisy or erroneous data. These points can be removed by point cloud filtering, thereby improving the accuracy of the model. The filtering methods used include sparsification, filters, morphological processing, etc.
[0092] Point cloud downsampling: There may be a large number of points in a point cloud, but only a portion of the points need to be selected for 3D modeling. Therefore, point cloud downsampling is performed to reduce the amount of computation. Downsampling methods include random sampling, equidistant sampling, and voxel sampling.
[0093] Point cloud normal vector estimation: In 3D modeling, it is necessary to calculate the normal vector of each point in the point cloud. The normal vector can help us better represent the details of the object's surface, thereby improving the accuracy of the model. The normal vector estimation methods used include spherical fitting, triangular patch fitting, and deep learning-based normal vector estimation.
[0094] Point cloud post-processing: After point cloud modeling, further processing is required, such as removing seams and repairing holes, to improve the visual effect of the model. Post-processing methods include surface smoothing, fine-grained surface processing, and surface texture processing.
[0095] Point cloud fusion: When processing multiple point cloud data, it is necessary to fuse the multiple point cloud data to obtain a better 3D model. The fusion methods used include point cloud registration, point cloud merging, and point cloud fusion.
[0096] Step S42, Real-time Model Generation and Update: Through real-time processing and algorithm optimization, the generation and update of the 3D model of earthwork engineering can be completed in a short time. In 3D modeling, gradient descent is used to optimize model parameters, such as smoothing and curve fitting, to meet the requirements of 3D modeling of earthwork engineering while maintaining the accuracy and precision of the model. The 3D model of earthwork engineering can distinguish ground structures such as mountains, rivers, ground, and buildings.
[0097] In some embodiments, the specific implementation process of step S15 is as follows:
[0098] Step S151, Multi-view image processing: By using a feature matching algorithm and the nearest neighbor matching principle to efficiently match feature points, the oblique photogrammetry data is automatically matched with corresponding feature points to establish the topological relationship of the image space.
[0099] Step S152, Aerial Triangulation and Sparse Point Cloud Generation: Aerial triangulation is performed using bundle adjustment to pair multiple sets of ground feature points with corresponding image points. Combining the true 3D coordinates of the ground feature points and multiple matched images, multiple collinearity condition equations are established, as shown in equations (1) and (2). Using the least squares method and the known coordinates of the ground feature points, the imaging parameters, i.e., the interior and exterior orientation elements, are solved. Then, through the determined imaging parameters and the collinearity condition equations, the accurate coordinates of other undetermined ground points in the 3D coordinate system are derived, forming a sparse 3D point cloud model.
[0100]
[0101] Where x, y, and z are the coordinates of the ground point; x s y s z s Here, represents the coordinates of the photography center in the ground-auxiliary coordinate system (i.e., the exterior orientation element); f is the focal length; Δx and Δy are the systematic error correction values, which are generally obtained by repeatedly measuring the image control points and statistically analyzing the observation errors of each control point in the image coordinate system. Based on this, the systematic error correction values are estimated using methods such as least squares or Kalman filtering; x0 and y0 are the interior orientation elements, and the direction cosines (a1, a2, a3, b1, b2, b3, c1, c2, c3) represent a 3×3 rotation matrix, where a1, a2, and a3 represent the rows of the rotation matrix, b1, b2, and b3 represent the columns of the rotation matrix, and c1, c2, and c3 represent the values of the rotation matrix;
[0102] Step S153, Multi-view Dense Matching and Ultra-High Density Point Cloud Generation: Based on the sparse point cloud model, a multi-view dense matching algorithm is used to generate ultra-high density point clouds.
[0103] First, the raw point cloud data is standardized and normalized to facilitate the training and prediction of the deep learning model. Feature extraction: Features are extracted from the point cloud using a CNN. Utilizing the extracted features, upsampling techniques are used to increase the number of points in the point cloud, thereby improving its resolution. Finally, the generated high-resolution point cloud is optimized and refined to improve its quality and accuracy, resulting in a more detailed and high-quality dense point cloud model, thus obtaining an ultra-high-density point cloud.
[0104] This process involves collecting and organizing multi-view imagery data acquired by drones, and using semantic segmentation algorithms to identify and classify terrain feature points in complex terrain areas. The location and attribute information of these feature points are labeled, including height and volume information. Height information refers to the elevation of features such as buildings, roads, and rivers; it is typically represented by discrete height values. Volume information refers to the volume of features, which is usually represented by continuous volume values, such as building volume, road length, and river area. Terrain feature points can include significant geomorphic features such as hillside edges, terrain intersections, and building corners, enabling the conversion from point clouds to semantic information and providing a richer description of terrain structure.
[0105] In some embodiments, the specific implementation process of step S42 is as follows:
[0106] According to step S22, based on the selection principle of complex terrain areas and UAV multi-view image data, the semantic segmentation algorithm is used to classify the ground point cloud data of complex terrain areas, divide the ground point cloud data into different semantic categories, and identify and classify the feature elements such as terrain contour lines, excavation faces, fill faces, buildings, roads, rivers, and vegetation that have obvious changes, so as to realize the conversion from point cloud to semantic information and provide a richer description of terrain structure.
[0107] The real ground point cloud data corrected by the lidar and the semantic information output by the semantic segmentation algorithm are imported into the 3D model of earthwork engineering generated in step S41 to correct the details and structure of the ground part. Specifically, a point cloud registration algorithm can be used to register the real-time monitored ground point cloud data with the ground point cloud data in the existing 3D model to achieve real-time monitoring and correction.
[0108] For point clouds identified as ground-type objects, generate corresponding 3D ground models to recreate the shape and details of the real ground.
[0109] For point clouds identified as ground structures such as personnel and construction machinery, they are marked and located, and corresponding 3D models and 3D entities corresponding to the point cloud data are established. Specifically, for personnel point clouds, human pose estimation methods can be used to correct and standardize the shape; for machinery point clouds, their approximate outline is extracted and modeled based on their specific shape (such as the rectangular shape of an excavator). In the 3D scene, entities can be objects such as personnel and construction machinery, or ground structures such as buildings and roads.
[0110] Labeling involves annotating and associating attributes with the 3D model and 3D entity of each generated feature element. This includes adding tags and descriptive information to each feature element, such as ground type, personnel role, and machinery type. For example, type tags are added to different categories of ground point clouds, such as fill surface / excavation surface; role tags are added to personnel point clouds, such as operator, inspector, etc.; and equipment type tags are added to machinery point clouds, such as excavator, loading truck, etc. Simultaneously, other descriptive information, such as name and ID number, is added to each feature element, linking it to the real-time 3D model of the earthwork project to enable the location and querying of feature elements.
[0111] In this invention, a multi-frame time-series input method is used for joint identification of moving targets such as construction machinery and personnel. Specifically, based on the continuous multi-frame image data sequence obtained by UAV oblique photogrammetry, the motion information and change patterns of the target between different frames are comprehensively considered, and a deep learning algorithm is used to process and learn the continuous multi-frame image data sequence to achieve accurate identification and classification of moving targets such as machinery and personnel.
[0112] Applying the method of the embodiments of the present invention to Figure 2 In the 3D modeling of the complex mountainous terrain shown, a DJI drone equipped with an L1 LiDAR was used. The drone flew at an altitude of 10m and a speed of 5m / s in three different areas of the same site, with areas measuring 1000 square meters, 2000 square meters, and 3000 square meters respectively. The implementation results of this embodiment are compared with commonly used drone 3D modeling optimization algorithms, as shown in Table 1.
[0113] Table 1 Comparison of different 3D modeling methods for earthwork projects
[0114]
[0115]
[0116] Example 2
[0117] This embodiment provides a method for calculating earthwork volume based on UAV measurement. The specific process is as follows:
[0118] Step 1: Construct a three-dimensional real-time model of the earthwork project according to Example 1;
[0119] Step 2: Using ground point cloud data from airborne lidar and a convolutional neural network based on an attention mechanism, obtain the location and attribute information of corrected complex terrain feature points. After calibrating the accuracy of the 3D terrain mesh obtained from UAV oblique photogrammetry, a density of 100–1000 points / m is obtained. 2 The point cloud data (which can calculate elevation and area) is imported into the established 3D model of earthwork engineering to obtain the first phase of earthwork data.
[0120] Step 3: Based on the site design elevation, i.e. the coordinate point data of the final finished surface required by the design documents, compare it with the current earthwork measurement data to determine the excavation and filling areas and the required excavation and filling earthwork volume;
[0121] Step 4: Utilize the ground point cloud data from the airborne lidar and a convolutional neural network based on an attention mechanism to update the 3D model point cloud data of the earthwork project in real time. Repeat the method used to determine the earthwork measurement data from the previous phase to generate the second phase of earthwork measurement data, i.e., the updated current earthwork measurement data. The processing results are shown below. Figure 3 , Figure 3 It reflects the site earthwork data at different times, and by comparing it with the site design elevation, it determines the updated cut and fill areas and the remaining cut and fill earthwork volume.
[0122] Repeat step 4 above at certain intervals, use real-time data from the drone to update the current earthwork measurement data in real time, and reflect the overall remaining proportion of earthwork by determining the ratio of the remaining excavation and filling volume to the initial excavation and filling volume in real time, so as to provide information support for construction personnel to manage the subsequent construction and schedule.
[0123] Example 3
[0124] This embodiment provides a method for monitoring the construction progress of earthwork projects based on UAV surveying, including the following steps:
[0125] Construct a real-time 3D model of the earthwork project according to Example 1;
[0126] Based on the real-time 3D model of the earthwork project and the 3D entities of personnel and machinery, the progress information of the earthwork project can be extracted according to the needs of management personnel, and the construction site situation can be fed back in real time. For example, combined with the method of Example 2, the completion status of excavation and filling operations can be measured and calculated to display the overall progress of earthwork construction; defining filling surfaces and excavation surfaces can be used to distinguish excavation and filling areas, determine whether the current excavation and filling operation is within the planned excavation and filling area, and issue a warning if it is not within the area; using ground point cloud data of airborne lidar and convolutional neural networks based on attention mechanism, the location and attribute information of corrected complex terrain feature points can be obtained to identify construction units such as personnel and construction machinery, which can provide reference objects for progress monitoring. The identification of different categories and the monitoring of real-time changes can help to adjust and improve the construction plan in a timely manner; long-term monitoring can measure the overall progress and verify the quality of construction management; based on the location of personnel and machinery in the excavation and filling construction area, it can be determined whether the location of construction personnel complies with safety regulations; intuitively monitor whether construction machinery is working in the correct construction area, and discover areas with abnormal excavation and filling volumes, providing a basis for management personnel to adjust and improve the construction plan in a timely manner.
[0127] The process of measuring and calculating the completion status of excavation and filling operations, in conjunction with the method of Example 2, is as follows:
[0128] By utilizing ground point cloud data from airborne lidar and a convolutional neural network based on an attention mechanism, the location and attribute information of complex terrain feature points are obtained after correction. Furthermore, the accuracy of the 3D terrain mesh obtained from UAV oblique photogrammetry is calibrated to achieve a density of 100–1000 points / m. 2 The point cloud data (area, elevation) is imported into the established 3D model of earthwork engineering, thereby creating a high-precision 3D model of earthwork.
[0129] Secondly, the coordinate point data of the final completed surface required by the site design elevation (i.e., the coordinate point data) are compared with the current earthwork measurement data to reflect the overall completion status of the earthwork project.
[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A real-time 3D modeling method for earthwork engineering based on UAV surveying, characterized in that, Includes the following steps: Step S1: Use UAVs and planned flight paths to acquire multi-view image data of the entire area, and combine oblique photogrammetry and 3D reconstruction algorithms to obtain a 3D terrain mesh of the entire area. Step S2: Using multi-view image data and semantic segmentation model, mark the location and attribute information of complex terrain feature points in the earthwork project area, and use airborne lidar to scan the area where the marked complex terrain feature points are located to obtain ground point cloud data of complex terrain. Step S3: Using ground point cloud data from airborne lidar and a convolutional neural network based on an attention mechanism, obtain the location and attribute information of the corrected complex terrain feature points, and calibrate the accuracy of the three-dimensional terrain grid of UAV oblique photogrammetry. Step S4: Utilize the location and attribute information of complex terrain feature points and high-precision three-dimensional terrain mesh to generate a refined three-dimensional model of earthwork engineering, and update and visualize the three-dimensional model of earthwork engineering in real time. The specific implementation process of step S3 is as follows: Step S31, Convolutional Neural Network Design and Training: For the task of recognizing and calibrating the accuracy of terrain feature points, a convolutional neural network model based on the attention mechanism is designed; Step S32, Terrain Feature Point Identification: The trained convolutional neural network model is used to identify the ground point cloud data of complex terrain acquired by the airborne lidar in step S2, and the location and attribute information of the corrected complex terrain feature points are obtained. Step S33, Measurement accuracy calibration: Based on the position and attribute information of the corrected terrain feature points, the accuracy of the three-dimensional terrain mesh measured by the UAV tilt measurement is calibrated; The specific implementation process of step S4 is as follows: Step S41, Model Optimization and Improvement: Based on the location and attribute information of complex terrain feature points and high-precision 3D terrain mesh, the 3D model generated by UAV oblique photogrammetry is optimized and improved, including removing redundant information, filling holes, smoothing, and correcting model deviations. Step S42, Real-time model generation and update: According to step S2, based on the selection principle of complex terrain areas and UAV multi-view image data, the semantic segmentation algorithm is used to classify the ground point cloud data of complex terrain areas, divide the ground point cloud data into different semantic categories, and identify and classify the feature elements with obvious changes. The feature elements include terrain outline, excavation face, fill face, buildings, roads, rivers, and vegetation. The real ground point cloud data corrected by the lidar and the semantic information output by the semantic segmentation algorithm are imported into the three-dimensional model of earthwork engineering generated in step S41 to correct the details and structure of the ground part.
2. The real-time 3D modeling method for earthwork engineering based on UAV surveying according to claim 1, characterized in that, The specific implementation process of step S1 is as follows: Step S11: Before starting the UAV measurement, conduct test flights and preliminary surveys. The preliminary surveys include terrain analysis, map study and data collection of the measurement area to determine the measurement requirements and select appropriate UAVs, sensors and measurement parameters. Step S12: Based on the terrain features and measurement requirements, formulate a flight path design scheme for the UAV, and determine the flight altitude, flight path spacing, and flight path direction; Step S13: The UAV flies along a predetermined route and at a set shooting frequency. It takes ground images from different angles and heights using oblique photogrammetry to obtain ground multi-view image data, and synchronizes it with precise time and space to form a ground multi-view image sequence. Step S14: Correct and process the acquired multi-view image data. Image processing includes image registration, geometric correction, color balance, and elimination of distortion and errors caused by flight motion and lighting changes. Step S15: Combine the multi-view image data acquired by the UAV with the oblique photography method to perform data processing and point cloud generation. Use a 3D reconstruction algorithm to transform the point cloud into a visualized 3D terrain mesh. Data processing includes meshing, surface reconstruction, and texture mapping.
3. The real-time 3D modeling method for earthwork engineering based on UAV surveying according to claim 2, characterized in that, The specific implementation process of step S15 is as follows: Step S151, Multi-view image processing: By using a feature matching algorithm and the nearest neighbor matching principle to efficiently match feature points, the oblique photogrammetry data is automatically matched with corresponding feature points to establish the topological relationship of the image space. Step S152, Aerial Triangulation and Sparse Point Cloud Generation: Aerial triangulation is performed using bundle adjustment to pair multiple sets of ground feature points with corresponding image points. Combining the true 3D coordinates of the feature points and multiple matched images, multiple collinearity condition equations are established, as shown in equations (1) and (2). Using the least squares method and the known coordinates of the ground feature points, the imaging parameters, i.e., the interior and exterior orientation elements, are solved. Then, through the determined imaging parameters and the collinearity condition equations, the accurate coordinates of other undetermined ground points in the 3D coordinate system are derived, forming a sparse 3D point cloud model. ； （1） ； （2） Where x, y, and z are the coordinates of the ground point; , , Here are the coordinates of the photography center in the ground-auxiliary coordinate system; f is the focal length. The system error correction value is generally obtained by repeatedly measuring the control points and statistically analyzing the observation error of each control point in the image coordinate system. Based on this, the system error correction value is estimated using methods such as least squares or Kalman filtering. x0 and y0 are interior orientation elements, and a1, a2, a3, b1, b2, b3, c1, c2, and c3 form a 3×3 rotation matrix, where a1, a2, and a3 represent the rows of the rotation matrix, b1, b2, and b3 represent the columns of the rotation matrix, and c1, c2, and c3 represent the values of the rotation matrix. Step S153, Multi-view Dense Matching and Ultra-High Density Point Cloud Generation: First, the original point cloud data is standardized and normalized to facilitate the training and prediction of the deep learning model. CNN is used to extract features from the point cloud. Using the extracted features, the number of points in the point cloud is increased through upsampling technology, thereby improving its resolution. Finally, the generated high-resolution point cloud is optimized and corrected to improve its quality and accuracy.
4. The real-time 3D modeling method for earthwork engineering based on UAV surveying according to claim 1, characterized in that, The specific implementation process of step S2 is as follows: Step S21: Based on the preliminary investigation and earthwork requirements, identify areas with complex terrain and arrange terrain feature points within these areas as reference points within the measurement area. Step S22: Collect and organize the multi-view image data acquired by the UAV, and use the semantic segmentation algorithm to identify and classify the terrain feature points in the complex terrain area, and label the location and attribute information of the terrain feature points, including height information and volume information. Step S23: Based on the initial 3D terrain grid and terrain feature point data, re-optimize the UAV's flight path design and use the airborne lidar to obtain the accurate coordinates and high-precision point cloud data of the area where the marked complex terrain feature points are located.
5. The real-time 3D modeling method for earthwork engineering based on UAV surveying according to claim 1, characterized in that, The specific implementation process of step S33 is as follows: The three-dimensional terrain point cloud data acquired by the UAV multi-view oblique photogrammetry in step S1 is compared with the position and attribute information of the corrected terrain feature points. The error of the measurement results is quantified by the least squares method. Then, the overall measurement results are corrected according to the error distribution law of different feature points, so as to correct and correct the systematic error of the UAV's preliminary measurement data.
6. The real-time three-dimensional modeling method for earthwork engineering based on UAV surveying according to claim 1, characterized in that, For point clouds identified as ground-type objects, generate corresponding 3D ground models to recreate the shape and details of the real ground. For point clouds identified as ground structures, they are marked and located, corresponding 3D models and 3D entities corresponding to the point cloud data are established, and annotations and attribute associations are performed. Labels and descriptive information are added to each feature element. Type labels such as fill surface and excavation surface are added to different categories of ground point clouds, role labels are added to personnel point clouds, and equipment type labels are added to machinery point clouds.
7. A method for calculating earthwork volume based on UAV surveying, characterized in that, The specific process is as follows: Step 1: Construct a real-time three-dimensional model of earthwork engineering based on UAV measurement according to any one of claims 1 to 6; Step 2: Using ground point cloud data from airborne lidar and a convolutional neural network based on attention mechanism, obtain the location and attribute information of the corrected complex terrain feature points. After calibrating the accuracy of the 3D terrain grid from UAV oblique photogrammetry, obtain point cloud data with a density of 100~1000 points / m² and import it into the established real-time 3D model of earthwork engineering to obtain the first phase of earthwork data. Step 3: Based on the site design elevation, i.e. the coordinate point data of the final finished surface required by the design documents, compare it with the current earthwork measurement data to determine the excavation and filling areas and the required excavation and filling earthwork volume; Step 4: Then, using the ground point cloud data of the airborne lidar and the convolutional neural network based on the attention mechanism, update the point cloud data of the three-dimensional model of the earthwork project in real time. Repeat the method of determining the earthwork measurement data of the previous period to generate the second earthwork measurement data, that is, the updated current earthwork measurement data. By comparing with the site design elevation, determine the updated excavation and filling area and the remaining excavation and filling earthwork volume. Repeat step 4 above at certain intervals, use the real-time data from the drone to update the current earthwork measurement data in real time, and reflect the remaining proportion of the earthwork project by determining the ratio of the remaining excavation and filling volume to the initial excavation and filling volume in real time.
8. A method for monitoring the construction progress of earthwork projects based on UAV surveying, characterized in that, Includes the following steps: A real-time three-dimensional model of earthwork engineering based on UAV measurement is constructed according to any one of claims 1 to 6. Based on the real-time 3D model of the earthwork project and the 3D entities of personnel and machinery, extract the progress information of the earthwork project and provide real-time feedback on the construction site situation, including: Measure and calculate the completion status of excavation and filling operations, and display the overall progress of earthwork construction; Define the fill surface and the excavation surface, distinguish the excavation and fill areas, determine whether the current excavation and fill operation is within the planned excavation and fill area, and issue a warning if it is not within the area; Based on the location of personnel and machinery within the excavation and filling construction area, determine whether the location of construction personnel complies with safety regulations; Monitor whether construction machinery is operating within the correct construction area.