Engineering intelligent aerial inspection method, device and storage medium based on drone
Through the UAV guard system and multi-source data matching technology, high-precision three-dimensional space scenarios are generated, which solves the problem of insufficient automation and intelligence of drone air patrol methods, realizes earthwork calculation and progress monitoring, and improves the engineering Intelligent management and decision-making support capabilities.
Patent Information
- Application Number
- CN202411269794.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-11
AI Technical Summary
The existing drone air patrol methods lack automation and intelligence, and the data is difficult to effectively integrate with multi-source data such as BIM, CAD and GIS. The intelligent analysis capabilities are insufficient, timely and accurate decision-making support cannot be provided, and multi-dimensional data is lacking. The comprehensive analysis ability cannot fully tap the value of data.
By deploying a UAV guard system, we can automatically perform preset route tasks, collect and process tilt photography data and RTK positioning data, generate high-precision two-dimensional orthophotographs and three-dimensional real-life models, and perform multi-source data matching, combining soil flow volume Comparing the calculation with multi-dimensional data analysis is carried out to generate engineering management optimization suggestions and safety risk warning information.
Improve the automation and efficiency of data collection, generate comprehensive and accurate three-dimensional spatial scenarios, enhance the integrity and practicality of data, realize the accuracy and progress monitoring of earthwork calculations, and improve the intelligence level of engineering management. and decision support capabilities reduce the possibility of human error.
Smart Images

Figure CN118778682B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image data processing, and in particular to an intelligent aerial inspection method, device and storage medium for engineering based on drones. Background Art
[0002] In recent years, with the continuous expansion of the scale and increasing complexity of construction projects, traditional manual inspection methods have been unable to meet the needs of modern engineering management. The development of drone technology has brought new opportunities for engineering inspections, which can quickly and efficiently obtain large-scale, high-precision engineering site data. At present, many engineering projects have begun to use drones for aerial inspections to assist engineering management by acquiring aerial images and videos. At the same time, the application of technologies such as BIM (Building Information Modeling), CAD (Computer-Aided Design) and GIS (Geographic Information System) has provided new means for the management and analysis of engineering data.
[0003] However, the existing drone aerial inspection methods still have some shortcomings. First, most systems lack automation and intelligence, and require a lot of manual intervention to control drone flight and data processing. Secondly, the acquired data is often isolated and difficult to effectively integrate with multi-source data such as BIM, CAD and GIS, which limits the application value of the data. Furthermore, the existing methods lack intelligent analysis capabilities in earthwork volume calculation, progress monitoring and safety risk warning, making it difficult to provide timely and accurate decision support for engineering management. Finally, the lack of comprehensive analysis capabilities for multi-dimensional data makes it impossible to fully tap the value of data and provide comprehensive optimization suggestions for engineering management. Summary of the invention
[0004] The present application provides a method, device and storage medium for drone-based engineering intelligent aerial inspection, which are used to improve the accuracy of drone-based engineering intelligent aerial inspection.
[0005] In a first aspect, the present application provides an engineering intelligent aerial inspection method based on a drone, and the engineering intelligent aerial inspection method based on a drone comprises:
[0006] Deploy a drone on-duty system at the construction site to obtain a drone control system that automatically executes preset route tasks, collect construction site data through the drone control system, and process the collected oblique photography data and RTK positioning data to obtain an original data set;
[0007] The original data set is model reconstructed to obtain a high-precision two-dimensional orthophoto and a three-dimensional real scene model, specifically including: performing image preprocessing on the original data set to obtain denoised image data, and performing color correction on the denoised image data to obtain corrected image data; extracting feature points from the corrected image data to obtain a feature point set, and performing feature matching on the feature point set to obtain matching point pairs; calculating the spatial position of the matching point pairs through a triangulation algorithm to obtain sparse point cloud data, and densely reconstructing the sparse point cloud data to obtain dense point cloud data; gridding the dense point cloud data to obtain 3D mesh data, and surface reconstruction of the 3D mesh data to obtain 3D curved surface data; texture mapping of the 3D curved surface data to obtain textured 3D data, and geometric correction of the textured 3D data to obtain the 3D real scene model; coordinate conversion of the RTK positioning data in the original data set to obtain geographic coordinate data, and geo-registration of the 3D real scene model according to the geographic coordinate data to obtain a geo-referenced 3D real scene model; orthographic projection of the geo-referenced 3D real scene model to obtain orthographic projection data, and resolution averaging of the orthographic projection data to obtain the high-precision 2D orthophoto;
[0008] Perform multi-source data matching on the three-dimensional real scene model to obtain a three-dimensional space scene including BIM data, CAD data and GIS information;
[0009] Calculating the earthwork volume of the three-dimensional space scene to obtain an earthwork volume analysis result;
[0010] Perform multi-time comparison on the three-dimensional space scene to obtain dynamic change data of the project progress and a visual progress report;
[0011] The three-dimensional space scene is subjected to multi-dimensional data analysis through the earthwork volume analysis results and the dynamic change data of the project progress, so as to obtain project management optimization suggestions and safety risk warning information.
[0012] In a second aspect, the present application provides an engineering intelligent aerial inspection device based on a drone, and the engineering intelligent aerial inspection device based on a drone includes:
[0013] A deployment module is used to deploy a drone guard system at the engineering site, obtain a drone control system that automatically executes the preset route mission, collect engineering site data through the drone control system, and process the collected oblique photography data and RTK positioning data to obtain an original data set;
[0014] The reconstruction module is used to reconstruct the model of the original data set to obtain a high-precision two-dimensional orthophoto and a three-dimensional real scene model, which specifically includes: performing image preprocessing on the original data set to obtain denoised image data, and performing color correction on the denoised image data to obtain corrected image data; extracting feature points from the corrected image data to obtain a feature point set, and performing feature matching on the feature point set to obtain matching point pairs; calculating the spatial position of the matching point pairs through a triangulation algorithm to obtain sparse point cloud data, and densely reconstructing the sparse point cloud data to obtain dense point cloud data; and gridding the dense point cloud data. Processing to obtain three-dimensional grid data, and reconstructing the surface of the three-dimensional grid data to obtain three-dimensional curved surface data; texture mapping the three-dimensional curved surface data to obtain textured three-dimensional data, and geometrically correcting the textured three-dimensional data to obtain the three-dimensional real scene model; coordinate conversion of the RTK positioning data in the original data set to obtain geographic coordinate data, and geo-registering the three-dimensional real scene model according to the geographic coordinate data to obtain a geographic reference three-dimensional real scene model; orthographically projecting the geographic reference three-dimensional real scene model to obtain orthographic projection data, and performing resolution averaging processing on the orthographic projection data to obtain the high-precision two-dimensional orthophoto;
[0015] A matching module is used to perform multi-source data matching on the three-dimensional real scene model to obtain a three-dimensional space scene containing BIM data, CAD data and GIS information;
[0016] A calculation module, used to calculate the earthwork volume of the three-dimensional space scene to obtain an earthwork volume analysis result;
[0017] A comparison module, used to compare the three-dimensional space scene over time to obtain dynamic change data of the project progress and a visual progress report;
[0018] The analysis module is used to perform multi-dimensional data analysis on the three-dimensional space scene through the earthwork volume analysis results and the dynamic change data of the project progress, so as to obtain project management optimization suggestions and safety risk warning information.
[0019] The third aspect of the present application provides a drone-based engineering intelligent aerial inspection device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory to enable the drone-based engineering intelligent aerial inspection device to perform the above-mentioned drone-based engineering intelligent aerial inspection method.
[0020] A fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer, the computer executes the above-mentioned drone-based engineering intelligent aerial inspection method.
[0021] In the technical solution provided by this application, by deploying a drone duty system and realizing automatic execution of preset route tasks, the automation and efficiency of data collection are greatly improved, the need for manual operation is reduced, and the consistency and reliability of data collection are ensured; high-precision oblique photography and RTK positioning technology are used, combined with model reconstruction algorithms, to achieve rapid generation of high-precision two-dimensional orthophotos and three-dimensional real-scene models, innovatively introduce multi-source data matching technology, deeply integrate BIM data, CAD data and GIS information with three-dimensional real-scene models, construct a comprehensive and accurate three-dimensional spatial scene, and greatly enhance the integrity and practicality of the data; in addition, in terms of earthwork volume calculation, high-precision digital terrain models and intelligent analysis algorithms are introduced, which significantly improves the accuracy and Efficiency provides a reliable basis for project progress management and cost control; at the same time, through multi-time comparison and intelligent analysis of three-dimensional spatial scenes, dynamic monitoring and visual display of project progress are realized, so that project managers can intuitively and timely grasp the progress of the project and quickly identify potential progress problems; more importantly, this method uses multi-dimensional data analysis technology to deeply integrate and analyze the earthwork volume analysis results, dynamic change data of project progress and three-dimensional spatial scenes, which can not only generate targeted project management optimization suggestions, but also provide timely and accurate safety risk warning information, greatly improving the intelligence level of project management and decision-making support capabilities; in addition, the full-process automation characteristics of this method greatly reduce the possibility of human errors and improve the overall quality and reliability of project management. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0023] Figure 1 This is a schematic diagram of an embodiment of an engineering intelligent aerial inspection method based on a drone in an embodiment of the present application;
[0024] Figure 2 This is a schematic diagram of an embodiment of an engineering intelligent aerial inspection device based on a drone in an embodiment of the present application;
[0025] Figure 3It is a structural schematic diagram of an engineering intelligent aerial inspection device based on a drone provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0026] The embodiments of the present application provide a method, device and storage medium for intelligent aerial inspection of engineering based on drones. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0027] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiments of the present application, an embodiment of the engineering intelligent aerial inspection method based on a drone includes:
[0028] Step S101: deploy a drone on-duty system at the engineering site, obtain a drone control system that automatically executes a preset route mission, collect engineering site data through the drone control system, and process the collected oblique photography data and RTK positioning data to obtain an original data set;
[0029] Step S102, reconstructing the model of the original data set to obtain a high-precision two-dimensional orthophoto and a three-dimensional real scene model;
[0030] Step S103: performing multi-source data matching on the three-dimensional real scene model to obtain a three-dimensional space scene including BIM data, CAD data and GIS information;
[0031] Step S104, calculating the earthwork volume of the three-dimensional space scene to obtain an earthwork volume analysis result;
[0032] Step S105, performing multi-time comparison on the three-dimensional space scene to obtain dynamic change data of the project progress and a visual progress report;
[0033] Step S106: Perform multi-dimensional data analysis on the three-dimensional space scene through the earthwork volume analysis results and the dynamic change data of the project progress to obtain project management optimization suggestions and safety risk warning information.
[0034] It is understandable that the execution subject of the present application can be an engineering intelligent aerial inspection device based on a drone, or a terminal or a server, which is not specifically limited here. The present application embodiment is described by taking a server as the execution subject as an example.
[0035] Specifically, the UAV-based engineering intelligent aerial inspection method first deploys the UAV guard system on the engineering site, which includes an automatic take-off and landing platform, charging equipment and data transmission module. The UAV control system automatically generates preset route tasks according to the terrain characteristics and construction progress of the engineering site to ensure comprehensive coverage of the entire engineering area. The UAV flies along the preset route to collect high-resolution oblique photography data and accurate RTK positioning data. Oblique photography technology obtains all-round image information of the engineering site through multi-angle shooting, while RTK positioning technology provides centimeter-level positioning accuracy. After these raw data are pre-processed by denoising, color correction and geometric correction, a high-quality raw data set is formed. Next, the raw data set is reconstructed. First, the corresponding points between images are identified by feature point extraction and matching algorithms, and then the three-dimensional point cloud is reconstructed using the principle of photogrammetry. After filtering, smoothing and gridding, the point cloud data generates a high-precision three-dimensional real scene model. At the same time, the oblique image is converted into a two-dimensional image of orthophoto projection through the orthorectification algorithm to obtain a high-precision two-dimensional orthophoto image. These models provide accurate geometric and texture information for subsequent analysis. In the multi-source data matching stage, the 3D real scene model is integrated with BIM data, CAD data and GIS information. Through feature matching and spatial registration algorithms, data from different sources are aligned to the same coordinate system. BIM data provides detailed information on engineering components, CAD data contains the precise dimensions of the design drawings, and GIS information provides the geographic environment background. This fusion of multi-source data generates a comprehensive 3D spatial scene containing rich semantic information.
[0036] Based on the comprehensive three-dimensional space scene, the earthwork volume is calculated. First, the terrain surface is extracted, and then compared with the design elevation surface to calculate the fill and cut volume. The soil characteristics and compaction coefficient of different regions are taken into account to ensure the accuracy of the earthwork volume calculation. At the same time, high-precision two-dimensional orthophotos are used to assist in identifying the boundaries of the earthwork area to improve the calculation accuracy. Multi-time comparison is achieved by comparing and analyzing the three-dimensional space scene data collected at different time points. First, the data of different periods are aligned, and then the changes in the project progress are identified through the change detection algorithm. Combined with BIM data, the progress tracking can be accurate to the component level. Not only is the dynamic change data of the project progress generated, but also an intuitive progress report is generated through visualization technology, including progress comparison charts and three-dimensional animations.
[0037] Finally, the earthwork analysis results and the dynamic change data of the project progress are integrated through multi-dimensional data analysis. Data mining and machine learning techniques are applied to analyze the project progress from multiple dimensions. For example, the possible risk of schedule delay can be predicted through the correlation analysis of earthwork change trends and progress critical paths. At the same time, combined with terrain data and weather information, safety risks such as slope stability and the impact of heavy rain are evaluated. These analysis results are ultimately transformed into specific project management optimization suggestions and safety risk warning information.
[0038] For example, in a large-scale water conservancy project, drones fly on preset routes every day to collect data from the project site. Through model reconstruction, a three-dimensional real-life model with centimeter-level accuracy is generated. In the multi-source data matching stage, this model is integrated with the BIM model of the water conservancy project, the river CAD design drawing, and the regional hydrological GIS data. The earthwork volume calculation shows that the actual excavation volume in a certain area is 15% more than planned, while the progress data shows that the construction period in this area is 7 days behind schedule. Multidimensional data analysis found that this deviation is related to the recent increase in rainfall, which has led to an increase in soil moisture content and increased the difficulty of excavation. Based on these data, the system generates management suggestions for adjusting the construction plan and adding drainage equipment, and issues early warnings for possible slope landslide risks.
[0039] In the embodiment of the present application, by deploying a drone duty system and automatically executing preset route tasks, the automation and efficiency of data collection are greatly improved, the need for manual operation is reduced, and the consistency and reliability of data collection are ensured; high-precision oblique photography and RTK positioning technology are used, combined with a model reconstruction algorithm, to achieve the rapid generation of high-precision two-dimensional orthophotos and three-dimensional real-scene models, innovatively introduce multi-source data matching technology, deeply integrate BIM data, CAD data and GIS information with three-dimensional real-scene models, construct a comprehensive and accurate three-dimensional spatial scene, and greatly enhance the integrity and practicality of the data; in addition, in terms of earthwork volume calculation, a high-precision digital terrain model and intelligent analysis algorithm are introduced to significantly improve the accuracy and efficiency of earthwork volume calculation , providing a reliable basis for project progress management and cost control; at the same time, through multi-time comparison and intelligent analysis of three-dimensional spatial scenes, dynamic monitoring and visual display of project progress are realized, so that project managers can intuitively and timely grasp the progress of the project and quickly identify potential progress problems; more importantly, this method uses multi-dimensional data analysis technology to deeply integrate and analyze the earthwork volume analysis results, dynamic change data of project progress and three-dimensional spatial scenes, which can not only generate targeted project management optimization suggestions, but also provide timely and accurate safety risk warning information, greatly improving the intelligence level of project management and decision-making support capabilities; in addition, the full-process automation characteristics of this method greatly reduce the possibility of human errors and improve the overall quality and reliability of project management.
[0040] In a specific embodiment, the process of executing step S101 may specifically include the following steps:
[0041] (1) Perform a laser radar scan on the construction site to obtain high-precision point cloud data, and use the point cloud segmentation algorithm to extract terrain features from the high-precision point cloud data to obtain a terrain feature model of the construction site;
[0042] (2) Conduct electromagnetic field simulation analysis on the terrain feature model of the project site to obtain the electromagnetic interference distribution map, and determine the optimal deployment position of the UAV guard system based on the electromagnetic interference distribution map to obtain the deployment coordinates of the guard system;
[0043] (3) Perform multi-objective optimization calculation on the deployment coordinates of the guard system to obtain a deployment plan that takes into account coverage, energy consumption, and communication quality. Install the drone guard system according to the deployment plan to obtain the deployed guard system network.
[0044] (4) Perform network topology analysis on the deployed guard system network to obtain a network topology diagram, and configure the mesh network protocol according to the network topology diagram to obtain a self-organizing drone control network;
[0045] (5) Perform multi-scale analysis of the project site through satellite image processing algorithms to obtain stratified data of the project area, and generate a stratified sampling strategy based on the stratified data of the project area to obtain a stratified route planning plan;
[0046] (6) Monte Carlo simulation is performed on the hierarchical route planning scheme to obtain the route reliability evaluation results. The flight parameters are optimized based on the route reliability evaluation results to obtain a UAV control system that automatically executes the preset route mission.
[0047] Specifically, the laser radar technology is first used to scan the engineering site with high precision. The laser radar obtains the three-dimensional coordinate information of the target by emitting laser pulses and measuring the reflection time, thereby generating high-precision point cloud data. These point cloud data contain detailed terrain information of the engineering site with an accuracy of up to centimeters. Subsequently, the point cloud segmentation algorithm is used to process the high-precision point cloud data. The algorithm divides the point cloud data into different terrain feature units, such as flat land, slopes, buildings, etc., through methods such as clustering analysis and edge detection. A terrain feature model of the engineering site is generated, which provides a basis for subsequent analysis. Next, the terrain feature model of the engineering site is subjected to electromagnetic field simulation analysis. This step uses numerical calculation techniques such as finite difference time domain method (FDTD) or moment method to simulate the propagation characteristics of electromagnetic waves in complex terrain. The analysis takes into account factors such as terrain undulations, building occlusion, and surrounding electromagnetic environment, and generates a detailed electromagnetic interference distribution map. Based on this distribution map, the optimal deployment position of the drone guard system is determined by comprehensively evaluating factors such as electromagnetic interference intensity, signal coverage, and terrain accessibility, and the precise deployment coordinates of the guard system are obtained.
[0048] The obtained deployment coordinates of the guard system are subjected to multi-objective optimization calculations, using genetic algorithms or particle swarm optimization algorithms, while considering multiple objectives such as coverage, energy consumption, and communication quality. During the optimization process, the coverage is quantified by calculating the percentage of the engineering area within the field of view of the drone, the energy consumption is estimated based on the flight distance and altitude change, and the communication quality is evaluated by signal strength and interference level. Through multiple iterative calculations, an optimal deployment plan that balances various indicators is obtained. According to this plan, the drone guard system, including the automatic take-off and landing platform, charging equipment, and data transmission module, is installed to form a complete guard system network. The network topology analysis of the deployed guard system network is performed. This step uses graph theory algorithms, such as the minimum spanning tree algorithm or the shortest path algorithm, to analyze the connection relationship and communication path between each guard system node. The analysis results generate a network topology diagram, which shows the connection strength and data flow direction between each node. Based on this topology diagram, the mesh network protocol is configured. Mesh network is a self-organizing and self-repairing network structure that can realize direct communication and data relay between drones. By configuring appropriate routing algorithms and data packet forwarding mechanisms, an efficient and reliable self-organizing drone control network is formed.
[0049] Subsequently, the project site was analyzed at multiple scales using satellite image processing algorithms. First, satellite images were decomposed into feature maps of different scales using multi-resolution analysis techniques, such as wavelet transform or pyramid decomposition. Then, different hierarchical structures of the project area, such as large geomorphic units, medium-sized construction areas, and small key facilities, were identified using image segmentation and feature extraction algorithms, such as region growing or edge detection. These analysis results formed hierarchical data of the project area. Based on these hierarchical data, a hierarchical sampling strategy was generated, that is, different sampling densities and flight altitudes were set in different hierarchical areas, so as to obtain a detailed hierarchical route planning scheme. Finally, a Monte Carlo simulation was performed on the hierarchical route planning scheme. The Monte Carlo method is a numerical calculation method based on random sampling, which is used here to evaluate the reliability of the route. During the simulation process, a large number of possible flight conditions, including variables such as wind speed, temperature, and battery status, were randomly generated, and the flight performance of the drone under these conditions was simulated. By statistically analyzing these simulation results, a quantitative evaluation of the route reliability was obtained. Based on these evaluation results, the flight parameters were optimized, including adjusting the flight speed, altitude, and route spacing, and finally a drone control system that can automatically execute the preset route mission was formed.
[0050] For example, in a large-scale wind farm construction project, a laser radar was first used to scan the 5 square kilometers of the project site, and high-precision point cloud data containing more than 1 billion points was obtained. Through the point cloud segmentation algorithm, three major wind turbine installation areas and two main transportation roads were identified. Electromagnetic field simulation analysis showed that there was strong electromagnetic interference in the mountaintop area, and the signal strength was attenuated by more than 60%. Based on these data, three locations on the mountainside were selected as deployment points for the drone guard system. The multi-objective optimization calculation took into account the coverage, energy consumption and communication quality of these three points, and finally determined a deployment plan that can cover 98% of the project area, with an average flight time of no more than 20 minutes and a communication signal strength always maintained above -75dBm. The network topology analysis showed that a triangular network structure was formed between the three guard systems, and the average data transmission delay was 15ms. The multi-scale analysis of satellite images divided the project area into three layers: large geomorphic units, wind turbine installation areas and fine operation areas, and set flight altitudes of 200m, 100m and 50m respectively. Monte Carlo simulations were performed 10,000 times, and the results showed that the planned routes could be completed safely under 95% of weather conditions. Based on these data, the flight parameters were finally optimized, including setting the average flight speed to 8m / s and adjusting the route spacing to 80% overlap of the field of view. In actual application, this system has greatly improved the monitoring efficiency of wind farm construction, shortening the daily inspection time from the original 4 hours to 1.5 hours, and improving the detection accuracy of abnormal situations.
[0051] In a specific embodiment, the process of executing step S102 may specifically include the following steps:
[0052] (1) Performing image preprocessing on the original data set to obtain denoised image data, and performing color correction on the denoised image data to obtain corrected image data;
[0053] (2) Extract feature points from the corrected image data to obtain a feature point set, and perform feature matching on the feature point set to obtain matching point pairs;
[0054] (3) The spatial position of the matching point pairs is calculated through the triangulation algorithm to obtain sparse point cloud data, and the sparse point cloud data is densely reconstructed to obtain dense point cloud data;
[0055] (4) Gridding the dense point cloud data to obtain three-dimensional grid data, and reconstructing the surface of the three-dimensional grid data to obtain three-dimensional surface data;
[0056] (5) Performing texture mapping on the three-dimensional surface data to obtain textured three-dimensional data, and performing geometric correction on the textured three-dimensional data to obtain a three-dimensional real scene model;
[0057] (6) Perform coordinate conversion on the RTK positioning data in the original data set to obtain geographic coordinate data, and perform geo-registration on the 3D real scene model based on the geographic coordinate data to obtain a geo-referenced 3D real scene model;
[0058] (7) Perform orthographic projection on the geo-referenced three-dimensional real-scene model to obtain orthographic projection data, and perform resolution averaging on the orthographic projection data to obtain a high-precision two-dimensional orthophoto.
[0059] Specifically, the original data set is subjected to image preprocessing, which includes removing image noise using algorithms such as Gaussian filtering or median filtering. For example, for Gaussian filtering, its two-dimensional convolution operation can be expressed as:
[0060]
[0061] in, is a Gaussian function, is the standard deviation, is the pixel coordinate. Through this filtering, the random noise in the image is effectively suppressed. Subsequently, the denoised image data is color corrected using techniques such as histogram equalization or white balance adjustment to ensure the accuracy and consistency of the image color.
[0062] Next, feature point extraction is performed on the corrected image data. Common algorithms include SIFT (Scale Invariant Feature Transform) or SURF (Speeded Up Robust Features). Taking SIFT as an example, its feature descriptor can be expressed as:
[0063]
[0064] in,( ) is the feature point coordinate, It's the scale. It's the direction. is a 128-dimensional feature vector. The extracted feature point set is then feature matched. By calculating the Euclidean distance or Hamming distance between feature descriptors, the corresponding points between different images are found to form matching point pairs. The spatial position of the matching point pairs is calculated by the triangulation algorithm to obtain sparse point cloud data. The basic principle of triangulation is to use the known camera parameters and the two-dimensional coordinates of the matching points in different images to calculate the position of the three-dimensional space points. This can be achieved by solving a system of linear equations. After obtaining the sparse point cloud, algorithms such as PMVS (Patch-based Multi-view Stereo) are used for dense reconstruction to generate high-density point cloud data.
[0065] Common methods for meshing dense point cloud data include Poisson surface reconstruction or March cube algorithm. This step converts discrete point cloud data into a continuous three-dimensional mesh surface. Subsequently, the three-dimensional mesh data is reconstructed using subdivision surfaces or NURBS techniques to obtain smooth three-dimensional surface data. Texture mapping is performed on the three-dimensional surface data to map the texture information in the original image to the surface of the three-dimensional model to enhance the visual realism of the model. After texture mapping, the textured three-dimensional data is geometrically corrected to correct the geometric distortion caused by the projection transformation and obtain an accurate three-dimensional real scene model.
[0066] The RTK positioning data in the original data set needs to be converted to a universal geographic coordinate system (such as WGS84). The general form of the coordinate conversion can be expressed as:
[0067]
[0068] in,( X,Y,Z ) is the target coordinate system coordinate, ( x,y,z ) is the original coordinate system coordinate, R is the rotation matrix, T is the translation vector. The converted geographic coordinate data is used to geo-reference the 3D real scene model to ensure the accurate positioning of the model in the real geographic space. Finally, the georeferenced 3D real scene model is orthographically projected to eliminate image deformation caused by terrain undulations and camera tilt. The data after orthographic projection is processed by resolution averaging to ensure the resolution of the entire image is consistent, thereby obtaining a high-precision 2D orthophoto.
[0069] For example, in an intelligent inspection of a large dam project, the drone collected 500 high-resolution images, each with 4000x3000 pixels. In the image preprocessing stage, a Gaussian filter with a standard deviation of 1.5 was used to remove noise, which effectively reduced the image noise level and improved the signal-to-noise ratio by about 3dB. Color correction adjusted the color temperature of the image from the original 7500K to the standard daylight color temperature of 5500K. In the feature point extraction stage, an average of 2000 SIFT feature points were extracted from each image, and feature matching found an average of 800 pairs of matching points between adjacent images. Triangulation generated about 1 million sparse point clouds, and the number of point clouds increased to 500 million after dense reconstruction. Gridding processing generated a model containing 10 million triangular facets, and after surface reconstruction, a three-dimensional surface model with centimeter-level accuracy was obtained. The coordinate conversion accuracy of RTK positioning data reached the centimeter level, ensuring the accurate positioning of the three-dimensional model in the real geographic space. The final orthophoto covers an area of 2 square kilometers with a ground resolution of 5 cm / pixel, providing high-precision data support for dam safety monitoring and construction progress assessment.
[0070] In a specific embodiment, the process of performing the step of calculating the spatial position of the matching point pair using a triangulation algorithm may specifically include the following steps:
[0071] (1) Estimate the camera position of the matching point pairs to obtain the camera space position data, and optimize the camera space position data to obtain accurate camera position data;
[0072] (2) Perform forward intersection calculation on the matching point pairs based on the precise camera position data to obtain three-dimensional spatial coordinate data, and then perform sparse reconstruction on the three-dimensional spatial coordinate data to obtain sparse point cloud data;
[0073] (3) Perform spatial clustering analysis on the sparse point cloud data to obtain a point cloud cluster set, and then perform local feature description on the point cloud cluster set to obtain a local feature descriptor;
[0074] (4) Select seed points from sparse point cloud data using local feature descriptors to obtain a seed point set, and perform regional growth on the seed point set to obtain extended point cloud data;
[0075] (5) Generate a depth map for the extended point cloud data to obtain a multi-view depth atlas, and perform perspective fusion through the multi-view depth atlas to obtain a fused depth map;
[0076] (6) The fused depth map is back-projected pixel by pixel to obtain initial dense point cloud data, and the initial dense point cloud data is noise filtered to obtain dense point cloud data.
[0077] Specifically, the spatial position of the camera is estimated by solving the PnP (Perspective-n-Point) problem using the two-dimensional coordinate information of the matching point pair. The solution to the PnP problem can be expressed by the following equation:
[0078]
[0079] in, R is the rotation matrix, t is the translation vector, is a two-dimensional image point, is the corresponding three-dimensional space point, is the projection function. By minimizing the projection error, the initial camera space position data is obtained. Subsequently, the camera space position data is optimized using the BundleAdjustment algorithm, which fine-tunes the camera parameters and 3D point coordinates by minimizing the reprojection error. Its objective function can be expressed as:
[0080]
[0081] here, represents the camera parameters, represents the coordinates of a three-dimensional point, is the observed image point, is the predicted image point, is the visibility indicator.
[0082] After optimization, accurate camera position data is obtained for forward intersection calculation. Forward intersection uses known camera parameters and image point coordinates to calculate the coordinates of three-dimensional space points through triangulation. It can be solved by linear least squares method:
[0083]
[0084] in, A is the coefficient matrix composed of camera parameters and image coordinates, X is the coordinate of the 3D point to be found. Solving this equation obtains the 3D spatial coordinate data to form a sparse point cloud. Perform spatial clustering analysis on the sparse point cloud data. Through iterative optimization, the point cloud data is divided into several point cloud clusters. For each point cloud cluster, local features can be described. Algorithms such as 3DSIFT or FPFH can be used to generate feature vectors that describe the geometric features of each local area.
[0085] Based on these local feature descriptors, points with significant features are selected as seed points. Seed point selection can be achieved by setting the threshold of the feature descriptor. Regional growth is performed on the selected seed points, that is, starting from the seed points, the surrounding points that meet the similarity conditions are gradually added to the current region to obtain the extended point cloud data. A depth map is generated for the extended point cloud data. This step projects the three-dimensional point cloud onto a two-dimensional plane to generate an image representing the depth value of each pixel. For each perspective, a depth map is generated to form a multi-perspective depth atlas. Then, the multi-perspective depth atlas is used to perform perspective fusion, comprehensively consider the depth information of different perspectives, solve problems such as occlusion and missing, and obtain a fused depth map.
[0086] Finally, the fused depth map is back-projected pixel by pixel, that is, the depth value of each pixel is converted back to the three-dimensional space coordinates to obtain the initial dense point cloud data. These data are noise filtered, and common methods include statistical outlier filtering or radius filtering to remove unreasonable points and finally obtain high-quality dense point cloud data.
[0087] For example, in an intelligent inspection of a large bridge construction project, a drone collected 1,000 high-resolution images. Through feature matching, about 500,000 matching point pairs were obtained. The PnP algorithm was used to preliminarily estimate the camera position, with an average reprojection error of 2.5 pixels. After optimization by Bundle Adjustment, the reprojection error was reduced to 0.8 pixels, and the camera position accuracy was improved to the centimeter level. The forward intersection calculation generated about 1 million three-dimensional spatial points to form the initial sparse point cloud. K-means clustering (k=50) divided these points into 50 regions, each containing an average of 20,000 points. The FPFH algorithm was used to extract a 33-dimensional feature vector for each region, and the 1% points with the largest eigenvalues were selected as seed points for regional growth, expanding the number of point clouds to 5 million points. In the depth map generation stage, a 1024x768 resolution depth map was generated for each view, and a high-precision depth map covering the entire bridge was obtained after fusion. Pixel-by-pixel back-projection generated about 200 million initial dense point clouds. After statistical outlier filtering (the threshold was set to 3 times the standard deviation), 180 million high-quality dense point cloud data points were finally obtained, providing a solid data foundation for accurate monitoring and quality assessment of bridge construction.
[0088] In a specific embodiment, the process of executing step S103 may specifically include the following steps:
[0089] (1) Extract feature points from the 3D real scene model to obtain a 3D feature point set, and calculate descriptors for the 3D feature point set to obtain a 3D feature descriptor;
[0090] (2) Extract geometric features from BIM data to obtain a BIM feature point set, and calculate descriptors for the BIM feature point set to obtain a BIM feature descriptor;
[0091] (3) Match the 3D feature descriptor and the BIM feature descriptor through the feature matching algorithm to obtain the initial matching point pair, and perform a geometric consistency check on the initial matching point pair to obtain the valid matching point pair;
[0092] (4) Calculate the transformation matrix from the 3D real scene model to the BIM data based on the effective matching point pairs to obtain the initial registration parameters, and iteratively optimize the initial registration parameters to obtain the refined registration parameters;
[0093] (5) Vectorizing the CAD data to obtain CAD vector data, and spatially transforming the CAD vector data according to the fine registration parameters to obtain aligned CAD data;
[0094] (6) Convert the GIS information into a coordinate system to obtain GIS data in a unified coordinate system, and superimpose the aligned CAD data with the GIS data in the unified coordinate system to obtain a comprehensive geographic information layer;
[0095] (7) The three-dimensional real-scene model, BIM data, aligned CAD data, and integrated geographic information layer are spatially integrated to obtain a three-dimensional spatial scene containing BIM data, CAD data, and GIS information.
[0096] Specifically, in the intelligent aerial inspection method of engineering based on drones, multi-source data fusion is a key link. First, feature points are extracted from the three-dimensional real scene model using the 3D-SIFT algorithm. This algorithm defines a scale space function in three-dimensional space:
[0097]
[0098] in, is a three-dimensional Gaussian kernel, is a three-dimensional image, is a scale parameter. By detecting local extreme points at different scales, a set of three-dimensional feature points is obtained. Descriptors are calculated for these feature points to form a three-dimensional feature descriptor. At the same time, geometric features are extracted from BIM data. BIM data contains geometric and semantic information of building components. By extracting features such as edges and corners, a set of BIM feature points is obtained. Descriptors are calculated for these feature points. The shape context descriptor is a method for describing the local features of points in a point cloud or three-dimensional model. It describes the geometric features of a point by considering the distribution of a point relative to its surrounding points. Specifically, its calculation process is as follows:
[0099] 1. For a given point p, consider a set of points around it. 2. Divide the space around point p into a number of bins, which are usually defined based on distance and angle. 3. For each bin, count the number of points that fall within the bin. 4. These numbers form a vector, which is the shape context descriptor of the point.
[0100] Next, the FLANN (Fast Nearest Neighbor Search Library) algorithm is used to match the 3D feature descriptor and the BIM feature descriptor. FLANN accelerates the nearest neighbor search by constructing a kd tree or a k-means tree. For each 3D feature descriptor, the BIM feature descriptor with the smallest Euclidean distance is found to form an initial matching point pair. Then a geometric consistency check is performed, and the RANSAC algorithm is used to eliminate incorrect matches to obtain valid matching point pairs.
[0101] Based on the valid matching point pairs, the transformation matrix from the 3D real scene model to the BIM data is calculated. It can be expressed as solving the rigid body transformation:
[0102]
[0103] in, is the error function, is a 3x1 translation vector, is a point in the 3D reality model. is the corresponding point in the BIM data, is the weight of each pair of matching points. Solving this least squares problem obtains the initial registration parameters, which are then iteratively optimized using the ICP (iterative closest point) algorithm to obtain the refined registration parameters.
[0104] Vectorize the CAD data and convert the rasterized CAD drawings into vector format. This includes edge detection, line segment extraction, and polygon fitting. After obtaining the CAD vector data, use the fine alignment parameters obtained above to perform spatial transformation to align the CAD data with the three-dimensional real-life model. GIS information usually uses different coordinate systems and requires coordinate system conversion. Common conversion methods include seven-parameter conversion or TIN interpolation. After conversion, GIS data in a unified coordinate system is obtained, which is then superimposed with the aligned CAD data to form a comprehensive geographic information layer.
[0105] Finally, the 3D real scene model, BIM data, aligned CAD data and comprehensive geographic information layer are spatially fused. It is necessary to consider the accuracy and resolution of different data sources and use multi-resolution fusion technology, such as wavelet transform or pyramid fusion, to finally obtain a high-precision 3D spatial scene containing BIM data, CAD data and GIS information.
[0106] For example, in the intelligent inspection of a large airport expansion project, drones collected 5,000 high-resolution images and reconstructed a 3D real-life model containing 1 billion points. 500,000 feature points were extracted from them using the 3D-SIFT algorithm, and each feature point was described by a 128-dimensional vector. The BIM model contains detailed information on major buildings such as the airport terminal and runway, and 200,000 feature points were extracted. The FLANN algorithm initially matched 100,000 pairs of matching points, and 50,000 pairs of valid matches were retained after RANSAC screening. The average error of the calculated initial registration parameters was 0.5 meters. After 50 ICP iterations, the average error of the fine registration parameters dropped to 0.05 meters. The CAD data contains a detailed plan of the airport, and 100,000 vector entities were obtained after vectorization processing. The GIS data covers 100 square kilometers of the airport surrounding area, including information such as terrain and water systems. The final fused 3D spatial scene has a centimeter-level accuracy, providing a comprehensive and accurate digital twin platform for progress monitoring, safety management, and environmental impact assessment of the airport expansion project.
[0107] In a specific embodiment, the process of executing step S104 may specifically include the following steps:
[0108] (1) Extract terrain from the three-dimensional space scene to obtain terrain surface data, and then grid the terrain surface data to obtain terrain grid data;
[0109] (2) Perform image segmentation on the high-precision two-dimensional orthophoto to obtain the earthwork area mask, and then clip the terrain grid data according to the earthwork area mask to obtain the earthwork calculation area;
[0110] (3) Generate triangulated network for the earthwork calculation area to obtain irregular triangulated network data, and perform volume integration on the irregular triangulated network data to obtain the original earthwork volume data;
[0111] (4) Extract the design elevation from the CAD data in the three-dimensional space scene to obtain the design elevation surface data, and generate a triangulated network from the design elevation surface data to obtain the design triangulated network data;
[0112] (5) Spatially compare the original earthwork volume data with the designed triangulated network data to obtain the cut-and-fill partition data, and calculate the volume difference of the cut-and-fill partition data to obtain the cut-and-fill volume data;
[0113] (6) Perform loose laying coefficient correction on the fill and cut volume data to obtain earthwork volume analysis results.
[0114] Specifically, in the intelligent aerial inspection method of engineering based on drones, earthwork volume calculation is a key link. First, terrain extraction is performed on the three-dimensional space scene, and the regional growing algorithm is used to identify ground points. The algorithm starts from the lowest point and gradually adds neighboring points with a height difference less than the threshold to the ground point set. Terrain extraction can be expressed as:
[0115]
[0116] Where T is the terrain point set, It is a point to be judged. Indicates the height of the point, is the lowest point, is the height threshold. After obtaining the terrain surface data, grid processing is performed, and terrain grid data is generated through the Delaunay triangulation algorithm. Next, the high-precision two-dimensional orthophoto is segmented, and a semantic segmentation algorithm based on deep learning, such as U-Net, is used to obtain the earthwork area mask. The loss function of U-Net can be expressed as:
[0117]
[0118] here, is the total number of pixels, is the true label, is the predicted value. According to the obtained earthwork area mask, the terrain grid data is clipped to obtain the earthwork calculation area. The earthwork calculation area is triangulated using the constrained Delaunay triangulation algorithm, which takes into account the regional boundary constraints while ensuring the Delaunay property. The generated irregular triangulated network data is used for subsequent volume calculations. The volume integral calculation formula is:
[0119]
[0120] in, is the total volume, n is the number of triangles, It is i The triangle j The height of the vertices, is the area of the triangle. This method is used to obtain the original earthwork volume data. The design elevation is extracted from the CAD data in the three-dimensional space scene. This step uses the CAD layer information and attribute data to extract the elevation points representing the design ground. These elevation points are interpolated to obtain continuous design elevation surface data. Then the same triangulation algorithm as terrain processing is used to generate the design triangulated network data.
[0121] The original earthwork volume data is spatially compared with the designed triangulated network data by calculating the height difference of each triangulated network unit. The comparison result can be expressed as:
[0122]
[0123] here, It is i The height difference of each unit, is the actual height, is the design height. A positive value indicates that excavation is required, and a negative value indicates that backfilling is required. Based on these height differences, the cut-and-fill partition data is obtained, and the volume difference of each partition is calculated to obtain the cut-and-fill volume data. Finally, the cut-and-fill volume data is corrected by the loose spreading coefficient. The loose spreading coefficient reflects the volume change of earthwork during excavation and backfilling. The correction formula is:
[0124]
[0125] in, is the corrected volume, is the original volume, It is the loose spreading coefficient. Different soil types have different loose spreading coefficients, which are usually determined through field sampling tests.
[0126] For example, in a large-scale water conservancy project, drones scanned a 100-hectare construction area with high precision and generated 3D point cloud data containing 500 million points. The terrain extraction algorithm identified 480 million ground points to form a terrain surface. After gridding, terrain grid data with 2 million triangular units was obtained. 2D orthophoto segmentation identified an 80-hectare earthwork area, corresponding to 160 million terrain points. The triangulation algorithm generated 1 million irregular triangles in this area. The volume integral calculation obtained the original earthwork volume of 5 million cubic meters. CAD data provided design elevation information and generated 900,000 design triangulation units. Spatial comparison found that 60% of the area required excavation, with a total excavation volume of 4 million cubic meters; 40% of the area required filling, with a total filling volume of 1 million cubic meters. Considering that the project area is mainly clay, a loose laying coefficient of 1.2 was used for correction, and the actual excavation volume was finally calculated to be 4.8 million cubic meters and the filling volume was 1.2 million cubic meters. These accurate earthwork volume data provide an important basis for project progress management, equipment scheduling and cost control, greatly improving the efficiency and accuracy of project management.
[0127] In a specific embodiment, the process of executing step S105 may specifically include the following steps:
[0128] (1) Divide the three-dimensional space scene into time series to obtain three-dimensional scene data at multiple time nodes, and align the three-dimensional scene data at multiple time nodes to obtain aligned time series three-dimensional scene data;
[0129] (2) Perform difference detection on the aligned time-series 3D scene data to obtain scene change area data, and perform semantic segmentation on the scene change area data to obtain engineering component-level change data;
[0130] (3) Use BIM data to mark the change data at the engineering component level to obtain component progress information, and perform statistical analysis on the component progress information to obtain the overall progress data of the project;
[0131] (4) Perform time series analysis on the overall project progress data to obtain progress trend data, and perform anomaly detection on the progress trend data to obtain progress anomaly point data;
[0132] (5) Compare the progress anomaly data with the project plan to obtain the progress deviation data, and visualize the progress deviation data to obtain a progress deviation heat map;
[0133] (6) Comprehensively process the progress deviation heat map and the overall project progress data to obtain the dynamic change data of the project progress and a visual progress report.
[0134] Specifically, the three-dimensional space scene is divided into time series, and the data collected by the drone at different time points are arranged in time order to form three-dimensional scene data of multiple time nodes. These data usually include point clouds, orthophotos, and three-dimensional models. In order to ensure that the data at different time points are comparable, these three-dimensional scene data need to be registered. The registration process uses the iterative closest point (ICP) algorithm to achieve accurate alignment of the three-dimensional scene by minimizing the distance between corresponding points at different time points, and obtain the aligned time-series three-dimensional scene data. Difference detection of the aligned time-series three-dimensional scene data is the next key step. The voxelization method is used to divide the three-dimensional space into a fixed-size cubic grid, and then compare the occupancy status of each voxel at different time points. In this way, the area where changes occur in the scene can be quickly identified, and the scene change area data can be obtained. Subsequently, the scene change area data is semantically segmented, and a deep learning model such as PointNet++ or 3D-UNet is used to divide the change area into different semantic categories, such as new components, removed components, shape changes, etc., so as to obtain engineering component-level change data.
[0135] The introduction of BIM (Building Information Modeling) data is the key to achieving accurate progress tracking. By annotating the change data at the engineering component level with BIM data, the detected changes can be associated with specific engineering components. It involves spatial registration and feature matching, and the changes in the three-dimensional scene correspond to the components in the BIM model one by one. After the annotation is completed, the component progress information containing the current status of each component is obtained. This information is statistically analyzed, such as calculating the completion rate of various components, the construction progress of each area, etc., and finally the overall progress data of the project reflecting the progress of the entire project is obtained. Time series analysis of the overall progress data of the project is an important means to predict and evaluate the progress of the project. This step uses time series prediction models, such as the autoregressive integrated moving average (ARIMA) model or the long short-term memory (LSTM) neural network, to model and predict the progress data and obtain progress trend data reflecting future trends. Next, the progress trend data is anomaly detected, and algorithms such as moving average deviation (MAD) or local anomaly factor (LOF) are used to identify time points that deviate significantly from the expected progress and obtain progress anomaly point data.
[0136] Comparing the progress anomaly data with the project plan is the key to evaluating the execution of the project. It involves aligning the actual progress with the planned progress on the timeline and calculating the progress deviation at each time point. Deviation calculation takes into account factors such as workload, duration and critical path. The obtained progress deviation data is visualized in the form of a heat map, using color depth to indicate the degree of deviation, red indicates progress lag, and green indicates progress ahead, so as to intuitively show the progress status of each part of the project. Finally, the progress deviation heat map and the overall progress data of the project are comprehensively processed. This step combines quantitative progress data with intuitive visual representation to generate a data set that fully reflects the dynamic changes in the project progress. At the same time, based on these data, a progress report containing text descriptions, charts and 3D visualizations is automatically generated to provide project managers with intuitive and detailed progress information.
[0137] For example, in a large bridge construction project, drones performed a comprehensive scan once a week, generating 3D point cloud data containing about 1 billion points. After 12 weeks, 3D scene data of 12 time points were accumulated, and the size of each data set was about 500GB. The ICP algorithm was used for registration, and the average registration error was controlled within 2cm. In the difference detection stage, a voxel size of 50cm×50cm×50cm was used to identify about 5 million changed voxels. Semantic segmentation classified these changes into categories such as new beams, pier construction, and temporary support removal. Combined with BIM data, the progress status of 3,000 components was determined. Time series analysis predicted the progress trend for the next 4 weeks, and the anomaly detection algorithm found 3 progress anomalies, 2 of which were delayed and 1 was ahead of schedule. Progress deviation analysis showed that the overall pier construction was delayed by 2 days, while the main beam installation was ahead of schedule by 1 day. The final progress report includes a 20-page PDF document with 15 progress charts and an interactive 3D model, which intuitively shows the overall progress and local details of the project.
[0138] In a specific embodiment, the process of executing step S106 may specifically include the following steps:
[0139] (1) Perform time series decomposition on the earthwork volume analysis results to obtain earthwork volume trend data, and perform anomaly detection on the earthwork volume trend data to obtain earthwork volume anomaly point data;
[0140] (2) Perform critical path analysis on the dynamic change data of the project progress to obtain the key node data of the progress, and then spatially and temporally associate the key node data of the progress with the abnormal point data of earthwork volume to obtain the earthwork-progress correlation data;
[0141] (3) Extract resource requirements from BIM data in the three-dimensional space scene to obtain resource allocation plan data, and match the resource allocation plan data with earthwork-progress association data to obtain resource allocation recommendation data;
[0142] (4) Analyze the terrain slope of the three-dimensional space scene to obtain high-risk area data, and overlay the high-risk area data with the dynamic change data of the project progress to obtain spatiotemporal safety risk data;
[0143] (5) Comprehensively process the resource allocation suggestion data and spatiotemporal security risk data to obtain multi-dimensional optimization solution data, and visualize the multi-dimensional optimization solution data to obtain interactive decision support charts;
[0144] (6) Perform natural language generation and processing on interactive decision support charts to obtain engineering management optimization suggestions and safety risk warning information.
[0145] Specifically, the earthwork volume analysis results were decomposed into three components: trend, seasonality, and residual using the seasonal trend decomposition (STL) method. The trend part reflects the long-term change trend of earthwork volume, the seasonal part captures the periodic changes, and the residual represents random fluctuations. Through this decomposition, earthwork volume trend data reflecting the long-term changes in earthwork volume were obtained. Subsequently, anomaly detection was performed on the earthwork volume trend data, and the moving average deviation (MAD) method was used to identify data points that significantly deviated from expectations, thereby obtaining earthwork volume anomaly point data. At the same time, a critical path analysis was performed on the dynamic change data of the project progress. This step uses the Project Evaluation and Review Technique (PERT) to calculate the earliest start time, the latest start time, and the total floating time of each task, thereby identifying the key tasks that have a significant impact on the total project duration and obtaining the progress key node data. These progress key node data were subjected to a spatiotemporal correlation analysis with the earthwork volume anomaly point data obtained previously, which involves the alignment of time series and the matching of spatial positions, and finally the earthwork-progress correlation data reflecting the relationship between earthwork volume changes and project progress were obtained.
[0146] Next, the resource requirements are extracted from the BIM data in the three-dimensional space scene. This step involves analyzing the attribute information of each component in the BIM model, including material type, construction process, and required equipment, so as to obtain detailed resource allocation plan data. These resource allocation plan data are matched with the earthwork-progress association data obtained previously, taking into account the relationship between resource requirements and actual project progress and earthwork volume changes, and finally obtaining resource allocation suggestion data, which provides a basis for project managers to optimize resource allocation. At the same time, the terrain slope analysis of the three-dimensional space scene is carried out, and the slope value of each grid unit is calculated using the digital elevation model (DEM), and steep areas that may have safety risks are identified according to the preset threshold to obtain high-risk area data. These high-risk area data are superimposed and analyzed with the dynamic change data of the project progress. This step takes into account the dynamic relationship between the project progress and terrain risk, and finally obtains spatiotemporal safety risk data reflecting potential safety hazards in different construction stages.
[0147] The resource allocation suggestion data and spatiotemporal safety risk data obtained above are processed comprehensively. This step uses a multi-objective optimization algorithm, such as genetic algorithm or particle swarm optimization, and considers multiple factors such as resource utilization efficiency, safety risk control and construction period targets to obtain multi-dimensional optimization solution data that balances various indicators. Subsequently, these multi-dimensional optimization solution data are visualized, and interactive data visualization technologies such as Tableau or D3.js are used to generate interactive decision support charts containing multi-level information, so that project managers can intuitively understand and explore various optimization solutions. Finally, the interactive decision support charts are processed by natural language generation. This step uses template-based natural language generation technology to convert key information in the chart into easy-to-understand text descriptions. It involves data interpretation, key point extraction and language generation, and finally obtains a text report containing specific engineering management optimization suggestions and safety risk warning information.
[0148] For example, in a large-scale water conservancy project, drones performed a comprehensive scan once a week and continuously monitored the changes in earthwork volume for 26 weeks. Time series decomposition showed that the earthwork volume showed an overall upward trend, but there was an abnormal increase in the 15th week, with a sudden increase of 50%. Critical path analysis identified five key nodes, of which the third node coincided with the time of abnormal earthwork volume growth. BIM data analysis showed that three excavators and 20 transport vehicles were required in this stage, but the actual configuration was insufficient. Terrain analysis found that there were three high-risk areas with slopes exceeding 30 degrees in the project area, one of which coincided with the area of abnormal earthwork volume growth in the 15th week. The multi-objective optimization algorithm generated five resource allocation plans, among which Plan 3 was expected to shorten the construction period by 5 days by adding two excavators and eight transport vehicles while ensuring safety. The interactive decision support chart intuitively shows the comparison of these five plans, including resource utilization, safety risk index and expected completion time. Based on this, the natural language generation system generated a 1,000-word management suggestion report, which focused on adding machinery and equipment in the 15th week and strengthening safety supervision in high-risk areas. It is expected that the risk of construction delays can be effectively controlled and the overall construction efficiency can be improved, which greatly improved the scientificity and efficiency of project management.
[0149] The above describes the engineering intelligent aerial inspection method based on a drone in the embodiment of the present application. The following describes the engineering intelligent aerial inspection device based on a drone in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of an engineering intelligent aerial inspection device based on a drone includes:
[0150] The deployment module 201 is used to deploy the UAV on-duty system at the engineering site, obtain the UAV control system that automatically executes the preset route mission, collect the engineering site data through the UAV control system, and process the collected oblique photography data and RTK positioning data to obtain the original data set;
[0151] The reconstruction module 202 is used to reconstruct the model of the original data set to obtain a high-precision two-dimensional orthophoto and a three-dimensional real scene model, which specifically includes: performing image preprocessing on the original data set to obtain denoised image data, and performing color correction on the denoised image data to obtain corrected image data; extracting feature points from the corrected image data to obtain a feature point set, and performing feature matching on the feature point set to obtain matching point pairs; calculating the spatial position of the matching point pairs through a triangulation algorithm to obtain sparse point cloud data, and densely reconstructing the sparse point cloud data to obtain dense point cloud data; and gridding the dense point cloud data. Processing to obtain three-dimensional grid data, and performing surface reconstruction on the three-dimensional grid data to obtain three-dimensional curved surface data; texture mapping the three-dimensional curved surface data to obtain textured three-dimensional data, and geometrically correcting the textured three-dimensional data to obtain the three-dimensional real scene model; coordinate conversion of the RTK positioning data in the original data set to obtain geographic coordinate data, and geo-registering the three-dimensional real scene model according to the geographic coordinate data to obtain a geographic reference three-dimensional real scene model; orthographically projecting the geographic reference three-dimensional real scene model to obtain orthographic projection data, and performing resolution averaging processing on the orthographic projection data to obtain the high-precision two-dimensional orthophoto;
[0152] A matching module 203 is used to perform multi-source data matching on the three-dimensional real scene model to obtain a three-dimensional space scene including BIM data, CAD data and GIS information;
[0153] A calculation module 204 is used to calculate the earthwork volume of the three-dimensional space scene to obtain an earthwork volume analysis result;
[0154] A comparison module 205 is used to compare the three-dimensional space scene over time to obtain dynamic change data of the project progress and a visual progress report;
[0155] The analysis module 206 is used to perform multi-dimensional data analysis on the three-dimensional space scene through the earthwork volume analysis results and the dynamic change data of the project progress, so as to obtain project management optimization suggestions and safety risk warning information.
[0156] Through the coordinated cooperation of the above-mentioned components, by deploying the drone duty system and realizing the automatic execution of preset route tasks, the automation and efficiency of data collection have been greatly improved, the need for manual operation has been reduced, and the consistency and reliability of data collection have been ensured; the use of high-precision oblique photography and RTK positioning technology, combined with the model reconstruction algorithm, has realized the rapid generation of high-precision two-dimensional orthophotos and three-dimensional real-scene models, and innovatively introduced multi-source data matching technology to deeply integrate BIM data, CAD data and GIS information with three-dimensional real-scene models, constructing a comprehensive and accurate three-dimensional spatial scene, greatly enhancing the integrity and practicality of the data; in addition, in terms of earthwork volume calculation, the introduction of high-precision digital terrain models and intelligent analysis algorithms has significantly improved the accuracy of earthwork volume calculation. The method improves the efficiency and effectiveness of the project, providing a reliable basis for project progress management and cost control; at the same time, through multi-time comparison and intelligent analysis of the three-dimensional space scene, dynamic monitoring and visual display of the project progress are realized, so that project managers can intuitively and timely grasp the progress of the project and quickly identify potential progress problems; more importantly, this method uses multi-dimensional data analysis technology to deeply integrate and analyze the earthwork volume analysis results, dynamic change data of the project progress and the three-dimensional space scene, which can not only generate targeted project management optimization suggestions, but also provide timely and accurate safety risk warning information, greatly improving the intelligent level of project management and decision-making support capabilities; in addition, the full-process automation characteristics of this method greatly reduce the possibility of human errors and improve the overall quality and reliability of project management.
[0157] The present application also provides an unmanned aerial vehicle-based engineering intelligent aerial inspection device, which includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the unmanned aerial vehicle-based engineering intelligent aerial inspection method in the above-mentioned embodiments.
[0158] Figure 33 is a schematic diagram of the structure of the engineering intelligent aerial inspection equipment based on drones provided in an embodiment of the present invention. The engineering intelligent aerial inspection equipment 300 based on drones may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be short-term storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the engineering intelligent aerial inspection equipment 300 based on drones. Furthermore, the processor 310 may be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the engineering intelligent aerial inspection equipment 300 based on drones.
[0159] The drone-based engineering intelligent aerial inspection device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 3 The structure of the UAV-based engineering intelligent aerial inspection equipment shown does not constitute a limitation of the UAV-based engineering intelligent aerial inspection equipment, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.
[0160] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the drone-based engineering intelligent aerial inspection method.
[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0162] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.
[0163] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for intelligent aerial inspection of engineering projects based on drones. It is characterized in that The engineering intelligent aerial inspection method based on drone includes: Deploy a drone on-duty system at the construction site to obtain a drone control system that automatically executes preset route tasks, collect construction site data through the drone control system, and process the collected oblique photography data and RTK positioning data to obtain an original data set; The original data set is model reconstructed to obtain a high-precision two-dimensional orthophoto and a three-dimensional real scene model, specifically including: performing image preprocessing on the original data set to obtain denoised image data, and performing color correction on the denoised image data to obtain corrected image data; extracting feature points from the corrected image data to obtain a feature point set, and performing feature matching on the feature point set to obtain matching point pairs; calculating the spatial position of the matching point pairs through a triangulation algorithm to obtain sparse point cloud data, and densely reconstructing the sparse point cloud data to obtain dense point cloud data; gridding the dense point cloud data to obtain 3D mesh data, and surface reconstruction of the 3D mesh data to obtain 3D curved surface data; texture mapping of the 3D curved surface data to obtain textured 3D data, and geometric correction of the textured 3D data to obtain the 3D real scene model; coordinate conversion of the RTK positioning data in the original data set to obtain geographic coordinate data, and geo-registration of the 3D real scene model according to the geographic coordinate data to obtain a geo-referenced 3D real scene model; orthographic projection of the geo-referenced 3D real scene model to obtain orthographic projection data, and resolution averaging of the orthographic projection data to obtain the high-precision 2D orthophoto; Perform multi-source data matching on the three-dimensional real scene model to obtain a three-dimensional space scene including BIM data, CAD data and GIS information; Calculating the earthwork volume of the three-dimensional space scene to obtain an earthwork volume analysis result; The three-dimensional space scene is compared over multiple time periods to obtain dynamic change data of the project progress and a visual progress report. The three-dimensional space scene is compared over multiple time periods to obtain dynamic change data of the project progress and a visual progress report, including: dividing the three-dimensional space scene into time series to obtain three-dimensional scene data of multiple time nodes, and registering the three-dimensional scene data of the multiple time nodes to obtain aligned time series three-dimensional scene data; performing difference detection on the aligned time series three-dimensional scene data to obtain scene change area data, and performing semantic segmentation on the scene change area data to obtain engineering component-level change data; through BI M data is used to mark the component-level change data of the project to obtain component progress information, and the component progress information is statistically analyzed to obtain the overall progress data of the project; time series analysis is performed on the overall progress data of the project to obtain progress trend data, and anomaly detection is performed on the progress trend data to obtain progress anomaly point data; the progress anomaly point data is compared with the project plan to obtain progress deviation data, and the progress deviation data is visualized to obtain a progress deviation heat map; the progress deviation heat map and the overall progress data of the project are comprehensively processed to obtain the dynamic change data of the project progress and a visualized progress report; The three-dimensional space scene is subjected to multi-dimensional data analysis through the earthwork volume analysis results and the dynamic change data of the project progress, so as to obtain project management optimization suggestions and safety risk warning information.
2. According to the method for intelligent aerial inspection of engineering based on drones in claim 1, It is characterized in that The UAV guard system is deployed on the engineering site to obtain a UAV control system that automatically executes the preset route mission. The engineering site data is collected through the UAV control system, and the collected oblique photography data and RTK positioning data are processed to obtain the original data set, including: Perform laser radar scanning on the construction site to obtain high-precision point cloud data, and extract terrain features from the high-precision point cloud data using a point cloud segmentation algorithm to obtain a terrain feature model of the construction site; Performing electromagnetic field simulation analysis on the terrain feature model of the engineering site to obtain an electromagnetic interference distribution map, and determining the optimal deployment position of the UAV guard system based on the electromagnetic interference distribution map to obtain the deployment coordinates of the guard system; Perform multi-objective optimization calculation on the deployment coordinates of the guard system to obtain a deployment plan that takes into account coverage, energy consumption and communication quality, and install the drone guard system according to the deployment plan to obtain a deployed guard system network; Performing a network topology analysis on the deployed guard system network to obtain a network topology diagram, and configuring a mesh network protocol according to the network topology diagram to obtain a self-organizing drone control network; Perform multi-scale analysis of the project site through satellite image processing algorithms to obtain stratified data of the project area, and generate a stratified sampling strategy based on the stratified data of the project area to obtain a stratified route planning solution; A Monte Carlo simulation is performed on the hierarchical route planning scheme to obtain a route reliability evaluation result, and flight parameters are optimized according to the route reliability evaluation result to obtain a UAV control system that automatically executes a preset route mission.
3. The engineering intelligent aerial inspection method based on unmanned aerial vehicle according to claim 1, It is characterized in that The spatial position calculation of the matching point pairs is performed by a triangulation algorithm to obtain sparse point cloud data, and the sparse point cloud data is densely reconstructed to obtain dense point cloud data, including: Performing camera position estimation on the matching point pairs to obtain camera space position data, and optimizing the camera space position data to obtain accurate camera position data; Performing forward intersection calculation on the matching point pair using the precise camera position data to obtain three-dimensional space coordinate data, and sparsely reconstructing the three-dimensional space coordinate data to obtain the sparse point cloud data; Performing spatial clustering analysis on the sparse point cloud data to obtain a point cloud cluster set, and performing local feature description on the point cloud cluster set to obtain a local feature descriptor; Selecting seed points from the sparse point cloud data using the local feature descriptor to obtain a seed point set, and performing region growing on the seed point set to obtain extended point cloud data; Generating a depth map for the extended point cloud data to obtain a multi-view depth atlas, and performing perspective fusion through the multi-view depth atlas to obtain a fused depth map; The fused depth map is back-projected pixel by pixel to obtain initial dense point cloud data, and the initial dense point cloud data is noise filtered to obtain the dense point cloud data.
4. The engineering intelligent aerial inspection method based on unmanned aerial vehicle according to claim 1, It is characterized in that The three-dimensional real scene model is matched with multiple sources of data to obtain a three-dimensional space scene containing BIM data, CAD data and GIS information, including: Extracting feature points from the three-dimensional real scene model to obtain a three-dimensional feature point set, and calculating descriptors from the three-dimensional feature point set to obtain a three-dimensional feature descriptor; Extracting geometric features from BIM data to obtain a BIM feature point set, and performing descriptor calculation on the BIM feature point set to obtain a BIM feature descriptor; The three-dimensional feature descriptor and the BIM feature descriptor are matched by a feature matching algorithm to obtain an initial matching point pair, and a geometric consistency check is performed on the initial matching point pair to obtain a valid matching point pair; Calculating a transformation matrix from the three-dimensional real scene model to the BIM data according to the effective matching point pairs to obtain initial registration parameters, and iteratively optimizing the initial registration parameters to obtain fine registration parameters; Performing vectorization processing on the CAD data to obtain CAD vector data, and performing spatial transformation on the CAD vector data according to the fine registration parameters to obtain aligned CAD data; Performing coordinate system conversion on the GIS information to obtain GIS data in a unified coordinate system, and superimposing the aligned CAD data with the GIS data in the unified coordinate system to obtain a comprehensive geographic information layer; The three-dimensional real scene model, the BIM data, the aligned CAD data and the comprehensive geographic information layer are spatially fused to obtain the three-dimensional spatial scene containing the BIM data, CAD data and GIS information.
5. The engineering intelligent aerial inspection method based on unmanned aerial vehicle according to claim 1, It is characterized in that The calculating the earthwork volume of the three-dimensional space scene to obtain the earthwork volume analysis result includes: Performing terrain extraction on the three-dimensional space scene to obtain terrain surface data, and performing gridding processing on the terrain surface data to obtain terrain grid data; Performing image segmentation on the high-precision two-dimensional orthophoto to obtain an earthwork area mask, and clipping the terrain grid data according to the earthwork area mask to obtain an earthwork calculation area; Generating a triangulated network for the earthwork calculation area to obtain irregular triangulated network data, and performing volume integration on the irregular triangulated network data to obtain original earthwork volume data; Extracting the design elevation from the CAD data in the three-dimensional space scene to obtain design elevation surface data, and generating a triangulated network from the design elevation surface data to obtain design triangulated network data; By spatially comparing the original earthwork volume data with the designed triangulated network data, fill-cut partition data is obtained, and volume difference calculation is performed on the fill-cut partition data to obtain fill-cut volume data; The fill and cut volume data is corrected for loose laying coefficient to obtain the earthwork volume analysis result.
6. The engineering intelligent aerial inspection method based on unmanned aerial vehicle according to claim 1, It is characterized in that The multi-dimensional data analysis of the three-dimensional space scene is performed through the earthwork volume analysis results and the engineering progress dynamic change data to obtain engineering management optimization suggestions and safety risk warning information, including: Performing time series decomposition on the earthwork volume analysis result to obtain earthwork volume trend data, and performing anomaly detection on the earthwork volume trend data to obtain earthwork volume abnormal point data; Performing a critical path analysis on the dynamic change data of the project progress to obtain key progress node data, and performing spatiotemporal association between the key progress node data and the earthwork volume abnormal point data to obtain earthwork-progress association data; Extracting resource requirements from the BIM data in the three-dimensional space scene to obtain resource allocation plan data, and matching the resource allocation plan data with the earthwork-progress association data to obtain resource allocation suggestion data; Performing terrain slope analysis on the three-dimensional space scene to obtain high-risk area data, and superimposing and analyzing the high-risk area data with the engineering progress dynamic change data to obtain spatiotemporal safety risk data; Comprehensively processing the resource allocation suggestion data and the spatiotemporal security risk data to obtain multi-dimensional optimization solution data, and visualizing the multi-dimensional optimization solution data to obtain interactive decision support charts; The interactive decision support chart is processed by natural language generation to obtain the engineering management optimization suggestion and safety risk warning information.
7. An intelligent aerial inspection device for engineering based on a drone, used to execute the intelligent aerial inspection method for engineering based on a drone as described in any one of claims 1 to 6, It is characterized in that The engineering intelligent aerial inspection device based on drone includes: A deployment module is used to deploy a drone guard system at the engineering site, obtain a drone control system that automatically executes the preset route mission, collect engineering site data through the drone control system, and process the collected oblique photography data and RTK positioning data to obtain an original data set; The reconstruction module is used to reconstruct the model of the original data set to obtain a high-precision two-dimensional orthophoto and a three-dimensional real scene model, which specifically includes: performing image preprocessing on the original data set to obtain denoised image data, and performing color correction on the denoised image data to obtain corrected image data; extracting feature points from the corrected image data to obtain a feature point set, and performing feature matching on the feature point set to obtain matching point pairs; calculating the spatial position of the matching point pairs through a triangulation algorithm to obtain sparse point cloud data, and densely reconstructing the sparse point cloud data to obtain dense point cloud data; and gridding the dense point cloud data. Processing to obtain three-dimensional grid data, and reconstructing the surface of the three-dimensional grid data to obtain three-dimensional curved surface data; texture mapping the three-dimensional curved surface data to obtain textured three-dimensional data, and geometrically correcting the textured three-dimensional data to obtain the three-dimensional real scene model; coordinate conversion of the RTK positioning data in the original data set to obtain geographic coordinate data, and geo-registering the three-dimensional real scene model according to the geographic coordinate data to obtain a geographic reference three-dimensional real scene model; orthographically projecting the geographic reference three-dimensional real scene model to obtain orthographic projection data, and performing resolution averaging processing on the orthographic projection data to obtain the high-precision two-dimensional orthophoto; A matching module is used to perform multi-source data matching on the three-dimensional real scene model to obtain a three-dimensional space scene containing BIM data, CAD data and GIS information; A calculation module, used to calculate the earthwork volume of the three-dimensional space scene to obtain an earthwork volume analysis result; A comparison module is used to perform a multi-time comparison on the three-dimensional space scene to obtain dynamic change data of the project progress and a visual progress report. The multi-time comparison on the three-dimensional space scene to obtain dynamic change data of the project progress and a visual progress report includes: dividing the three-dimensional space scene into time series to obtain three-dimensional scene data of multiple time nodes, and aligning the three-dimensional scene data of the multiple time nodes to obtain aligned time-series three-dimensional scene data; performing difference detection on the aligned time-series three-dimensional scene data to obtain scene change area data, and performing semantic segmentation on the scene change area data to obtain engineering component-level change data; The component-level change data of the project is annotated through BIM data to obtain component progress information, and the component progress information is statistically analyzed to obtain the overall progress data of the project; the overall progress data of the project is time-series analyzed to obtain progress trend data, and the progress trend data is detected to obtain abnormal progress point data; the abnormal progress point data is compared with the project plan to obtain progress deviation data, and the progress deviation data is visualized to obtain a progress deviation heat map; the progress deviation heat map and the overall progress data of the project are comprehensively processed to obtain the dynamic change data of the project progress and a visualized progress report; The analysis module is used to perform multi-dimensional data analysis on the three-dimensional space scene through the earthwork volume analysis results and the dynamic change data of the project progress, so as to obtain project management optimization suggestions and safety risk warning information.
8. A computer-readable storage medium having instructions stored thereon, It is characterized in that When the instructions are executed by the processor, the engineering intelligent aerial inspection method based on a drone as described in any one of claims 1-6 is implemented.
Citation Information
Patent Citations
Method for managing remedied earthwork volume of polluted site by combining aerial survey of unmanned aerial vehicle and BIM technology
CN113112057A
Scene point cloud processing method and device, unmanned aerial vehicle, telemetry terminal and storage medium
CN113920274A
Earthwork volume calculation method based on unmanned aerial vehicle three-dimensional scanning live-action modeling
CN115909091A
Method for assisting earth excavation construction based on oblique photography of unmanned aerial vehicle
CN116448080A
Landslide risk monitoring and early warning method and system based on live-action three dimensions
CN116504032A
