BIM construction site inspection system based on augmented reality of unmanned aerial vehicle and control method thereof

Abstract

The invention relates to the technical field of road project construction site management, in particular to a BIM construction site inspection system based on augmented reality of an unmanned aerial vehicle and a control method thereof, and the method comprises the steps: 1, system initialization: constructing a three-dimensional building information model B (x, y, z) of a construction site on a BIM platform, and recording a building structure, equipment layout and related information, associating the model with an unmanned aerial vehicle inspection module and an AR visualization module to realize data synchronization; and 2, task setting: inputting inspection task parameters through a user interaction interface, including an inspection area omega, a task frequency ft and sensor configuration S = {s1, s2,..., sn}. According to the invention, the unmanned aerial vehicle + AR + BIM combination greatly improves the inspection efficiency and accuracy, reduces the manual inspection time and cost, and improves the inspection efficiency and accuracy. Meanwhile, the risk that personnel enter a dangerous area is reduced through unmanned aerial vehicle inspection, the safety of the inspection process is improved, the intelligent scheduling module can adjust resource allocation in real time according to the inspection result, and the utilization rate and efficiency of resources are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of highway project construction site management, and in particular to a BIM construction site inspection system based on unmanned aerial vehicle augmented reality and a control method thereof. Background Art

[0002] In the process of road construction, site inspection plays a vital role. It not only ensures that the quality of the project meets the standards, but also monitors the construction progress and ensures that the project can be completed on time by timely discovering and correcting defects and errors in the construction. In addition, site inspection also involves strict control of construction safety, preventing accidents, protecting the lives of workers, managing construction personnel, and improving work efficiency. Environmental protection is also an important part of site inspection to ensure that the impact of construction activities on the environment is minimized. Site inspection is a key link in maintaining the quality, safety, progress and environmental protection of road construction;

[0003] Traditional site inspection methods mainly rely on manual labor, which is not only inefficient but also has safety risks. In recent years, drone technology has been widely used in site inspections due to its high-altitude operation capabilities, flexibility and low cost.

[0004] However, relying solely on drones for inspections still has problems such as non-intuitive data interpretation and inaccurate problem location. At the same time, BIM technology, as an important tool for the digital transformation of the construction industry, can provide full life cycle information of construction projects, but its application in site inspections is still insufficient. To address the above problems, a BIM site inspection system based on drone augmented reality and its control method are proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a BIM site inspection system based on drone augmented reality and its control method, so as to solve the problems of non-intuitive data interpretation and inaccurate problem location when relying solely on drones for inspection. At the same time, BIM technology, as an important tool for the digital transformation of the construction industry, can provide full life cycle information of construction projects, but its application in site inspections is still insufficient.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A BIM site inspection system based on drone augmented reality and a control method thereof, comprising:

[0008] Step 1: System initialization: Build a three-dimensional building information model B (x, y, z) of the construction site on the BIM platform, record the building structure, equipment layout and related information, and associate the model with the drone inspection module and AR visualization module to achieve data synchronization;

[0009] Step 2: Task setting: Enter the inspection task parameters through the user interface, including the inspection area Ω, task frequency f t and sensor configuration S = {s 1 ,s 2 ,…,s n}, the UAV generates the best inspection path R according to the optimization algorithm to cover the inspection area Ω;

[0010] Step 3: Data collection and transmission: The drone performs the task according to the inspection path R and collects environmental data D = {d 1 ,d 2 ,…,d k}, and transmit it to the BIM data analysis module and AR visualization module in real time through wireless network, and the data transmission needs to ensure low latency and high throughput;

[0011] Step 4: Data processing and analysis: The BIM data analysis module uses convolutional neural networks to extract image features F img The image features are matched with the BIM model features B through the feature matching algorithm FLANN, and the matching degree M(f i ,b j ):

[0012]

[0013] In the formula, f i is the image feature, b j For BIM model features, when the matching degree is lower than the threshold, it is judged as abnormal and an inspection report is generated, indicating the abnormal location and suggestions;

[0014] Step 5: Data visualization: The AR visualization module integrates the inspection data with the BIM model and uses augmented reality equipment to present the construction site status in real time. The fusion model satisfies the following formula:

[0015] V(x,y,z)=α·B(x,y,z)+(1-α)·D(x,y,z)

[0016] Where α is the transparency parameter and V is the final rendered view;

[0017] Step 6: Resource Scheduling: The intelligent scheduling module is based on the abnormal distribution A={a 1 ,a 2 ,…,a m}, priority is assigned based on the severity of the exception and the availability of resources. The task scheduling formula is:

[0018]

[0019] Prioritize high-priority tasks and dynamically adjust resource allocation plans to achieve timely problem repair and resource optimization.

[0020] As further optimized content of the present invention, among them: in the step three, an adaptive protocol based on latency and throughput is used to optimize the data transmission path, so that data can be transmitted with low latency and high throughput in different areas of the construction site environment. The protocol dynamically adjusts the data routing strategy according to the real-time network status of the construction site to ensure the priority of image data and sensor data.

[0021] As a further optimization content of the present invention, in the step four, an improved multi-level convolutional neural network model is used to perform deep feature extraction and anomaly detection on images for different types of building structures and environmental conditions, and the multi-level convolutional neural network model includes multiple convolutional layers and pooling layers.

[0022] As a further optimization of the present invention, in step 4, trend analysis is performed in combination with historical data of the construction site, potential abnormal situations in the future are predicted through a time series prediction algorithm, and preventive measures are taken in advance.

[0023] As further optimized content of the present invention, wherein: the step 2 further includes using a machine learning algorithm to predict the changing trend of inspection demand based on historical inspection data and construction progress of the construction site, and optimizing the inspection path and task frequency based on the prediction results.

[0024] As a further optimization of the present invention, in step six, resources are scheduled based on a particle swarm optimization algorithm, taking into account various constraints on the construction site, the constraints are: resource availability, task urgency, environmental changes, and dynamically generating the optimal resource allocation strategy.

[0025] As a further optimized content of the present invention, wherein: the system comprises:

[0026] UAV inspection module: used to perform site inspection tasks and collect images, videos and environmental data;

[0027] AR visualization module: used to integrate drone-collected data with BIM models, and to display the three-dimensional model of the construction site and data overlay in real time through augmented reality equipment;

[0028] BIM data analysis module: used to analyze the data collected by drones, match it with the BIM model, identify anomalies and generate inspection reports;

[0029] Intelligent scheduling module: used to dynamically allocate construction site resources based on the results of inspection reports;

[0030] User interface: used for task setting, report viewing and parameter adjustment.

[0031] The drone inspection module includes:

[0032] Equipped with high-definition cameras and infrared thermal imagers to collect high-definition images and temperature data on the construction site;

[0033] Automatic route planning function generates the best inspection route according to the inspection area and task requirements.

[0034] As a further optimized content of the present invention, the AR visualization module supports real-time display of the three-dimensional model of the construction site and superimposed inspection data through augmented reality glasses, tablet computers or other terminal devices.

[0035] As further optimized content of the present invention, the BIM data analysis module adopts a convolutional neural network algorithm to process the image data collected by the drone, and realizes the precise matching of abnormal data with the BIM model through the FLANN feature matching algorithm; the intelligent scheduling module can automatically schedule the maintenance team and required equipment based on the abnormal type, location and severity of the inspection report, and generate a dynamically optimized task schedule.

[0036] As further optimized content of the present invention, the user interaction interface is constructed using HTML5 and WebGL technologies, supporting multi-platform operations of PC, tablet computer and mobile terminal.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. In the present invention, the combination of drone + AR + BIM greatly improves the efficiency and accuracy of inspections, reduces the time and cost of manual inspections, and at the same time, drone inspections reduce the risk of personnel entering dangerous areas and improve the safety of the inspection process. In addition, the intelligent scheduling module can adjust resource allocation in real time according to the inspection results, thereby improving resource utilization and efficiency. The combination of AR and BIM technologies can also realize the intelligentization and visualization of construction site management, providing strong technical support for construction site management;

[0039] 2. In the present invention, by constructing and updating the three-dimensional building information model of the construction site on the BIM platform in real time, the system can accurately record the building structure, equipment layout and construction progress, ensure the synchronization and accuracy of the data, and the drone inspection module collects high-definition images and environmental data, and combines the convolutional neural network and FLANN feature matching algorithm to detect abnormal conditions in the construction site in real time and accurately locate them, greatly improving the accuracy and response speed of the inspection. In addition, through the AR visualization module, the system integrates the real-time inspection data with the BIM model to provide an intuitive and interactive three-dimensional display, which improves the convenience of operation and inspection effect of on-site staff. The intelligent scheduling module automatically allocates resources according to the inspection report to ensure efficient execution of tasks and timely repair of problems, and optimizes the intelligent level of construction site management;

[0040] 3. In the present invention, by combining technologies such as adaptive data transmission protocol, improved multi-level convolutional neural network model, machine learning prediction algorithm and particle swarm optimization algorithm, the intelligence and flexibility of the system are greatly improved. In terms of data transmission, a low-latency, high-throughput adaptive protocol is adopted to ensure the real-time transmission of data in a complex construction site environment, avoiding the problems of transmission delay and data loss. In terms of data processing and analysis, the system can accurately predict and identify potential abnormal situations through deep learning and historical data trend analysis, take preventive measures in advance, and reduce the safety risks of the construction site. Furthermore, through the resource scheduling algorithm based on particle swarm optimization, the system can dynamically adjust the resource allocation plan according to various constraints, improve resource utilization efficiency, ensure that high-priority tasks are processed in a timely manner, reduce resource waste and construction stagnation, and improve the operation efficiency and management level of the entire construction site. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of a BIM site inspection system control method based on UAV augmented reality of the present invention;

[0042] Figure 2 This is a system block diagram of a BIM construction site inspection system based on drone augmented reality in the present invention. DETAILED DESCRIPTION

[0043] See also Figure 1-2 , the present invention provides a technical solution:

[0044] A BIM site inspection system based on drone augmented reality and a control method thereof, comprising: Step 1: System initialization: constructing a three-dimensional building information model B (x, y, z) of the site on the BIM platform, recording the building structure, equipment layout and related information, associating the model with a drone inspection module and an AR visualization module to achieve data synchronization;

[0045] Step 2: Task setting: Enter the inspection task parameters through the user interface, including the inspection area Ω, task frequency f t and sensor configuration S = {s 1 ,s 2 ,…,s n}, the UAV generates the best inspection path R according to the optimization algorithm to cover the inspection area Ω;

[0046] Step 3: Data collection and transmission: The drone performs the task according to the inspection path R and collects environmental data D = {d 1 ,d 2 ,…,d k}, and transmit it to the BIM data analysis module and AR visualization module in real time through wireless network, and the data transmission needs to ensure low latency and high throughput;

[0047] Step 4: Data processing and analysis: The BIM data analysis module uses convolutional neural networks to extract image features F img The image features are matched with the BIM model features B through the feature matching algorithm FLANN, and the matching degree M(f i ,b j ):

[0048]

[0049] In the formula, f i is the image feature, b j For BIM model features, when the matching degree is lower than the threshold, it is judged as abnormal and an inspection report is generated, indicating the abnormal location and suggestions;

[0050] Step 5: Data visualization: The AR visualization module integrates the inspection data with the BIM model and uses augmented reality equipment to present the construction site status in real time. The fusion model satisfies the following formula:

[0051] V(x,y,z)=α·B(x,y,z)+(1-α)·D(x,y,z)

[0052] Where α is the transparency parameter and V is the final rendered view;

[0053] Step 6: Resource Scheduling: The intelligent scheduling module is based on the abnormal distribution A={a 1 ,a 2 ,…,a m}, priority is assigned based on the severity of the exception and the availability of resources. The task scheduling formula is:

[0054]

[0055] Prioritize high-priority tasks, dynamically adjust resource allocation plans, and implement timely problem repair and resource optimization. By building and updating BIM models in real time, the high accuracy and real-time nature of construction site data are ensured, the efficiency of inspection tasks is improved, the time and cost of manual inspections are reduced, and the level of intelligence of construction site management is enhanced.

[0056] As a technical solution for further implementation of this solution, in step three, an adaptive protocol based on latency and throughput is used to optimize the data transmission path, so that data can be transmitted with low latency and high throughput in different areas of the construction site environment. The protocol dynamically adjusts the data routing strategy according to the real-time network status of the construction site to ensure the priority of image data and sensor data. By optimizing the data transmission path, real-time data transmission is guaranteed, especially in a complex construction site environment, avoiding data delay and loss problems, ensuring the timeliness and integrity of inspection data, and improving the reliability and stability of the system.

[0057] As a technical solution for further implementation of this solution, in step 4, an improved multi-level convolutional neural network model is used to perform deep feature extraction and anomaly detection on images for different types of building structures and environmental conditions. The multi-level convolutional neural network model includes multiple convolutional layers and pooling layers. The improved multi-level convolutional neural network improves the accuracy and efficiency of image processing, and can more accurately detect abnormal conditions on the construction site, especially in complex environments. Accurate defect detection of different types of building structures and facilities ensures the accuracy and timeliness of inspection results;

[0058] As a technical solution for further implementation of this plan, in step 4, trend analysis is performed in combination with historical data of the construction site, and potential abnormal situations in the future are predicted through time series prediction algorithms, and preventive measures are taken in advance. By combining historical data for trend analysis and abnormality prediction, the system can actively identify potential risks and take measures in advance, reducing sudden problems on the construction site, helping to prevent and allocate resources in time, and improving the safety and management efficiency of the construction site;

[0059] As a technical solution for further implementation of this solution, step 2 further includes using machine learning algorithms to predict the changing trend of inspection demand based on historical inspection data and construction progress of the construction site, and optimizing the inspection path and task frequency based on the prediction results. The machine learning algorithm is used to predict inspection demand, and the priority and frequency of inspection tasks can be dynamically adjusted according to the actual construction progress of the construction site and historical inspection data, thereby improving the accuracy and work efficiency of inspections, reducing unnecessary inspection tasks, and saving resources and time;

[0060] As a technical solution for further implementation of this solution, in step six, a particle swarm optimization algorithm is used to schedule resources. Taking into account multiple constraints on the construction site, the constraints are: resource availability, task urgency, and environmental changes, and the optimal resource allocation strategy is dynamically generated. The particle swarm optimization algorithm can quickly generate the optimal resource allocation plan under multiple constraints, ensuring that high-priority tasks are processed in a timely manner, while effectively utilizing existing resources, improving the intelligent level of construction site management, and reducing resource waste and costs;

[0061] As a technical solution for further implementing this solution, the system includes:

[0062] UAV inspection module: used to perform site inspection tasks and collect images, videos and environmental data;

[0063] AR visualization module: used to integrate drone-collected data with BIM models, and to display the three-dimensional model of the construction site and data overlay in real time through augmented reality equipment;

[0064] BIM data analysis module: used to analyze the data collected by drones, match it with the BIM model, identify anomalies and generate inspection reports;

[0065] Intelligent scheduling module: used to dynamically allocate construction site resources based on the results of inspection reports;

[0066] User interface: used for task setting, report viewing and parameter adjustment.

[0067] The drone inspection module includes:

[0068] Equipped with high-definition cameras and infrared thermal imagers to collect high-definition images and temperature data on the construction site;

[0069] Automatic route planning function generates the best inspection route according to the inspection area and task requirements. The system integrates multiple modules and has efficient data collection, processing and display functions. It can quickly respond to site inspection needs and provide intuitive and real-time inspection results, greatly improving the automation level of site management and the execution efficiency of inspection tasks.

[0070] As a technical solution for further implementation of this solution, the AR visualization module supports real-time display of the three-dimensional model of the construction site and superimposed inspection data through augmented reality glasses, tablet computers or other terminal devices. By supporting multiple terminal devices, the AR visualization module provides a flexible operation mode that can meet the needs of different users, improves the applicability and convenience of the system, and enables construction site managers to view real-time data and make decisions more conveniently;

[0071] As a technical solution for further implementation of this plan, the BIM data analysis module uses a convolutional neural network algorithm to process image data collected by drones, and uses the FLANN feature matching algorithm to achieve accurate matching of abnormal data with the BIM model. The intelligent scheduling module can automatically schedule the maintenance team and required equipment based on the abnormal type, location and severity of the inspection report, and generate a dynamically optimized task schedule. This module uses deep learning algorithms and precise feature matching technology to make abnormality detection and resource scheduling more efficient and accurate, thereby reducing manual intervention, ensuring timely detection and rapid resolution of problems, optimizing the site management process, and improving overall work efficiency.

[0072] As a technical solution for further implementation of this plan, the user interaction interface is built with HTML5 and WebGL technology, supporting multi-platform operations on PCs, tablets and mobile terminals. The use of HTML5 and WebGL technology ensures the high compatibility and ease of operation of the system interface, supports multi-platform access, and facilitates users to view data, adjust settings and conduct decision analysis on different devices, thereby improving the system's usability and user experience.

[0073] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. The above is only a preferred implementation of the present invention. It should be pointed out that due to the limitations of textual expression and the objective existence of infinite specific structures, ordinary technicians in this technical field can make several improvements, modifications or changes without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the protection scope of the present invention.

Claims

1. A control method for a BIM site inspection system based on drone augmented reality, characterized in that: include: Step 1: System initialization: Build a three-dimensional building information model B (x, y, z) of the construction site on the BIM platform, record the building structure, equipment layout and related information, and associate the model with the drone inspection module and AR visualization module to achieve data synchronization; Step 2: Task setting: Enter the inspection task parameters through the user interface, including the inspection area Ω, task frequency f t and sensor configuration S = {s1,s2,…,s n }, the UAV generates the best inspection path R according to the optimization algorithm to cover the inspection area Ω; Step 3: Data collection and transmission: The drone performs the task according to the inspection path R and collects environmental data D = {d1, d2, …, d k }, and transmit it to the BIM data analysis module and AR visualization module in real time through wireless network, and the data transmission needs to ensure low latency and high throughput; Step 4: Data processing and analysis: The BIM data analysis module uses convolutional neural networks to extract image features F img The image features are matched with the BIM model features B through the feature matching algorithm FLANN, and the matching degree M(f i ,b j ): In the formula, f i is the image feature, b j For BIM model features, when the matching degree is lower than the threshold, it is judged as abnormal and an inspection report is generated, indicating the abnormal location and suggestions; Step 5: Data visualization: The AR visualization module integrates the inspection data with the BIM model and uses augmented reality equipment to present the site status in real time. The fusion model satisfies the following formula: V(x,y,z)=α·B(x,y,z)+(1-α)·D(x,y,z) Where α is the transparency parameter and V is the final rendered view; Step 6: Resource Scheduling: The intelligent scheduling module allocates resources according to the abnormal distribution A={a1,a2,…,a m }, priority is assigned based on the severity of the exception and the availability of resources. The task scheduling formula is: Prioritize high-priority tasks and dynamically adjust resource allocation plans to achieve timely problem repair and resource optimization.

2. The control method of a BIM site inspection system based on drone augmented reality according to claim 1 is characterized in that: In step three, an adaptive protocol based on latency and throughput is used to optimize the data transmission path, so that data can be transmitted with low latency and high throughput in different areas of the construction site environment. The protocol dynamically adjusts the data routing strategy according to the real-time network status of the construction site to ensure the priority of image data and sensor data.

3. The control method of a BIM site inspection system based on drone augmented reality according to claim 1 is characterized in that: In the step 4, an improved multi-level convolutional neural network model is used to perform deep feature extraction and anomaly detection on the image for different types of building structures and environmental conditions. The multi-level convolutional neural network model includes multiple convolutional layers and pooling layers.

4. The control method of a BIM site inspection system based on drone augmented reality according to claim 1 is characterized in that: In step 4, trend analysis is performed in combination with historical data of the construction site, potential abnormal situations in the future are predicted through a time series prediction algorithm, and preventive measures are taken in advance.

5. The control method of a BIM site inspection system based on drone augmented reality according to claim 1 is characterized in that: The step 2 further includes using a machine learning algorithm to predict the changing trend of inspection demand based on historical inspection data and construction progress of the construction site, and optimizing the inspection path and task frequency based on the prediction results.

6. The control method of the BIM site inspection system based on drone augmented reality according to claim 1 is characterized by: In step six, a particle swarm optimization algorithm is used to schedule resources, taking into account various constraints on the construction site, including resource availability, task urgency, and environmental changes, to dynamically generate an optimal resource allocation strategy.

7. A BIM site inspection system based on drone augmented reality according to any one of claims 1 to 6, characterized in that: The system comprises: UAV inspection module: used to perform site inspection tasks and collect images, videos and environmental data; AR visualization module: used to integrate drone-collected data with BIM models, and to display the three-dimensional model of the construction site and data overlay in real time through augmented reality equipment; BIM data analysis module: used to analyze the data collected by drones, match it with the BIM model, identify anomalies and generate inspection reports; Intelligent scheduling module: used to dynamically allocate construction site resources based on the results of inspection reports; User interface: used for task setting, report viewing and parameter adjustment. The drone inspection module includes: Equipped with high-definition cameras and infrared thermal imagers to collect high-definition images and temperature data on the construction site; Automatic route planning function generates the best inspection route according to the inspection area and task requirements.

8. The BIM site inspection system based on drone augmented reality according to claim 7 is characterized by: The AR visualization module supports real-time display of the construction site three-dimensional model and superimposed inspection data through augmented reality glasses, tablet computers or other terminal devices.

9. The BIM site inspection system based on drone augmented reality according to claim 7 is characterized by: The BIM data analysis module uses a convolutional neural network algorithm to process the image data collected by the drone, and uses the FLANN feature matching algorithm to achieve accurate matching of abnormal data with the BIM model. The intelligent scheduling module can automatically schedule the maintenance team and required equipment based on the abnormality type, location and severity of the inspection report, and generate a dynamically optimized task schedule.

10. The BIM site inspection system based on drone augmented reality according to claim 7, characterized in that: The user interaction interface is constructed using HTML5 and WebGL technologies, and supports multi-platform operations on PCs, tablet computers, and mobile terminals.

Images