Construction progress dynamic optimization method and system based on BIM and computer vision

Abstract

The application discloses a construction progress dynamic optimization method and system based on BIM and computer vision, and particularly relates to the technical field of building construction management, and comprises the following steps: automatically registering a BIM model and a construction site image; processing the construction site image by using a visual recognition algorithm to generate a visual recognition result; constructing a four-dimensional dynamic BIM model, and mapping the visual recognition result to a corresponding component in real time through multi-feature similarity calculation; monitoring progress deviation by using key path dynamic identification and deviation propagation matrix, and combining a Bayesian network and a Monte Carlo simulation to predict risks. The application integrates automatic registration, dynamic monitoring, risk prediction and intelligent decision-making into one construction progress optimization system by fusing BIM and computer vision technology, solves the problems of low efficiency of traditional manual data collection, lag of progress monitoring, ambiguity of risk prediction and extensive resource allocation, and realizes accurate monitoring of construction progress, risk early warning and optimal allocation of resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of building construction management, more particularly, the present application relates to a construction progress dynamic optimization method and system based on BIM and computer vision. BACKGROUND

[0002] In the process of building construction management, dynamic monitoring and optimization of construction progress has always been a difficult point of engineering management.

[0003] The traditional construction progress management method mainly relies on manual inspection and experience judgment, which has problems such as low data collection efficiency, poor real-time performance, large error, etc., and cannot accurately reflect the actual progress situation of the construction site, nor can it timely respond to various risks and deviations in the construction process.

[0004] With the development of BIM (Building Information Modeling) technology and computer vision technology, how to effectively combine the two to realize the dynamic optimization of construction progress has become a technical problem to be solved in the field. SUMMARY

[0005] The present application aims to provide a construction progress dynamic optimization method and system based on BIM and computer vision to solve the problems of low efficiency, poor real-time performance, inaccurate risk prediction, etc. in the prior art, and to realize accurate monitoring, risk prediction and dynamic optimization of construction progress.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a construction progress dynamic optimization method based on BIM and computer vision, comprising the following steps:

[0007] Step one, obtain BIM model point cloud data, collect construction site images, establish a unified spatial coordinate system, automatically register the BIM model and the construction site images;

[0008] Step two, use a visual recognition algorithm to process the construction site images, identify component state, equipment operating parameters, quality defects and safety violations, and generate visual recognition results;

[0009] Step three, build a four-dimensional dynamic BIM model, and map the visual recognition results to the corresponding components in real time through multi-feature similarity calculation;

[0010] Step four, use key path dynamic identification and deviation propagation matrix to monitor progress deviation, and combine Bayesian network and Monte Carlo simulation to predict risks;

[0011] Step five, generate a dynamic optimization scheme of construction sequence and resource allocation through a hierarchical reinforcement learning architecture.

[0012] Specifically, based on step one, the BIM model is automatically registered with the construction site image, and the automatic registration method is as follows:

[0013] S11, feature acquisition: using a laser scanner to sample point cloud data of the BIM model, and acquiring point cloud data containing three-dimensional coordinates; collecting RGB-D images of the construction site through a binocular camera, extracting depth information to generate site point cloud data;

[0014] S12, feature extraction: using the FPFH algorithm to extract the fast point feature histogram descriptor of the point cloud data and the site point cloud data respectively, and establishing an initial correspondence relationship through bidirectional nearest neighbor search; using the iterative closest point algorithm to optimize the spatial transformation parameters, calculating the rotation matrix and translation vector to be optimized through the objective function, and minimizing the function to register the BIM model point cloud and the site point cloud.

[0015] Specifically, based on step two, the construction site image is processed using a visual recognition algorithm, which includes the following steps:

[0016] S21, multi-modal feature acquisition: acquiring RGB images of the construction site through an industrial camera, and extracting channel features; simultaneously acquiring depth maps output by the ZED camera, and generating depth features through bilateral filtering preprocessing;

[0017] S22, feature fusion processing: using ResNet50 as the backbone network to extract the semantic features of the RGB image, using 3D convolution layers to extract the spatial distance information of the depth features, and using a fusion function to perform nonlinear transformation on the features;

[0018] S23, spatio-temporal attention calculation: inputting the fusion features of a continuous set of frames into a gated recurrent unit, and calculating the time series attention weight of the t-th frame.

[0019] Specifically, the visual recognition algorithm further includes:

[0020] Quality defect feature acquisition and processing: using a U-Net network to perform semantic segmentation on the RGB image, inputting channels including RGB three channels and a depth single channel, and outputting a pixel-level defect probability map; training model parameters through a defect type database, and calculating defect severity;

[0021] Safety hazard feature acquisition and processing: presetting building construction safety inspection standards, identifying real-time construction safety features from the construction site image based on a YOLOv8 target detection network, and generating a set number of anchor boxes through K-means clustering;

[0022] Then, whether there is a safety hazard is determined through spatio-temporal correlation analysis.

[0023] Specifically, based on step three, the construction of the four-dimensional dynamic BIM model includes:

[0024] S31, state feature acquisition: geometric completion degree data of the component is read through the RFID tag reading member, construction quality indicators are collected, and a component quality state vector is generated; the construction process type and start / end time are extracted from the construction log and recorded as a construction operation vector; human data, equipment data, and material data on the construction site are collected and recorded as a resource input vector;

[0025] S32, model construction processing: a state transition function is established, and LSTM network training is adopted, with the input being the component quality state vector, the construction operation vector, and the resource input vector in the set time region, and the output being the next time state prediction; a resource-schedule association model is established through the construction condition vector, the preset resource efficiency function, the construction progress completion amount, and the maximum construction progress completion amount.

[0026] Specifically, the visual recognition result is mapped to the corresponding component in real time through multi-feature similarity calculation, and the real-time mapping process includes:

[0027] Mapping feature acquisition: geometric features, texture features, and material features of the component are extracted from the visual recognition result, and corresponding attribute parameters are extracted from the corresponding component in the BIM model;

[0028] Mapping processing: corresponding weight coefficients are set for the geometric features, texture features, and similarity material features; a multi-feature fusion similarity function is constructed, and the fused feature dimensions include geometric features, texture features, and material features; the geometric feature similarity is calculated by the Euclidean distance, the texture feature similarity is calculated by the LBP texture histogram algorithm, and the material feature similarity is calculated by the HSV color space analysis; corresponding weight coefficients are set for the geometric features, texture features, and similarity material features; the geometric feature similarity, the texture feature similarity, and the similarity material feature similarity are weighted and fused with the set weight coefficients to obtain a comprehensive similarity; the comprehensive similarity is used to measure the matching degree of the visual recognition result and the BIM component features;

[0029] Based on the multi-feature fusion similarity, the mapping confidence is calculated, specifically: the confidence calculation covers the corresponding dimensions of geometry, texture, and material, and the final confidence is obtained by averaging the feature similarities after processing the features by the feature extraction function; a confidence threshold is set, and when the calculated confidence is lower than the confidence threshold, the BIM engineer triggers manual verification through the WebGL interactive interface.

[0030] Specifically, the critical path is dynamically identified and the progress deviation is monitored by the deviation propagation matrix, and the monitoring method of the progress deviation is as follows:

[0031] Progress feature acquisition: Obtain the duration data of each construction activity through the task list of the BIM model, and identify the actual start time and actual end time of the construction activity through the field image; In the dynamic identification of the critical path, for any construction activity, the earliest start time is determined by the maximum value of the earliest start time of all previous construction activities plus its own duration;

[0032] For any construction activity, the latest start time is determined by subtracting the minimum value of its own duration from the latest start time of all subsequent construction activities;

[0033] Deviation processing: Construct a deviation propagation matrix: the element value in the matrix is determined according to the logical relationship of the construction process: if a process is a previous process of another process, the corresponding matrix element value is set to 1; If there is a resource sharing relationship between the two processes, the corresponding matrix element value is set to 0.5; If there is no above-mentioned association, the corresponding matrix element value is set to 0;

[0034] By comparing the actual construction progress with the planned progress, the delay time of each process is calculated, and the current process delay vector is calculated;

[0035] The deviation propagation matrix and the current process delay vector are operated to calculate the execution deviation propagation, and the delay time distribution results of the subsequent processes affected are output, realizing the dynamic monitoring and propagation simulation of the construction progress deviation.

[0036] Specifically, based on step four, the risk is predicted based on the Bayesian network and Monte Carlo simulation, specifically:

[0037] Risk feature acquisition: Extract historical construction data from the project management system; By setting the number of historical risk event data, K2 algorithm is used for training and learning to generate the conditional probability table of the Bayesian network; Based on the conditional probability table, the Bayesian network is constructed;

[0038] Risk processing: Set the number of simulations, and for each simulation, generate a risk scenario based on the Bayesian network and calculate the corresponding risk value; Set the risk threshold; Determine whether the risk scenario of each simulation triggers the risk through the indicator function; The proportion of risk events in all simulation results is taken as the probability of occurrence of construction risk; Generate and output a set number of high-probability risk scenarios.

[0039] Specifically, based on step five, a dynamic optimization scheme of construction sequence and resource allocation is generated through a hierarchical reinforcement learning architecture, which includes:

[0040] Decision feature acquisition: The decision features are divided into upper-level features and lower-level features; The upper-level features include the current construction stage and the critical path activity state, and the lower-level features include the resource demand of each process and the material inventory level; And preset the reward function;

[0041] Algorithm processing: in the hierarchical decision-making architecture, the upper layer strategy adopts a convolutional neural network, the input is a construction phase feature map, and the output is a construction phase conversion probability; the lower layer strategy adopts a fully connected network, the input is a resource feature vector, and the output is a resource allocation scheme;

[0042] Actor network: designed as a 3-layer fully connected layer structure, 256 neurons are set in each layer, and used for outputting decision action probability;

[0043] Critic network: using a value function approximation method to evaluate the value of the current decision;

[0044] Training process: set the learning rate and discount factor, train multiple rounds through the PPO algorithm, and until the fluctuation of the cumulative reward value meets the preset fluctuation threshold; during the training, the current construction state, decision action and reward feedback are combined to dynamically update the parameters of the Actor and Critic networks, and the construction sequence and resource allocation strategy are optimized.

[0045] The application also proposes a system using the above method, comprising the following modules:

[0046] A data acquisition module is used to acquire BIM model point cloud data and construction site RGB-D images, and to establish a unified spatial coordinate system;

[0047] A visual recognition module is used to process construction site images using a visual recognition algorithm to identify component status, equipment operating parameters, quality defects and safety violations, and to generate visual recognition results;

[0048] A four-dimensional BIM modeling module is used to construct a four-dimensional dynamic BIM model, and to map the visual recognition results to the corresponding components in real time through multi-feature similarity calculation;

[0049] A progress deviation monitoring module is used to monitor construction progress deviation in real time through a key path dynamic identification algorithm and a deviation propagation matrix, and to generate delay time distribution;

[0050] A risk prediction module is used to integrate Bayesian networks and Monte Carlo simulation algorithms to predict construction risks based on historical risk data and real-time environmental parameters;

[0051] A dynamic optimization decision module is used to generate a dynamic optimization scheme for construction sequence and resource allocation through a hierarchical reinforcement learning architecture.

[0052] Technical effects and advantages of the application:

[0053] 1. The application fuses BIM and computer vision technology, automatically registers models and site images through laser scanning and visual recognition, and obtains component status, quality defects and other data in real time, solving the problems of low efficiency and large errors in traditional manual data collection.

[0054] 2. The application breaks through the limitations of poor real-time performance and inaccurate risk prediction of traditional methods by constructing a key path dynamic monitoring and Bayesian network risk prediction system, combining bias propagation matrix and Monte Carlo simulation to analyze progress deviation in real time and quantify risk probability.

[0055] 3. The application relies on a hierarchical reinforcement learning architecture and a four-dimensional BIM model to dynamically generate construction sequences and resource allocation optimization schemes, enabling precise quantification of resource input and progress association, and solving the problems of extensive resource allocation and lagging progress optimization in traditional management.

[0056] In summary, the application integrates BIM and computer vision technology to build a construction progress optimization system that integrates automatic registration, dynamic monitoring, risk prediction and intelligent decision-making, solving the problems of low efficiency of traditional manual data collection, lagging progress monitoring, fuzzy risk prediction and extensive resource allocation, and achieving precise monitoring of construction progress, risk warning and optimal allocation of resources. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The flowchart of the construction progress dynamic optimization method based on BIM and computer vision proposed by the application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0059] As shown in the drawings, the construction progress dynamic optimization method based on BIM and computer vision includes the following steps: Figure 1

[0060] Step one, obtain BIM model point cloud data through a laser scanner, collect construction site images through a binocular camera, establish a unified spatial coordinate system, and automatically register the BIM model and the construction site images;

[0061] Step two, process the construction site images using a visual recognition algorithm to identify component status, equipment operating parameters, quality defects and safety violations, and generate visual recognition results;

[0062] Step three, construct a four-dimensional dynamic BIM model and map the visual recognition results to the corresponding components in real time through multi-feature similarity calculation;

[0063] ​Step four, monitor progress deviation by using critical path dynamic identification and deviation propagation matrix, combine with Bayesian network and Monte Carlo simulation to predict risk;

[0064] Step five, generate dynamic optimization scheme of construction sequence and resource allocation through hierarchical reinforcement learning architecture.

[0065] In this application, based on step one, the BIM model is automatically registered with the construction site image, and the automatic registration method is as follows:

[0066] S11, feature acquisition: use FAROS70 laser scanner to sample point cloud data of BIM model, obtain point cloud data containing three-dimensional coordinates, denoted as , n represents the number of points in the BIM model point cloud data P, and the three-dimensional coordinate information of the i-th point in the BIM model point cloud data is denoted as ; collect the RGB-D image of the construction site by ZED2i binocular camera, extract the depth information to generate the site point cloud data, denoted as , m represents the number of points in the site point cloud data, and the three-dimensional coordinate information of the j-th point in the site point cloud data is denoted as ;

[0067] S12, feature extraction: use FPFH algorithm to extract the fast point feature histogram descriptor FP and FQ of the point cloud data P and the site point cloud data Q respectively, and establish the initial correspondence relationship by bidirectional nearest neighbor search , where is the nearest neighbor in FQ, and is the nearest neighbor in FP; Use the iterative closest point algorithm to optimize the space transformation parameters, calculate the rotation matrix and translation vector t to be optimized by the objective function

[0068] , and minimize the function to register the BIM model point cloud and the site point cloud; where the weight matrix , , is the weight distribution of the i-th point in the point cloud registration. , , , , , is an adjustable parameter for adjusting the weight distribution of point cloud registration.

[0069] ​It should be noted that the automatic registration method first acquires BIM model point cloud data and field point cloud data by FAROS 70 laser scanner and ZED 2i binocular camera respectively; then the FPFH algorithm is used to extract features, and the bidirectional nearest neighbor search is used to establish an initial corresponding relationship; finally, the iterative closest point algorithm is used to optimize the rotation matrix and the translation vector according to the target function containing the curvature and the normal vector, so as to realize point cloud registration and lay a data foundation for subsequent construction progress correlation analysis.

[0070] In the present application, based on step two, a visual recognition algorithm is used to process the construction site image, and the visual recognition algorithm processing specifically includes:

[0071] S21, multi-modal feature acquisition: acquiring the RGB image of the construction site by Basler acA2500-14gc industrial camera, and extracting channel features ; synchronously acquiring the depth map output by the ZED camera, and generating depth features by bilateral filtering preprocessing ; wherein H is the image height, W is the image width, represents the channel feature matrix of the RGB image, represents the depth feature matrix;

[0072] S22, feature fusion processing: using ResNet50 as the backbone network to extract the semantic features of the RGB image, using the 3D convolution layer to extract the spatial distance information of the depth features, and using the fusion function , wherein , are the weight matrices of feature fusion respectively, b1 represents the bias vector for adjusting the overall offset when calculating the RGB image semantic features and the depth features, is an activation function, which is used for nonlinear transformation of the fused features;

[0073] S23, spatio-temporal attention calculation: inputting the fusion features of the continuous T frames into the gated recurrent unit, and calculating the time sequence attention weight of the t frame by ; wherein is the hidden state of the previous moment, W represents the weight matrix of the linear transformation of the hidden state of the previous moment , U represents the weight matrix of the linear transformation of the current frame fusion features , and b2 is the bias vector for offset adjustment of the linear transformation result when calculating the time sequence attention weight of the previous moment hidden state and the current frame fusion features.

[0074] ​It should be noted that the visual recognition algorithm processing flow in this application first uses a Basler industrial camera and a ZED camera to acquire RGB images and depth maps of the construction site, respectively, and obtains multimodal features through preprocessing; then, semantic and spatial distance information is extracted using a ResNet50 backbone network and 3D convolutional layers, and feature fusion is achieved through a fusion function containing weight matrices, bias vectors and activation functions; finally, the fused features of T consecutive frames are input into a gated recurrent unit, and the temporal attention weight is calculated by combining the hidden state of the previous time step with the features of the current frame through a specific formula, thus completing the entire process of visual recognition processing of construction site images from multimodal feature acquisition and fusion to spatiotemporal attention calculation, providing accurate image feature support for subsequent construction progress analysis.

[0075] In this application, the visual recognition algorithm also includes:

[0076] Quality Defect Feature Acquisition and Processing: A U-Net network is used for semantic segmentation of RGB images. The input channels include three RGB channels and one depth channel, and the output is a pixel-level defect probability map. The model parameters were trained using a defect type database (containing 12 types of defect samples such as cracks, exposed reinforcement, and honeycombing). The formula for calculating the defect severity D is as follows: Where N represents the total number of defective pixels. This represents the defect probability of the i1th defective pixel, i.e., the likelihood that this pixel is a quality defect. The value is in the range [0,1], with a higher probability of being a defect as the value is closer to 1. This represents the defect weight of the i-th defect pixel. The defect weight is preset based on the defect location (e.g., load-bearing parts such as the bottom of a beam and the corner of a column have a higher weight coefficient, while non-load-bearing parts have a lower weight coefficient) and the defect type (e.g., different crack widths have different weights). It is used to reflect the difference in the degree of impact of defects of different locations and types on the overall quality (the severity of the defect is a comprehensive index calculated from the relevant information of the defect pixel, used to quantitatively assess the severity of the quality defect).

[0077] Safety Hazard Characteristics Acquisition and Handling: Pre-set construction safety inspection standards, and record the construction rules in the construction safety monitoring standards as... (These are pre-defined criteria for judging safety, such as rules like "construction workers must wear safety helmets" and "scaffolding erection must meet spacing requirements") Based on the YOLOv8 target detection network, real-time construction safety features are identified from construction site images, and a set number of anchor boxes are generated through K-means clustering.

[0078] Then through To achieve spatiotemporal correlation analysis, among which, This indicates the result of the safety hazard assessment, and is a Boolean value. If the calculation result is true (understood as 1), it is determined that there is a safety violation, and if it is false (understood as 0), it is determined that there is no safety violation. represents a matching function for judging whether the real-time recognized construction safety features violate the construction rules in the construction safety inspection standard , if they violate, return true, otherwise return false; wherein represents a set of real-time construction safety features identified from the construction site image, including target categories (such as "person without safety helmet", "illegal erection of scaffold", etc.), target positions (position information in the image or construction site space) and continuous frame number (the number of image frames in which the safety feature continuously appears); represents a function for calculating the number of frames in which the real-time construction safety feature continuously appears, i.e., the duration of the safety feature in consecutive image frames; represents a set frame number, which is a time threshold for determining a safety violation, when the calculated continuous frame number is greater than , it means that the safety feature may constitute a safety violation (need to be jointly determined with the Match result) to avoid false identification of safety violations due to temporary misidentification; ∧ represents a logical AND operator, only when returns true (there is a rule violation) and (the continuous frame number exceeds the set frame number threshold) are both true, then it is determined that there is a safety violation (true), if either condition is not met, then it is determined that there is no safety violation (false).

[0079] It should be noted that by further expanding the visual recognition algorithm, the Basler industrial camera and ZED camera are used to collect multi-modal features and pre-process them, and ResNet50, 3D convolution layer and fusion function are used to realize feature fusion, then input the gated recurrent unit to calculate the temporal attention weight combining the hidden state and the current frame feature to complete the basic image visual recognition. At the same time, quality defects and safety hazard identification are also expanded, U-Net network semantic segmentation is used to output defect probability map from RGB image and calculate defect severity based on preset weight, YOLOv8 target detection network is used to identify safety features, and K-means clustering anchor box and spatio-temporal correlation analysis logic (matching safety rules, determining continuous frame number) are combined to accurately identify quality defects and safety hazards, providing comprehensive and accurate image features and risk information support for construction progress optimization and construction management.

[0080] In the present application, based on steps three, the construction of the four-dimensional dynamic BIM model specifically includes:

[0081] S31, state feature acquisition: the geometric completion degree data of the component is read through the RFID tag reading member, construction quality indicators (such as concrete strength collected through a rebound hammer) are collected again, and a component quality state vector is generated and denoted as ; the construction operation vector is denoted as by extracting the construction process type and start / end time from the construction log; the resource input vector is denoted as by collecting human data (such as the number of team members obtained through a face recognition attendance system), equipment data (such as tower crane operation parameters obtained through GPS positioning), and material data (such as the amount of cement / steel obtained through a ground scale weighing) on the construction site;

[0082] S32, model construction processing: a state transition function is established, and an LSTM network is trained, the input is the component quality state vector, the construction operation vector, and the resource input vector in a set time region, and the output is the state prediction at the next time;

[0083] A resource-schedule association model is established, and the model expression is: , is a construction condition vector, environmental data such as precipitation and temperature are obtained through a weather station, g is a resource efficiency function, which can be fitted based on a set number of historical resource-schedule data through an XGBoost algorithm, represents the construction progress completion amount, represents the maximum construction progress completion amount.

[0084] It should be noted that by constructing a four-dimensional dynamic BIM model, the component quality state (geometric completion degree, concrete strength, etc.), construction operation (process type and time), and resource input (human, equipment, and material data) three types of vectors are obtained through RFID tags, construction logs, face recognition, GPS positioning, and ground scale weighing; then, the state transition function is trained using an LSTM network, and the next time state is predicted based on the input of the three types of vectors in a set time region; at the same time, a resource-schedule association model is established, a construction condition vector (environmental data such as precipitation and temperature) is introduced, a resource efficiency function is fitted based on an XGBoost algorithm, and the association between resource input and construction progress is quantified by combining the construction progress completion amount and the maximum completion amount, thereby providing model support for dynamic optimization of construction progress;

[0085] The four-dimensional dynamic BIM model of the application integrates time-dimensional dynamic construction data, and through an automatic registration process, progress and site parameters are mapped in real time, the construction process is dynamically simulated, the four-dimensional association of space, time, and dynamic data is realized, the design model is expanded to a construction dynamic management model, and it is different from the traditional static mode.

[0086] In the present application, the visual recognition result is mapped to the corresponding component in real time through multi-feature similarity calculation, and the real-time mapping process includes:

[0087] Mapping feature acquisition: extract the geometric features (such as length, angle), texture features (such as surface roughness), and material features (such as color threshold of concrete / steel) of the component from the visual recognition result V, and extract the corresponding attribute parameters from the corresponding component in the BIM model;

[0088] Mapping processing: set corresponding weight coefficients q1, q2, q3 for geometric features, texture features, and similarity material features; construct a multi-feature fusion similarity function, and weight and fuse the geometric feature similarity, texture feature similarity, and similarity material feature similarity with the set weight coefficients to obtain a comprehensive similarity , wherein is the geometric feature similarity, is the texture feature similarity, is the material feature similarity;

[0089] Based on the multi-feature fusion similarity, the mapping confidence is calculated , and the calculation formula is , wherein n1=3 (three dimensions of geometry / texture / material), represents the m1th feature extraction function, represents the i2th component feature in the visual recognition result, represents the j2th component feature in the BIM component; set a confidence threshold δ, and if the confidence is lower than the confidence threshold δ, trigger the BIM engineer to manually verify through the WebGL interactive interface.

[0090] It should be noted that through the real-time mapping process proposed in the present application, geometric, texture, and material features are first extracted from the visual recognition result and the corresponding component in the BIM model; then weight coefficients are set for each feature, a multi-feature fusion similarity function is constructed, and a comprehensive similarity is calculated by weighting to measure the matching degree of the visual recognition component and the BIM component; then the mapping confidence is calculated based on the comprehensive similarity, and a confidence threshold is set. When the confidence is lower than the threshold, the BIM engineer triggers the manual verification through the WebGL interactive interface, so as to realize the accurate and reliable mapping of the visual recognition result to the BIM component, and provide accurate component association data support for construction progress dynamic optimization.

[0091] In the present application, the key path is dynamically identified and the progress deviation is monitored by using the deviation propagation matrix, and the monitoring method of the progress deviation is as follows:

[0092] Progress feature acquisition: activity duration D(i3) is obtained through the task list of the BIM model, and the actual start / end time is obtained through on-site image recognition; in the dynamic identification of the critical path, the earliest start time , the latest start time , wherein is the set of immediately preceding processes of activity number j3, is the set of immediately subsequent processes of activity number i3;

[0093] Deviation processing: the elements of the deviation propagation matrix A are determined through the process logical relationship: if process i3 is the immediately preceding process of process j3, then ; if there is a resource sharing relationship, then ; otherwise ; the current process delay vector Δt is calculated through the time difference between the actual progress and the planned progress, and the deviation propagation calculation formula is The delay time distribution of the subsequent process is output , and the progress deviation propagation analysis is realized.

[0094] It should be noted that the construction progress deviation monitoring proposed in the present application first obtains the activity duration from the BIM model task list, combines the actual start / end time identified through on-site image recognition, uses the critical path method, calculates the earliest start time and the latest start time of the activity through the immediately preceding process set and the immediately subsequent process set, and determines the construction critical path; then a deviation propagation matrix is constructed, the matrix elements are determined according to the process logical relationship (immediately preceding process, resource sharing, etc.), the current process delay vector (the time difference between the actual progress and the planned progress) is combined, and the delay time distribution of the subsequent process is output through matrix operation, so as to realize dynamic identification and propagation analysis of the progress deviation, and provide data support for construction progress optimization and adjustment.

[0095] In the present application, based on step four, the progress deviation is monitored by using the critical path dynamic identification and the deviation propagation matrix, and the risk is predicted by combining the Bayesian network and the Monte Carlo simulation, specifically:

[0096] Risk feature acquisition: historical progress deviation data is extracted from the project management system, specifically, steel / concrete material shortage records are obtained from the material management system, and precipitation probability during the construction period is obtained from the meteorological database; the variables include progress deviation (discretized into high, medium and low states), material shortage (binary state of yes or no), and precipitation (binary state of yes or no); by setting the number of historical risk event data groups, the K2 algorithm is used to learn the conditional probability table P of the Bayesian network, thereby forming the Bayesian network BN=(G,P), which is used to depict the correlation between risk variables;

[0097] Risk processing: perform Monte Carlo simulation, set the number of simulations; generate risk scenarios (i.e. combination of states of each risk variable) according to the Bayesian network BN, calculate the corresponding risk value (quantified as progress delay days); set the risk threshold (determined based on the proportion or absolute days of the contract duration), determine whether the risk scenario triggers the risk through the indicator function I (I = 1 if and only if the risk value of the i4th simulation , otherwise I = 0), calculate the risk occurrence probability based on all simulation results , the calculation formula is ; wherein is a preset risk threshold, which is a critical value preset for judging whether a risk occurs, for example, it can be a standard of progress delay days determined based on the contract duration, when the risk value obtained by simulation reaches or exceeds the risk threshold, it is determined that a risk has occurred; is the risk value obtained by the i4th Monte Carlo simulation;

[0098] and generate and output a set number of high-probability risk scenarios (such as the probability that the combination of “rain + steel shortage” causes progress delay ≥ 10 days), realizing construction risk prediction.

[0099] It should be noted that based on step four progress deviation monitoring, historical progress deviation, material shortage, precipitation and other risk related data are first extracted from the project management system through multiple channels (material management, meteorological database, etc.), and after discretization processing, the Bayesian network (BN) is learned and constructed using the K2 algorithm to depict the correlation between risk variables; then Monte Carlo simulation is performed, risk scenarios are generated according to the BN and risk values (progress delay days) are quantified, a risk threshold is set, a risk trigger is determined through an indicator function, simulation results are counted to calculate the risk occurrence probability, and high-probability risk scenarios (such as “rain + steel shortage” causing progress delay cases) are output, realizing accurate construction risk prediction and providing risk warning support for progress optimization decision-making.

[0100] In the present application, based on step five, a dynamic optimization scheme of construction sequence and resource allocation is generated through a hierarchical reinforcement learning architecture, which includes:

[0101] Decision feature acquisition: divided into upper layer decision features and lower layer decision features; wherein the upper layer decision features include the current construction stage (foundation / main body / decoration), the key path activity state, the lower layer decision features include the resource demand of each process (manpower gap, equipment idle rate), the material inventory level; and a reward function is preset, specifically, a reward of +10 for 1 day of progress advance, a penalty of -20 for 1 day of delay, a reward of +5 for every 1% reduction in resource waste rate;

[0102] Algorithm processing: in the hierarchical decision-making architecture, the upper layer strategy adopts the CNN network (input is the stage feature map) to output the stage transition probability, and the lower layer strategy adopts the fully connected network (input is the resource feature vector) to output the resource allocation scheme; the Actor network structure is a 3-layer fully connected layer (256 neurons per layer), and the Critic network adopts the value function approximation method; the parameter update formula learning rate , discount factor γ = 0.95, and the PPO algorithm is trained for a set number of rounds until the cumulative reward fluctuates, wherein is the Actor network parameter, is the Critic network parameter, is the decision action, s is the current state, is the next state, r is the reward value, π is the policy function, and V() is the value function.

[0103] It should be noted that based on step five, a hierarchical reinforcement learning architecture is constructed to generate the construction sequence and resource allocation scheme. First, decision-making features are extracted hierarchically, the upper layer focuses on the construction phase and the key path state, and the lower layer focuses on the process resource demand and material inventory; a preset reward function is used to guide the optimization of progress advance / delay and resource waste rate. In terms of algorithm processing, the upper layer uses a CNN network to output stage transition probability, and the lower layer uses a fully connected network to output resource allocation; the Actor-Critic network cooperates, is trained by the PPO algorithm, and dynamically updates parameters according to the state, action, and reward, iteratively optimizes the policy, realizes the dynamic adaptation and optimization of construction decision-making, and helps the efficient cooperation of construction progress and resource management.

[0104] In the present application, a system using the above method is also proposed, comprising:

[0105] a data acquisition module for acquiring BIM model point cloud data and construction site RGB-D images, and establishing a unified spatial coordinate system;

[0106] a visual recognition module for processing construction site images using a visual recognition algorithm to identify component state, equipment operating parameters, quality defects, and safety violations, and generate visual recognition results;

[0107] a four-dimensional BIM modeling module for constructing a four-dimensional dynamic BIM model and mapping the visual recognition results to the corresponding components in real time through multi-feature similarity calculation;

[0108] a progress deviation monitoring module for monitoring construction progress deviation in real time through a key path dynamic recognition algorithm and a deviation propagation matrix, and generating delay time distribution;

[0109] a risk prediction module for integrating Bayesian networks and Monte Carlo simulation algorithms to predict construction risks based on historical risk data and real-time environmental parameters;

[0110] a dynamic optimization decision module for generating a dynamic optimization scheme of the construction sequence and resource allocation through a hierarchical reinforcement learning framework.

[0111] The above formulas are all calculated by dimensionless values, which can be realized by standardization, etc. and will not be described in detail here. The formulas are obtained by software simulation based on a large amount of measured data, and the preset parameters are configured by a person skilled in the art according to the actual working conditions.

[0112] The embodiment can be realized by software, hardware, firmware or a combination thereof. When realized in the form of software, it can be integrated into a computer program product, which contains computer instructions or programs and can realize the processes and functions of the present application after being loaded and executed. The computer can be a general-purpose / special-purpose device or a network device, and the instructions can be stored in a computer readable medium (such as a magnetic, optical or semiconductor medium) and transmitted through a wired / wireless manner.

[0113] It should be noted that the sequence numbers of the steps do not represent the execution sequence, which is determined according to the functional logic. As known by those skilled in the art, each unit and algorithm step can be realized by electronic hardware or a combination of hardware and software, and the specific implementation depends on the application scenario and design constraints. The system and device disclosed in the present application can be realized by other division methods, and the functional units can be integrated or exist independently.

[0114] If the function is realized and sold in the form of a software unit, it can be stored in a U disk, a hard disk, a ROM, a RAM or other computer readable storage medium, and the medium contains instructions to drive the device to execute the method steps of the present application.

[0115] As understood by those of ordinary skill in the art, the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the specific function is executed by hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can choose different methods to realize the described functions for each specific application, and such implementation methods should not be considered as beyond the protection scope of the present application.

Claims

1. A method for construction progress dynamic optimization based on BIM and computer vision, characterized in that, Comprising the following steps: Step one, obtaining BIM model point cloud data, collecting construction site images, establishing a unified spatial coordinate system, automatically registering the BIM model and the construction site images; Step two, processing the construction site images using a visual recognition algorithm to identify component status, equipment operating parameters, quality defects, and safety violations, and generating visual recognition results; The visual recognition algorithm processing specifically includes: S21, multi-modal feature acquisition: acquire RGB images of the construction site through an industrial camera and extract channel features; simultaneously acquire depth maps output by the ZED camera and generate depth features through bilateral filtering preprocessing; S22, feature fusion processing: use ResNet50 as the backbone network to extract semantic features of the RGB image, extract spatial distance information of the depth features through a 3D convolution layer, and perform nonlinear transformation on the semantic features of the RGB image and the depth features through a fusion function; S23, spatio-temporal attention calculation: input the fusion features of the continuous set of frames into the gated recurrent unit, and calculate the time sequence attention weight of the t frame; Step three, constructing a four-dimensional dynamic BIM model and mapping the visual recognition results to the corresponding components in real time through multi-feature similarity calculation; Step four, monitoring progress deviation using key path dynamic identification and deviation propagation matrix, and predicting risks by combining Bayesian networks and Monte Carlo simulation; Step five, generating a dynamic optimization scheme for construction sequence and resource allocation through a hierarchical reinforcement learning architecture.

2. The BIM and computer vision based construction schedule dynamic optimization method of claim 1, wherein, Based on step one, the BIM model and the construction site images are automatically registered, and the automatic registration method is as follows: S11, feature acquisition: use a laser scanner to sample the BIM model point cloud to obtain point cloud data containing three-dimensional coordinates; acquire RGB-D images of the construction site through a binocular camera and extract depth information to generate site point cloud data; S12, feature extraction: use the FPFH algorithm to extract the fast point feature histogram descriptor of the point cloud data and the site point cloud data, respectively, and establish an initial correspondence relationship through bidirectional nearest neighbor search; Optimize the spatial transformation parameters using the iterative closest point algorithm, calculate the rotation matrix and translation vector to be optimized through the objective function, and minimize the function to register the BIM model point cloud and the site point cloud.

3. The BIM and computer vision based construction schedule dynamic optimization method of claim 1, wherein, The visual recognition algorithm further includes: Quality defect feature acquisition and processing: use the U-Net network to perform semantic segmentation on the RGB image, the input channel contains RGB three channels and a depth single channel, and the output is a pixel-level defect probability map; train the model parameters through a defect type database and calculate the defect severity; Safety hazard feature acquisition and processing: preset the building construction safety inspection standard, identify real-time construction safety features from the construction site images based on the YOLOv8 target detection network, and generate a specified number of anchor boxes through K-means clustering; Then determine whether there is a safety hazard through spatio-temporal correlation analysis.

4. The BIM and computer vision based construction schedule dynamic optimization method of claim 1, wherein, Based on step three, the construction of the four-dimensional dynamic BIM model specifically includes: S31, state feature acquisition: through the RFID tag reading member's geometric completion degree data, and then collecting construction quality indicators to generate the component quality state vector; through extracting the construction process type and start / end time from the construction log, it is recorded as the construction operation vector; through collecting human data, equipment data, and material data on the construction site, it is recorded as the resource input vector; S32, model construction processing: establish a state transition function and train it using an LSTM network, with the input being the component quality state vector, construction operation vector, and resource input vector in a specified time region, and the output being the next time state prediction; establish a resource-schedule association model through the construction condition vector, the preset resource efficiency function, the construction progress completion amount, and the maximum construction progress completion amount.

5. The BIM and computer vision based construction schedule dynamic optimization method of claim 1, wherein, Through multi-feature similarity calculation, the visual recognition result is mapped to the corresponding component in real time, and the real-time mapping process includes: Mapping feature acquisition: extract the geometric features, texture features, and material features of the component from the visual recognition result, and extract the corresponding attribute parameters from the corresponding component in the BIM model; Mapping processing: set corresponding weight coefficients for geometric features, texture features, and similarity material features; construct a multi-feature fusion similarity function, with the fused feature dimensions including geometric features, texture features, and material features; calculate the geometric feature similarity by Euclidean distance, the texture feature similarity by LBP texture histogram algorithm, and the material feature similarity by HSV color space analysis; set corresponding weight coefficients for geometric features, texture features, and similarity material features; weight and fuse the geometric feature similarity, texture feature similarity, and similarity material feature similarity with their set weight coefficients to obtain the comprehensive similarity; the comprehensive similarity is used to measure the matching degree of the visual recognition result and the BIM component features; Based on the multi-feature fusion similarity, the mapping confidence is calculated, which is: the confidence calculation covers the corresponding dimensions of geometry, texture, and material, and the final confidence is obtained by averaging the feature similarities after processing by the feature extraction function of each dimension; set a confidence threshold, when the calculated confidence is lower than the confidence threshold, trigger the BIM engineer to perform manual verification through the WebGL interactive interface. Use key path dynamic identification and deviation propagation matrix to monitor progress deviation, and the monitoring method of progress deviation is as follows:

6. The BIM and computer vision based construction schedule dynamic optimization method of claim 1, wherein, Progress feature acquisition: obtain the duration data of each construction activity through the task list of the BIM model, and obtain the actual start time and actual end time of the construction activity through on-site image recognition; in key path dynamic identification, for any construction activity, the earliest start time is determined by adding the maximum value in the duration of all previous construction activities to the earliest start time of the previous construction activities; For any construction activity, the latest start time is determined by subtracting the minimum value in the duration of itself from the latest start time of all subsequent construction activities; ​ Deviation processing: construct deviation propagation matrix: the element value in the matrix is determined according to the logical relationship of construction process: if a process is the immediate previous process of another process, the corresponding matrix element value is set to 1; if there is a resource sharing relationship between two processes, the corresponding matrix element value is set to 0.5; if there is no correlation, the corresponding matrix element value is set to 0; By comparing the actual construction progress with the planned progress, the delay time of each process is calculated, and the current process delay vector is calculated; The deviation propagation matrix and the current process delay vector are operated to calculate the execution deviation propagation, and the delay time distribution results of the subsequent processes affected are output, realizing the dynamic monitoring and propagation simulation of construction progress deviation.

7. The BIM and computer vision based construction schedule dynamic optimization method of claim 1, wherein, Based on step four, the risk is predicted by combining Bayesian network and Monte Carlo simulation, specifically: Risk feature acquisition: extract historical construction data from the project management system; By setting the number of historical risk event data, K2 algorithm is used for training and learning to generate the conditional probability table of Bayesian network; Based on the conditional probability table, the Bayesian network is constructed; Risk processing: set the number of simulations, and generate risk scenarios according to the Bayesian network in each simulation, and calculate the corresponding risk value; Set the risk threshold; Determine whether the risk scenario of each simulation triggers the risk through the indicator function; Statistical proportion of risk events in all simulation results as the probability of construction risk occurrence; Generate and output a certain number of high-probability risk scenarios.

8. The BIM and computer vision based construction schedule dynamic optimization method of claim 1, wherein, Based on step five, the dynamic optimization scheme of construction sequence and resource allocation is generated through the hierarchical reinforcement learning architecture, which includes: Decision feature acquisition: divide the decision features into upper and lower features; Among them, the upper features include the current construction stage, the key path activity state, and the lower features include the resource demand of each process and the material inventory level; And preset the reward function; Algorithm processing: in the hierarchical decision architecture, the upper strategy adopts convolutional neural network, the input is the construction stage feature map, and the output is the construction stage transition probability; The lower strategy adopts fully connected network, the input is the resource feature vector, and the output is the resource allocation scheme; Actor network: designed as a 3-layer fully connected layer structure, with 256 neurons in each layer, used to output decision action probability Critic network: uses value function approximation method to evaluate the value of the current decision; Training process: set learning rate and discount factor, train multiple rounds through PPO algorithm until the cumulative reward value fluctuation meets the preset fluctuation threshold; In the training, combined with the current construction state, decision action and reward feedback, the parameters of Actor and Critic networks are dynamically updated to optimize the construction sequence and resource allocation strategy.

9. A BIM and computer vision based construction progress dynamic optimization system employing the method of any one of claims 1-8, characterized in that, It includes the following modules: Data acquisition module, used to acquire BIM model point cloud data and construction site RGB-D image, and establish a unified spatial coordinate system; Visual recognition module, used to process construction site images using visual recognition algorithm, recognize component state, equipment operating parameters, quality defects and safety violations, and generate visual recognition results; Four-dimensional BIM modeling module, used to construct four-dimensional dynamic BIM model, and map visual recognition results to corresponding components through multi-feature similarity calculation; A progress deviation monitoring module is configured to monitor the construction progress deviation in real time by a critical path dynamic identification algorithm and a deviation propagation matrix, and generate a delay time distribution; A risk prediction module is configured to integrate a Bayesian network and a Monte Carlo simulation algorithm, and predict the construction risk based on historical risk data and real-time environmental parameters; A dynamic optimization decision module is configured to generate a dynamic optimization scheme of the construction sequence and resource allocation by a hierarchical reinforcement learning architecture.

Images