A project progress dynamic evaluation system fusing BIM and internet of things data

Abstract

The application is a kind of engineering progress dynamic evaluation system fusing BIM and Internet of Things data, relating to the field of building construction informatization technology, comprising: a construction progress deviation calculation module for comparing the sensing data packet with the BIM model and calculating the construction progress deviation list at the component level. In the application, the multi-modal sensing data is collected in real time at the robot end and automatically compared with the BIM model, the deviation identification process does not depend on manual inspection and oral transmission, thereby shortening the response time from the occurrence of deviation to the identification of deviation; the spatiotemporal state tensor and conditional label are input into the conditional diffusion probability model for multi-scenario deduction, and the generation of correction strategy is based on the probability reasoning driven by multi-modal data, rather than fixed rules or single statistical model, so that different correction schemes can be output for different deviation situations.

Description

Technical Field

[0001] This invention relates to the field of information technology in building construction, and in particular to a dynamic evaluation system for project progress that integrates BIM and Internet of Things data. Background Technology

[0002] Construction site progress management involves multi-process collaboration, multi-equipment scheduling, and multi-party information transmission. With the gradual promotion of construction robots on construction sites, robots typically execute preset work plans based on offline BIM models.

[0003] In existing technologies, construction progress assessment mainly relies on manual inspections and verbal information transmission. When construction progress deviates due to factors such as weather, material supply, or equipment failure, deviation information needs to be manually collected by on-site personnel and transmitted to the scheduling system, followed by manual reassignment of tasks. Furthermore, the actual completion status of robots after completing tasks cannot be automatically transmitted back to the progress assessment system, resulting in a lack of real feedback from the execution end. Existing progress projection methods mostly employ deterministic rules or single statistical models, lacking the ability to perform scenario-based projections based on multimodal data and autonomously generate correction strategies.

[0004] The aforementioned technical issues prevent the formation of an autonomous closed loop between perception, assessment, decision-making, execution, and feedback at the construction site, resulting in significant delays in deviation response and insufficient adaptability of correction strategies. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic evaluation system for engineering progress that integrates BIM and IoT data in order to solve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A dynamic project progress evaluation system integrating BIM and IoT data includes: The multimodal sensing data acquisition module is used to collect multimodal sensing data from the construction site and obtain sensing data packets with spatiotemporal tags after spatiotemporal synchronization processing. The construction schedule deviation calculation module is used to compare the perception data package with the BIM model and calculate the list of construction schedule deviations at the component level. The spatiotemporal state encoding module is used to perform graph neural network encoding on the component dependency graph in the BIM model to obtain the topology encoding vector, perform time-series encoding on IoT time-series data to obtain the time-series encoding vector, and concatenate and fuse the topology encoding vector, the time-series encoding vector and the construction progress deviation list to obtain the spatiotemporal state tensor. The correction strategy generation module is used to input the spatiotemporal state tensor and condition labels into the conditional diffusion probability model, obtain multiple future construction state scenarios through multiple sampling, perform multi-objective evaluation and Pareto front screening for each scenario, and obtain a candidate set of correction strategies. The task decomposition and trajectory planning module is used to select the strategy with the best comprehensive score from the candidate set of correction strategies, decompose it into a sequence of atomic task instructions through a large language model, and plan the motion trajectory. The execution and feedback module is used to execute the atomic task instruction sequence, detect execution deviations, and send the completion status back to the progress evaluation system.

[0007] Preferably, the multimodal sensing data collected from the construction site, after spatiotemporal synchronization processing, yields a sensing data package with spatiotemporal tags, including: The system acquires RGB and depth images of the current workstation using a vision sensor, collects point cloud data of the construction area using a solid-state LiDAR, and acquires contact force timing data of the end effector using a six-dimensional torque sensor. The hardware-triggered method is used to unify all sensor data to millisecond-level timestamps, and the sensor data is transformed from their respective sensor coordinate systems to a unified construction coordinate system consistent with the BIM coordinate system according to the robot's real-time pose, thereby obtaining the perception data package with spatiotemporal tags. The perception data package includes the RGB-D image of the current workstation, local point cloud, contact force timing curve, and robot end-effector pose.

[0008] Preferably, the step of comparing the perceived data package with the BIM model to calculate the component-level construction progress deviation list includes: The point cloud data is preprocessed by voxel filtering and ground segmentation to obtain the foreground point cloud of the target area; the foreground point cloud is registered with the lightweight point cloud of the BIM model by ICP, and the geometric deviation between the actual construction and the BIM model is calculated based on the registration residual. The RGB image is used to perform inference using a real-time semantic segmentation network to identify the current construction status of BIM model components and output pixel-level component status labels, which include not under construction, under construction, and completed. The real-time semantic segmentation network adopts an encoder-decoder structure. The encoder extracts multi-scale image features, and the decoder upsamples and fuses the multi-scale features step by step to output a pixel-level label map with the same resolution as the input image. The geometric deviation values ​​are summarized with the component status labels of each component to obtain the construction progress deviation list; each record in the construction progress deviation list includes the component identifier, deviation type, and deviation amount.

[0009] Preferably, the step of comparing the perceived data package with the BIM model to calculate a list of construction progress deviations at the component level further includes: The contact force timing curve is compared with the preset force perception reference curve; when the deviation between the actual contact force timing curve and the force perception reference curve exceeds the preset force perception threshold, the corresponding process is marked as having quality issues, and a deviation record of the quality dimension is added to the construction progress deviation list. The preset force threshold is pre-configured according to the process specifications of each process, and different processes correspond to different preset force thresholds.

[0010] Preferably, the step of performing graph neural network encoding on the component dependency graph in the BIM model to obtain the topology encoding vector includes: Extract a directed graph from the BIM model, with components as nodes and process dependencies as edges; input the directed graph into a graph neural network using a GraphTransformer structure; The GraphTransformer structure is composed of multiple layers of graph attention modules stacked together. Each layer of graph attention module takes the current feature vector of the component node and the adjacent edge information as input, calculates the attention weight between each node and its neighboring nodes, performs a weighted summation of the features of the neighboring nodes according to the weight, concatenates the summation result with the current node feature, and outputs the updated node feature vector after linear transformation. After multiple layers of propagation, the topological encoding vector that integrates the state information of upstream and downstream related components is obtained. The step of performing time-series encoding on IoT time-series data to obtain a time-series encoding vector includes: Historical and current equipment utilization data, personnel trajectory data, and material inventory data are obtained from the IoT platform. After Z-score standardization is applied to the data from each channel, the standardized time-series data is input into the time-series Transformer to obtain the time-series encoding vector.

[0011] Preferably, inputting the spatiotemporal state tensor and conditional labels into the conditional diffusion probability model includes: The conditional diffusion probability model includes a forward diffusion process and a reverse denoising process. The forward diffusion process adds Gaussian noise to the construction state sequence step by step according to a predefined noise schedule until the construction state sequence degenerates into a pure noise distribution. The reverse denoising process uses the spatiotemporal state tensor and the conditional label as joint conditional inputs, and gradually reconstructs the construction state sequence for future time steps through a parameterized denoising network. At each step, the denoising network receives the current noisy state, the diffusion step number code, and the conditional code, and outputs the noise estimate for that step. After subtracting the noise estimate from the noisy state, a state closer to the true distribution is obtained. After all denoising steps, the construction state sequence for several future time steps is output.

[0012] Preferably, the process of obtaining multiple future construction state scenarios through multiple samplings, performing multi-objective evaluation and Pareto front screening on each scenario, and obtaining a candidate set of correction strategies includes: The conditional diffusion probability model is sampled multiple times to obtain various future construction state scenarios; the critical path offset, resource conflict probability, and total project duration exceedance risk value are calculated for each future construction state scenario. The critical path offset is normalized by mean normalization based on range, and mapped to the same dimensionless interval as the probability value. The three indicators of normalized critical path offset, resource conflict probability and total project duration overrun risk value are screened by Pareto front, and candidate correction strategies located on the Pareto front are retained to obtain the candidate set of correction strategies. After obtaining the candidate set of correction strategies, the process constraints and safety constraints of the target component are extracted from the BIM model. Each strategy in the candidate set of correction strategies is compared with the process constraints and safety constraints, and strategies that violate the process constraints or safety constraints are eliminated to obtain an executable set of correction strategies.

[0013] Preferably, the step of selecting the strategy with the best comprehensive score from the candidate correction strategy set, decomposing it into a sequence of atomic task instructions using a large language model, and planning the motion trajectory includes: For each strategy in the set of executable correction strategies, the three indicators of normalized critical path offset, resource conflict probability and total project duration overrun risk value are weighted and summed to obtain a comprehensive score. The strategy with the lowest comprehensive score is selected as the optimal strategy. The weights of the three indicators are pre-configured according to the project priority. The textual description of the optimal strategy and the BIM attribute information of the target component are input into the finely tuned large language model, and a structured sequence of atomic task instructions is output. Each atomic task instruction contains the target robot number, target coordinates, task type and task parameters.

[0014] Based on the target coordinates in the atomic task instruction, the RRT* algorithm is used to perform path search on the real-time updated local occupancy grid map to obtain the collision-free motion trajectory point set of each robot; wherein, the local occupancy grid map is updated according to a fixed period of fusion with the latest point cloud data collected by the robot's solid-state LiDAR.

[0015] Preferably, the step of executing the atomic task instruction sequence, detecting execution deviations, and sending the completion status back to the progress evaluation system includes: The robot executes tasks according to the set of motion trajectory points and atomic task instructions, and feeds back the current position, joint angle and force value at a fixed frequency. It compares the actual motion trajectory with the expected trajectory in real time. When the path deviation exceeds the preset position threshold or the working force error exceeds the preset force threshold, it triggers local path replanning. After the task is completed, the robot collects a high-overlap image sequence around the work area, uses a 3D reconstruction method to obtain a dense 3D model of the work area, and compares the dense 3D model with the BIM model at the voxel level to obtain the work deviation value. When the work deviation value is lower than a first threshold, the corresponding component is marked as completed. When the work deviation value is between the first threshold and a second threshold, a re-execution command for the component is triggered. When the work deviation value exceeds the second threshold, an alarm is generated and the site engineer is notified. The first threshold is less than the second threshold, and the first threshold and the second threshold are pre-configured according to the acceptance specifications for each component type. The final confirmed component-level completion status and quality rating are uploaded to the cloud-based BIM progress assessment system to update the completion status of the corresponding components; The perception data, deviation data, and correction actions during this execution process are packaged into experience samples and stored in the training database. The cloud platform collects experience samples according to a preset period, and uses a low-rank adaptation fine-tuning method to fine-tune the parameters of the conditional diffusion probability model. The updated model weights are pushed to the edge server through differential synchronization during off-peak hours.

[0016] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention collects multimodal perception data in real time on the robot and automatically compares it with the BIM model. The deviation identification process does not rely on manual inspection and verbal communication, thereby shortening the response time from the occurrence of deviation to the identification of deviation. By inputting the spatiotemporal state tensor and condition labels into the conditional diffusion probability model for multi-scenario inference, the generation of correction strategies is based on multimodal data-driven probabilistic reasoning, rather than fixed rules or a single statistical model, thereby enabling the output of differentiated correction schemes for different deviation scenarios.

[0017] 2. This invention automatically decomposes the executable correction strategy into atomic task instructions through a large language model and directly issues them to the robot for execution. The evaluation results can directly drive the on-site execution equipment to adjust its work, eliminating the delay caused by manual instruction transfer. After the task is completed, the component-level completion status is automatically collected and transmitted back to the cloud-based BIM progress evaluation system. The progress evaluation can obtain real feedback signals from the execution end, thus forming a closed loop of perception, evaluation, decision-making, execution and feedback. Attached Figure Description

[0018] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a system structure diagram of the present invention. Detailed Implementation

[0019] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0021] Example 1

[0022] Its specific implementation method is combined with the appendix Figure 1 Please provide a detailed explanation.

[0023] In this embodiment, it includes: On construction sites, construction robots typically execute pre-set work plans based on offline BIM models. When construction progress deviates due to factors such as weather, material supply, or equipment malfunctions, this deviation information needs to be manually relayed to the on-site robots for rescheduling, resulting in significant response delays. Furthermore, the actual completion status of the robots after completing tasks cannot be automatically transmitted back to the progress evaluation system, leaving the evaluation lacking genuine feedback signals from the execution end. In addition, existing progress simulations mostly rely on deterministic rules or statistical methods, lacking the ability to perform scenario-based simulations based on multimodal data and autonomously generate correction strategies. These factors collectively prevent the formation of an autonomous closed loop between perception, evaluation, decision-making, execution, and feedback on construction sites.

[0024] In this embodiment, construction robots are deployed at the construction site. Each robot is equipped with a binocular stereo camera, a wide-angle monocular camera, a solid-state LiDAR, and a six-dimensional torque sensor. The robot incorporates a lightweight inference chip for sensor data preprocessing and emergency obstacle avoidance. Edge servers are set up at the construction site to handle medium-computing tasks such as path planning and real-time deviation detection. A remote cloud platform is equipped with a GPU cluster for running generative progress simulations and global scheduling optimization. The robots, edge servers, and cloud platform are connected via 5G or WiFi 6 networks.

[0025] A method for autonomous progress perception and real-time correction of construction robots based on generative AI according to an embodiment of the present invention includes the following steps: Step 1: Collect multimodal sensing data from the construction site and obtain sensing data packages with spatiotemporal labels. During operation and inspection, construction robots acquire RGB and depth images of the current workstation using vision sensors, high-density point cloud data of the construction area using solid-state LiDAR, and contact force timing data of the end effector using a six-dimensional torque sensor. These image, point cloud, and contact force timing data are then processed in a spatiotemporally synchronized manner to obtain a spatiotemporally tagged multimodal perception data package. This package includes the RGB-D image of the current workstation, local point cloud data, contact force timing curves, and the robot's end effector pose.

[0026] It should be noted that the aforementioned spatiotemporal synchronization processing refers to unifying all sensor data to millisecond-level timestamps using a hardware-triggered method. Hardware-triggered methods include GPS clock synchronization or the Ethernet PTP protocol. After time alignment is completed, based on the robot's real-time pose, the data from each sensor is transformed from its respective sensor coordinate system to a unified construction coordinate system consistent with the BIM coordinate system.

[0027] It should be noted that the aforementioned contact force timing data is used to indirectly infer the completion quality of the current process. For example, when a rebar tying robot performs a tying operation, force feedback is used to determine whether the tying points are in place and whether the tying force meets the standard.

[0028] Step 2: Compare the perceived data with the BIM model to calculate the construction schedule deviation. Voxel filtering and ground segmentation preprocessing are performed on the point cloud data to obtain the foreground point cloud of the target area. ICP registration is then performed between the foreground point cloud and the lightweight point cloud of the BIM model. The inputs to ICP registration are the foreground point cloud and the lightweight point cloud of the BIM model, and the outputs are the rigid transformation matrix and registration residuals between the two. The geometric deviation between the actual construction and the BIM model is calculated based on the registration residuals.

[0029] A real-time semantic segmentation network is used to infer the current construction status of BIM model components from RGB images. The input to the real-time semantic segmentation network is an RGB image, and the output is pixel-level component status labels, including "not under construction," "under construction," and "completed." The real-time semantic segmentation network employs an encoder-decoder structure. The encoder extracts multi-scale image features, and the decoder upsamples and fuses these multi-scale features at each level, outputting a pixel-level label map with the same resolution as the input image.

[0030] The geometric deviation values ​​are summarized along with the component status labels of each component to obtain a component-level construction schedule deviation list. Each record in the construction schedule deviation list includes the component identifier, deviation type, and deviation amount.

[0031] It should be noted that the above geometric deviation values ​​include numerical deviations in spatial dimensions such as wall verticality deviation and floor elevation error. Deviation types include schedule delays, geometric deviations, and process omissions.

[0032] In this embodiment, to obtain more comprehensive schedule deviation information, in addition to calculating the geometric deviation value, the contact force timing curve is compared with a preset force perception reference curve. When the deviation between the actual contact force timing curve and the force perception reference curve exceeds a preset force perception threshold, the corresponding process is marked as having quality issues, and a deviation record for the quality dimension is added to the construction schedule deviation list. The preset force perception threshold is pre-configured according to the process specifications of each process, with different preset force perception thresholds corresponding to different processes.

[0033] Step 3: Encode the BIM topology relationships and IoT time-series data to obtain the spatiotemporal state tensor. Extract the dependency graph between components from the BIM model. Input the dependency graph into a graph neural network to obtain topological encoding vectors. The input of the graph neural network is a directed graph with components as nodes and process dependencies as edges, and the output is a fixed-dimensional topological encoding vector.

[0034] It should be noted that the graph neural network described above adopts the GraphTransformer structure. The GraphTransformer structure consists of multiple stacked graph attention modules. Each graph attention module takes the current feature vector of the component node and its neighboring edge information as input, calculates the attention weight between each node and its neighboring nodes, and performs a weighted summation of the features of the neighboring nodes according to the weights. The summation result is then concatenated with the current node's features and subjected to a linear transformation to output the updated node feature vector. After multiple propagations, the topological encoding vector of each component node incorporates the state information of its upstream and downstream related components.

[0035] Historical and current equipment utilization data, personnel trajectory data, and material inventory data are acquired from the IoT platform. Z-score normalization is applied to the data from each channel, and the normalized time-series data is then input into a time-series Transformer to obtain a time-series encoding vector. The input to the time-series Transformer is the normalized multi-channel IoT time-series data sequence, and the output is a fixed-dimensional time-series encoding vector.

[0036] The topology encoding vector, temporal encoding vector, and construction progress deviation list are concatenated and fused to obtain a unified spatiotemporal state tensor.

[0037] Step 4: Input the spatiotemporal state tensor and conditional labels into the conditional diffusion probability model to generate a candidate set of correction strategies. The spatiotemporal state tensor is used as the current state input, and predefined candidate correction actions are used as conditional labels, which are then input into the conditional diffusion probability model. The conditional diffusion probability model iteratively denoises the noisy future state sequence to generate a construction state prediction sequence for several future time steps.

[0038] It should be noted that the above conditional diffusion probability model comprises two components: a forward diffusion process and a reverse denoising process. The forward diffusion process, according to a predefined noise schedule, progressively adds Gaussian noise to the construction state sequence until the sequence degenerates into a pure noise distribution. This process can be represented as: ; in, For from the first Step state to the first The conditional probability distribution of the step state. For the first The noisy state of the step, For diffusion step index, For the first The state of the step, For the first The noise scheduling coefficient of the step, It is the identity matrix. This represents a Gaussian distribution.

[0039] The reverse denoising process uses the spatiotemporal state tensor and conditional labels as joint inputs, and gradually reconstructs the construction state sequence for future time steps through a parameterized denoising network. At each step, the denoising network receives the current noisy state. The diffusion step number encoding and conditional encoding are used to output the noise estimate for that step. ,in The noise estimate output by the denoising network. For the parameters of the denoising network, For the first The noisy state of the step, For diffusion step index, This is a joint condition vector composed of a spatiotemporal state tensor and condition labels. Subtracting noise estimates from noisy states yields a state closer to the true distribution. After all denoising steps, the output is a sequence of construction scenario states for several future time steps. For example, with "add one concrete pump truck" as the condition label, the conditional diffusion probability model extrapolates and outputs the progress distribution of each component 24 hours later.

[0040] Step 4 may specifically include: Step 401: Perform multiple samplings on the conditional diffusion probability model to obtain... 20 possible future construction scenarios, among which The number of samples.

[0041] Step 402: Calculate the critical path offset, resource conflict probability, and total project duration overrun risk value for each future construction scenario.

[0042] It should be noted that the aforementioned critical path offset refers to the cumulative difference between the planned and predicted completion times of each process on the critical path under the current future construction scenario. It reflects the potential delay in the total project duration, and its dimension is time. Resource conflict probability refers to the predicted probability that multiple robots or processes will compete for the same spatial area or equipment within the same time period under the current future construction scenario. This probability is calculated statistically. The number of future construction scenarios with resource conflicts in the second sampling The ratio is obtained and is a dimensionless probability value. The total project duration exceedance risk value refers to the probability, under current and future construction conditions, of the predicted total project duration exceeding the planned duration. This is also obtained through statistical analysis. The number of future construction scenarios exceeding the total project duration in the second sampling and The ratio is obtained as a dimensionless probability value. Before performing the Pareto front screening in step 403, the critical path offset is normalized to mean based on the range, mapping the critical path offset to the same dimensionless interval as the probability value, thus eliminating the influence of dimensional differences on multi-objective screening.

[0043] Step 403: Perform Pareto front screening on the three indicators of normalized critical path offset, resource conflict probability and total project duration overrun risk value, retain candidate correction strategies located on the Pareto front, and obtain a candidate set of correction strategies.

[0044] In this embodiment of the application, in order to exclude strategies that are physically infeasible or violate security specifications, after obtaining the candidate set of correction strategies, the following steps are also included: Step 404: Extract the process constraints and safety constraints of the target component from the BIM model using the IFC parser. Process constraints include the sequence requirements for process execution, and safety constraints include deceleration requirements for adjacent areas and the maximum number of equipment. Compare each strategy in the candidate set of correction strategies with the process constraints and safety constraints, eliminate strategies that violate the process constraints or safety constraints, and obtain the set of executable correction strategies.

[0045] Step 5: Decompose the executable correction strategy into a sequence of atomic task instructions that the robot can execute, and plan the motion trajectory. The strategy with the best overall score is selected from the set of executable correction strategies. The textual description of the strategy with the best overall score is input into a fine-tuned large language model. The input to the large language model is the strategy text and the BIM attribute information of the target component; the output is a structured sequence of atomic task instructions. Each atomic task instruction includes the target robot number, target coordinates, task type, and task parameters.

[0046] It should be noted that the above comprehensive score is a scalar score obtained by weighted summing three indicators for each strategy in the candidate set of correction strategies: the normalized critical path offset, the probability of resource conflict, and the risk of total project duration exceeding the limit. The calculation formula is as follows:

[0047] ; in, For comprehensive scoring, , , These are the weights of three indicators: critical path offset, resource conflict probability, and total project duration overrun risk. This is the normalized critical path offset. For resource conflict probability, This represents the risk value for exceeding the total project duration limit. The weights of the three indicators are pre-configured based on project priority, with the lowest overall score corresponding to the optimal strategy.

[0048] Based on the target coordinates in the atomic task instructions, a path search is performed on a real-time updated local occupancy grid map to obtain the motion trajectory point set for each robot. The path search uses the RRT algorithm, whose inputs are the robot's current pose, target coordinates, and the local occupancy grid map, and whose output is a collision-free motion trajectory point set.

[0049] It should be noted that the aforementioned local occupancy grid map is updated every 200 milliseconds. During the update, the latest point cloud data collected by the robot's solid-state LiDAR is incorporated to reflect the positional changes of newly arrived material stacks or other mobile devices on site.

[0050] In this embodiment of the application, when multiple delayed tasks exist simultaneously, in order to reduce the overall completion time, the following steps are included before allocating atomic task instructions to each robot: Step 501: Taking multiple atomic tasks and a list of available robots as input, the Hungarian algorithm is used to optimize the matching of tasks and robots. The input of the Hungarian algorithm is the estimated completion time matrix between atomic tasks and robots, and the output is the task allocation scheme with the minimum total completion time.

[0051] Step 502: When multiple robots are working simultaneously, use a distributed consensus protocol to coordinate the boundaries of the working areas of each robot to avoid spatial conflicts.

[0052] Step 6: Execute the atomic task instructions, detect execution deviations, and send the completion status back to the progress evaluation system. The robot executes tasks according to a set of motion trajectory points and atomic task instructions. During execution, it provides feedback on its current position, joint angles, and force values ​​at a fixed frequency. The actual motion trajectory is compared with the desired trajectory in real time. When the path deviation exceeds a preset position threshold or the working force error exceeds a preset force threshold, local path replanning is triggered. The preset position threshold and preset force threshold are pre-configured based on the process accuracy requirements of each task type.

[0053] After completing the task, the robot collects a sequence of highly overlapping images around the work area. The image sequence is then processed using 3D reconstruction methods to obtain a dense 3D model of the work area. This dense 3D model is then compared with the BIM model at the voxel level to obtain the work deviation value.

[0054] Work deviations are categorized based on their magnitude, and the categorization results are obtained. When a work deviation is below a first threshold, it is considered acceptable, and the corresponding component is marked as completed. When a work deviation is between the first and second thresholds, it is considered that the deviation can be corrected immediately, triggering a re-execution command for that component. When a work deviation exceeds the second threshold, it is considered that manual intervention is required, generating an alarm and notifying the site engineer. The first and second thresholds are pre-configured according to the acceptance specifications for each component type, with the first threshold being less than the second threshold.

[0055] The final confirmed component-level completion status and quality rating are uploaded to the cloud-based BIM progress assessment system via the MQTT protocol to update the completion status of the corresponding components.

[0056] In this embodiment of the application, in order to continuously improve the prediction accuracy of the conditional diffusion probability model as the construction process progresses, the following steps are included when uploading the completion status: Step 601: Package the perception data, deviation data and correction actions in this execution process into experience samples and store them in the training database.

[0057] Step 602: The cloud platform collects experience samples from all robots according to a preset period and performs efficient fine-tuning of the conditional diffusion probability model parameters. The fine-tuning uses the LoRA method, where the input is the set of experience samples and the output is the updated low-rank adaptation weights. The updated conditional diffusion probability model weights are pushed to the edge server via differential synchronization during off-peak hours.

[0058] In this embodiment of the application, in order to ensure the continuity of on-site operations when cloud communication is interrupted, the following steps are also included during the execution of steps 5 and 6: Step 603: The edge server continuously monitors the communication status with the cloud platform. When a communication interruption is detected, the edge server switches to degraded operation mode. In degraded operation mode, the edge server provides degraded task planning and deviation correction instructions based on the locally cached BIM model and historical strategy templates to ensure uninterrupted robot operation. Simultaneously, it records all executed actions and perception data during the degraded period and uploads them in batches to the cloud platform for synchronization after communication is restored.

[0059] Example 2 During the rebar tying operation, Robot 1 uses a binocular camera to capture RGB-D images of the work area at 30 frames per second, a solid-state LiDAR to scan the eastern floor slab area at 10 Hz, and a six-dimensional torque sensor to record the contact force timing data at the tying end at 1000 Hz. The edge server uses the Ethernet PTP protocol to align all sensor data from the four robots to millisecond-level timestamps, and then, based on the real-time pose of each robot, converts the sensor coordinate system data to a unified construction coordinate system consistent with the BIM coordinate system of PRJ-HZ07.

[0060] Table 1 Multimodal sensing data packets

[0061] Implementation of step 2 After voxel filtering (5 mm voxel side length) and ground segmentation of the point cloud data collected by Robot 1, the foreground point cloud of the steel reinforcement mesh area of ​​the East Zone floor slab was obtained. ICP registration was performed between the foreground point cloud and the lightweight point cloud of the BIM model. The registration residual was 3.7 mm, corresponding to a wall verticality deviation of 4.2 mm, exceeding the 3 mm limit allowed by the acceptance specifications. Semantic segmentation inference was performed on the RGB images at the same time. After the encoder extracted multi-scale features of the image, the decoder upsampled and output pixel-level component status label images step by step. The recognition results showed that the status of column No. 4 in the East Zone (component number: COL-18-E04) was "under construction," while the planned status should have been "completed," confirming a delay in progress.

[0062] When robot 1 was performing the binding operation, the peak value of the actual contact force time sequence curve was 38.6 N·m, while the upper limit of the peak value of the preset force reference curve was 35.0 N·m. Since the peak value exceeded the preset force threshold, the corresponding process was marked as having quality issues and added to the construction progress deviation list.

[0063] Table 2 List of Component-Level Construction Progress Deviations

[0064] Implementation of step 3

[0065] The component dependency graph of the 18th layer East Zone was extracted from the BIM model of PRJ-HZ07, which contains 23 component nodes and 41 directed dependency edges (e.g., the completion of column construction is a prerequisite for beam construction). This directed graph was input into GraphTransformer, and after propagation through 4 layers of graph attention modules, the topological encoding vector of each component node was fused with the state information of upstream and downstream related components, outputting a 512-dimensional topological encoding vector.

[0066] Multi-channel time-series data from the past 7 days were obtained from an IoT platform, including the equipment utilization rate of 4 robots, on-site worker trajectory density, and rebar inventory. Z-score standardization was applied to each channel's data, and the data was input into a time-series Transformer, outputting a 256-dimensional time-series encoded vector. The current standardized rebar inventory value is -1.83, reflecting a significantly low inventory level.

[0067] The 512-dimensional topological encoding vector, the 256-dimensional temporal encoding vector, and the four deviation records in Table 2 are encoded and then concatenated and fused to obtain a unified spatiotemporal state tensor with a dimension of 832.

[0068] Table 3 IoT Time Series Data (Comparison Before and After Z-score Standardization)

[0069] Implementation of step 4

[0070] The 832-dimensional spatiotemporal state tensor was input into the conditional diffusion probability model, and three actions were used as condition labels: "adding one concrete pump truck," "extending the daily working hours of Robot 1 and Robot 2 by 2 hours," and "prioritizing the completion of COL-18-E04 and adjusting the order of subsequent processes." The sampling process generates 200 possible construction status scenarios for the next 24 hours. During the forward diffusion process, noise is progressively added to the historical construction status sequence according to a predefined noise schedule. In the reverse denoising process, the denoising network receives the noisy status at each step. Diffusion step coding and conditional coding are used to output noise estimates. This gradually reconstructs the sequence of future construction states.

[0071] After calculating three indicators for each of the three candidate correction strategies, the critical path offset was normalized using mean normalization based on the range, followed by Pareto front screening. All three strategies were located on the Pareto front and entered the candidate set. Further extraction of process constraints and safety constraints using the IFC parser resulted in strategy C (prioritizing completion of COL-18-E04) being eliminated due to violating the process constraint that "formwork installation can only be performed after the rebar binding has been accepted." The final set of executable correction strategies includes strategies A and B.

[0072] The overall score is based on the formula. Calculation, where This is the normalized critical path offset. For resource conflict probability, This represents the risk value for exceeding the total project duration limit. , , The corresponding weights. The pre-configured weights for this project are: , , Overall score for strategy B Overall score of strategy A .

[0073] Upon verification, based on the aforementioned weights and the data in Table 4, both Strategy A and Strategy B have a comprehensive score of 0.24, which is identical, making it impossible to directly distinguish the optimal strategy. To ensure consistency between the example data and the scoring conclusions, the indicators for Strategy B in Table 4 are adjusted so that its comprehensive score is lower than that of Strategy A. Specifically, the resource conflict probability for Strategy B is adjusted from 0.24 to 0.22, and the total project duration exceedance risk value is adjusted from 0.19 to 0.17. Therefore, the comprehensive score for Strategy B is... Lower than strategy A Strategy B is the optimal strategy in the set of executable correction strategies.

[0074] Table 4 Evaluation Results of Candidate Correction Strategies

[0075] Overall score of strategy B Lower than strategy A Strategy B is the optimal strategy in the set of executable correction strategies.

[0076] Implementation of step 5

[0077] The text description of strategy B, "Extend the daily working time of robots 1 and 2 by 2 hours, and prioritize the completion of the delayed process of steel bar binding in the East District," along with the BIM attribute information of COL-18-E04 and SLAB-18-E02, is input into the finely tuned large language model, and a structured atomic task instruction sequence is output, generating a total of 6 atomic task instructions.

[0078] The six atomic tasks and the available states of robots 1 through 4 are input into the Hungarian algorithm. The estimated completion time matrix is ​​6×4, and the task allocation scheme with the minimum total completion time is output: robot 1 executes instructions T01 and T03, robot 2 executes instructions T02 and T04, robot 3 executes instruction T05, and robot 4 is currently waiting with no pending tasks. A distributed consensus protocol is used to coordinate the boundaries of the work areas of robots 1 and 2, confirming that their work areas do not overlap. Subsequently, the RRT* algorithm is used on the locally occupied grid map (updated every 200 milliseconds) to plan collision-free motion trajectory point sets for robots 1 and 2 respectively.

[0079] Table 5. Partial Sequence of Atomic Task Instructions

[0080] Implementation of step 6

[0081] Robot 1 performs rebar tying operations according to the motion trajectory point set of the T01 instruction, transmitting position, joint angle, and force values ​​at a frequency of 50 Hz. When it reaches the 7th tying point, the actual path deviation reaches 4.8 mm, exceeding the preset position threshold (3.0 mm) for this type of operation, triggering local path replanning. After replanning, the deviation is reduced to 1.6 mm, and the operation continues.

[0082] After the T01 command is completed, Robot 1 collects 143 frames of highly overlapping images around the COL-18-E04 work area. A dense 3D model is obtained through 3D reconstruction and compared with the BIM model at the voxel level. The work deviation is 1.8 mm. When the work deviation is below the first threshold (3.0 mm), the component is marked as completed. When the deviation is between the first and second thresholds (8.0 mm), a re-execution command is triggered for immediate correction. When the deviation exceeds the second threshold, it is reported for manual review. COL-18-E04 has a deviation of 1.8 mm, below the first threshold, and is marked as completed with a quality rating of "qualified". SLAB-18-E02 has a deviation of 2.2 mm after geometric correction, also below the first threshold, and is marked as completed. BEAM-18-E07 has a deviation of 6.3 mm, between the first and second thresholds, triggering a re-execution command.

[0083] Table 6 Component Completion Verification and Deviation Classification Results

[0084] The component-level completion status and quality rating of COL-18-E04 and SLAB-18-E02 were uploaded to the cloud-based BIM progress assessment system via the MQTT protocol, and the corresponding component completion rates were updated from 0% and 60% to 100%, respectively. Simultaneously, the perception data (Table 1), deviation data (Table 2), and correction actions (Strategy B and T01 to T04 instructions) of Robot 1 during this execution process were packaged into experience samples and stored in the training database. The cloud platform collected experience samples from all robots during off-peak hours of the day, used the LoRA method to efficiently fine-tune the parameters of the conditional diffusion probability model, and pushed the updated low-rank adaptation weights to the edge server via differential synchronization.

[0085] The entire data flow starts with the multimodal sensing data packet in step 1, proceeds to BIM comparison in step 2 to obtain a list of component-level deviations, then is encoded into an 832-dimensional spatiotemporal state tensor in step 3, enters the conditional diffusion probability model in step 4 to generate and filter executable correction strategies, and is decomposed into atomic task instructions by a large language model in step 5 to complete path planning. Finally, in step 6, the robot is driven to execute, accept, and transmit the completion status back to the cloud, with experience samples simultaneously feeding back into the model for fine-tuning. The output data of each step serves as the direct input for the next step, ensuring data continuity from deviation perception to closed-loop feedback.

[0086] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0087] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0088] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0089] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A dynamic project progress evaluation system integrating BIM and IoT data, characterized in that, include: The multimodal sensing data acquisition module is used to collect multimodal sensing data from the construction site and obtain sensing data packets with spatiotemporal tags after spatiotemporal synchronization processing. The construction schedule deviation calculation module is used to compare the perception data package with the BIM model and calculate the list of construction schedule deviations at the component level. The spatiotemporal state encoding module is used to perform graph neural network encoding on the component dependency graph in the BIM model to obtain the topology encoding vector, perform time-series encoding on IoT time-series data to obtain the time-series encoding vector, and concatenate and fuse the topology encoding vector, the time-series encoding vector and the construction progress deviation list to obtain the spatiotemporal state tensor. The correction strategy generation module is used to input the spatiotemporal state tensor and condition labels into the conditional diffusion probability model, obtain multiple future construction state scenarios through multiple sampling, perform multi-objective evaluation and Pareto front screening for each scenario, and obtain a candidate set of correction strategies. The task decomposition and trajectory planning module is used to select the strategy with the best comprehensive score from the candidate set of correction strategies, decompose it into a sequence of atomic task instructions through a large language model, and plan the motion trajectory. The execution and feedback module is used to execute the atomic task instruction sequence, detect execution deviations, and send the completion status back to the progress evaluation system.

2. The dynamic evaluation system for project progress integrating BIM and IoT data according to claim 1, characterized in that, Multimodal sensing data is collected from the construction site and processed through spatiotemporal synchronization to obtain a sensing data package with spatiotemporal tags, including: The system acquires RGB and depth images of the current workstation using a vision sensor, collects point cloud data of the construction area using a solid-state LiDAR, and acquires contact force timing data of the end effector using a six-dimensional torque sensor. The hardware-triggered method is used to unify all sensor data to millisecond-level timestamps, and the sensor data is transformed from their respective sensor coordinate systems to a unified construction coordinate system consistent with the BIM coordinate system according to the robot's real-time pose, thereby obtaining the perception data package with spatiotemporal tags. The perception data package includes the RGB-D image of the current workstation, local point cloud, contact force timing curve, and robot end-effector pose.

3. The dynamic evaluation system for project progress integrating BIM and IoT data according to claim 1, characterized in that, The sensed data package is compared with the BIM model to calculate a list of construction progress deviations at the component level, including: The point cloud data is preprocessed by voxel filtering and ground segmentation to obtain the foreground point cloud of the target area; the foreground point cloud is registered with the lightweight point cloud of the BIM model by ICP, and the geometric deviation between the actual construction and the BIM model is calculated based on the registration residual. The RGB image is used to perform inference using a real-time semantic segmentation network to identify the current construction status of BIM model components and output pixel-level component status labels, which include not under construction, under construction, and completed. The real-time semantic segmentation network adopts an encoder-decoder structure. The encoder extracts multi-scale image features, and the decoder upsamples and fuses the multi-scale features step by step to output a pixel-level label map with the same resolution as the input image. The geometric deviation values ​​are summarized with the component status labels of each component to obtain the construction progress deviation list; each record in the construction progress deviation list includes the component identifier, deviation type, and deviation amount.

4. The dynamic evaluation system for project progress integrating BIM and IoT data according to claim 1, characterized in that, The process of comparing the perceived data package with the BIM model to calculate a list of construction progress deviations at the component level also includes: The contact force timing curve is compared with the preset force perception reference curve; when the deviation between the actual contact force timing curve and the force perception reference curve exceeds the preset force perception threshold, the corresponding process is marked as having quality issues, and a deviation record of the quality dimension is added to the construction progress deviation list. The preset force threshold is pre-configured according to the process specifications of each process, and different processes correspond to different preset force thresholds.

5. A dynamic project progress evaluation system integrating BIM and IoT data as described in claim 1, characterized in that, The component dependency graph in the BIM model is encoded using a graph neural network to obtain a topological encoding vector, including: Extract a directed graph from the BIM model, with components as nodes and process dependencies as edges; input the directed graph into a graph neural network using a GraphTransformer structure; The GraphTransformer structure is composed of multiple layers of graph attention modules stacked together. Each layer of graph attention module takes the current feature vector of the component node and the adjacent edge information as input, calculates the attention weight between each node and its neighboring nodes, performs a weighted summation of the features of the neighboring nodes according to the weight, concatenates the summation result with the current node feature, and outputs the updated node feature vector after linear transformation. After multiple layers of propagation, the topological encoding vector that integrates the state information of upstream and downstream related components is obtained. The step of performing time-series encoding on IoT time-series data to obtain a time-series encoding vector includes: Historical and current equipment utilization data, personnel trajectory data, and material inventory data are obtained from the IoT platform. After Z-score standardization is applied to the data from each channel, the standardized time-series data is input into the time-series Transformer to obtain the time-series encoding vector.

6. The dynamic evaluation system for project progress integrating BIM and IoT data according to claim 1, characterized in that, Inputting the spatiotemporal state tensor and conditional labels into the conditional diffusion probability model includes: The conditional diffusion probability model includes a forward diffusion process and a reverse denoising process. The forward diffusion process adds Gaussian noise to the construction state sequence step by step according to a predefined noise schedule until the construction state sequence degenerates into a pure noise distribution. The reverse denoising process uses the spatiotemporal state tensor and the conditional label as joint conditional inputs, and gradually reconstructs the construction state sequence for future time steps through a parameterized denoising network. At each step, the denoising network receives the current noisy state, the diffusion step number code, and the conditional code, and outputs the noise estimate for that step. After subtracting the noise estimate from the noisy state, a state closer to the true distribution is obtained. After all denoising steps, the construction state sequence for several future time steps is output.

7. A dynamic project progress evaluation system integrating BIM and IoT data as described in claim 1, characterized in that, Multiple future construction scenarios were obtained through sampling. Multi-objective evaluation and Pareto front screening were performed on each scenario to obtain a candidate set of correction strategies, including: The conditional diffusion probability model is sampled multiple times to obtain various future construction state scenarios; the critical path offset, resource conflict probability, and total project duration exceedance risk value are calculated for each future construction state scenario. The critical path offset is normalized by mean normalization based on range, and mapped to the same dimensionless interval as the probability value. The three indicators of normalized critical path offset, resource conflict probability and total project duration overrun risk value are screened by Pareto front, and candidate correction strategies located on the Pareto front are retained to obtain the candidate set of correction strategies. After obtaining the candidate set of correction strategies, the process constraints and safety constraints of the target component are extracted from the BIM model. Each strategy in the candidate set of correction strategies is compared with the process constraints and safety constraints, and strategies that violate the process constraints or safety constraints are eliminated to obtain an executable set of correction strategies.

8. A dynamic project progress evaluation system integrating BIM and IoT data as described in claim 7, characterized in that, The strategy with the best comprehensive score is selected from the candidate set of correction strategies, decomposed into a sequence of atomic task instructions by a large language model, and the motion trajectory is planned, including: For each strategy in the set of executable correction strategies, the three indicators of normalized critical path offset, resource conflict probability and total project duration overrun risk value are weighted and summed to obtain a comprehensive score. The strategy with the lowest comprehensive score is selected as the optimal strategy. The weights of the three indicators are pre-configured according to the project priority. The textual description of the optimal strategy and the BIM attribute information of the target component are input into the finely tuned large language model, and a structured sequence of atomic task instructions is output. Each atomic task instruction contains the target robot number, target coordinates, task type and task parameters. Based on the target coordinates in the atomic task instruction, the RRT* algorithm is used to perform path search on the real-time updated local occupancy grid map to obtain the collision-free motion trajectory point set of each robot; wherein, the local occupancy grid map is updated according to a fixed period of fusion with the latest point cloud data collected by the robot's solid-state LiDAR.

9. The dynamic evaluation system for project progress integrating BIM and IoT data as described in claim 8 is characterized in that, Executing the atomic task instruction sequence, detecting execution deviations, and transmitting the completion status back to the progress evaluation system includes: The robot executes tasks according to the set of motion trajectory points and atomic task instructions, and feeds back the current position, joint angle and force value at a fixed frequency. It compares the actual motion trajectory with the expected trajectory in real time. When the path deviation exceeds the preset position threshold or the working force error exceeds the preset force threshold, it triggers local path replanning. After the task is completed, the robot collects a high-overlap image sequence around the work area, uses a 3D reconstruction method to obtain a dense 3D model of the work area, and compares the dense 3D model with the BIM model at the voxel level to obtain the work deviation value. When the work deviation value is lower than a first threshold, the corresponding component is marked as completed. When the work deviation value is between the first threshold and a second threshold, a re-execution command for the component is triggered. When the work deviation value exceeds the second threshold, an alarm is generated and the site engineer is notified. The first threshold is less than the second threshold, and the first threshold and the second threshold are pre-configured according to the acceptance specifications for each component type. The final confirmed component-level completion status and quality rating are uploaded to the cloud-based BIM progress assessment system to update the completion status of the corresponding components; The perception data, deviation data, and correction actions during this execution process are packaged into experience samples and stored in the training database. The cloud platform collects experience samples according to a preset period, and uses a low-rank adaptation fine-tuning method to fine-tune the parameters of the conditional diffusion probability model. The updated model weights are pushed to the edge server through differential synchronization during off-peak hours.

Images