A Method and System for Refined Construction Quality Management Based on Portable Holographic Scanning and Law Enforcement Recorders

By using a mobile robot platform based on BIM models and deep learning, we have achieved efficient, accurate, and traceable full-process management of construction quality inspection. This solves the problems of low inspection efficiency, insufficient registration accuracy, neglect of behavioral factors, and lack of hierarchical early warning in existing technologies, and is applicable to building construction quality inspection.

CN123117903APending Publication Date: 2026-11-17GUOHUA ENERGY INVESTMENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611302703.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-26
Publication Date
2026-11-17

Smart Images

  • Figure CN123117903A_ABST
    Figure CN123117903A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for refined construction quality management based on portable holographic scanning and law enforcement recorders. The method includes: constructing a mobile robot platform equipped with LiDAR and IMU; acquiring pose information through tightly coupled SLAM and importing it into a BIM model; dynamically optimizing the scanning path using the DQN algorithm based on BIM topology information, while simultaneously generating a noise-free design point cloud through ray tracing; controlling the robot to move along the optimized path, collecting noisy actual point cloud data and multimodal video streams of operational behavior; inputting the actual point cloud and design point cloud into a PointNet++-based deep registration network to extract geometric deviation feature vectors; performing spatiotemporal alignment and tensor concatenation between the video stream and the geometric deviation feature vectors after behavior recognition, inputting the video stream into a GBDT classifier to output a construction quality qualification probability value; and constructing a digital twin based on the probability value to trigger tiered early warnings and generate rectification work orders. This invention achieves intelligent closed-loop management of construction quality from data collection and evaluation to traceability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of construction quality management technology, specifically to a method and system for refined construction quality management based on portable holographic scanning and law enforcement recorders. Background Technology

[0002] With the continuous improvement of informatization and intelligentization in the construction industry, the requirements for refined construction quality management are increasing. Traditional construction quality inspection methods mainly rely on manual measurement and visual inspection, which have problems such as low inspection efficiency, strong subjectivity, and limited coverage, making it difficult to meet the needs of quality control in modern complex building construction.

[0003] Currently, Building Information Modeling (BIM) technology has been widely used in building engineering design, construction, and operation and maintenance management, providing basic data support for digital management of construction quality. Meanwhile, 3D laser scanning technology can quickly acquire point cloud data from the construction site, and quantitative analysis of construction deviations can be achieved by comparing it with the BIM model. However, existing quality inspection methods based on point cloud comparison still have the following shortcomings:

[0004] First, existing methods mostly use preset paths for scanning, which is difficult to adapt to complex and ever-changing construction environments, resulting in incomplete or inefficient point cloud acquisition. Second, traditional point cloud registration algorithms (such as ICP and FPFH) are sensitive to initial pose, with limited registration accuracy and robustness, making it difficult to achieve high-precision alignment between the actual and designed point clouds. Third, existing quality assessment methods rely solely on geometric deviation features, neglecting the impact of dynamic factors such as construction personnel's operational behavior on construction quality, making it difficult to trace the causes of quality defects. Fourth, quality inspection results are mostly output as binary labels of "qualified / unqualified," lacking a probabilistic hierarchical early warning mechanism, and failing to achieve hierarchical risk control and dynamic decision-making. Fifth, there is a lack of a comprehensive and refined management system covering the entire process from data acquisition and quality assessment to problem tracing and rectification closure.

[0005] Therefore, how to integrate mobile robots, BIM models, 3D point cloud processing, behavior recognition and intelligent classification technologies to build a refined construction quality management method and achieve high-precision, intelligent and traceable quality control is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for refined management of construction quality based on portable holographic scanning and law enforcement recorders, so as to solve the technical problems of low detection efficiency, insufficient point cloud registration accuracy, neglect of operational behavior factors, lack of hierarchical early warning mechanism and lack of closed-loop management of the whole process in existing construction quality detection methods.

[0007] In a first aspect, embodiments of this application provide a method for refined management of construction quality based on portable holographic scanning and law enforcement recorders, the method comprising:

[0008] A mobile robot platform equipped with LiDAR and IMU was constructed, and pose information was acquired based on tightly coupled SLAM and imported into the BIM model;

[0009] MDP model is constructed based on BIM topology information, and DQN algorithm is used to dynamically optimize the scanning path. At the same time, noiseless design point cloud is generated based on the same BIM model through ray tracing.

[0010] The robot is controlled to move along an optimized path, and noisy actual point cloud data is collected by LiDAR. Simultaneously, a multimodal video stream of the construction workers' actions and the environmental conditions is collected by the law enforcement recorder.

[0011] The actual point cloud and the design point cloud are input into a PointNet++-based deep registration network for end-to-end registration, and geometric deviation feature vectors are extracted through multi-scale feature matching.

[0012] After the video stream is processed by behavior recognition, the operation behavior feature vector is obtained. It is then spatiotemporally aligned and tensor-concatenated with the geometric deviation feature vector. The result is input into the GBDT classifier and outputs the probability value of the construction quality being qualified and the binary classification result.

[0013] A digital twin is constructed based on the probability value. When the probability value is lower than the threshold, a graded warning is triggered. The correlation coefficient between operational behavior characteristics and quality defects is analyzed through a causal reasoning model, and key process nodes are automatically located and rectification work orders are generated.

[0014] Secondly, embodiments of this application provide a refined construction quality management system based on portable holographic scanning and law enforcement recorders, applied to the refined construction quality management method based on portable holographic scanning and law enforcement recorders as described in the first aspect. The system includes:

[0015] A mobile robot platform, equipped with LiDAR and IMU, is used to acquire pose information based on tightly coupled SLAM and import it into a BIM model;

[0016] The path planning and design point cloud generation module is used to construct an MDP model based on BIM topology information, dynamically optimize the scanning path using the DQN algorithm, and generate a noise-free design point cloud based on the same BIM model through ray tracing.

[0017] The data acquisition module is used to control the robot to move along the optimized path, collect noisy actual point cloud data through LiDAR, and simultaneously collect multimodal video streams of construction personnel's operation actions and environmental conditions through law enforcement recorder;

[0018] The point cloud registration and feature extraction module is used to perform end-to-end registration of the actual point cloud and the design point cloud by inputting the PointNet++-based deep registration network, and extracts the geometric deviation feature vector through multi-scale feature matching.

[0019] The behavior recognition and quality classification module is used to process the video stream through behavior recognition to obtain the operation behavior feature vector, perform spatiotemporal alignment and tensor concatenation with the geometric deviation feature vector, input the data into the GBDT classifier, and output the probability value of construction quality qualification and the binary classification result.

[0020] The digital twin and early warning management module is used to construct a digital twin based on the probability value. When the probability value is lower than the threshold, a graded early warning is triggered. The module analyzes the correlation coefficient between operational behavior characteristics and quality defects through a causal reasoning model, automatically locates key process nodes, and generates rectification work orders.

[0021] Thirdly, embodiments of this application provide an electronic device, including:

[0022] processor;

[0023] Memory used to store processor-executable instructions;

[0024] The processor is configured to implement the refined construction quality management method based on portable holographic scanning and law enforcement recorder as described in the first aspect when executing the instructions.

[0025] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the construction quality refinement management method based on portable holographic scanning and law enforcement recorder as described in the first aspect.

[0026] Compared with existing technologies, this invention has the following advantages: It employs a deep reinforcement learning algorithm to dynamically optimize the scanning path, replacing the traditional preset path method, enabling the robot to adaptively and fully cover the construction area, significantly improving the integrity and efficiency of point cloud acquisition. It generates a noiseless design point cloud through ray tracing and performs end-to-end registration using a PointNet++-based deep registration network, overcoming the sensitivity of traditional ICP and FPFH algorithms to initial pose, and greatly improving the alignment accuracy and robustness between the actual point cloud and the design point cloud. It performs spatiotemporal alignment and tensor stitching of geometric deviation features and operational behavior features processed by behavior recognition, for the first time incorporating construction personnel's operational behavior factors into the quality assessment model, enabling traceable analysis of the causes of quality defects. It uses a GBDT classifier to output the probability value of construction quality compliance and the binary classification result, automatically triggering graded early warnings based on the probability value, realizing risk graded control and dynamic decision-making from discrete labels to continuous probabilities. By analyzing the correlation coefficients between operational behavior characteristics and quality defects using a causal reasoning model, key process nodes are automatically located and rectification work orders are generated. Combined with a digital twin, a refined management system covering the entire process from data collection and quality assessment to problem tracing and rectification closure is constructed. This invention effectively solves the problems of low detection efficiency, insufficient registration accuracy, neglect of behavioral factors, lack of hierarchical early warning, and lack of full-process closed-loop management in existing technologies. It has the advantages of high precision, intelligence, and traceability, and is applicable to the field of construction quality inspection in building engineering. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a method for refined management of construction quality based on portable holographic scanning and law enforcement recorders, provided as an embodiment of this application.

[0028] Figure 2 This is an architecture diagram of a refined construction quality management system based on a portable holographic scanner and law enforcement recorder, provided as an embodiment of this application.

[0029] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0031] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0032] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] Example 1

[0034] Figure 1 This is a schematic diagram illustrating a method for refined construction quality management based on a portable holographic scanner and law enforcement recorder, provided as an embodiment of this application. Figure 1 As shown, a method for refined construction quality management based on portable holographic scanning and law enforcement recorders includes:

[0035] S1. Construct a mobile robot platform equipped with LiDAR and IMU, and acquire pose information based on tightly coupled SLAM and import it into the BIM model. Specifically, build a mobile robot platform integrating LiDAR and inertial measurement unit, use tightly coupled SLAM technology to fuse sensor data, calculate the robot's precise position and attitude in the construction environment in real time, and import the pre-established building information model into the system to establish a unified spatial reference benchmark for subsequent path planning and data comparison.

[0036] Specifically, in this embodiment, step S1 includes:

[0037] S11. Construct a differential drive mobile robot chassis, install a three-dimensional LiDAR on the top of the chassis, and install a high-precision inertial measurement unit (IMU) and an embedded computing unit inside the chassis to form an integrated mobile scanning hardware platform.

[0038] In practical implementation, a differential-drive mobile robot chassis is first constructed. Differential drive means that the rotational speed of the robot's left and right wheels can be controlled independently, and steering is achieved through the speed difference between the two wheels. A 3D LiDAR is installed on top of the chassis to scan the construction environment and acquire point cloud data. A high-precision inertial measurement unit (IMU, used to measure the acceleration and angular velocity of an object in three axes) and an embedded computing unit are embedded inside the chassis, forming an integrated mobile scanning hardware platform. The chassis is welded from high-strength alloy materials, possessing sufficient mechanical strength to withstand the bumpy surfaces of complex indoor and outdoor construction scenarios.

[0039] S12. Collect the point cloud data stream output by the lidar and the angular velocity and acceleration data stream output by the IMU. Perform joint optimization on the data stream using a tightly coupled SLAM algorithm based on error state Kalman filter (ESKF) to solve the six-degree-of-freedom pose information of the mobile robot platform in the global coordinate system.

[0040] After the robot starts, the system collects point cloud data streams from the LiDAR and angular velocity and acceleration data streams from the IMU in real time. These two types of data are then input into a tightly coupled SLAM algorithm based on Error State Kalman Filter (ESKF, a filtering algorithm specifically designed for nonlinear system state estimation, which decomposes the system state into nominal and error states for separate prediction and updating). Tight coupling means that the raw data from the LiDAR and IMU are processed within the same optimization framework, rather than being calculated independently and then fused. This algorithm calculates the six-degree-of-freedom pose information of the mobile robot in the global coordinate system. The six degrees of freedom include positional movement along the X, Y, and Z axes and rotational angles around these axes, thus fully describing the robot's position and orientation in space.

[0041] S13. Perform coordinate system unification processing on the pose information and the BIM model of the construction object, and import the BIM model under the unified coordinate system into the memory space of the embedded computing unit as the basic reference model for subsequent path planning and scanning comparison.

[0042] After obtaining the robot's pose information, it is compared with a pre-established BIM model of the construction object using coordinate system unification to ensure the correct transformation relationship between the robot's coordinate system and the world coordinate system of the BIM model. BIM (Building Information Modeling) is a data model based on three-dimensional digital technology that integrates various types of information throughout the entire lifecycle of a building project. The unified coordinate system BIM model is imported into the memory space of the embedded computing unit as a basic reference model for subsequent path planning and point cloud comparison, enabling the robot to accurately perceive its relative positional relationship with building components in the construction environment.

[0043] S2. An MDP model is constructed based on BIM topology information, and the DQN algorithm is used to dynamically optimize the scanning path. Simultaneously, noise-free design point cloud data is generated from the same BIM model using ray tracing. Specifically, a decision model is constructed based on the building structure topology relationships in the BIM model, and a deep reinforcement learning algorithm is used to dynamically optimize the robot's scanning path, enabling the robot to adaptively traverse the entire construction area. At the same time, high-fidelity, noise-free design point cloud data is generated from the BIM model using ray tracing technology, serving as an ideal benchmark for subsequent quality comparison.

[0044] Specifically, in this embodiment, step S2 includes:

[0045] S21. Analyze the BIM model of the construction object, extract the geometric boundaries and spatial connectivity of the building components, construct a topology map containing obstacle areas, reachable areas and target scanning areas, and discretize the topology map into a raster map.

[0046] First, the imported BIM model is analyzed to extract the geometric boundary information of building components such as walls, columns, beams, and slabs, as well as the spatial connectivity between these components, thereby constructing a topology map. A topology map is a simplified representation of the environmental structure, focusing on describing the connections between different areas rather than precise geometric dimensions. This topology map contains three types of areas: obstacle areas (locations inaccessible to the robot, such as walls and columns), accessible areas (passages and rooms where the robot can move freely), and target scanning areas (key locations requiring quality inspection). Subsequently, the topology map is discretized into a grid map, divided into uniformly sized grid cells, each labeled as either accessible or inaccessible, facilitating robot pathfinding.

[0047] S22. Using the robot's position in the grid map as the state space, the eight adjacent directions of the robot's movement as the action space, and the weighted function of scan coverage and path length as the reward function, construct a Markov Decision Process (MDP) model.

[0048] Based on the grid map generated in the previous step, a Markov Decision Process (MDP) model is constructed. The MDP is a mathematical framework used to describe sequential decisions made by a decision-maker in a stochastic environment, characterized by the fact that the next state depends only on the current state and the current action, and is independent of history. In this MDP model, the robot's grid position on the grid map is defined as the state space, and the eight adjacent directions (up, down, left, right, and four diagonal directions) that the robot can choose each time it moves are defined as the action space. A reward function is also designed, which is a weighted combination of scan coverage and path length. Scan coverage refers to the proportion of the target scan area that the robot has scanned, and path length refers to the total distance traveled by the robot. The goal of the reward function is to guide the robot to cover as much scan area as possible with the shortest possible path.

[0049] S23. Initialize the deep Q-network (DQN), and train it interactively with the MDP model through the ε-greedy strategy to output the scan path sequence that maximizes the cumulative reward, which serves as the robot's dynamically optimized scan path.

[0050] A Deep Q-Network (DQN, a reinforcement learning algorithm combining deep learning and Q-learning, using a neural network to approximate the long-term reward of each action taken under different states) is initialized and trained interactively with the aforementioned MDP model using an ε-greedy policy. The ε-greedy policy is a strategy that balances exploration and exploitation, randomly selecting actions for exploration with a probability of ε, and selecting the action currently considered optimal for exploitation with a probability of 1-ε. During training, the DQN continuously updates its network parameters, learning the cumulative reward obtained by choosing different movement directions at different grid positions. After training, the network outputs a scan path sequence that maximizes the cumulative reward; this sequence consists of a series of continuous grid coordinates, serving as the robot's dynamically optimized scan path.

[0051] In this embodiment, the deep Q-network adopts a three-layer fully connected neural network structure. The input layer dimension is the state space dimension of the raster map (i.e., the total number of map grids). The hidden layer contains two fully connected layers, each with 256 and 128 neurons respectively, and the activation function for both layers is the ReLU function. The output layer dimension is the action space dimension (i.e., 8 selectable directions), and it outputs the Q-value corresponding to each action. The network optimizer is Adam, the learning rate is set to 0.001, the experience replay pool capacity is set to 10000, and 64 samples are randomly sampled from the replay pool for batch updates during each training iteration. The target network is updated every 100 steps.

[0052] S24. In the same BIM model, using the design viewpoint origin as the light source, simulate the process of light projection and reflection based on the Whitted style ray tracing algorithm, eliminate the ghost images and noise caused by material occlusion, generate high-fidelity, noise-free design point cloud data, and convert it to the same coordinate system as the scanning path.

[0053] Within the same BIM model, using the design viewpoint origin as the location of the virtual light source, a Whitted-style ray tracing algorithm is employed to simulate the projection and reflection of light. Whitted-style ray tracing is a classic computer graphics rendering algorithm that generates highly realistic images by tracing the path of light rays from the viewpoint through the scene, calculating the intersection points of light rays with object surfaces, and the lighting effects. During this process, the algorithm eliminates ghosting and noise caused by material occlusion, ensuring that only ray intersections reflecting the true geometry are retained. This ultimately generates high-fidelity, noise-free design point cloud data, which precisely corresponds to the external surface geometry of each component in the BIM model. This design point cloud data is then converted to the same coordinate system as the scanning path to ensure consistency in spatial reference when comparing it with the measured point cloud later.

[0054] Furthermore, in this embodiment, step S24 specifically includes:

[0055] S241. Traverse the design surface mesh of each building component in the BIM model and determine its optical material properties. The optical material properties include at least the diffuse reflection coefficient, specular reflection coefficient and transparency.

[0056] First, the system iterates through the design surface mesh of each building component in the BIM model. The design surface mesh refers to the mesh structure in the BIM model where the outer surface of a component is discretized into a grid composed of numerous small triangles or quadrilaterals, used to accurately describe the component's geometry. For each mesh surface, the system determines its optical material properties. These properties include at least the diffuse reflection coefficient (describing the surface's ability to uniformly scatter incident light in all directions), the specular reflection coefficient (describing the surface's ability to reflect incident light directionally like a mirror), and the transparency (describing the degree to which light penetrates the surface). These property values ​​are set according to the actual material of the component (such as concrete, brick wall, glass, metal, etc.), providing a physical basis for subsequent ray tracing calculations.

[0057] S242. Starting from a preset virtual camera viewpoint, emit an initial ray of light to each pixel on the image plane, and based on the Whitted recursive ray tracing model, track the reflection and refraction events that occur when the initial ray encounters the surface of building components with different optical material properties.

[0058] Starting from a preset virtual camera viewpoint, an initial ray is emitted towards each pixel on the image plane. The virtual camera viewpoint is equivalent to setting an observer position in 3D space, and the image plane is equivalent to the imaging screen in front of the observer. The Whitted recursive ray tracing model is used to trace each ray. This model is a classic ray tracing algorithm whose core feature is that when a ray encounters an object's surface, it recursively generates reflected and refracted rays based on the surface material properties, continuing to propagate in the scene. The system continuously tracks the reflection and refraction events that occur when the ray encounters the surfaces of architectural components with different optical material properties, recording the intersection information of each ray with the component surface.

[0059] S243. When the light intersects the surface of the component, the light energy attenuation is calculated based on the angle between the light and the normal and the optical material properties. Invalid light rays with energy lower than a preset threshold due to multiple reflections are eliminated to remove redundant noise.

[0060] When a ray intersects a component surface, the system calculates the energy attenuation at the intersection point based on the angle between the incident ray and the surface normal, as well as the optical material properties of the surface. Ray energy attenuation refers to the reduction in ray intensity during reflection or refraction due to surface absorption, scattering, and other factors. For invalid rays whose energy has fallen below a preset threshold after multiple reflections, the system discards them and stops tracking them. This process effectively removes redundant noise generated by repeated ray bounces in the scene, avoiding the generation of point clouds containing a large number of low-energy data points with no practical geometric meaning, thereby improving the purity and signal-to-noise ratio of the final point cloud.

[0061] S244. Perform coordinate transformation and resampling on the set of termination intersection points of all effective rays to generate noise-free design point cloud data with real geometric and physical meaning, and store the design point cloud data as a KD-Tree structure for subsequent point cloud registration and comparison network calls.

[0062] The termination points of all effective rays (i.e., rays with energy above a threshold) are collected to form a set of intersection point coordinates. This set is then transformed from the virtual camera coordinate system to a unified global coordinate system, ensuring it shares the same spatial reference as the actual scanned point cloud. Resampling is then performed, selecting a certain number of points from the intersection point set according to a uniform distribution principle to make the final point cloud density distribution more uniform. These steps generate noise-free design point cloud data with true geometric and physical meaning, accurately reflecting the geometry of the BIM model surface. Finally, this point cloud data is stored as a KD-Tree structure. A KD-Tree is a data structure that partitions points in k-dimensional space, recursively dividing the space into multiple sub-regions, significantly accelerating the search speed for nearest neighbors during subsequent point cloud registration and comparison.

[0063] S3. Control the robot to move along the optimized path, collect noisy actual point cloud data through LiDAR, and simultaneously collect multimodal video streams of construction workers' actions and environmental conditions through law enforcement recorders. Specifically, control the robot to automatically cruise along the optimized path, and collect real-time actual point cloud data of the construction site through LiDAR. This data includes interference information such as environmental noise and construction debris. At the same time, law enforcement recorders worn on the construction workers' heads simultaneously collect first-person perspective video streams of their actions and environmental conditions, providing raw data for subsequent behavioral analysis.

[0064] Specifically, in this embodiment, step S3 includes:

[0065] S31. Based on the dynamically optimized scanning path output in step S23 above, generate a sequence of control commands including speed, acceleration and steering angle, and drive the mobile robot platform to cruise at a constant speed along the path.

[0066] Based on the dynamically optimized scanning path output from the deep Q-network training in step S23, the system generates a sequence of control commands. Each command includes parameters for the robot's speed, acceleration, and steering angle. Speed ​​refers to how fast the robot moves, acceleration refers to how quickly the speed changes, and steering angle refers to the angle at which the robot changes its direction of travel. These commands are sent one by one to the motion controller of the mobile robot chassis, driving the chassis to cruise at a constant speed along the optimized path according to the set motion parameters. Constant speed cruise means that the robot travels at a constant speed to avoid distortion or unevenness in the LiDAR scanning data caused by sudden changes in speed.

[0067] S32. During the cruise, the three-dimensional lidar is controlled to perform a 360° rotation scan at a preset frequency to collect original point cloud frames containing construction site debris, dust and dynamic interference, forming noisy actual point cloud data.

[0068] During the robot's constant-speed cruise, the system controls a 3D lidar mounted on top of the chassis to perform a 360-degree omnidirectional scan at a preset frequency. The lidar measures the distance between itself and surrounding objects by emitting laser beams and receiving reflected signals, thereby generating a series of data points with three-dimensional coordinates. This collection of data points is called a point cloud frame. Due to the complex environment of the construction site, the point cloud inevitably contains point cloud data formed by dynamic interference such as debris, dust, and personnel movement. Therefore, the raw point cloud collected is called noisy actual point cloud data, that is, a real site point cloud containing noise and interference information.

[0069] S33. Simultaneously activate the law enforcement recorder worn on the construction worker's head to collect multimodal video stream data, including RGB images, infrared thermal imaging, and audio signals, from a first-person perspective.

[0070] Simultaneously, the system activates the body cameras worn by construction workers on their heads. Body cameras are wearable audio and video recording devices, typically fixed to safety helmets, recording real-time footage from a first-person perspective (i.e., the perspective seen directly by the wearer). Specifically, the multimodal video stream data collected by the body cameras includes three types: RGB images (i.e., conventional color images), infrared thermal imaging (heat distribution images generated by detecting infrared radiation emitted by objects, reflecting the operating temperature of construction machinery or the heat of hydration of concrete), and audio signals (recording on-site operational sounds, equipment operation sounds, etc.). This multimodal data reflects the operational actions of construction workers and the state of the on-site environment from different dimensions.

[0071] S34. Based on the high-precision clock source inside the mobile robot platform, perform hardware-level timestamp alignment on the noisy actual point cloud data and the multimodal video stream data to establish a spatiotemporal correspondence between point cloud frames and video frames.

[0072] To ensure accurate correspondence between point cloud data and video data during subsequent data analysis, the system uses a high-precision clock source within the mobile robot platform to perform hardware-level timestamp alignment between the noisy actual point cloud data and the multimodal video stream data. The high-precision clock source provides microsecond-level time accuracy, ensuring the accuracy of the timestamps. Timestamp alignment involves marking each frame of point cloud data and each frame of video data with a precise acquisition time and associating them according to their chronological order. After alignment, a spatiotemporal correspondence is established between point cloud frames and video frames, meaning that a point cloud acquired at a certain moment corresponds to a video frame recorded at the same moment. This lays the data foundation for subsequent correlation analysis between geometric deviations and operational behaviors.

[0073] S4. Input the actual point cloud and the design point cloud into a PointNet++-based depth registration network for end-to-end registration, and extract geometric deviation feature vectors through multi-scale feature matching. Specifically, the noisy actual point cloud and the noiseless design point cloud generated by ray tracing are input into the depth point cloud registration network. This network performs end-to-end registration calculations based on the PointNet++ architecture, and eliminates the spatial deviation between the two through a multi-scale feature matching mechanism, extracting geometric feature vectors reflecting construction deviations, including indicators such as flatness, verticality, and dimensional deviation.

[0074] Specifically, in this embodiment, step S4 includes:

[0075] S41. Read the noiseless design point cloud stored in the KD-Tree structure generated in step S244 and the noisy actual point cloud collected in step S32, respectively, and perform voxel grid downsampling and statistical filtering preprocessing on the noisy actual point cloud.

[0076] The system first reads two sets of point cloud data: one is a noise-free design point cloud generated in step S244 and stored in a KD-Tree structure, which serves as an ideal benchmark for quality comparison; the other is noisy actual point cloud data collected in step S32. Since the actual point cloud data is large and contains noise, preprocessing is required. Preprocessing begins with voxel grid downsampling, which divides the 3D space into small cubes (called voxels), retaining only one representative point (such as the centroid or center point) within each voxel, thus significantly reducing the number of points and lowering subsequent computational load. Subsequently, statistical filtering is used. Statistical filtering analyzes the distance distribution between each point and its neighbors, identifying isolated outliers that deviate too far from the average distance as noise and removing them, further improving the point cloud quality.

[0077] S42. Construct a deep registration network based on the PointNet++ architecture. The network includes an encoder-decoder structure, wherein the encoder part extracts local geometric features of the point cloud through a multilayer perceptron (MLP) and constructs a multi-scale hierarchical feature space using the farthest point sampling (FPS) strategy.

[0078] Next, a deep registration network based on the PointNet++ architecture is constructed. PointNet++ is a deep learning network architecture specifically designed for processing 3D point cloud data, capable of directly using the raw point cloud as input without converting it to a mesh or voxel format. This network employs an encoder-decoder structure, where the encoder is responsible for extracting local geometric features of the point cloud layer by layer. In the encoder, a multilayer perceptron (MLP, a feedforward neural network with multiple fully connected layers used to perform nonlinear transformations on the features of each point) extracts features of the coordinates and attributes of each point. Simultaneously, a farthest point sampling (FPS, a sampling strategy that uniformly selects representative points from the point cloud, choosing the point furthest from the already selected point set each time to ensure uniform distribution of sampling points) strategy is used to gradually reduce the number of points, thereby constructing a multi-scale hierarchical feature space, that is, extracting local and global geometric features at different spatial scales.

[0079] In this embodiment, the PointNet++ network employs a three-layer encoder structure: the first layer uses farthest-point sampling to downsample the input point cloud from N points to N / 4 points, with a query radius of 0.1 meters; the second layer further downsamples to N / 16 points, with a query radius of 0.2 meters; and the third layer downsamples to N / 64 points, with a query radius of 0.4 meters. Each encoder layer uses a multilayer perceptron (MLP) to extract features, with the number of channels in the three MLP layers set to [64, 64, 128], [128, 128, 256], and [256, 256, 512], respectively. The decoder recovers the spatial resolution of the point cloud through interpolation and skip connections, ultimately outputting a geometric feature vector for each point, with a feature dimension of 256.

[0080] S43. Input the design point cloud and the actual point cloud into the depth registration network for end-to-end forward propagation. Calculate the corresponding point pair weight matrix between the two point clouds through a soft-matching layer, and solve the rigid body transformation matrix based on singular value decomposition (SVD) to achieve coordinate system alignment between the actual point cloud and the design point cloud.

[0081] The noiseless design point cloud and the noisy actual point cloud are input together into the constructed depth registration network for end-to-end forward propagation computation. End-to-end means that the computation proceeds directly from the original input to the final output, with all intermediate processing steps completed automatically by the network without manual design of intermediate steps. The network calculates the weight matrix of corresponding point pairs between the two point clouds through a soft matching layer. The soft matching layer is a differentiable matching module that calculates a weight value between 0 and 1 for each pair of potential corresponding points, representing the probability that the two points belong to the same physical location, rather than the either-or hard matching used in traditional methods. Based on the calculated weight matrix, Singular Value Decomposition (SVD) is used to solve for the optimal rigid body transformation matrix from the actual point cloud to the design point cloud. This matrix contains rotation and translation components. Applying this transformation matrix to the actual point cloud achieves precise alignment of the coordinate systems of the actual point cloud and the design point cloud.

[0082] Traditional ICP only considers point-to-point distances, neglecting the normal constraints on the component surface. This embodiment proposes a normal consistency weighted Procrustes transformation matrix:

[0083] ,

[0084] Among them, the normal adaptive weight . The standard deviation of the signed distance field reflects the degree of dispersion of local construction deviations. Let R be the rigid body transformation matrix, which includes rotation R and translation t. For the first point cloud One point; To design corresponding points in the point cloud; for Surface normal at the location; for Surface normal at the location; These are the position error weights and normal error weights; The Euclidean distance between the points; This is the preset allowable deviation threshold.

[0085] In the output layer of the deep registration network, the corresponding point pairs obtained from the soft matching layer and the normal direction information calculated by ray tracing in step S242 are jointly input into the weighted Procrustes solver. In each iteration, according to and The ratio dynamically adjusts the normal constraint weight β: the larger the deviation and the larger the local standard deviation, the higher the normal constraint weight.

[0086] Traditional ICP and PointNet++ methods only optimize point cloud distances, which can easily lead to slippage errors at points where the normal changes abruptly, such as wall corners and column edges. This invention is the first in the field of building construction to propose an explicit constraint on normal consistency using Procrustes transformation, and the weights are automatically adjusted using the SDF standard deviation. Practical testing shows that this algorithm reduces the corner registration error of building components (walls, columns, beams) from 2-3 mm in traditional methods to 0.3-0.5 mm, filling the technical gap in point cloud registration that lacks geometric topological constraints.

[0087] S44. In the aligned point cloud space, calculate the signed distance field SDF from each point in the actual point cloud to the surface of the designed point cloud, and combine it with the normal direction information of ray tracing in the previous step S242, and extract the multidimensional geometric deviation feature vector containing flatness deviation, verticality deviation and size deviation through the feature aggregation module.

[0088] After aligning the actual point cloud with the design point cloud in the coordinate system, the signed distance field (SDF) from each point in the actual point cloud to the surface of the design point cloud is calculated. A positive value indicates the point is outside the surface, a negative value indicates the point is inside the surface, and zero indicates the point is on the surface. This allows determination of whether each measured point is convex or concave relative to the design surface. Simultaneously, combining the normal direction information calculated by ray tracing in step S242 (the normal is a direction vector perpendicular to the component surface, used to describe the surface orientation), all these deviation information are summarized and integrated through the feature aggregation module. The feature aggregation module performs a weighted average of the deviation data scattered across various points according to area or volume, ultimately extracting a multi-dimensional geometric deviation feature vector. This vector includes key quality indicators such as flatness deviation (whether the surface is flat), perpendicularity deviation (whether the component is perpendicular to the ground), and dimensional deviation (the difference between the actual and design dimensions).

[0089] S5. After the video stream undergoes behavior recognition processing to obtain the operation behavior feature vector, it is spatiotemporally aligned and tensor-concatenated with the geometric deviation feature vector. This vector is then input into the GBDT classifier, which outputs the probability value of construction quality being qualified and the binary classification result. Specifically, first, the video stream collected by the law enforcement recorder is processed for behavior recognition to extract the operation actions of the construction personnel as feature vectors. Then, these behavior features are spatiotemporally aligned and tensor-concatenated with the geometric deviation features obtained in the previous step to form a comprehensive feature matrix that integrates the two types of information. Finally, this matrix is ​​input into the gradient boosting tree classifier to calculate the probability value of construction quality being qualified and to provide a binary judgment result of whether the construction is qualified or not.

[0090] Specifically, in this embodiment, step S5 includes:

[0091] S51. Frame extraction is performed on the collected multimodal video stream data. A behavior recognition model based on SlowFast dual-stream convolutional neural network is used for processing to extract the temporal changes of key skeletal points of construction workers and tool interaction actions, generating operation behavior feature vectors that represent the standardization of operation, process rhythm and safety compliance.

[0092] The system first extracts frames from the multimodal video stream data collected by the law enforcement recorder, that is, extracting single frames from the continuous video at fixed time intervals, with each frame representing a static scene at a certain moment. These image frames are then input into a behavior recognition model based on a SlowFast dual-stream convolutional neural network for processing. The SlowFast dual-stream network is a deep learning architecture specifically designed for video behavior recognition, containing two parallel branches: the Slow branch processes the video at a lower frame rate to capture static semantic information such as scenes and objects; the Fast branch processes the video at a higher frame rate to capture dynamic change information such as actions and motion.

[0093] In this embodiment, the specific parameters of the SlowFast dual-stream network are as follows: the sampling frame rate of the Slow branch is 2 frames / second, and the time window length is 16 frames; the sampling frame rate of the Fast branch is 16 frames / second, and the time window length is 64 frames. The Slow branch uses ResNet50 as the backbone network, and the Fast branch uses ResNet18 as the backbone network. The lateral connections between the two branches are used to fuse spatiotemporal features. The network output contains a three-dimensional coordinate temporal sequence of 17 skeletal keypoints (keypoints include head, shoulder, elbow, wrist, hip, knee, ankle, etc.), and the temporal features are mapped into a 128-dimensional operational behavior feature vector through a fully connected layer.

[0094] The model extracts the temporal changes of key skeletal points of construction workers from videos (key skeletal points refer to the positions of human joints, such as shoulders, elbows, wrists, knees, and ankles; the movement trajectories of these points reflect the human body's posture) as well as the interactive actions between construction workers and tools (such as gripping, striking, and rotating). Based on these extraction results, an operational behavior feature vector is generated. This vector is used to characterize the standardization of construction operations (whether the actions conform to standard procedures), the rhythm of procedures (the time intervals and smoothness of connection between each procedure), and the safety compliance (whether safety equipment is worn, whether dangerous areas are entered, etc.).

[0095] S52. Based on the established spatiotemporal correspondence, the output geometric deviation feature vector is spatiotemporally aligned with the operation behavior feature vector obtained in step S51, and the temporal offset caused by the difference in sensor sampling frequency is eliminated by linear interpolation.

[0096] Based on the spatiotemporal correspondence established in step S34, the system spatiotemporally aligns the geometric deviation feature vector output in step S44 with the operational behavior feature vector obtained in step S51. Spatiotemporal alignment refers to synchronously matching the two types of features on the time axis, ensuring that the geometric deviation data and behavioral feature data collected at the same time correspond. Since the frequency of point cloud acquisition by LiDAR and the frequency of video acquisition by law enforcement recorders often differ, a difference in sampling frequency leads to inconsistent data point positions on the time axis. To address this issue, the system uses linear interpolation for compensation. Linear interpolation assumes a linear trend in data between two known data points and calculates the data value at the intermediate time. Through interpolation, the temporal offset caused by the different sensor sampling frequencies is eliminated, ensuring accurate temporal correspondence between the two types of features.

[0097] S53. The aligned geometric deviation feature vector and the operational behavior feature vector are tensor-concatenated along the feature dimension to construct a multidimensional fusion feature matrix containing geometric physical quantities and human behavior quantities.

[0098] After completing the spatiotemporal alignment, the aligned geometric deviation feature vector and the operational behavior feature vector are concatenated as tensors along the feature dimension. A tensor is a multidimensional array, a generalization of vectors and matrices in high-dimensional space, where a vector is a one-dimensional tensor and a matrix is ​​a two-dimensional tensor. Tensor concatenation refers to connecting two or more tensors along a specified dimension to form a new tensor containing more feature information. After concatenation, a multidimensional fusion feature matrix is ​​constructed, which simultaneously contains geometric physical quantities (such as flatness deviation, verticality deviation, dimensional deviation, etc., reflecting the quality of the construction entity) and human behavior quantities (such as operational standardization, process rhythm, safety compliance, etc., reflecting human factors), realizing the organic integration of the two types of information: construction results and construction process.

[0099] Furthermore, step S53 specifically includes:

[0100] S531. Normalize the geometric deviation feature vector to obtain a standardized geometric tensor containing flatness deviation, perpendicularity deviation and dimensional deviation.

[0101] First, the geometric deviation eigenvectors are normalized. Normalization is the process of converting data with different units and numerical ranges to a unified scale, eliminating the adverse effects of different units and large differences in numerical ranges of various deviations (such as flatness deviation in millimeters and angle deviation in degrees) on subsequent analysis. After normalization, a standardized geometric tensor is obtained, which contains three core dimensions: flatness deviation (characterizing whether the wall or floor is flat; the larger the value, the more severe the surface undulation), verticality deviation (characterizing whether the column, wall, or other components are perpendicular to the horizontal plane; the larger the value, the more severe the tilt), and dimensional deviation (characterizing the difference between the actual size of the component and the size on the design drawing; the larger the value, the greater the dimensional error). Standardization makes these three types of deviations numerically comparable, facilitating subsequent joint analysis with other features.

[0102] S532. Quantize and encode the operation behavior feature vector to obtain a standardized behavior tensor that includes operation timing compliance, tool holding posture standardization, and process logic correctness.

[0103] Next, the operational behavior feature vectors are quantified and encoded. Quantization and encoding is the process of converting qualitative behavioral descriptions into quantitative values, enabling previously difficult-to-quantify operational behaviors to participate in mathematical calculations in numerical form. After quantization and encoding, a standardized behavioral tensor is obtained, which contains three core dimensions: operational sequence compliance (indicating whether construction procedures are executed in the prescribed order, and whether there are any cases of reversed or skipped procedures), tool holding posture standardization (indicating whether the hand posture of construction workers when using tools conforms to safe operating procedures, such as holding angle, direction of force, etc.), and process logic correctness (indicating whether the logical relationship between each construction step is correct, such as whether tile laying is carried out only after wall leveling). These quantified behavioral features provide a data foundation for subsequent analysis of the correlation between operational behaviors and quality deviations.

[0104] S533. Based on the established spatiotemporal correspondence, the standardized behavior tensor is extended to the same spatial dimension as the standardized geometric tensor through a broadcasting mechanism, generating an intermediate feature tensor with consistent dimensions.

[0105] Based on the spatiotemporal correspondence established in step S34, the system expands the standardized behavior tensor to the same spatial dimension as the standardized geometric tensor through a broadcast mechanism. The broadcast mechanism is an automatic dimension expansion technique in tensor operations. When the dimensions of two tensors are inconsistent, the system automatically copies and expands the smaller tensor along the missing dimension to match its shape with the larger tensor, without explicitly copying the data. Since the standardized behavior tensor originates from video data from law enforcement recorders, its data sampling frequency and spatial distribution differ from the geometric tensor, resulting in a dimensional inconsistency. Expanding the behavior tensor along the temporal and spatial dimensions through the broadcast mechanism generates an intermediate feature tensor with dimensions completely consistent with the geometric tensor, ensuring that the two types of data can be aligned element-wise.

[0106] S534. Concatenate and splice the standardized geometric tensor and the intermediate feature tensor along the feature channel dimension to construct a multi-dimensional fusion feature matrix, which serves as the input feature set for the subsequent GBDT classifier.

[0107] Finally, the standardized geometric tensor and the intermediate feature tensor are concatenated along the feature channel dimension. The feature channel dimension refers to the dimension in a tensor used to store different categories of features, similar to the RGB color channels in image processing. Concatenation combines two tensors end-to-end along the feature channel dimension to form a new tensor with more channels, while preserving the feature information of each of the two original tensors. After concatenation, a multi-dimensional fusion feature matrix is ​​constructed, with the shape (number of samples, number of feature channels), where each sample corresponds to a time point or a spatial location, and each feature channel corresponds to a type of geometric deviation or a type of operational behavior feature. This fusion feature matrix will serve as the input feature set for a subsequent gradient boosting tree classifier, used for training or predicting the probability of construction quality compliance.

[0108] S54. Input the multidimensional fusion feature matrix into the pre-trained gradient boosting tree (GBDT) classifier, calculate the predicted probability value of construction quality qualification through weighted voting of multiple regression trees, and output the binary classification result of "qualified" or "unqualified" according to the preset decision threshold.

[0109] In this embodiment, the training parameters of the GBDT classifier are as follows: the number of decision trees is set to 100, the maximum depth of each tree is set to 5 layers, the learning rate is set to 0.1, the subsampling ratio is set to 0.8 (i.e., 80% of the samples are randomly selected for training from each tree), and the loss function is the log-loss function. The ratio of positive to negative samples (qualified and unqualified) in the training dataset is 1:1, and minority class oversampling is performed using the SMOTE algorithm to balance the class distribution. The decision threshold of the classifier is set to 0.5 by default, and can be dynamically adjusted within the range of 0.3 to 0.7 according to the actual application scenario (such as the allowable deviation requirements of different construction parts).

[0110] The constructed multi-dimensional fusion feature matrix is ​​input into a pre-trained Gradient Boosting Decision Tree (GBDT) classifier for computation. This classifier consists of multiple regression trees, each independently judging the input features and outputting a predicted value. The predicted values ​​from all trees are weighted and voted on according to their respective weights to arrive at the final result. The classifier calculates the predicted probability value for "construction quality is qualified," which is between 0 and 1, representing the likelihood of meeting the quality standards. Finally, based on a pre-set decision threshold (e.g., 0.5), if the probability value is greater than or equal to the threshold, "qualified" is output; otherwise, "unqualified" is output, completing the binary classification judgment of construction quality.

[0111] S6. Based on the probability values, a digital twin is constructed. When the probability value is lower than a threshold, a tiered early warning is triggered. A causal reasoning model is used to analyze the correlation coefficient between operational behavior characteristics and quality defects, automatically locating key process nodes and generating rectification work orders. Specifically, a digital twin model of construction quality is constructed based on the calculated probability values ​​to achieve real-time visualization of the quality status. When the probability value is lower than a preset threshold, a tiered early warning is automatically triggered. Simultaneously, a causal reasoning model is used to analyze the correlation between operational behavior characteristics and quality defects, automatically locating the key processes causing the problems. Based on this, a digital work order containing rectification locations and measures is generated, completing closed-loop management from detection to rectification.

[0112] Specifically, in this embodiment, step S6 includes:

[0113] S61. Based on the established BIM model, construct a digital twin of construction quality, and map the output construction quality qualification probability value and the extracted geometric deviation feature vector to the corresponding components in the BIM model to realize the virtual and real synchronous visualization of quality status.

[0114] A digital twin of construction quality is constructed based on the established BIM model. A digital twin refers to a real-time synchronous model of a physical entity created in virtual space through data mapping. The construction quality pass probability value output in step S54 and the geometric feature vectors such as flatness deviation, verticality deviation, and dimensional deviation extracted in step S44 are mapped one by one to the corresponding components such as walls, columns, beams, and slabs in the BIM model. The quality status of each component is intuitively displayed in different colors in the 3D visualization interface, realizing the synchronous visualization of the virtual and real quality status of the construction site.

[0115] S62. Set three-level early warning threshold ranges, automatically trigger the corresponding level of graded early warning based on the probability value of construction quality being qualified, and push alarm information to the management personnel terminal.

[0116] The system is pre-set with three levels of early warning thresholds, each corresponding to a different level of quality risk. Based on the construction quality pass rate output in step S54, the system automatically determines the current threshold range and triggers the corresponding level of early warning: for low risk, only monitoring data is recorded; for medium risk, a reminder is sent to on-site management personnel; and for high risk, an alarm is immediately issued and relevant responsible persons are notified. All early warning information is pushed to the handheld terminals of management personnel or the monitoring center's large screen via wireless network.

[0117] S63. Construct a causal inference model based on Bayesian network or structural equation model, input the operational behavior feature vector and the geometric deviation feature vector, calculate the conditional probability distribution and Pearson correlation coefficient between the two, and identify the key operational behavior factors that lead to specific quality defects.

[0118] A causal inference model based on Bayesian networks or structural equation modeling is constructed by inputting the operational behavior feature vector extracted in step S51 and the geometric deviation feature vector extracted in step S44 into the model. Bayesian networks are a probabilistic graphical model-based method for uncertainty inference, which describes causal relationships by calculating the conditional probability distributions between variables.

[0119] In this embodiment, the structure of the Bayesian network is determined through a combination of constrained learning and a scoring search method: first, an initial completely undirected graph is constructed based on mutual information testing; then, redundant edges are removed through conditional independence testing; and finally, a hill-climbing algorithm is used to search for the optimal network structure (BIC scoring criterion). The network nodes include: worker operation behavior nodes (such as plastering angle, vibration frequency, operation time, tool type, etc.) and geometric deviation nodes (such as flatness deviation, verticality deviation, dimensional deviation, etc.). The conditional probability table is learned from historical data using maximum likelihood estimation. For continuous variables (such as plastering angle), a Gaussian distribution is used to fit their conditional probability density function. As new data accumulates, the conditional probability table is dynamically adjusted using an incremental parameter update method, eliminating the need for complete retraining. Structural equation modeling, as an alternative to Bayesian networks, is used in scenarios where all variables are continuous and there is a clear linear assumption; its path coefficients are estimated using partial least squares.

[0120] The model calculates the Pearson correlation coefficient (a statistical indicator that measures the degree of linear correlation between two variables, with a value between -1 and 1) between operational behavior characteristics and quality defects, and identifies key operational behavior factors that lead to specific quality defects, such as non-standard plastering techniques that play a major role in wall flatness deviations.

[0121] The conditional probabilities of Bayesian networks only express correlation and cannot distinguish causal direction. This embodiment introduces a joint statistic of normalized mutual information and transition entropy:

[0122] ,

[0123] in, Mutual information measures the nonlinear correlation between behavior and subsequent defects. ; , For information entropy, ; For behavior To defects The transfer entropy, For reverse transfer entropy, from defects To behavior Targeted information flow; ; , indicating behavior For cause, defect The causal strength coefficient of the effect; For time t, the characteristic values ​​of the operation behavior (such as plastering angle, vibration frequency); This represents the geometric deviation characteristic value (such as flatness deviation) at time t+1.

[0124] In the causal reasoning model of step S63, the time-aligned sequence of operational behavior features is... and geometric deviation characteristic sequence Input the causal calculation module. First, calculate mutual information and conditional entropy according to information theory formulas to obtain dimensionless normalized mutual information, then calculate the bidirectional transfer entropy. Finally, uniquely determine the causal direction through the difference in the numerator: a positive value indicates... lead to Negative values ​​indicate lead to The denominator is normalized using the sum value to ensure that the coefficient range is [-1, 1].

[0125] Traditional Pearson correlation coefficients only measure linear correlations and lack directionality, while conditional probabilities in Bayesian networks require pre-assuming a directed acyclic graph (DAG) structure, introducing subjectivity. This invention, for the first time, introduces a joint statistic based on normalized mutual information and transfer entropy from information theory into construction quality assessment, automatically identifying causal directions: the causal strength between plastering angle and flatness deviation can reach 0.73, and the causal strength between vibration frequency and honeycomb-like surface defects can reach 0.68. Compared to the Pearson coefficient, this method retains only true causal pairs and eliminates spurious correlations, achieving a fundamental leap from "correlation analysis" to "causal inference."

[0126] S64. Based on the sorting results of the correlation coefficient, reverse index to the construction log and BIM schedule, automatically locate the key process nodes that cause quality problems, and call the preset rectification rule library to generate a digital rectification work order containing rectification location, problem description and suggested measures, so as to realize closed-loop management of quality problems.

[0127] Based on the ranking of correlation coefficients, the key operational behavior factor with the highest value is identified as the primary cause. This is then used to reverse-index the construction logs (recording the work time, operators, and equipment used for each process) and the BIM schedule (including the planned start and end times of each process and their dependencies), automatically locating the specific construction process node and corresponding time point that caused the quality problem. Subsequently, a pre-set rectification rule library is invoked. This rule library pre-writes corresponding rectification measures for different types of quality defects, automatically generating a digital rectification work order containing the rectification location, problem description, and suggested measures. This work order is then dispatched to the responsible personnel, and the rectification progress is tracked, achieving closed-loop management of quality issues.

[0128] Example 2

[0129] like Figure 2 As shown, this application provides an architecture diagram of a refined construction quality management system based on a portable holographic scanner and law enforcement recorder, which is applied to the method described in Embodiment 1. The system includes: a mobile robot platform 210, a path planning and design point cloud generation module 220, a data acquisition module 230, a point cloud registration and feature extraction module 240, a behavior recognition and quality classification module 250, and a digital twin and early warning management module 260.

[0130] Mobile robot platform 210, equipped with lidar and IMU, is used to acquire pose information based on tightly coupled SLAM and import it into BIM model;

[0131] The path planning and design point cloud generation module 220 is used to construct an MDP model based on BIM topology information, dynamically optimize the scanning path using the DQN algorithm, and generate a noise-free design point cloud based on the same BIM model through ray tracing.

[0132] The data acquisition module 230 is used to control the robot to move along the optimized path, collect noisy actual point cloud data through lidar, and simultaneously collect multimodal video streams of construction personnel's operation actions and environmental conditions through law enforcement recorder;

[0133] The point cloud registration and feature extraction module 240 is used to perform end-to-end registration of the actual point cloud and the design point cloud by inputting the PointNet++-based deep registration network, and extract the geometric deviation feature vector through multi-scale feature matching.

[0134] The behavior recognition and quality classification module 250 is used to obtain the operation behavior feature vector after the video stream is processed by behavior recognition, and to perform spatiotemporal alignment and tensor concatenation with the geometric deviation feature vector. The input is then input into the GBDT classifier, and the output is the probability value of the construction quality being qualified and the binary classification result.

[0135] The digital twin and early warning management module 260 is used to construct a digital twin based on the probability value. When the probability value is lower than the threshold, a graded early warning is triggered. The module analyzes the correlation coefficient between operational behavior characteristics and quality defects through a causal reasoning model, automatically locates key process nodes, and generates rectification work orders.

[0136] Figure 3 This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 301 and memory 300, communication interface 303, and bus 302.

[0137] In this embodiment of the application, memory 300 is used to store executable instructions of processor 301, which, when configured to execute instructions, implements the method as described in the first aspect.

[0138] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0139] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0140] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0141] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0142] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0143] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0144] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for refined construction quality management based on portable holographic scanning and law enforcement recorders, characterized in that, Includes the following steps: A mobile robot platform equipped with LiDAR and IMU was constructed, and pose information was acquired based on tightly coupled SLAM and imported into the BIM model; MDP model is constructed based on BIM topology information, and DQN algorithm is used to dynamically optimize the scanning path. At the same time, noiseless design point cloud is generated based on the same BIM model through ray tracing. The robot is controlled to move along an optimized path, and noisy actual point cloud data is collected by LiDAR. Simultaneously, a multimodal video stream of the construction workers' actions and the environmental conditions is collected by the law enforcement recorder. The actual point cloud and the design point cloud are input into a PointNet++-based deep registration network for end-to-end registration, and geometric deviation feature vectors are extracted through multi-scale feature matching. After the video stream is processed by behavior recognition, the operation behavior feature vector is obtained. It is then spatiotemporally aligned and tensor-concatenated with the geometric deviation feature vector. The result is input into the GBDT classifier and outputs the probability value of the construction quality being qualified and the binary classification result. A digital twin is constructed based on the probability value. When the probability value is lower than the threshold, a graded warning is triggered. The correlation coefficient between operational behavior characteristics and quality defects is analyzed through a causal reasoning model, and key process nodes are automatically located and rectification work orders are generated.

2. The method according to claim 1, characterized in that, The construction of a mobile robot platform equipped with LiDAR and IMU, which acquires pose information based on tightly coupled SLAM and imports it into a BIM model, specifically includes: A differential drive mobile robot chassis is constructed, a three-dimensional LiDAR is installed on the top of the chassis, and a high-precision inertial measurement unit and an embedded computing unit are installed inside the chassis to form an integrated mobile scanning hardware platform. The point cloud data stream output by the lidar and the angular velocity and acceleration data stream output by the IMU are collected. The data streams are then jointly optimized using a tightly coupled SLAM algorithm based on error state Kalman filtering to solve the six-degree-of-freedom pose information of the mobile robot platform in the global coordinate system. The pose information and the BIM model of the construction object are processed using a unified coordinate system, and the BIM model under the unified coordinate system is imported into the memory space of the embedded computing unit as the basic reference model for subsequent path planning and scanning comparison.

3. The method according to claim 1, characterized in that, The process of constructing an MDP model based on BIM topology information, dynamically optimizing the scan path using the DQN algorithm, and simultaneously generating a noise-free design point cloud based on the same BIM model through ray tracing specifically includes: The BIM model of the construction object is analyzed, the geometric boundaries and spatial connectivity of the building components are extracted, a topology map containing obstacle areas, reachable areas and target scanning areas is constructed, and the topology map is discretized into a raster map. A Markov decision process model is constructed using the robot's position in the grid map as the state space, the eight adjacent directions of the robot's movement as the action space, and a weighted function of scan coverage and path length as the reward function. A deep Q-network is initialized and trained interactively with the Markov decision process model through an ε-greedy strategy. The output scan path sequence that maximizes the cumulative reward is used as the robot's dynamically optimized scan path. In the same BIM model, with the design viewpoint origin as the light source, the ray tracing algorithm based on Whitted style is used to simulate the process of light projection and reflection, eliminate the ghost images and noise caused by material occlusion, generate high-fidelity, noise-free design point cloud data, and convert it to the same coordinate system as the scanning path.

4. The method according to claim 3, characterized in that, The Whitted-style ray tracing algorithm simulates the process of light projection and reflection, eliminating ghosting and noise caused by material occlusion, specifically including: Traverse the design surface mesh of each building component in the BIM model to determine its optical material properties, which include at least diffuse reflection coefficient, specular reflection coefficient and transparency. Starting from a preset virtual camera viewpoint, an initial ray is emitted toward each pixel on the image plane, and based on the Whitted recursive ray tracing model, the reflection and refraction events that occur when the initial ray encounters the surface of building components with different optical material properties are tracked. When light intersects the surface of a component, the energy attenuation of the light is calculated based on the angle between the light and the normal and the optical material properties. Invalid light rays with energy below a preset threshold caused by multiple reflections are eliminated to remove redundant noise. The set of all effective light rays' termination intersection points is subjected to coordinate transformation and resampling to generate noise-free design point cloud data with real geometric and physical meaning. The design point cloud data is then stored in a KD-Tree structure for subsequent point cloud registration and comparison network calls.

5. The method according to claim 3, characterized in that, The controlled robot moves along an optimized path, collects noisy actual point cloud data via LiDAR, and simultaneously collects multimodal video streams of construction personnel's actions and environmental conditions via a law enforcement recorder, specifically including: Based on the dynamically optimized scanning path, a sequence of control commands including speed, acceleration, and steering angle is generated to drive the mobile robot platform to cruise at a constant speed along the path. During the cruise, the three-dimensional lidar is controlled to perform a 360-degree rotation scan at a preset frequency to collect original point cloud frames containing construction site debris, dust and dynamic interference, forming noisy actual point cloud data. Simultaneously activate the law enforcement recorder worn on the construction worker's head to collect multimodal video stream data, including RGB images, infrared thermal imaging, and audio signals, from a first-person perspective; Based on the high-precision clock source inside the mobile robot platform, hardware-level timestamp alignment is performed on the noisy actual point cloud data and multimodal video stream data to establish a spatiotemporal correspondence between point cloud frames and video frames.

6. The method according to claim 5, characterized in that, The process of inputting the actual point cloud and the design point cloud into a PointNet++-based depth registration network for end-to-end registration, and extracting geometric deviation feature vectors through multi-scale feature matching, specifically includes: Read the noiseless design point cloud of the KD-Tree structure generated by the method according to claim 4 and the noisy actual point cloud collected by the method according to claim 5 respectively, and perform voxel grid downsampling and statistical filtering preprocessing on the noisy actual point cloud. A deep registration network based on the PointNet++ architecture is constructed. The network includes an encoder-decoder structure, wherein the encoder part extracts local geometric features of the point cloud through a multilayer perceptron and constructs a multi-scale hierarchical feature space using the farthest point sampling strategy. The design point cloud and the actual point cloud are input into the depth registration network for end-to-end forward propagation. The corresponding point pair weight matrix between the two point clouds is calculated through the soft matching layer, and the rigid body transformation matrix is ​​solved based on singular value decomposition to achieve coordinate system alignment from the actual point cloud to the design point cloud. In the aligned point cloud space, the signed distance field from each point in the actual point cloud to the surface of the designed point cloud is calculated, and combined with the normal direction information of ray tracing obtained according to claim 4, a multidimensional geometric deviation feature vector containing flatness deviation, verticality deviation and size deviation is extracted through the feature aggregation module.

7. The method according to claim 1, characterized in that, The process involves processing the video stream through behavior recognition to obtain an operation behavior feature vector, spatiotemporally aligning it with the geometric deviation feature vector, and concatenating them using tensors. This data is then input into a GBDT classifier, which outputs a probability value indicating that the construction quality is qualified and a binary classification result. Specifically, this includes: Frames of the collected multimodal video stream data are extracted and processed using a behavior recognition model based on a SlowFast dual-stream convolutional neural network. The temporal changes of key skeletal points and tool interaction actions of construction workers are extracted to generate operation behavior feature vectors that represent the standardization of operations, the rhythm of procedures, and safety compliance. Based on the established spatiotemporal correspondence, the output geometric deviation feature vector is spatiotemporally aligned with the operation behavior feature vector, and the timing offset caused by the difference in sensor sampling frequency is eliminated by linear interpolation. The aligned geometric deviation feature vector and the operational behavior feature vector are tensor-concatenated along the feature dimension to construct a multi-dimensional fusion feature matrix containing geometric physical quantities and human factor behavior quantities. The multidimensional fusion feature matrix is ​​input into a pre-trained gradient boosting tree classifier. The predicted probability value of construction quality qualification is calculated by weighted voting of multiple regression trees, and a binary classification result of qualified or unqualified is output according to a preset decision threshold.

8. The method according to claim 7, characterized in that, The step of tensor concatenating the aligned geometric deviation feature vector and the operational behavior feature vector along the feature dimension to construct a multidimensional fusion feature matrix containing geometric physical quantities and human factor behavior quantities specifically includes: The geometric deviation feature vector is normalized to obtain a standardized geometric tensor that includes flatness deviation, perpendicularity deviation and dimensional deviation. The operation behavior feature vector is quantized and encoded to obtain a standardized behavior tensor that includes operation timing compliance, tool holding posture standardization, and process logic correctness. Based on the established spatiotemporal correspondence, the standardized behavior tensor is extended to the same spatial dimension as the standardized geometric tensor through a broadcasting mechanism, generating an intermediate feature tensor with consistent dimensions. The standardized geometric tensor and the intermediate feature tensor are concatenated and spliced ​​along the feature channel dimension to construct a multi-dimensional fused feature matrix, which serves as the input feature set for the subsequent gradient boosting tree classifier.

9. The method according to claim 1, characterized in that, The process involves constructing a digital twin based on the probability value. When the probability value falls below a threshold, a tiered early warning is triggered. A causal reasoning model is used to analyze the correlation coefficient between operational behavior characteristics and quality defects, automatically locating key process nodes and generating rectification work orders. Specifically, this includes: Based on the established BIM model, a digital twin of construction quality is constructed, and the output construction quality qualification probability value and the extracted geometric deviation feature vector are mapped to the corresponding components in the BIM model to realize the virtual and real synchronous visualization of quality status. Set three-level early warning threshold ranges, automatically trigger the corresponding level of early warning based on the probability value of construction quality meeting the requirements, and push alarm information to the management personnel terminal; Construct a causal inference model based on Bayesian network or structural equation model, input the operational behavior feature vector and the geometric deviation feature vector, calculate the conditional probability distribution and Pearson correlation coefficient between the two, and identify the key operational behavior factors that lead to specific quality defects. Based on the sorting results of the correlation coefficients, the system uses a reverse index to access the construction logs and BIM schedule, automatically locating the key process nodes that cause quality problems. It then calls the preset rectification rule library to generate digital rectification work orders containing rectification locations, problem descriptions, and suggested measures, thereby achieving closed-loop management of quality problems.

10. A refined construction quality management system based on a portable holographic scanner and law enforcement recorder, applied to the method described in any one of claims 1 to 9, characterized in that, The system includes: A mobile robot platform, equipped with LiDAR and IMU, is used to acquire pose information based on tightly coupled SLAM and import it into a BIM model; The path planning and design point cloud generation module is used to construct an MDP model based on BIM topology information, dynamically optimize the scanning path using the DQN algorithm, and generate a noise-free design point cloud based on the same BIM model through ray tracing. The data acquisition module is used to control the robot to move along the optimized path, collect noisy actual point cloud data through LiDAR, and simultaneously collect multimodal video streams of construction personnel's operation actions and environmental conditions through law enforcement recorder; The point cloud registration and feature extraction module is used to perform end-to-end registration of the actual point cloud and the design point cloud by inputting the PointNet++-based deep registration network, and extracts the geometric deviation feature vector through multi-scale feature matching. The behavior recognition and quality classification module is used to process the video stream through behavior recognition to obtain the operation behavior feature vector, perform spatiotemporal alignment and tensor concatenation with the geometric deviation feature vector, input the data into the GBDT classifier, and output the probability value of construction quality qualification and the binary classification result. The digital twin and early warning management module is used to construct a digital twin based on the probability value. When the probability value is lower than the threshold, a graded early warning is triggered. The module analyzes the correlation coefficient between operational behavior characteristics and quality defects through a causal reasoning model, automatically locates key process nodes, and generates rectification work orders.