A construction monitoring method

Abstract

The present application provides a construction monitoring method, comprising: acquiring multi-view video image data of a construction site, fusing the multi-view video image data using an image stitching algorithm to obtain a panoramic three-dimensional scene model of the construction site; acquiring image frames in the panoramic three-dimensional scene model, analyzing the image frames using a pre-trained work type recognition model to obtain position information of workers and equipment of each work type; acquiring the position information of workers of each work type, analyzing the posture of each worker using a pre-trained human posture estimation model to obtain motion status information of each worker; acquiring the position information and motion status information of workers of each work type, and judging the specific working status of workers of each work type in combination with a pre-established work type operation knowledge base.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a construction monitoring method. Background Art

[0002] On construction sites, coordination and collaboration between various types of workers are crucial. Traditional manual oversight models struggle to fully capture real-time on-site dynamics, leading to issues such as poor coordination between different types of workers and cross-task conflicts. While automated monitoring using computer vision technology can capture real-time visual data, accurately identifying the work status of different types of workers from massive amounts of video images and conducting effective logical correlation and analysis remains a pressing technical challenge.

[0003] Specifically, due to the complex and ever-changing construction environment, the clothing, equipment, and materials used by workers in different trades vary greatly, and the work scenes are often interfered by factors such as occlusion and lighting changes, making it difficult to ensure the accuracy of trade identification. In addition, there are complex logical relationships between different trades. How to reasonably infer the progress and connection of each trade based on the identification results, and predict possible conflicts, is also a technical challenge. At the same time, due to the dynamic and uncertain nature of the construction site, the visual monitoring system needs to have a certain degree of adaptability and robustness to cope with various changes and emergencies on site and continuously output reliable monitoring results. This places higher demands on the design and implementation of the algorithm. Summary of the Invention

[0004] The present invention provides a construction monitoring method, which mainly includes:

[0005] Acquire multi-view video image data of the construction site, and fuse the multi-view video image data using an image stitching algorithm to obtain a panoramic three-dimensional scene model of the construction site;

[0006] Acquire image frames in the panoramic three-dimensional scene model, analyze the image frames using a pre-trained work type recognition model, and obtain location information of workers and equipment of each work type;

[0007] Obtaining the position information of the workers of each type of work, analyzing the posture of each worker using a pre-trained human posture estimation model, and obtaining the motion state information of each worker;

[0008] Obtaining the location information and action status information of the workers of each type of work, and combining it with a pre-established work operation knowledge base to determine the specific working status of the workers of each type of work;

[0009] Obtaining the specific working status of workers in each type of work, continuously tracking each worker using a visual tracking algorithm with an attention mechanism, and if occlusion occurs, obtaining contextual information and target features before and after the occlusion, inputting the contextual information and target features into a pre-trained occlusion processing model, re-identifying the target, and restoring the status information;

[0010] Obtain the operation status information of each type of work, use the spatiotemporal data association analysis algorithm to construct a logical dependency relationship diagram between different types of work, and based on the preset work type collaboration rules, determine whether there is a conflict in the connection of different types of work in time and space. If there is a conflict, generate an early warning message and send it to the on-site management personnel terminal;

[0011] The latest operation status information of each type of work at the construction site is obtained, the type of work recognition model and the human posture estimation model are optimized using an incremental learning algorithm, and the occlusion processing model is updated using an online learning algorithm to improve the robustness and generalization ability of the system.

[0012] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0013] The present invention discloses a method for intelligent monitoring of construction sites based on multi-view video images. The method constructs a panoramic three-dimensional scene model through an image stitching algorithm, analyzes the status of workers using work type recognition and human posture estimation models, and determines the specific work status in combination with a work type knowledge base. A visual tracking algorithm with an attention mechanism is used to continuously monitor each type of work, and an occlusion processing model is used to solve the occlusion problem. Logical dependencies between types of work are constructed through spatiotemporal data association analysis, and conflicts are judged and warnings are generated based on preset rules. The present invention also uses incremental learning and online learning algorithms to continuously optimize each model to improve the robustness and generalization ability of the system. The method realizes all-round intelligent monitoring of the construction site, effectively improving the efficiency and safety of construction management. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The present invention is a flow chart of a construction monitoring method.

[0015] Figure 2 It is a schematic diagram of a construction monitoring method of the present invention.

[0016] Figure 3 This is another schematic diagram of a construction monitoring method of the present invention. DETAILED DESCRIPTION

[0017] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0018] like Figure 1-3 In this embodiment, a construction monitoring method may specifically include:

[0019] S101 , obtaining multi-view video image data of a construction site, and fusing the multi-view video image data using an image stitching algorithm to obtain a panoramic three-dimensional scene model of the construction site.

[0020] By arranging multiple video acquisition devices at the construction site, video image data from different angles of the construction site are obtained; the obtained multi-angle video image data are preprocessed, including using median filtering to denoise the image and using the histogram equalization algorithm to enhance the image to improve the image quality and prepare for subsequent image stitching; according to the preset image stitching rules, the preprocessed multi-angle video images are stitched using the SIFT feature extraction and RANSAC feature matching algorithms to obtain a preliminary panoramic image; wherein the SIFT algorithm is used to extract the scale-invariant features of the image, and the RANSAC algorithm is used to eliminate erroneous feature matching points to improve the stitching accuracy; for the preliminary panoramic image obtained by stitching, the SIFT feature extraction and RANSAC feature matching are used to optimize the stitching effect of the panoramic image, eliminate stitching gaps and overlapping areas, and obtain a seamless high-quality panoramic image; based on the 3D laser scanning data collected at the construction site, the 3D spatial information of the construction site is obtained; using point cloud data processing software such as CloudCompare, the 3D laser scanning data is filtered and registered to obtain a regular 3D point cloud model; based on the 3D point cloud model, the 3D point cloud model is processed by De L The Aunay triangulation algorithm is used to reconstruct the surface of the point cloud and generate a three-dimensional mesh model of the construction site; the quality and detail of the three-dimensional model are optimized through mesh simplification and subdivision operations; texture mapping technology is used to map the optimized panoramic image as a texture to the surface of the three-dimensional mesh model, and through UV unfolding and texture coordinate mapping, an accurate correspondence between the panoramic image and the three-dimensional model is achieved, thereby improving the realism and visual effect of the scene model; the generated panoramic three-dimensional scene model of the construction site is stored and managed, and a multi-level Level of Detail (LOD) model is constructed using the 3DTi l es three-dimensional data organization format to achieve efficient loading and rendering of large-scale three-dimensional scenes; based on Web three-dimensional visualization technology, a three-dimensional visualization platform for construction sites is developed for use in construction monitoring and progress tracking applications; the Web three-dimensional visualization technology includes WebGL and Three.js.

[0021] For example, eight high-definition cameras were deployed at the construction site, capturing video images from different angles. The captured video images were preprocessed using a 5x5 median filter to denoise the images, and histogram equalization was used to stretch the image grayscale range to 0-255 to enhance image contrast. Based on pre-set image stitching rules, SIFT features were extracted from the images, and RANSAC matching was performed to remove incorrect matching points. The images were then stitched together to form a preliminary panoramic image. SIFT features were extracted again from the stitched image and RANSAC matching was performed. The pixel values ​​of the stitched areas were weighted and fused to eliminate stitching gaps, resulting in a seamless, high-quality panoramic image. Point cloud data captured by a 3D laser scanner was noise-filtered and registered using CloudCompare software. A 3D mesh model of the construction site was reconstructed using the Delaunay triangulation algorithm. Edge collapse and loop subdivision algorithms were used to simplify and refine the mesh model, optimizing model quality. The panoramic image was then mapped onto the 3D model surface using UV unwrapping to achieve precise image-to-geometric mapping. Finally, the panoramic 3D model was stored in 3D ile es format, a multi-level LOD model was constructed, and a Web 3D visualization platform was developed based on WebGL and Three.js to achieve panoramic roaming and interactive monitoring of the construction site.

[0022] S102: Acquire image frames in the panoramic three-dimensional scene model, analyze the image frames using a pre-trained work type recognition model, and obtain location information of workers and equipment of various work types.

[0023] Image frames from a panoramic 3D scene model are acquired and analyzed using a pre-trained YOLOv5 object detection model to obtain the location information of workers and equipment for each type of work within the image frames. A 3D spatial coordinate system is constructed based on the acquired panoramic 3D scene model. The location information of workers and equipment identified in the image frames is mapped to the 3D spatial coordinate system to obtain the spatial coordinates of the workers and equipment within the 3D scene. The Euclidean distance between the 3D coordinates of the workers and equipment is calculated, and a preset safety distance threshold is obtained. A determination is made as to whether the calculated Euclidean distance is less than the safety distance threshold. If so, the worker and equipment are considered to pose a safety hazard. Based on the location coordinates of workers of each type of work, a K-means clustering algorithm is used to cluster workers of different types of work to determine clustering areas for workers of each type of work. A preset threshold for the number of people in each cluster is obtained, and the number of workers within each clustered area is counted. A determination is made as to whether the number of workers exceeds the threshold. If so, the clustered area is considered to pose a safety hazard due to the concentration of workers of that type of work. Based on the workers' positional trajectories, a Kalman filter algorithm is used to track their movements in real time to obtain their real-time location information. The safety hazard information between workers and equipment, the gathering area information of workers of various types, and the safety hazard information of gathering areas exceeding the number of people are integrated with the real-time location information of workers of various types, and visualized through a 3D visualization engine to provide intuitive decision-making support for safety management on the construction site.

[0024] For example, a frame is extracted from the panoramic 3D scene model every second, with a resolution of 1920x1080 pixels per frame. Object detection is performed on each frame using the YOLOv5s model pre-trained on the COCO dataset, with a detection threshold of 6 and an NMS threshold of 5. The YOLOv5s model, consisting of a CSPDarknet53 backbone network and a PANet backbone network, can detect 80 common objects in real time. Detection results show that eight different types of workers, including electricians, welders, and scaffolders, as well as five types of construction equipment, such as tower cranes and excavators, were identified at the construction site. Based on the coordinate information of the panoramic 3D model, the bounding box centers of the workers and equipment detected in the image coordinate system were mapped to a 3D spatial coordinate system to obtain their X, Y, and Z coordinates. The Euclidean distance between each worker and adjacent equipment was calculated, with a safety distance threshold set to 2 meters. If the distance is less than 2 meters, a safety hazard is identified and flagged. The spatial coordinates of the workers were clustered using the K-means clustering algorithm. The optimal number of clusters, K = 5, was selected based on the elbow rule, dividing the workers into five clustering areas. The number of people in each area was counted, and the threshold was set to 10. Areas with more than 10 people were identified as illegal clusters and a warning message was generated. The Kalman filter algorithm was applied to track the position of each worker, establishing a motion model with a matching probability of 8. The worker's position coordinates at the next moment were predicted in real time, matched with the detection results of the new frame, and the worker's motion trajectory was updated. Finally, various safety hazard information and worker trajectories were integrated into a 3D visualization engine, providing a panoramic roaming display of the construction site. Areas with safety hazards were marked with red warning boxes, allowing managers to intuitively understand the safety situation at the construction site.

[0025] S103, obtaining the position information of the workers of each type of work, using a pre-trained human posture estimation model to analyze the posture of each worker, and obtaining the motion state information of each worker. Specifically, the following steps are included:

[0026] S401. Obtain the real-time position coordinate information of workers of various types of work, pre-process the position coordinate information, and convert it into a format suitable for input into the human posture estimation model; S402. Input the pre-processed position coordinate information into the pre-trained OpenPose human posture estimation model, and analyze the input position coordinate information through the model to obtain the human body key point coordinate information of each worker; S403. Calculate the Euclidean distance and angle feature parameters between each key point based on the obtained human body key point coordinate information, and construct the posture feature vector of the worker; S404. Input the constructed posture feature vector into the pre-trained support vector machine (SVM) posture classification model, and obtain the human body key point coordinate information of each worker through model reasoning. Specific action status category; S405, conduct frequency statistics on the action status category of each operator to obtain the overall action status distribution of operators of each type of work; S406, judge whether there are abnormal actions or dangerous actions in the action status distribution of operators of each type of work according to the preset abnormal action threshold and dangerous action rules, and trigger the early warning mechanism if so; S407, continuously track the position changes of operators, update the position coordinate information in real time, and repeat steps S402 to S406 to realize real-time monitoring of operators' actions and risk early warning; S408, use visualization tools to display the action status distribution of operators in real time, and generate action status monitoring reports to provide decision support for managers.

[0027] For example, the real-time position coordinates of workers in various trades are first obtained from a panoramic 3D scene model and converted into the input format required by the OpenPose human pose estimation model. Specifically, the worker's position coordinates are sampled at a frequency of 5 frames per second, their X, Y, and Z coordinates are extracted, and the coordinates are normalized to the range [-1, 1]. The pre-processed position coordinates are then input into an OpenPose model pre-trained on the COCO dataset. This model uses the VGG-19 network as its backbone network, which can detect 18 key points of the human body from the input image, including the top of the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. Through forward propagation, the model outputs the coordinates of the 18 key points for each worker. Next, based on the obtained key point coordinates, feature parameters such as the Euclidean distance and angle between the key points are calculated. For example, the Euclidean distance between the left shoulder and left elbow and the cosine of the angle between the left thigh and left calf are calculated, constructing a pose feature vector containing 36 elements. This feature vector is input into an SVM multi-classification model using a Gaussian kernel function, and the operator's action state category, such as standing, bending, kneeling, and sitting cross-legged, is obtained through a one-vs-one voting strategy. The frequency of each operator's action state category in the past 30 seconds is counted to obtain a histogram of the overall action state distribution for each type of work. If the frequency of abnormal actions (such as sitting for a long time) of a single operator exceeds 10 times, or if three consecutive operators perform the same dangerous action (such as climbing), an early warning prompt is triggered. The system updates the operator's position coordinates every minute and repeats the above posture analysis process to achieve real-time monitoring of the operator's actions. Finally, the operator's action state is rendered in real time into a panoramic scene through a three-dimensional visualization engine, and a monitoring report containing a pie chart of the action distribution of each type of work and abnormal action warning information is generated and uploaded to the construction management system for management personnel to review.

[0028] S104: Obtain the position information and action status information of the workers of each type of work, and determine the specific working status of the workers of each type of work in combination with a pre-established work operation knowledge base.

[0029] Real-time location coordinates of workers in various trades are collected using the Global Positioning System (GPS) or indoor positioning systems (such as UWB and RFID). Acceleration and angular velocity data are also collected during the work process using inertial measurement unit (IMU) sensors worn by the workers. The collected location coordinates and IMU motion data are synchronized in time and space, aligned, and cleaned and normalized to produce time series data in a unified format. The preprocessed location and motion time series data are then input into a pre-established knowledge base for trade operations for comparison and analysis. This knowledge base contains standard motion templates and location areas for each trade, as well as sequence constraints for different processes. The dynamic time warping (DTW) algorithm is used to calculate the similarity between the worker's actual motion sequence and the standard motion template. The system then determines whether the worker's location coordinate sequence falls entirely within the standard operating area for the corresponding trade. If both the position and motion match exceed a preset threshold, the worker is considered to be in a normal working state. Otherwise, the worker is considered to be in an abnormal state, and a warning alert is issued based on the type of abnormality (e.g., non-standard motion, out-of-bounds position, etc.). Based on the operator's position change trajectory and action change sequence, the Hidden Markov Model (HMM) algorithm is used to identify the process and determine the specific work process the operator is currently performing. Combined with the standard working hours for different processes of each type of work, the weighted sum of the operator's working time on each process is taken to obtain a comprehensive workload index. The actual workload of the operator is compared with the expected workload, and the operator's work efficiency level is calculated based on the difficulty coefficient of each type of work. For personnel with low efficiency or insufficient workload, improvement suggestions are pushed through the mobile app. For personnel who have repeatedly encountered abnormal conditions, the supervisor is notified by text message or phone call to intervene and handle the situation. The abnormal conditions and handling results are recorded for performance appraisal and optimization and improvement.

[0030] For example, GPS positioning modules and UWB positioning tags are installed on workers' helmets and work clothes to collect workers' real-time location coordinates at a frequency of once per second. Simultaneously, workers wear smart watches equipped with built-in triaxial accelerometers and triaxial gyroscopes, which collect acceleration and angular velocity data at a frequency of 50 Hz during work. GPS coordinates, UWB coordinates, and IMU data are synchronized using timestamps and uniformly converted into three-dimensional coordinates in the ENU coordinate system with the southwest corner of the construction site as the origin. Kalman filtering is performed on the collected data to remove high-frequency noise and outliers, and the data is scaled to the [0, 1] interval using the minimum-maximum normalization method. Next, the preprocessed position and motion time series data is input into a work knowledge base built on a Neo4j graph database. This knowledge base stores standard motion templates and location area information for 10 major work types, such as rebar workers, carpenters, and concrete workers, as well as the timing constraints for 30 processes, such as pouring, vibrating, and curing. By calculating the Euclidean distance and the dynamic time warping algorithm, the similarity score between the worker's actual action sequence and the standard template is obtained. The Ray Casting algorithm is also used to determine whether the worker's position coordinates are within the polygonal work area and output a Boolean value. If the action similarity score is greater than 85 and the location Boolean value is True, the worker is considered to be in a normal working state. Otherwise, the worker is considered to be in an abnormal state. If the action score is less than 7, a "non-standard action" message is displayed, and if the location Boolean value is False, a "position out of bounds" message is displayed. The worker's position and action sequences for the last 30 minutes are then input into a pre-trained hidden Markov model and decoded using the Viterbi algorithm to obtain the most likely process state sequence. Based on the process duration quotas in the knowledge base, the exponentially weighted average method is used to calculate the worker's work time for each process. Assuming the quota for the pouring process is 5 hours / cubic meter, and the worker poured a total of 7 cubic meters of concrete, spending 6 hours actually, their efficiency coefficient for the pouring process is 5 × 7 ÷ 6 = 125. Finally, the efficiency coefficients of each process are weighted and averaged, with weights determined by the difficulty coefficient of the work type, to determine the worker's overall work efficiency level. If the work efficiency falls below 8, improvement suggestions for improving efficiency are pushed to the worker via the mobile app. If the individual's work efficiency falls below 6 and position and movement anomalies occur more than three times, an alert text message is sent to the project manager, and the assessment score is recorded in the performance management system.

[0031] S105. Obtain the specific working status of the workers in each type of work, and use a visual tracking algorithm with an attention mechanism to continuously track the workers in each type of work; if occlusion occurs, obtain the context information and target features before and after the occlusion, input the context information and target features into a pre-trained occlusion processing model, re-identify the target and restore the status information.

[0032] Acquire real-time video image data of workers in various types of work, and preprocess the image data, wherein the preprocessing includes image denoising and background removal to obtain preprocessed image data;

[0033] Performing target detection on the preprocessed image data using the YOLOv5 algorithm based on the attention mechanism to identify the operator target in the image data;

[0034] The DeepSORT algorithm is used to track the operator target to obtain the motion trajectory and status information of the operator target;

[0035] If occlusion occurs during the tracking process, context information and target features within a certain time range before and after the occlusion occurs are obtained, and the context information and target features are input into the pre-trained MaskRCNN model for processing to re-identify the occluded target;

[0036] Based on the re-identified target, combined with the state information before and after the occlusion, the state change information of the target during the occlusion period is restored through the LSTM time series prediction model;

[0037] Fusing the recovered target state information with the state information before and after the occlusion to obtain the complete state trajectory information of the target;

[0038] Analyzing the state trajectory information of the operator target, calculating the speed and acceleration of the trajectory, and determining whether the speed and acceleration exceed a preset normal range threshold. If the threshold is exceeded, it is determined to be an abnormal state and trigger an early warning;

[0039] Comparing the status information of the operator target with the work standards of the type of work to which the operator belongs that have been pre-entered into the system, determining the similarity between the status information and the work standards by calculating the Euclidean distance, and judging whether the work behavior of the operator target meets the standard requirements;

[0040] During the comparison process, the DTW dynamic time warping algorithm is used to eliminate the difference in time scale between the state information and the operation standard, thereby improving the accuracy of the compliance judgment.

[0041] For example, a high-definition camera is used to capture real-time video of a construction site with a frame rate of 30 fps and a resolution of 1920×1080. The captured video image is denoised using Gaussian filtering, and background culling is performed using a mixture of Gaussian background modeling to obtain a foreground image containing only the worker targets. The preprocessed image is then fed into the attention-based YOLOv5 object detection model. This model uses the CSPDarknet53 as its backbone network, incorporates spatial pyramid pooling (SPP) and path aggregation network (PANet) modules, and incorporates a channel-wise attention mechanism during training, effectively improving detection accuracy. This model identifies the worker targets in the image and outputs their category and location coordinates. The detected worker targets are then tracked using the DeepSORT multi-target tracking algorithm, which combines the Kalman filter and the Hungarian algorithm to obtain target trajectory and state information. During the tracking process, if occlusion occurs, contextual information and target appearance features from the 5 seconds before and after the occlusion are extracted and fed into a pre-trained MaskRCNN instance segmentation model. The model uses the FPN feature pyramid structure and RoIAlign alignment method to accurately segment the outline of the occluded target. The segmentation results and the target state information before and after occlusion are input into the LSTM network. The position and velocity changes of the target during occlusion are restored through time series prediction, and weighted fusion is performed with the state before and after occlusion to obtain the complete target state trajectory. Finally, the state trajectory of the target is analyzed, the velocity curve is smoothed by median filtering, and its first-order and second-order derivatives are calculated. If the speed exceeds 3m / s or the acceleration exceeds 5m / s 2 , it is determined to be an abnormal state and triggers an alert. Simultaneously, the operator's status trajectory is compared with the pre-entered operating standards through a DTW distance calculation, and the distance value is mapped to the interval [0, 1] using the minimum-maximum normalization method. If the normalized distance is less than 2, the operating behavior is determined to meet the standard requirements.

[0042] S106. Obtain the operation status information of each type of work, and use a spatiotemporal data association analysis algorithm to construct a logical dependency relationship diagram between different types of work; based on preset work collaboration rules, determine whether there is a conflict in the connection of different types of work in time and space; if there is a conflict, generate an early warning message and send it to the on-site management personnel terminal.

[0043] Real-time work status information for each type of work is obtained, including worker location, work content, and progress. GPS positioning and video surveillance are used to collect data and convert it into spatiotemporal data. Spatiotemporal data association analysis algorithms, such as the spatiotemporal density clustering algorithm DBSCAN, are used to analyze the temporal and spatial logical dependencies between different types of work and construct a logical dependency graph. Based on a pre-defined knowledge base of work collaboration rules, a rule-based reasoning engine is used to determine whether there are any conflicts in the temporal and spatial connections between different types of work in the graph. If a conflict exists, an early warning message is automatically generated using the early warning rule library and the reasoning engine, and pushed to the on-site manager's mobile app. Based on this early warning information and the actual site conditions, managers can promptly adjust and optimize the work plans and collaboration methods for each type of work to eliminate conflicts. By continuously monitoring changes in the work status of each type of work, updating spatiotemporal data in real time, and dynamically updating the logical dependency graph between types of work, the system achieves real-time optimization of work collaboration. Collect on-site feedback data from the collaboration process of various types of work, and continuously optimize the work type collaboration rule knowledge base and early warning rule base through machine learning algorithms such as support vector machines (SVM), improve the accuracy and timeliness of early warnings, and realize intelligent management of work type collaboration.

[0044] For example, first, Beidou high-precision GPS positioning equipment and high-definition video surveillance equipment deployed at the construction site collect real-time location coordinates, work content, and progress information for workers in various trades, with data updated every five seconds. The collected heterogeneous data is cleaned, converted, and loaded using ETL tools, converted into a unified spatiotemporal data format, and stored in a spatiotemporal database. Then, the spatiotemporal data is clustered using the spatiotemporal density clustering algorithm DBSCAN, setting the radius parameter eps to 10 meters and the minimum number of samples minPts to 5. Hotspots of activity across time and space for different trades are identified, and the spatiotemporal distance matrix between these hotspots is calculated. Based on a pre-defined knowledge base of trade collaboration rules, a Droo ls-based inference engine is used to determine whether hotspots for different trades overlap in time and space. When the spatial and temporal distance between the hotspots of rebar tying and concrete pouring is detected to be less than 5 meters, and the temporal overlap exceeds 30%, an early warning rule is triggered, automatically generating a warning message: "Rebar tying and concrete pouring conflict; please adjust work plan." This warning message is pushed to the project manager's mobile app via a message queue. After reviewing the warning message, the project manager, taking into account the on-site construction progress and resource availability, postpones the concrete pouring work by two hours and adjusts the rebar tying area, eliminating the conflict. Simultaneously, the optimized and adjusted work plan is updated to the system, and changes in work status are monitored in real time. The work collaboration map is updated every hour. Finally, on-site problem feedback data from the collaboration process of each work type is collected and classified using the support vector machine (SVM) algorithm. When the training set reaches 1,000 items, the accuracy of the work type collaboration warning can reach over 85%, continuously optimizing the system's intelligence level.

[0045] S107. Obtain the latest operation status information of each type of work at the construction site, and use an incremental learning algorithm to optimize the type of work recognition model and the human posture estimation model; use an online learning algorithm to update the occlusion processing model to improve the robustness and generalization ability of the system.

[0046] Real-time video image data of workers of various trades at the construction site is obtained, and image noise is removed through median filtering to improve image quality; the workers in the image are identified by trade based on the pre-established convolutional neural network model to obtain the location information of workers of each trade; the OpenPose human pose estimation model is used to perform posture analysis on the identified images of workers of various trades to obtain key point coordinate information; the working posture of workers of various trades is judged based on the key point coordinate information to determine their current working status; if a worker of each trade is occluded, the occluded area is repaired using an image restoration model based on a generative adversarial network, and the pose is re-estimated; the occluded repaired image and its corresponding pose estimation result are added to the training dataset, and the convolutional neural network's trade recognition model is fine-tuned and optimized; real-time video images of the construction site are continuously obtained, and the occlusion processing model and trade recognition model are continuously optimized by regularly updating the training dataset to improve the robustness and generalization ability of the system in complex environments.

[0047] For example, high-definition cameras deployed at the construction site first capture real-time video data at 30 frames per second. The captured image data is then preprocessed using a median filter algorithm, with a filter window size of 5×5, to remove salt-and-pepper noise and improve image quality. The preprocessed image data is then fed into a pre-trained convolutional neural network model. This model, using the ResNet-50 architecture, was pre-trained on the ImageNet dataset and then fine-tuned on a dataset of construction worker images. This model can identify 10 common trades, including rebar workers, carpenters, and electricians, with an accuracy rate exceeding 95%. For each identified worker image, the OpenPose human pose estimation model is used for pose analysis. By extracting the coordinates of 18 key points on the human body, the worker's presence is determined. If a worker is occluded, an image inpainting model based on a generative adversarial network is used to inpaint the occluded area. The inpainted image is then fed back into the convolutional neural network for trade identification and pose estimation. The occluded and repaired images and their corresponding pose estimation results were added to the training dataset. The convolutional neural network's work type recognition model was fine-tuned using the Adam optimization algorithm with a learning rate of 0.01. Each iteration involved 500 samples and 10 epochs of optimization. By continuously acquiring real-time video images from the construction site and updating the training dataset daily, the system's robustness and generalization capabilities were continuously improved. Ultimately, the system achieved accurate identification and status assessment of various work types in complex construction environments, with a stable recognition accuracy rate exceeding 98%.

[0048] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the concept of this application. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A construction monitoring method, characterized in that: The method comprises: Acquire multi-view video image data of the construction site, and fuse the multi-view video image data using an image stitching algorithm to obtain a panoramic three-dimensional scene model of the construction site; Acquire image frames in the panoramic three-dimensional scene model, analyze the image frames using a pre-trained work type recognition model, and obtain location information of workers and equipment of each work type; Obtaining the position information of the workers of each type of work, analyzing the posture of each worker using a pre-trained human posture estimation model, and obtaining the motion state information of each worker; Obtaining the location information and action status information of the workers of each type of work, and combining it with a pre-established work operation knowledge base to determine the specific working status of the workers of each type of work; Obtaining the specific working status of workers in each type of work, continuously tracking each worker using a visual tracking algorithm with an attention mechanism, and if occlusion occurs, obtaining contextual information and target features before and after the occlusion, inputting the contextual information and target features into a pre-trained occlusion processing model, re-identifying the target, and restoring the status information; Obtain the operation status information of each type of work, use the spatiotemporal data association analysis algorithm to construct a logical dependency relationship diagram between different types of work, and based on the preset work type collaboration rules, determine whether there is a conflict in the connection of different types of work in time and space. If there is a conflict, generate an early warning message and send it to the on-site management personnel terminal; The latest operation status information of each type of work at the construction site is obtained, the type of work recognition model and the human posture estimation model are optimized using an incremental learning algorithm, and the occlusion processing model is updated using an online learning algorithm to improve the robustness and generalization ability of the system.

2. The method according to claim 1, characterized in that The method of acquiring multi-view video image data of the construction site and fusing the multi-view video image data using an image stitching algorithm to obtain a panoramic three-dimensional scene model of the construction site includes: By arranging multiple video acquisition devices at the construction site, video image data from different angles of the construction site can be obtained; Preprocess the acquired multi-angle video image data, including using median filtering to remove noise from the image and enhancing the image through histogram equalization algorithm to improve image quality and prepare for subsequent image stitching; According to the preset image stitching rules, SIFT feature extraction and RANSAC feature matching algorithm are used to stitch the pre-processed multi-angle video images to obtain a preliminary panoramic image; Among them, the SIFT algorithm is used to extract scale-invariant features of the image, and the RANSAC algorithm is used to eliminate incorrect feature matching points and improve the stitching accuracy; For the initial panoramic image obtained by stitching, SIFT feature extraction and RANSAC feature matching are used to optimize the stitching effect of the panoramic image, eliminate stitching gaps and overlapping areas, and obtain a seamless high-quality panoramic image; Obtain three-dimensional spatial information of the construction site based on the three-dimensional laser scanning data collected at the construction site; Use point cloud data processing software such as CloudCompare to filter and register the 3D laser scanning data to obtain a regular 3D point cloud model; Based on the 3D point cloud model, the surface of the point cloud is reconstructed using the Delaunay triangulation algorithm to generate a 3D mesh model of the construction site. Optimize the quality and detail of 3D models through mesh simplification and subdivision operations; Using texture mapping technology, the optimized panoramic image is used as a texture and mapped to the surface of the 3D mesh model. Through UV unfolding and texture coordinate mapping, the panoramic image is accurately mapped to the 3D model, improving the realism and visual effects of the scene model. The generated panoramic 3D scene model of the construction site is stored and managed. 3DTiles 3D data organization format is used to build a multi-level LOD model to achieve efficient loading and rendering of large-scale 3D scenes. Based on Web 3D visualization technology, a construction site 3D visualization platform is developed for use in construction monitoring and progress tracking applications; the Web 3D visualization technology includes WebGL and Three.js.

3. The method according to claim 1, characterized in that The step of acquiring image frames in the panoramic three-dimensional scene model and analyzing the image frames using a pre-trained work type recognition model to obtain location information of workers and equipment of each work type includes: Acquire image frames from the panoramic 3D scene model and analyze them using the pre-trained YOLOv5 target detection model to obtain the location information of workers and equipment in each type of work. Based on the acquired panoramic 3D scene model, a 3D spatial coordinate system is constructed, and the position information of the workers and equipment identified in the image frames is mapped to the 3D spatial coordinate system to obtain the spatial position coordinates of the workers and equipment in the 3D scene; Calculate the Euclidean distance between the position coordinates of the operator and the equipment in three-dimensional space, obtain a preset safety distance threshold, and determine whether the calculated Euclidean distance is less than the safety distance threshold. If so, it is determined that there is a safety hazard between the operator and the equipment; According to the location coordinates of workers in different types of work, the K-means clustering algorithm is used to cluster workers in different types of work and obtain the clustering areas of workers in different types of work; Obtain the preset regional population threshold, count the number of workers in each gathering area, and determine whether the number of workers exceeds the regional population threshold. If so, it is determined that the gathering area has a safety hazard of workers gathering together; According to the position change trajectory of the operator, the Kalman filter algorithm is used to track the movement trajectory of the operator in real time to obtain the real-time position information of the operator; The safety hazard information between workers and equipment, the gathering area information of workers of various types, and the safety hazard information of gathering areas exceeding the number of people are integrated with the real-time location information of workers of various types, and visualized through a 3D visualization engine to provide intuitive decision-making support for safety management on the construction site.

4. The method according to claim 1, wherein The position information of the workers of each type of work is obtained, and the posture of each worker is analyzed using a pre-trained human posture estimation model to obtain the motion state information of each worker, including: S401, obtaining real-time position coordinate information of workers in various types of work, pre-processing the position coordinate information, and converting it into a format suitable for input into a human posture estimation model; S402: Input the pre-processed position coordinate information into the pre-trained OpenPose human pose estimation model, and analyze the input position coordinate information through the model to obtain the coordinate information of the key points of the human body of each operator; S403, calculating the Euclidean distance and angle feature parameters between the key points based on the acquired coordinate information of the key points of the human body, and constructing a posture feature vector of the operator; S404: Input the constructed posture feature vector into a pre-trained support vector machine (SVM) posture classification model, and obtain the specific action state category of each operator through model reasoning; S405: Perform frequency statistics on the action status category of each operator to obtain the overall action status distribution of operators of each type of work; S406. Based on the preset abnormal action threshold and dangerous action rules, determine whether the action status distribution of workers in each type of work has abnormal or dangerous actions, and if so, trigger the early warning mechanism; S407: Continuously track the position changes of the operator, update the position coordinate information in real time, and repeat steps S402 to S406 to achieve real-time monitoring of the operator's actions and risk warning; S408. Display the distribution of the operator's action status in real time through visualization tools, and generate an action status monitoring report to provide decision support for managers.

5. The method according to claim 1, wherein The acquisition of the position information and action status information of the workers of each type of work, combined with a pre-established work operation knowledge base, to determine the specific working status of the workers of each type of work, includes: Obtaining real-time location coordinates of workers in various occupations through a global positioning system (GPS) or an indoor positioning system, including UWB and RFID. At the same time, the inertial measurement unit (IMU) sensor worn by the operator collects acceleration and angular velocity data during the operation; The collected position coordinate information and IMU motion data are synchronized in time and space and the coordinate system is unified. The data is cleaned and normalized to obtain time series data in a unified format. Input the pre-processed position and motion timing data into the pre-established work operation knowledge base for comparison and analysis; The knowledge base contains standard action templates and location area information for various types of work, as well as the sequence constraints of different processes; The dynamic time warping (DTW) algorithm is used to calculate the similarity between the operator's actual action sequence and the standard action template; At the same time, it is determined whether the position coordinate sequence of the operators all falls within the standard operating area of ​​the corresponding type of work; If the matching degree of position and action exceeds the preset threshold, it is judged that the operator is in a normal working state; Otherwise, it is judged as an abnormal state and a corresponding warning prompt is issued according to the abnormal type; the abnormal type includes irregular action and position out of bounds; Based on the operator's position change trajectory and action change sequence, the Hidden Markov Model (HMM) algorithm is used to identify the process and determine the specific work process the operator is currently performing. Combined with the standard working hours for different processes of each type of work, the working hours of the operators in each process are weighted and summed to obtain the comprehensive workload index; Compare the actual workload of the operators with the expected workload, and calculate the operator's work efficiency level based on the difficulty coefficient of each type of work; For those with low efficiency or insufficient workload, improvement suggestions are pushed through the mobile app; For personnel who have experienced abnormal situations multiple times, the supervisor will be notified via SMS or phone call to intervene and handle the situation. The abnormal situations and handling results will be recorded for performance appraisal and optimization and improvement.

6. The method according to claim 1, wherein The specific working status of the workers of each type of work is obtained, and the visual tracking algorithm of the attention mechanism is used to continuously track the workers of each type of work; If occlusion occurs, context information and target features before and after the occlusion are obtained, and the context information and target features are input into a pre-trained occlusion processing model to re-identify the target and restore state information, including: Acquire real-time video image data of workers in various types of work, and preprocess the image data, wherein the preprocessing includes image denoising and background removal to obtain preprocessed image data; Performing target detection on the preprocessed image data using the YOLOv5 algorithm based on the attention mechanism to identify the operator target in the image data; The DeepSORT algorithm is used to track the operator target to obtain the motion trajectory and status information of the operator target; If occlusion occurs during the tracking process, context information and target features within a certain time range before and after the occlusion occurs are obtained, and the context information and target features are input into the pre-trained MaskRCNN model for processing to re-identify the occluded target; Based on the re-identified target, combined with the state information before and after the occlusion, the state change information of the target during the occlusion period is restored through the LSTM time series prediction model; Fusing the recovered target state information with the state information before and after the occlusion to obtain the complete state trajectory information of the target; Analyzing the state trajectory information of the operator target, calculating the speed and acceleration of the trajectory, and determining whether the speed and acceleration exceed a preset normal range threshold. If the threshold is exceeded, it is determined to be an abnormal state and trigger an early warning; Comparing the status information of the operator target with the work standards of the type of work to which the operator belongs that have been pre-entered into the system, determining the similarity between the status information and the work standards by calculating the Euclidean distance, and judging whether the work behavior of the operator target meets the standard requirements; During the comparison process, the DTW dynamic time warping algorithm is used to eliminate the difference in time scale between the state information and the operation standard, thereby improving the accuracy of the compliance judgment.

7. The method according to claim 1, characterized in that The operation status information of each type of work is obtained, and a spatiotemporal data association analysis algorithm is used to construct a logical dependency relationship diagram between different types of work; based on the preset type of work collaboration rules, it is determined whether there is a conflict in the connection of different types of work in time and space; If there is a conflict, an early warning message is generated and sent to the on-site manager terminal, including: Obtain real-time operation status information for each type of work, including the location of workers, work content, and work progress. Use GPS positioning and video surveillance to collect data and convert it into spatiotemporal data. Adopt spatiotemporal data association analysis algorithms, including the spatiotemporal density clustering algorithm DBSCAN, to analyze the logical dependencies between different types of work in time and space, and construct a logical dependency graph between different types of work; Based on the preset knowledge base of work collaboration rules, a rule-based reasoning engine is used to determine whether there are conflicts in the temporal and spatial connections of different work types in the graph. If there is a conflict, an early warning message is automatically generated through the early warning rule library and inference engine, and the early warning message is pushed to the mobile terminal app of the on-site manager; Based on the early warning information and the actual situation on site, managers can timely adjust and optimize the work plans and collaboration methods of each type of work to eliminate connection conflicts; Continuously monitor the changes in the operating status of each type of work, update spatiotemporal data in real time, and dynamically update the logical dependency graph between types of work to achieve real-time optimization of work collaboration; Collect on-site feedback data from the collaboration process of various types of work, and continuously optimize the work collaboration rule knowledge base and early warning rule base through the support vector machine (SVM) machine learning algorithm to improve the accuracy and timeliness of early warnings and realize intelligent management of work collaboration.

8. The method according to claim 1, characterized in that The method includes obtaining the latest operation status information of each type of work at the construction site, optimizing the type of work recognition model and the human posture estimation model using an incremental learning algorithm, and updating the occlusion processing model using an online learning algorithm to improve the robustness and generalization ability of the system, including: Acquire real-time video image data of various workers at the construction site, remove image noise through median filtering, and improve image quality; The pre-established convolutional neural network model is used to identify the types of workers in the image and obtain the location information of each type of worker. The OpenPose human pose estimation model is used to analyze the poses of the identified workers in various occupations to obtain key point coordinate information. Determine the working posture of each worker based on the key point coordinate information and determine their current working status; If workers are obscured, an image restoration model based on a generative adversarial network is used to repair the obscured area and re-estimate the pose. The occluded and repaired images and their corresponding pose estimation results are added to the training dataset, and the convolutional neural network's work type recognition model is fine-tuned and optimized using the gradient descent algorithm. Continuously acquire real-time video images of the construction site, and continuously optimize the occlusion processing model and work type recognition model by regularly updating the training dataset, thereby improving the system's robustness and generalization capabilities in complex environments.

Images