Building construction monitoring method

Abstract

The invention provides a building construction monitoring method, which comprises the steps of obtaining multi-view video image data of a construction site, and performing fusion processing on the multi-view video image data by adopting an image splicing algorithm to obtain a panoramic three-dimensional scene model of the construction site; acquiring an image frame in the panoramic three-dimensional scene model, and analyzing the image frame by adopting a pre-trained work type identification model to obtain position information of operating personnel and equipment of each work type; acquiring position information of the operating personnel of each type of work, and analyzing postures of the operating personnel by adopting a pre-trained human body posture estimation model to obtain action state information of the operating personnel; and acquiring the position information and the action state information of the operating personnel of each type of work, and judging the specific operation state of the operating personnel of each type of work in combination with a pre-established work type operation knowledge base.

Description

Technical Field

[0001] The invention relates to the field of information technology, and in particular to a building construction monitoring method. Background Art

[0002] At a construction site, coordination and cooperation between different types of work is crucial. The traditional manual supervision model is difficult to fully grasp the real-time dynamics of the site, and is prone to problems such as poor connection between different types of work and cross-operation conflicts. Although the use of computer vision technology for automated monitoring can obtain real-time visual data on site, how to accurately identify the working status of different types of work from massive video images and make effective logical associations and analyses is a technical problem that needs to be solved urgently.

[0003] Specifically, due to the complex and ever-changing construction environment, the clothing, equipment, and materials of workers in different types of work vary greatly, and there are often interferences from factors such as occlusion and lighting changes in the work scene, which makes it difficult to ensure the accuracy of work type identification. In addition, there are intricate logical relationships between different types of work. How to reasonably infer the progress and connection of each type of work based on the recognition results, and predict possible conflicts, is also a technical challenge. At the same time, due to the dynamics and uncertainty of the construction site, the visual monitoring system needs to have a certain degree of adaptability and robustness, be able to cope with various changes and emergencies on the site, and continuously output reliable monitoring results, which puts higher requirements on the design and implementation of the algorithm. Summary of the invention

[0004] The present invention provides a construction monitoring method, which mainly includes:

[0005] Acquire multi-view video image data of the construction site, and use an image stitching algorithm to fuse the multi-view video image data to obtain a panoramic three-dimensional scene model of the construction site;

[0006] Acquire image frames in the panoramic three-dimensional scene model, analyze the image frames using a pre-trained work type recognition model, and obtain location information of workers and equipment of each work type;

[0007] Acquire the position information of the workers of each type of work, use the pre-trained human posture estimation model to analyze the posture of each worker, and obtain the action state information of each worker;

[0008] Obtaining the position information and action status information of the workers of each type of work, and judging the specific working status of the workers of each type of work in combination with the pre-established work type and working knowledge base;

[0009] Obtain the specific working status of the workers of each type of work, and continuously track the workers of each type of work using a visual tracking algorithm with an attention mechanism. If occlusion occurs, obtain the context information and target features before and after the occlusion, input the context information and target features into a pre-trained occlusion processing model, re-identify the target and restore the status information;

[0010] Obtain the operation status information of each type of work, use the spatiotemporal data association analysis algorithm to construct a logical dependency diagram between different types of work, and based on the preset type of work collaboration rules, determine whether there is a conflict in the connection of different types of work in time and space. If there is a conflict, generate an early warning message and send it to the on-site management personnel terminal;

[0011] The latest operation status information of each type of work at the construction site is obtained, the type of work recognition model and the human posture estimation model are optimized by using an incremental learning algorithm, and the occlusion processing model is updated by using an online learning algorithm to improve the robustness and generalization ability of the system.

[0012] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0013] The present invention discloses a method for intelligent monitoring of construction sites based on multi-view video images. The method constructs a panoramic three-dimensional scene model through an image stitching algorithm, analyzes the status of workers using work type recognition and human posture estimation models, and determines the specific work status in combination with a work type operation knowledge base. A visual tracking algorithm with an attention mechanism is used to continuously monitor each type of work, and an occlusion processing model is used to solve the occlusion problem. Logical dependencies between types of work are constructed through spatiotemporal data association analysis, and conflicts are judged based on preset rules and warnings are generated. The present invention also uses incremental learning and online learning algorithms to continuously optimize each model to improve the robustness and generalization ability of the system. The method realizes all-round intelligent monitoring of the construction site, effectively improving the efficiency and safety of construction management. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The present invention is a flow chart of a construction monitoring method.

[0015] Figure 2 It is a schematic diagram of a construction monitoring method of the present invention.

[0016] Figure 3 It is another schematic diagram of a construction monitoring method of the present invention. DETAILED DESCRIPTION

[0017] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0018] like Figure 1-3 In this embodiment, a construction monitoring method may specifically include:

[0019] S101, acquiring multi-view video image data of a construction site, and fusing the multi-view video image data using an image stitching algorithm to obtain a panoramic three-dimensional scene model of the construction site.

[0020] By arranging multiple video acquisition devices at the construction site, video image data from different angles of the construction site are obtained; the obtained multi-angle video image data are preprocessed, including using median filtering to denoise the image, and using the histogram equalization algorithm to enhance the image to improve the image quality, in preparation for subsequent image stitching; according to the preset image stitching rules, the preprocessed multi-angle video images are stitched using the SIFT feature extraction and RANSAC feature matching algorithms to obtain a preliminary panoramic image; wherein the SIFT algorithm is used to extract the scale-invariant features of the image, and the RANSAC algorithm is used to remove erroneous feature matching points to improve the stitching accuracy; for the preliminary panoramic image obtained by stitching, the stitching effect of the panoramic image is optimized by SIFT feature extraction and RANSAC feature matching, and the stitching gaps and overlapping areas are eliminated to obtain a seamless high-quality panoramic image; according to the 3D laser scanning data collected at the construction site, the 3D spatial information of the construction site is obtained; point cloud data processing software such as CloudCompare is used to filter and align the 3D laser scanning data to obtain a regular 3D point cloud model; based on the 3D point cloud model, the 3D point cloud model is processed by De l The aunay triangulation algorithm is used to reconstruct the surface of the point cloud and generate a three-dimensional mesh model of the construction site; the quality and detail expression of the three-dimensional model are optimized through mesh simplification and subdivision operations; the texture mapping technology is used to map the optimized panoramic image as a texture to the surface of the three-dimensional mesh model, and the precise correspondence between the panoramic image and the three-dimensional model is achieved through UV expansion and texture coordinate mapping, thereby improving the realism and visual effect of the scene model; the generated panoramic three-dimensional scene model of the construction site is stored and managed, and the 3DTi l es three-dimensional data organization format is used to construct a multi-level LOD model to achieve efficient loading and rendering of large-scale three-dimensional scenes; based on Web three-dimensional visualization technology, a three-dimensional visualization platform for the construction site is developed for use in construction monitoring and progress tracking applications; the Web three-dimensional visualization technology includes WebGL and Three.js.

[0021] Exemplarily, 8 high-definition cameras are arranged at the construction site to collect video image data of the construction site from different angles. The acquired video image data is preprocessed, and the image is denoised by using a 5x5 median filter, and the image grayscale range is stretched to 0-255 by a histogram equalization algorithm to enhance the image contrast. According to the preset image stitching rules, the SIFT features of the image are extracted, and the RANSAC algorithm is used for feature matching, and the wrong matching points are removed, and the image is stitched into a preliminary panoramic image. For the stitching result, the SIFT features are extracted again and RANSAC matching is performed. The pixel values ​​of the stitching area are weighted and fused to eliminate the stitching gaps, and a seamless high-quality panoramic image is obtained. Using the point cloud data obtained by the 3D laser scanner, the CloudCompare software is used for noise filtering and point cloud registration, and the 3D mesh model of the construction site is reconstructed by the De launay triangulation algorithm. The mesh model is simplified and refined by using algorithms such as edge collapse and loop subdivision to optimize the model quality. The panoramic image is mapped to the surface of the 3D model through UV expansion to achieve accurate correspondence between the image and the geometric model. Finally, the panoramic 3D model is stored in 3D iles format, a multi-level LOD model is constructed, and a Web 3D visualization platform is developed based on WebGL and Three.js to realize panoramic roaming and interactive monitoring of the construction site.

[0022] S102: Acquire image frames in the panoramic three-dimensional scene model, and use a pre-trained work type recognition model to analyze the image frames to obtain location information of workers and equipment of each work type.

[0023] The image frames in the panoramic three-dimensional scene model are obtained. The pre-trained YOLOv5 target detection model is used to analyze the obtained image frames to obtain the position information of the workers and equipment of each type of work in the image frame. According to the obtained panoramic three-dimensional scene model, a three-dimensional space coordinate system is constructed, and the position information of the workers and equipment identified in the image frame is mapped to the three-dimensional space coordinate system to obtain the spatial position coordinates of the workers and equipment in the three-dimensional scene. The Euclidean distance between the position coordinates of the workers and equipment in the three-dimensional space is calculated, and a preset safety distance threshold is obtained. It is determined whether the calculated Euclidean distance is less than the safety distance threshold. If so, it is determined that the workers and equipment have safety hazards. According to the position coordinates of workers of each type of work, the K-means clustering algorithm is used to cluster workers of different types of work to obtain the gathering areas of workers of each type of work. The preset regional population threshold is obtained, the number of workers in each gathering area is counted, and it is determined whether the number of people exceeds the regional population threshold. If so, it is determined that the gathering area has a safety hazard of workers of the type of work gathering. According to the position change trajectory of the workers, the Kalman filter algorithm is used to track the movement trajectory of the workers in real time to obtain the real-time position information of the workers. The safety hazard information between workers and equipment, the gathering area information of workers of various types, and the safety hazard information of gathering areas exceeding the threshold number of people are integrated with the real-time location information of workers of various types, and visualized through a 3D visualization engine to provide intuitive decision-making support for safety management at the construction site.

[0024] Exemplarily, a frame of image is extracted from the panoramic 3D scene model every 1 second, and the resolution of each frame is 1920x1080 pixels. The YOLOv5s model pre-trained on the COCO dataset is used to detect the target of each frame, and the detection threshold is set to 6 and the NMS threshold is set to 5. The YOLOv5s model consists of a CSPDarknet53 backbone network and a PANet neck network, and can detect 80 common targets in real time. The detection results show that a total of 8 types of workers of different types of work are identified at the construction site, including electricians, welders, scaffolders, etc., and 5 types of construction equipment, such as tower cranes and excavators. According to the coordinate information of the panoramic 3D model, the bounding box center of the workers and equipment detected in the image coordinate system is mapped to the 3D space coordinate system to obtain their XYZ coordinates. The Euclidean distance between each worker and the adjacent equipment is calculated, and the safety distance threshold is set to 2 meters. If the distance is less than 2 meters, it is determined that there is a safety hazard and marked. The spatial coordinates of the workers are clustered using the K-means clustering algorithm. The optimal cluster number K=5 is selected according to the elbow rule, that is, the workers are divided into 5 clustering areas. The number of people in each area is counted, and the threshold is set to 10 people. For areas with more than 10 people, it is determined to be an illegal gathering and a warning message is generated. The Kalman filter algorithm is applied to track the position of each worker, and a motion model with a matching probability of 8 is established. The position coordinates of the next moment are predicted in real time, matched with the detection results of the new frame, and the motion trajectory of the workers is updated. Finally, various safety hazard information and worker trajectories are integrated into the three-dimensional visualization engine, and a panoramic roaming display of the construction site is performed. The areas with safety hazards are marked with red warning boxes, which facilitates managers to intuitively control the safety situation of the construction site.

[0025] S103, obtaining the position information of the workers of each type of work, using a pre-trained human posture estimation model to analyze the posture of each worker, and obtaining the action state information of each worker. Specifically, the following steps are included:

[0026] S401. Obtain the real-time position coordinate information of workers of various types of work, pre-process the position coordinate information, and convert it into a format suitable for input into the human posture estimation model; S402. Input the pre-processed position coordinate information into the pre-trained OpenPose human posture estimation model, and analyze the input position coordinate information through the model to obtain the human body key point coordinate information of each worker; S403. Calculate the Euclidean distance and angle feature parameters between each key point based on the obtained human body key point coordinate information, and construct the posture feature vector of the worker; S404. Input the constructed posture feature vector into the pre-trained support vector machine (SVM) posture classification model, and obtain the posture feature vector of each worker through model reasoning. Specific action state categories; S405. Perform frequency statistics on the action state categories of each operator to obtain the overall action state distribution of operators of each type of work; S406. According to the preset abnormal action threshold and dangerous action rules, determine whether there are abnormal actions or dangerous actions in the action state distribution of operators of each type of work, and if so, trigger the early warning mechanism; S407. Continuously track the position changes of operators, update the position coordinate information in real time, and repeat steps S402 to S406 to achieve real-time monitoring of operators' actions and risk early warning; S408. Use visualization tools to display the action state distribution of operators in real time, and generate an action state monitoring report to provide decision support for managers.

[0027] Exemplarily, first, the real-time position coordinate information of workers of various types of work is obtained from the panoramic three-dimensional scene model, and converted into the input format required by the OpenPose human posture estimation model. Specifically, the position coordinates of the workers are sampled at a frequency of 5 frames per second, their XYZ coordinate values ​​are extracted, and the coordinate values ​​are normalized to the range of [-1,1]. Then, the preprocessed position coordinate information is input into the OpenPose model pre-trained on the COCO dataset. The model uses VGG-19 as the backbone network, which can detect 18 human key points from the input image, including the top of the head, neck, shoulders, elbows, wrists, hips, knees and ankles. Through forward propagation, the model outputs the coordinates of 18 key points of each worker. Then, according to the acquired key point coordinates, the characteristic parameters such as the Euclidean distance and angle between the key points are calculated. For example, the Euclidean distance between the left shoulder and the left elbow is calculated, the cosine value of the angle between the left thigh and the left calf is calculated, and a posture feature vector containing 36 elements is constructed. The feature vector is input into the SVM multi-classification model using the Gaussian kernel function, and the action state categories of the operators are obtained through the One-vs-One voting strategy, such as standing, bending, kneeling, sitting cross-legged, etc. The frequency of the action state categories of each operator in the past 30 seconds is counted to obtain the overall action state distribution histogram of each type of work. If the frequency of abnormal actions (such as sitting for a long time) of a single operator exceeds 10 times, or three consecutive operators have the same dangerous actions (such as climbing), an early warning prompt is triggered. The system updates the position coordinates of the operator every 1 minute and repeats the above posture analysis process to realize real-time monitoring of the operator's actions. Finally, the action state of the operator is rendered in real time into the panoramic scene through the three-dimensional visualization engine, and a monitoring report containing a pie chart of the action distribution of each type of work and abnormal action warning information is generated and uploaded to the construction management system for management personnel to review.

[0028] S104, obtaining the position information and action status information of the workers of each type of work, and judging the specific working status of the workers of each type of work in combination with a pre-established work type operation knowledge base.

[0029] The real-time position coordinate information of workers in various types of work is obtained through the global positioning system (GPS) or indoor positioning system (such as UWB, RFID, etc.). At the same time, the acceleration and angular velocity data during the operation are collected through the inertial measurement unit (IMU) sensor worn by the workers. The collected position coordinate information and IMU motion data are synchronized in time and space and the coordinate system is unified, and the data is cleaned and normalized to obtain time series data in a unified format. The pre-processed position and motion time series data are input into the pre-established work type operation knowledge base for comparison and analysis. The knowledge base contains standard motion templates and position area information for each type of work, as well as the sequence constraints of different processes. The dynamic time warping (DTW) algorithm is used to calculate the similarity between the actual motion sequence of the worker and the standard motion template. At the same time, it is determined whether the position coordinate sequence of the worker all falls within the standard operation area of ​​the corresponding type of work. If the matching degree of the position and motion exceeds the preset threshold, the worker is judged to be in a normal working state. Otherwise, it is judged to be an abnormal state, and a corresponding early warning prompt is issued according to the abnormal type (such as irregular motion, position out of bounds, etc.). According to the position change trajectory and action change sequence of the operator, the Hidden Markov Model (HMM) algorithm is used to identify the process and determine the specific process that the operator is currently performing. Combined with the standard working hours quota of different processes of each type of work, the weighted sum of the operator's working time in each process is obtained to obtain a comprehensive workload index. The actual workload of the operator is compared with the expected workload, and the operator's work efficiency level is calculated in combination with the difficulty coefficient of each type of work. For personnel with low efficiency or unsaturated workload, improvement suggestions are pushed through mobile APP. For personnel who have repeatedly encountered abnormal conditions, the supervisor is notified by SMS or phone to intervene and handle, and the abnormal conditions and handling results are recorded for performance appraisal and optimization and improvement.

[0030] For example, first, by installing GPS positioning modules and UWB positioning tags on the workers' helmets and work clothes, the real-time position coordinates of the workers are collected at a frequency of once per second. At the same time, the smart watch worn by the workers has a built-in three-axis accelerometer and a three-axis gyroscope, which collects acceleration and angular velocity data during the operation at a frequency of 50Hz. The GPS coordinates, UWB coordinates and IMU data are synchronized and aligned through timestamps, and uniformly converted into three-dimensional coordinates in the ENU coordinate system with the southwest corner of the construction site as the origin. The collected data is Kalman filtered to remove high-frequency noise and outliers, and the minimum-maximum normalization method is used to scale the data to the [0,1] interval. Then, the pre-processed position and action timing data are input into the work type operation knowledge base built based on the Neo4j graph database. The knowledge base stores the standard action templates and location area information of 10 major types of work such as rebar workers, carpenters, and concrete workers, as well as the timing constraints of 30 processes such as pouring, vibrating, and curing. By calculating the Euclidean distance and dynamic time warping algorithm, the similarity score between the actual action sequence of the operator and the standard template is obtained. At the same time, the Ray Casting algorithm is used to determine whether the position coordinates of the operator are within the polygonal operation area and output a Boolean value. If the action similarity score is greater than 85 and the position Boolean value is True, the operator is judged to be in a normal working state. Otherwise, it is judged to be an abnormal state. If the action score is less than 7, it prompts "non-standard action", and if the position Boolean value is Fail, it prompts "position out of bounds". Then, the position sequence and action sequence of the operator in the last 30 minutes are input into the pre-trained hidden Markov model, and the most likely process state sequence is obtained by decoding with the Viterbi algorithm. Then, according to the process time quota in the knowledge base, the exponential weighted average method is used to calculate the operation time of the operator in each process. Assuming that the quota working time of the pouring process is 5 hours / cubic meter, the operator pours 7 cubic meters of concrete cumulatively, and the actual time is 6 hours, then the efficiency coefficient of the pouring process is 5×7÷6=125. Finally, the efficiency coefficients of each process are weighted averaged, and the weights are determined according to the difficulty coefficient of the work type to obtain the comprehensive work efficiency level of the operator. If the work efficiency is lower than 8, the mobile APP will push improvement suggestions for "improving efficiency". If the individual work efficiency is lower than 6, and the number of abnormal positions and movements exceeds 3 times, a warning SMS will be sent to the project manager, and the assessment score will be recorded in the performance management system.

[0031] S105. Obtain the specific working status of the workers of each type of work, and use a visual tracking algorithm with an attention mechanism to continuously track the workers of each type of work; if occlusion occurs, obtain the context information and target features before and after the occlusion, input the context information and target features into a pre-trained occlusion processing model, re-identify the target and restore the status information.

[0032] Acquire real-time video image data of workers of various types of work, and preprocess the image data, wherein the preprocessing includes image denoising and background removal, to obtain preprocessed image data;

[0033] Using the YOLOv5 algorithm based on the attention mechanism to perform target detection on the preprocessed image data, and identify the operator target in the image data;

[0034] The DeepSORT algorithm is used to track the operator target to obtain the motion trajectory and status information of the operator target;

[0035] If occlusion occurs during the tracking process, context information and target features within a certain time range before and after the occlusion occurs are obtained, and the context information and target features are input into a pre-trained MaskRCNN model for processing to re-identify the occluded target;

[0036] According to the re-identified target, combined with the state information before and after the occlusion, the state change information of the target during the occlusion period is restored through the LSTM time series prediction model;

[0037] The restored target state information is merged with the state information before and after the occlusion to obtain the complete state trajectory information of the target;

[0038] Analyze the state trajectory information of the operator target, and determine whether the speed and acceleration exceed a preset normal range threshold by calculating the speed and acceleration of the trajectory. If the speed and acceleration exceed the threshold, it is determined to be an abnormal state and trigger an early warning;

[0039] Compare the status information of the operator target with the operation standard of the type of work to which the operator belongs that is pre-entered into the system, determine the similarity between the status information and the operation standard by calculating the Euclidean distance, and judge whether the operation behavior of the operator target meets the standard requirements;

[0040] During the comparison process, the DTW dynamic time warping algorithm is used to eliminate the difference in time scale between the state information and the operation standard, thereby improving the accuracy of the compliance judgment.

[0041] For example, first, a real-time video image of the construction site is obtained through a high-definition camera, with a video frame rate of 30fps and a resolution of 1920×1080. The collected video image is denoised by Gaussian filtering, and the background is removed by using a mixed Gaussian background modeling method to obtain a foreground image containing only the operator target. Then, the preprocessed image is input into the YOLOv5 target detection model based on the attention mechanism. The model uses CSPDarknet53 as the backbone network, introduces the spatial pyramid pooling (SPP) and path aggregation network (PANet) modules, and adds a channel attention mechanism during the training process, which can effectively improve the detection accuracy. The operator target in the image is identified by the model, and its category and position coordinate information are output. Then, the DeepSORT multi-target tracking algorithm is used to track the detected operator target. The algorithm combines the Kalman filter and the Hungarian algorithm to obtain the target's motion trajectory and state information. During the tracking process, if occlusion occurs, the context information and target appearance features within 5 seconds before and after the occlusion are extracted and input into the pre-trained MaskRCNN instance segmentation model. The model uses the FPN feature pyramid structure and RoIAlign alignment method to accurately segment the outline of the occluded target. The segmentation result and the target state information before and after occlusion are input into the LSTM network. The position and speed changes of the target during occlusion are restored through time series prediction, and weighted fusion is performed with the state before and after occlusion to obtain the complete target state trajectory. Finally, the state trajectory of the target is analyzed, the speed curve is smoothed by median filtering, and its first-order and second-order derivatives are calculated. If the speed exceeds 3m / s or the acceleration exceeds 5m / s 2 , it is determined to be an abnormal state and triggers an early warning. At the same time, the state trajectory of the operator and the pre-entered operation standard are calculated by DTW distance, and the minimum-maximum normalization method is used to map the distance value to the [0,1] interval. If the normalized distance is less than 2, it is determined that the operation behavior meets the standard requirements.

[0042] S106. Obtain the operation status information of each type of work, and use the spatiotemporal data association analysis algorithm to construct a logical dependency diagram between different types of work; based on the preset type of work collaboration rules, determine whether there is a conflict in the connection of different types of work in time and space; if there is a conflict, generate an early warning message and send it to the on-site management personnel terminal.

[0043] Obtain the real-time operation status information of each type of work, including the location of the operator, the content of the operation, the progress of the operation, etc., collect data by means of GPS positioning and video monitoring, and convert it into spatiotemporal data. Use spatiotemporal data association analysis algorithms, such as the spatiotemporal density clustering algorithm DBSCAN, to analyze the logical dependencies between different types of work in time and space, and construct a logical dependency graph between types of work. According to the preset knowledge base of type of work collaboration rules, use the rule-based reasoning engine to determine whether there is a conflict in the connection between different types of work in time and space in the graph. If there is a conflict, the early warning information is automatically generated through the early warning rule library and the reasoning engine, and the early warning information is pushed to the mobile terminal App of the on-site manager. According to the early warning information and the actual situation on site, the manager timely adjusts and optimizes the operation plan and collaboration mode of each type of work to eliminate the connection conflict. Continuously monitor the changes in the operation status of each type of work, update the spatiotemporal data in real time, dynamically update the logical dependency graph between types of work, and realize the real-time optimization of type of work collaboration. Collect on-site feedback data from the collaboration process of various types of work, and continuously optimize the work collaboration rule knowledge base and early warning rule base through machine learning algorithms such as support vector machines (SVM), improve the accuracy and timeliness of early warnings, and realize intelligent management of work collaboration.

[0044] For example, first, the real-time location coordinates, work content, work progress and other information of workers of various types of work are collected through Beidou high-precision GPS positioning equipment and high-definition video monitoring equipment deployed at the construction site, and the data is updated every 5 seconds. The collected heterogeneous data is cleaned, converted and loaded through ETL tools, converted into a unified spatiotemporal data format, and stored in the spatiotemporal database. Then, the spatiotemporal data is clustered using the spatiotemporal density clustering algorithm DBSCAN, the radius parameter eps is set to 10 meters, the minimum number of samples minPts is set to 5, the activity hot spots of different types of work in time and space are identified, and the spatiotemporal distance matrix between different hot spots is calculated. According to the preset knowledge base of work collaboration rules, the reasoning engine based on Droo ls is used to determine whether there are overlapping conflicts in the hot spots of different types of work in time and space. When it is detected that the time-space distance between the hot spots of the steel bar binding and concrete pouring work is less than 5 meters, and the time overlap exceeds 30%, the warning rule is triggered, and the warning information "The connection between steel bar binding and concrete pouring conflicts, please adjust the work plan" is automatically generated and pushed to the project manager's mobile app through the message queue. After checking the warning information, the project manager postponed the concrete pouring operation for 2 hours based on the on-site construction progress and resource conditions, adjusted the steel bar binding operation area, and eliminated the conflict between the types of work. At the same time, the optimized and adjusted work plan is updated to the system, the operation status changes are monitored in real time, and the type of work collaboration map is updated every 1 hour. Finally, the feedback data of on-site problems in the collaboration process of each type of work is collected, and the support vector machine SVM algorithm is used for classification training. When the training set reaches 1,000, the accuracy of the type of work collaboration warning can reach more than 85%, and the intelligence level of the system is continuously optimized.

[0045] S107, obtaining the latest operation status information of each type of work at the construction site, optimizing the type of work recognition model and the human body posture estimation model using an incremental learning algorithm; and updating the occlusion processing model using an online learning algorithm to improve the robustness and generalization ability of the system.

[0046] Real-time video image data of workers of various types at the construction site are obtained, and image noise is removed by median filtering to improve image quality; the types of workers in the image are identified according to the pre-established convolutional neural network model to obtain the position information of workers of various types; the OpenPose human posture estimation model is used to perform posture analysis on the identified images of workers of various types to obtain key point coordinate information; the working posture of workers of various types is judged according to the key point coordinate information to determine their current working status; if workers of various types are occluded, the image restoration model based on the generative adversarial network is used to repair the occluded area and re-estimate the posture; the occluded repaired image and its corresponding posture estimation results are added to the training data set, and the gradient descent algorithm is used to fine-tune and optimize the convolutional neural network's work type recognition model; real-time video images of the construction site are continuously obtained, and the occlusion processing model and work type recognition model are continuously optimized by regularly updating the training data set to improve the robustness and generalization ability of the system in complex environments.

[0047] Exemplarily, first, the real-time video image data of the construction site is obtained at a speed of 30 frames per second through the high-definition camera deployed at the construction site. Then, the median filtering algorithm is used to preprocess the acquired image data, and the filter window size is set to 5×5 to remove the salt and pepper noise in the image and improve the image quality. Next, the preprocessed image data is input into the pre-trained convolutional neural network model. The model adopts the ResNet-50 network structure, is pre-trained on the ImageNet dataset, and then fine-tuned on the construction worker image dataset to achieve recognition of 10 common types of work such as steel workers, carpenters, and electricians, with an accuracy rate of more than 95%. For the images of each type of worker identified, the OpenPose human posture estimation model is used for posture analysis, and the coordinate information of 18 key points of the human body is extracted to determine whether it is in the construction state. If the worker is occluded, the image restoration model based on the generative adversarial network is used to repair the occluded area, and the repaired image is re-input into the convolutional neural network for type recognition and posture estimation. The occluded and repaired images and their corresponding pose estimation results are added to the training data set. The Adam optimization algorithm is used to fine-tune the convolutional neural network's work type recognition model with a learning rate of 0.01. Each iteration uses 500 samples and optimizes 10 epochs. By continuously acquiring real-time video images of the construction site and updating the training data set once a day, the robustness and generalization ability of the system are continuously improved, and ultimately accurate recognition and status judgment of various types of workers in complex construction environments are achieved, with the recognition accuracy rate being stabilized at more than 98%.

[0048] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept of the present application. For example, the above features are replaced with the technical features with similar functions disclosed in the present application (but not limited to) to form a technical solution.

Claims

1. A construction monitoring method, characterized in that: The method comprises: Acquire multi-view video image data of the construction site, and use an image stitching algorithm to fuse the multi-view video image data to obtain a panoramic three-dimensional scene model of the construction site; Acquire image frames in the panoramic three-dimensional scene model, analyze the image frames using a pre-trained work type recognition model, and obtain location information of workers and equipment of each work type; Acquire the position information of the workers of each type of work, use the pre-trained human posture estimation model to analyze the posture of each worker, and obtain the action state information of each worker; Obtaining the position information and action status information of the workers of each type of work, and judging the specific working status of the workers of each type of work in combination with the pre-established work type and working knowledge base; Obtain the specific working status of the workers of each type of work, and continuously track the workers of each type of work using a visual tracking algorithm with an attention mechanism. If occlusion occurs, obtain the context information and target features before and after the occlusion, input the context information and target features into a pre-trained occlusion processing model, re-identify the target and restore the status information; Obtain the operation status information of each type of work, use the spatiotemporal data association analysis algorithm to construct a logical dependency diagram between different types of work, and based on the preset type of work collaboration rules, determine whether there is a conflict in the connection of different types of work in time and space. If there is a conflict, generate an early warning message and send it to the on-site management personnel terminal; The latest operation status information of each type of work at the construction site is obtained, the type of work recognition model and the human posture estimation model are optimized by using an incremental learning algorithm, and the occlusion processing model is updated by using an online learning algorithm to improve the robustness and generalization ability of the system.

2. The method according to claim 1, characterized in that The method of acquiring multi-view video image data of the construction site and fusing the multi-view video image data using an image stitching algorithm to obtain a panoramic three-dimensional scene model of the construction site includes: By arranging multiple video acquisition devices at the construction site, video image data from different angles of the construction site can be obtained; Preprocess the acquired multi-angle video image data, including using median filtering to remove noise from the image and enhancing the image through histogram equalization algorithm to improve the image quality and prepare for subsequent image stitching; According to the preset image stitching rules, SIFT feature extraction and RANSAC feature matching algorithm are used to stitch the preprocessed multi-angle video images to obtain a preliminary panoramic image; Among them, the SIFT algorithm is used to extract the scale-invariant features of the image, and the RANSAC algorithm is used to eliminate erroneous feature matching points and improve the stitching accuracy; For the initial panoramic image obtained by stitching, SIFT feature extraction and RANSAC feature matching are used to optimize the stitching effect of the panoramic image, eliminate stitching gaps and overlapping areas, and obtain a seamless high-quality panoramic image. Obtain the three-dimensional spatial information of the construction site based on the three-dimensional laser scanning data collected at the construction site; Use point cloud data processing software such as CloudCompare to filter and register the 3D laser scanning data to obtain a regular 3D point cloud model; Based on the 3D point cloud model, the surface of the point cloud is reconstructed through the Delaunay triangulation algorithm to generate a 3D mesh model of the construction site; Optimize the quality and detail of 3D models through mesh simplification and subdivision operations; Using texture mapping technology, the optimized panoramic image is used as a texture and mapped to the surface of the 3D mesh model. Through UV unfolding and texture coordinate mapping, the precise correspondence between the panoramic image and the 3D model is achieved, thus improving the realism and visual effect of the scene model. The generated panoramic 3D scene model of the construction site is stored and managed, and the 3DTiles 3D data organization format is used to build a multi-level LOD model to achieve efficient loading and rendering of large-scale 3D scenes; Based on Web 3D visualization technology, a construction site 3D visualization platform is developed for use in construction monitoring and progress tracking applications; the Web 3D visualization technology includes WebGL and Three.js.

3. The method according to claim 1, characterized in that The step of acquiring image frames in the panoramic three-dimensional scene model and analyzing the image frames using a pre-trained work type recognition model to obtain location information of workers and equipment of each work type includes: Obtain image frames in the panoramic 3D scene model, and use the pre-trained YOLOv5 target detection model to analyze the acquired image frames to obtain the location information of operators and equipment of various types of work in the image frames; A three-dimensional space coordinate system is constructed based on the acquired panoramic three-dimensional scene model, and the position information of the operator and equipment identified in the image frame is mapped to the three-dimensional space coordinate system to obtain the spatial position coordinates of the operator and equipment in the three-dimensional scene; Calculate the Euclidean distance between the position coordinates of the operator and the equipment in the three-dimensional space, obtain a preset safety distance threshold, and determine whether the calculated Euclidean distance is less than the safety distance threshold. If so, it is determined that there is a safety hazard between the operator and the equipment; According to the location coordinates of workers in different types of work, the K-means clustering algorithm is used to cluster workers in different types of work and obtain the clustering areas of workers in different types of work; Obtain the preset regional population threshold, count the number of workers in each gathering area, and determine whether the number of workers exceeds the regional population threshold. If so, determine that there is a safety hazard of workers gathering in the gathering area; According to the position change trajectory of the operator, the Kalman filter algorithm is used to track the movement trajectory of the operator in real time to obtain the real-time position information of the operator; The safety hazard information between workers and equipment, the gathering area information of workers of various types, and the safety hazard information of gathering areas exceeding the number of people are integrated with the real-time location information of workers of various types, and visualized through a 3D visualization engine to provide intuitive decision-making support for safety management at the construction site.

4. The method according to claim 1, characterized in that: The position information of the workers of each type of work is obtained, and the posture of each worker is analyzed by using a pre-trained human posture estimation model to obtain the action state information of each worker, including: S401, obtaining real-time position coordinate information of workers of various types of work, preprocessing the position coordinate information, and converting it into a format suitable for input into a human posture estimation model; S402, inputting the preprocessed position coordinate information into a pre-trained OpenPose human posture estimation model, and analyzing the input position coordinate information through the model to obtain the human body key point coordinate information of each operator; S403, according to the acquired coordinate information of the key points of the human body, calculating the Euclidean distance and angle characteristic parameters between the key points, and constructing the posture characteristic vector of the operator; S404, inputting the constructed posture feature vector into a pre-trained support vector machine (SVM) posture classification model, and obtaining the specific action state category of each operator through model reasoning; S405, performing frequency statistics on the action status category of each operator to obtain the overall action status distribution of operators of each type of work; S406. According to the preset abnormal action threshold and dangerous action rules, determine whether there are abnormal actions or dangerous actions in the action state distribution of workers of various types of work, and if so, trigger the early warning mechanism; S407, continuously tracking the position changes of the operator, updating the position coordinate information in real time, and repeatedly executing steps S402 to S406 to achieve real-time monitoring of the operator's actions and risk warning; S408. Display the distribution of the operating personnel's action status in real time through visualization tools, and generate an action status monitoring report to provide decision support for managers.

5. The method according to claim 1, characterized in that The obtaining of the position information and action status information of the operators of each type of work, and combining with a pre-established type of work knowledge base to determine the specific working status of the operators of each type of work, includes: Obtaining real-time location coordinate information of workers in various types of work through the global positioning system (GPS) or indoor positioning system; the indoor positioning system includes UWB and RFID; At the same time, the inertial measurement unit (IMU) sensors worn by the operators collect acceleration and angular velocity data during the operation; The collected position coordinate information and IMU motion data are synchronized in time and space and the coordinate system is unified, and the data is cleaned and normalized to obtain time series data in a unified format; Input the pre-processed position and motion timing data into the pre-established work operation knowledge base for comparison and analysis; The knowledge base contains standard action templates and location area information for various types of work, as well as the sequence constraints of different processes; The similarity between the operator's actual action sequence and the standard action template is calculated using the dynamic time warping (DTW) algorithm; At the same time, it is determined whether the position coordinate sequence of the operator is completely within the standard operation area of ​​the corresponding type of work; If the matching degree of the position and the action exceeds the preset threshold, it is judged that the operator is in a normal working state; Otherwise, it is judged as an abnormal state, and a corresponding warning prompt is issued according to the abnormal type; the abnormal type includes irregular action and position out of bounds; According to the position change trajectory and action change sequence of the operator, the Hidden Markov Model (HMM) algorithm is used to identify the process and determine the specific process the operator is currently performing. Combined with the standard working hours of different processes of each type of work, the working hours of the operators in each process are weighted and summed to obtain the comprehensive workload index; Compare the actual workload of operators with the expected workload, and calculate the operator's work efficiency level based on the difficulty coefficient of each type of work; For those who are inefficient or have insufficient workload, improvement suggestions are pushed through the mobile app; For personnel who have experienced abnormal situations multiple times, the supervisor will be notified via SMS or phone call to intervene and handle the situation. The abnormal situations and handling results will be recorded for performance appraisal and optimization and improvement.

6. The method according to claim 1, characterized in that The specific working status of the workers of each type of work is obtained, and the visual tracking algorithm of the attention mechanism is used to continuously track the workers of each type of work; If occlusion occurs, context information and target features before and after the occlusion are obtained, and the context information and target features are input into a pre-trained occlusion processing model to re-identify the target and restore the state information, including: Acquire real-time video image data of workers of various types of work, and preprocess the image data, wherein the preprocessing includes image denoising and background removal, to obtain preprocessed image data; Using the YOLOv5 algorithm based on the attention mechanism to perform target detection on the preprocessed image data, and identify the operator target in the image data; The DeepSORT algorithm is used to track the operator target to obtain the motion trajectory and status information of the operator target; If occlusion occurs during the tracking process, context information and target features within a certain time range before and after the occlusion occurs are obtained, and the context information and target features are input into a pre-trained MaskRCNN model for processing to re-identify the occluded target; According to the re-identified target, combined with the state information before and after the occlusion, the state change information of the target during the occlusion period is restored through the LSTM time series prediction model; The restored target state information is merged with the state information before and after the occlusion to obtain the complete state trajectory information of the target; Analyze the state trajectory information of the operator target, and determine whether the speed and acceleration exceed a preset normal range threshold by calculating the speed and acceleration of the trajectory. If the speed and acceleration exceed the threshold, it is determined to be an abnormal state and trigger an early warning; Compare the status information of the operator target with the operation standard of the type of work to which the operator belongs that is pre-entered into the system, determine the similarity between the status information and the operation standard by calculating the Euclidean distance, and judge whether the operation behavior of the operator target meets the standard requirements; During the comparison process, the DTW dynamic time warping algorithm is used to eliminate the difference in time scale between the state information and the operation standard, thereby improving the accuracy of the compliance judgment.

7. The method according to claim 1, characterized in that The operation status information of each type of work is obtained, and a spatiotemporal data association analysis algorithm is used to construct a logical dependency relationship diagram between different types of work; based on the preset type of work collaboration rules, it is determined whether there is a conflict in the connection of different types of work in time and space; If there is a conflict, an early warning message is generated and sent to the site manager terminal, including: Obtain real-time operation status information of each type of work, including the location of the operator, the content of the operation and the progress of the operation, collect data using GPS positioning and video monitoring, and convert it into spatiotemporal data; Adopt spatiotemporal data association analysis algorithms, including the spatiotemporal density clustering algorithm DBSCAN, to analyze the logical dependencies between different types of work in time and space, and construct a logical dependency graph between types of work; According to the preset knowledge base of work type collaboration rules, the rule-based reasoning engine is used to determine whether there is a conflict in the connection of different work types in time and space in the graph; If there is a conflict, the warning information is automatically generated through the warning rule library and reasoning engine, and the warning information is pushed to the mobile terminal app of the on-site manager; Based on the early warning information and the actual situation on site, the management personnel will timely adjust and optimize the work plans and collaboration methods of each type of work to eliminate connection conflicts; Continuously monitor the changes in the operation status of each type of work, update the spatiotemporal data in real time, dynamically update the logical dependency graph between types of work, and achieve real-time optimization of work collaboration; Collect on-site feedback data from the collaboration process of each type of work, and continuously optimize the work collaboration rule knowledge base and early warning rule base through the support vector machine (SVM) machine learning algorithm, improve the accuracy and timeliness of the early warning, and realize the intelligent management of work collaboration.

8. The method according to claim 1, characterized in that The latest operation status information of each type of work at the construction site is obtained, and the type of work recognition model and the human body posture estimation model are optimized by using an incremental learning algorithm; the occlusion processing model is updated by using an online learning algorithm to improve the robustness and generalization ability of the system, including: Obtain real-time video image data of various types of workers at the construction site, remove image noise through median filtering, and improve image quality; According to the pre-established convolutional neural network model, the types of work of the personnel in the image are identified to obtain the location information of the personnel in each type of work; The OpenPose human posture estimation model is used to analyze the posture of the identified images of workers in various types of work to obtain the coordinate information of key points; Determine the working posture of each type of worker based on the key point coordinate information and determine their current working status; If a worker is blocked, an image restoration model based on a generative adversarial network is used to repair the blocked area and re-estimate the pose. The occluded and repaired images and their corresponding pose estimation results are added to the training data set, and the convolutional neural network work type recognition model is fine-tuned and optimized using the gradient descent algorithm. Continuously acquire real-time video images of the construction site, and continuously optimize the occlusion processing model and work type recognition model by regularly updating the training data set to improve the robustness and generalization ability of the system in complex environments.

Images