Safety monitoring system and method based on construction site AI

Abstract

The invention discloses a safety monitoring system and method based on construction site AI. The system comprises an edge end, a transmission layer and a cloud platform. The edge end comprises an image acquisition module, a lightweight reasoning module and a local alarm module; the transmission layer encrypts and compresses the preprocessed image and the alarm information output by the edge end and transmits the preprocessed image and the alarm information to the cloud platform; the cloud platform comprises an image storage module, a depth reasoning module, a visual management module and a database. According to the method, the lightweight model is fixedly deployed on the edge side and is responsible for real-time target and posture recognition, and first-time response to potential safety hazards is ensured; complex analysis tasks such as an SORT trajectory tracking algorithm and the like are placed in the cloud, and deep behavior analysis is carried out on continuous image frames. The response of the system to the real-time requirement is guaranteed, frequent switching and maintenance of special models in multiple stages are avoided, and the overall stability and execution efficiency of the system are improved.

Description

Technical Field

[0001] This invention belongs to the field of safety monitoring technology, specifically relating to a construction site AI-based safety monitoring system and method. Background Technology

[0002] The construction industry is one of the important pillar industries of the national economy. Its working environment is complex and dynamic, with frequent cross-operations of personnel, machinery and materials, making it a high-incidence area for safety accidents for a long time. Traditional manual safety inspection methods are limited by human resources, subjective judgment and continuous working time, making it difficult to achieve all-weather, all-round real-time monitoring. Missed reports and false reports occur frequently, and cannot meet the high requirements of modern large-scale projects for refined and intelligent safety management.

[0003] In recent years, with the rapid development of technologies such as artificial intelligence and the Internet of Things, construction safety monitoring systems based on intelligent sensing and data analysis have gradually been applied. A search revealed Chinese patent CN118780943B, which proposes an AI-based construction safety early warning system and method. This system integrates multiple sensing terminals, including sensor networks, cameras, and wearable devices, and combines them with corresponding AI algorithm models for different construction stages to achieve the identification and early warning of multi-dimensional risks at construction sites. Such systems, to a certain extent, improve the coverage and responsiveness of safety management.

[0004] However, existing technologies still have several significant drawbacks in practical deployment and application: First, the system heavily relies on heterogeneous sensing devices such as lidar, ground-penetrating radar, various environmental sensors, and wearable devices, resulting in high hardware costs and complex deployment and maintenance, posing a significant barrier, especially for small and medium-sized construction sites or projects with limited budgets. Second, to adapt to the risk characteristics of different construction stages, such as foundation construction, main structure construction, and decoration, multiple dedicated AI models need to be pre-installed and switched according to the stage. This not only leads to high model training costs and long cycles but also introduces additional complexity at the model management and scheduling level, thus limiting the overall robustness and real-time performance of the system. Furthermore, existing systems mostly focus on judging macro-level environmental risks or the status of specific equipment, such as intrusion detection based on electronic fences or monitoring the status of distribution boxes. However, they lack continuous and accurate visual perception and analysis capabilities for subtle and variable unsafe behaviors of construction workers, such as not properly fastening safety ropes, resting near edges, and climbing ladders with two people, making it easy to miss such high-risk human-caused hazards.

[0005] The aforementioned problems severely restrict the large-scale promotion and application of intelligent safety monitoring technology in real construction site environments: high equipment dependence increases system costs and technical barriers; complex multi-model switching mechanisms reduce system response speed and stability; and insufficient identification of key risk dimensions such as personnel behavior directly leads to the system's failure to fundamentally reverse the blind spots and lags in traditional safety management. Therefore, there is an urgent need in this field for an innovative construction site safety monitoring system and method that can achieve comprehensive, accurate, and real-time safety monitoring at a lower cost and in a more efficient manner, especially one that can deeply integrate image perception and AI analysis, significantly reduce dependence on external sensors, and achieve integrated identification of personnel behavior and equipment status.

[0006] In existing technologies, target detection and pose estimation are usually treated as two independent modules, which leads to problems such as computational redundancy, lack of information sharing, and error accumulation. At the same time, traditional trajectory tracking algorithms only focus on the target's motion trajectory and lack a deep understanding of the interaction behavior between targets and the environmental context. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a construction site AI-based safety monitoring system and method, including an edge terminal, a transmission layer, and a cloud platform. The edge terminal includes an image acquisition module, a lightweight inference module, and a local alarm module. The transmission layer encrypts and compresses the pre-processed images and alarm information output from the edge terminal and transmits them to the cloud platform. The cloud platform includes an image storage module, a deep inference module, a visualization management module, and a database. This invention deploys a lightweight model fixed at the edge, responsible for real-time target and posture recognition, ensuring immediate response to safety hazards. Complex analysis tasks such as the SORT trajectory tracking algorithm are placed in the cloud for deep behavioral analysis of continuous image frames. This ensures the system's responsiveness to real-time requirements and avoids frequent switching and maintenance of dedicated models for multiple stages, improving the overall stability and execution efficiency of the system.

[0008] The technical solution adopted by this invention to solve its technical problem is as follows: A construction site AI-based safety monitoring system includes: an edge terminal, a transmission layer, and a cloud platform; The edge device includes an image acquisition module, a lightweight inference module, and a local alarm module. The image acquisition module acquires the original image stream and generates a preprocessed image stream after preprocessing. The lightweight inference module is equipped with a lightweight AI model trained on the construction site scene. It performs real-time target detection and posture recognition on the preprocessed image stream and outputs the safety hazard identification results to the local alarm module and the transmission layer. The transmission layer encrypts and compresses the pre-processed image stream and alarm information output from the edge and transmits them to the cloud platform. The cloud platform includes an image storage module, a deep inference module, a visualization management module, and a database. The image storage module stores preprocessed image streams transmitted via the transport layer. The deep inference module is equipped with a target trajectory tracking algorithm to perform behavioral analysis on the preprocessed image streams in the image storage module, identify persistent security risks, and output the behavioral analysis results to the visualization management module and the database.

[0009] Preferably, the image acquisition module acquires raw images at a rate of 25 frames per second using a 4K high-definition camera; The original image data is represented as a matrix sequence. ,in The time index is used, H and W are the image height and width respectively, and C is the number of channels; Image acquisition module for Distortion removal, Gaussian noise reduction, and contrast enhancement are performed sequentially, and the preprocessed image stream is output. ; The distortion correction is based on the camera intrinsic parameter matrix. and distortion coefficient vector This is accomplished through a mapping function, making ; Indicates the original image The distortion-corrected image obtained after distortion correction operation. This represents the original image based on the camera intrinsic parameter matrix K and the distortion coefficient vector d. The mapping function used for distortion correction; The Gaussian noise reduction uses a Gaussian filter kernel. Perform convolution operations: ,in The standard deviation of the filter. Indicates the distortion-reduced image Gaussian filter kernel The denoised image obtained after performing a convolution operation; The contrast enhancement process is achieved through adaptive histogram equalization, improving... Contrast to obtain preprocessed image stream .

[0010] Preferably, the lightweight inference module receives the preprocessed image stream. And load the progressive multi-granularity visual understanding network PG-Net, which has been specifically trained for construction site scenarios, to perform target detection and pose recognition; The PG-Net uses a lightweight Hybrid-SwinTransformer as its backbone network, employing CNN to extract detailed features in shallow layers and SwinTransformer to capture global dependencies in deep layers. The PG-Net network structure contains a two-stage progressive decoder: the first stage outputs the coarse bounding box and basic class of the target, and the second stage adaptively fuses the candidate region features with the multi-scale features of the backbone network to output the accurate target bounding box, fine-grained class, and heat map of 18 key points of the human body in parallel. The network is trained using a multi-task collaborative loss function:

[0011] in The consistency constraint loss is used to penalize unreasonable situations such as missing key points within the detection box or key point groups located outside the detection box. Indicates the total loss. The weighting coefficients represent the detection loss. Indicates detection loss, The weighting coefficients represent the attitude estimation loss. This represents the attitude estimation loss. The weighting coefficients representing the consistency constraint loss; Recognition results It includes target category, location, and attitude risk labels, and outputs them to the local alarm module and transmission layer.

[0012] Preferably, the local alarm module receives the recognition result output by the lightweight inference module. If the result contains safety hazards, the sound and light alarm device and the voice playback device will be triggered. The audible and visual alarm signals are generated by a pulse control circuit, with a sound pressure level of not less than 90 decibels and a light signal brightness of not less than 200 lumens. The audio playback content consists of pre-recorded safety warning statements; Finally, the local alarm module will... Encapsulate the corresponding image stream segment into an alarm message Send to the transport layer.

[0013] Preferably, the transport layer receives the image stream sent from the edge end. and alarm messages First, the data is encrypted using the AES-256 algorithm, and then the image stream is compressed using H.265 encoding. The compressed data stream is represented as follows: The compressed data stream is encapsulated using the RTSP protocol and transmitted to the cloud platform via the 5G network; the transport layer uses QoS mechanisms to distinguish data priorities, assigning the highest priority to warning data; the transport layer outputs the encrypted and compressed data stream. Image storage module in the cloud.

[0014] Preferably, the image storage module receives the data stream sent by the transport layer. Decrypt and decode it to restore it to image data. ; Locally deployed hard drives at the edge store all image data in a cyclic overwrite manner, with a retention period of 30 days. The cloud only stores image clips associated with the alarm, with the clip length being 10 minutes before and after the alarm time. Storing image data The output is sent to the deep inference module for further analysis.

[0015] Preferably, the deep inference module receives the output from the image storage module. It also employs the STAR-Engine, a spatiotemporal context-aware behavioral reasoning engine, for target trajectory tracking and behavior analysis. The STAR-Engine introduces a re-identification module based on the SORT algorithm to solve the target occlusion problem, and constructs a spatiotemporal graph to model the interaction behavior between targets; The spatiotemporal graph is input into the spatiotemporal graph convolutional network ST-GCN to learn the feature evolution of nodes and edges, and to identify group behavior and high-risk scenarios, including personnel entering equipment blind spots, group gathering risks, and illegal cross-operations. Nodes are the detection targets in each frame, and edges include spatial edges and temporal edges. The engine makes probabilistic predictions about risks in the next few seconds based on learned time-series patterns; The deep inference module outputs a set of target trajectories. Behavioral risk labels and predicted risk information are transmitted to the visualization management module and database.

[0016] Preferably, the visualization management module receives the target trajectory set output by the deep inference module. The behavioral analysis results are displayed in real time through a web page, dynamically updating the construction site layout, equipment status, and alarm statistics.

[0017] A construction site AI-based safety monitoring method includes the following steps: Step 1: Acquire image streams from the construction site via the edge and preprocess the image streams; Step 2: Use the lightweight inference module to perform real-time target detection and personnel posture recognition on the preprocessed image stream to determine whether there are any safety hazards; Step 3: When a safety hazard is identified, a local audible and visual alarm is triggered and an alarm message is generated. At the same time, the image stream and alarm message are encrypted and compressed and then sent to the cloud platform through the transport layer. Step 4: The cloud platform stores the received image stream and uses a deep inference model to track the target trajectory and analyze continuous behavior of the image data to further identify security risks; Step 5: Display real-time monitoring screens, alarm statistics and analysis results through a visual management interface, and store relevant alarm records and device status in the database, supporting multi-dimensional queries and analysis.

[0018] The beneficial effects of this invention are as follows: (1) The security monitoring system of the present invention adopts a layered processing mechanism of edge and cloud. The edge deploys a lightweight AI model to be responsible for real-time target and posture recognition to ensure that security risks are responded to in the first time. The cloud relies on powerful computing power to perform complex analysis tasks, avoiding frequent switching and maintenance of multi-stage dedicated models, and improving the overall stability and execution efficiency of the system.

[0019] (2) The system is based on pure visual analysis and relies solely on 4K ultra-high-definition cameras to collect image streams. It does not require the introduction of additional dedicated sensing hardware. While achieving comparable or even better risk identification coverage, it significantly improves the economy and scalability in small and medium-sized construction sites.

[0020] (3) The system uses Progressive Multi-Granularity Visual Understanding Network (PG-Net) to complete target detection and pose recognition simultaneously in a single forward propagation. The computational load is reduced by about 35% compared to the traditional serial model. Furthermore, the consistency constraint loss ensures the spatial logical consistency between the detection box and the human body key points, reducing the error accumulation of independent models.

[0021] (4) The system relies on the spatiotemporal context-aware behavior reasoning engine (STAR-Engine) to upgrade the traditional trajectory tracking algorithm to a spatiotemporal behavior analysis module driven by graph neural networks. It can identify hidden risks that traditional methods cannot detect and also has the ability to predict risks in the next few seconds.

[0022] (5) Through the collaborative work of PG-Net and STAR-Engine, the system constructs a full-chain intelligent security monitoring process from static target discovery to dynamic behavior understanding and risk prediction, which significantly improves the system's risk identification accuracy and early warning capability. Attached Figure Description

[0023] Figure 1 This is a system framework diagram of the present invention.

[0024] Figure 2 This is a flowchart of the method of the present invention.

[0025] Figure 3 This is the first input image for an embodiment of the present invention.

[0026] Figure 4 This is the second input image in an embodiment of the present invention.

[0027] Figure 5 This is the third input image in an embodiment of the present invention.

[0028] Figure 6 This is the fourth input image in an embodiment of the present invention.

[0029] Figure 7 This is the first output image of an embodiment of the present invention.

[0030] Figure 8 This is the second output image of an embodiment of the present invention.

[0031] Figure 9 This is the third output image of an embodiment of the present invention.

[0032] Figure 10 This is the fourth output image of an embodiment of the present invention.

[0033] Figure 11 This is the F1 confidence curve for target detection in this invention.

[0034] Figure 12 This is a statistical chart showing the size and quantity of the target instances of the present invention.

[0035] Figure 13 This is the normalized confusion matrix diagram for target detection in this invention.

[0036] Figure 14 This is a scatter plot showing the target size distribution of the present invention.

[0037] Figure 15 This is the absolute confusion matrix diagram for target detection in this invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] This invention proposes a "Progressive Multi-Granularity Visual Understanding Network" (PG-Net), which replaces the traditional YOLOv8s+OpenPose serial structure with an end-to-end multi-task collaborative architecture. By sharing the backbone network and progressive decoder, it can simultaneously complete target detection and pose recognition in a single forward propagation, significantly improving computational efficiency and recognition accuracy.

[0040] This invention proposes a "Spatiotemporal Context-Aware Behavior Reasoning Engine" (STAR-Engine), which upgrades the traditional SORT algorithm to a spatiotemporal behavior analysis module driven by a graph neural network. By constructing a spatiotemporal graph and using a spatiotemporal graph convolutional network (ST-GCN) to learn the interaction behavior between targets, it can identify complex group risk scenarios and achieve risk prediction.

[0041] A construction site AI-based safety monitoring system, such as Figures 1-2 As shown, it includes the edge, transport layer, and cloud platform; The edge device includes an image acquisition module, a lightweight inference module, and a local alarm module; The transport layer encrypts and compresses the image stream and alarm information output from the edge and transmits them to the cloud platform. The cloud platform includes an image storage module, a deep inference module, a visualization management module, and a database; The lightweight inference module is equipped with a lightweight AI model specifically trained for construction site scenarios. It performs real-time target detection and posture recognition on the pre-processed image stream and outputs the safety hazard identification results to the local alarm module and the transmission layer. The deep inference module is equipped with a target trajectory tracking algorithm. It performs behavior analysis on continuous image frames in the image storage module, identifies persistent safety risks, and outputs the behavior analysis results to the visualization management module and the database. The system reduces its reliance on multiple types of external sensors through a collaborative processing mechanism of lightweight and deep models, enabling efficient and multi-level identification and response to construction site safety risks.

[0042] The image acquisition module captures a raw image stream at a rate of 25 frames per second using a 4K high-definition camera, and the resulting raw image data is represented as a matrix sequence. Where the time index t corresponds to the frame sequence order, H is the image height, W is the image width, and C is the number of channels. This module performs preprocessing after acquiring the raw image data. In the preprocessing stage, distortion correction is first performed based on the camera intrinsic parameter matrix. and distortion coefficient vector The original image is geometrically corrected using a mapping function to obtain a distortion-free image. The intrinsic parameter matrix K contains the focal length and principal point coordinates, and the distortion coefficient vector d describes the radial and tangential distortion parameters. Gaussian noise reduction is then performed using a Gaussian filter kernel. right Perform a convolution operation to obtain ,in The standard deviation of the filter kernel controls the smoothness. The convolution operator * represents the filtering operation in the spatial domain. This step effectively suppresses random noise and preserves edge contours. Finally, contrast enhancement is performed using an adaptive histogram equalization method. The brightness distribution is remapped to generate a contrast-enhanced image. The result is higher local contrast and texture clarity. Through the above steps, the output... It preserves scene details and structural features, and uses them as input to a lightweight inference module to achieve subsequent real-time target detection and pose recognition.

[0043] The lightweight inference module receives pre-processed images. It loads a Progressive Multi-Granularity Visual Understanding Network (PG-Net) specifically trained for construction site scenarios to achieve real-time object detection and pose recognition. PG-Net uses a lightweight Hybrid-SwinTransformer as its backbone network, employing CNNs to extract detailed features in shallow layers and SwingTransformers to capture global dependencies in deeper layers, effectively balancing the extraction of local details and global contextual information.

[0044] The PG-Net network employs a two-stage progressive decoder architecture: the first stage performs coarse localization, outputting a rough bounding box and basic category of the target (such as "person" or "equipment"); the second stage performs fine-grained recognition, adaptively fusing the candidate region features output from the first stage with the multi-scale features of the backbone network, and then inputting them into the second decoder head to output in parallel accurate target bounding boxes and fine-grained categories (such as "person not wearing a safety helmet") as well as a heat map of 18 key points of the human body.

[0045] The network is trained using a multi-task collaborative loss function:

[0046] in To detect the loss, CIOU regression loss and focus loss are combined; For attitude estimation loss, weighted heatmap MSE loss is used; The innovative consistency constraint loss is used to punish unreasonable situations such as "missing key points within the detection box" or "key point groups located outside the detection box", forcing the detection and pose estimation tasks to supervise each other and optimize collaboratively.

[0047] The lightweight inference module ultimately outputs the recognition results. It includes target category, precise boundary location, and attitude risk label, and outputs them simultaneously to the local alarm module and the transmission layer. Compared to the traditional YOLOv8s+OpenPose serial architecture, PG-Net reduces computation by approximately 35% while maintaining accuracy through end-to-end multi-task collaborative design, and particularly improves the ability to identify small targets and occluded targets.

[0048] The lightweight inference module also integrates the PG-Net pose recognition model to extract key points from the detected human body area, obtaining a set of two-dimensional coordinates for 18 key points on the human body. Each of them This indicates the position of the corresponding joint. To determine whether a dangerous movement exists, a set of preset dangerous posture templates are defined. And calculate the matching degree between the input key points and the template:

[0049] Where n=18, Represents the Euclidean distance, measuring the deviation between the input keypoints and the template keypoints. When the matching degree... A value greater than 0.85 is considered a dangerous posture. The final identification result is expressed as follows: It includes target category, boundary location, and attitude risk label, and is simultaneously output to the local alarm module and transmission layer to provide input information for subsequent safety risk response.

[0050] The local alarm module receives the recognition results output from the lightweight inference module. ,in This includes the target category, boundary location, and attitude risk label at time t. When the result indicates a safety hazard, the local alarm module immediately triggers the audible and visual alarm device and the voice playback device. The audible and visual alarm signals are generated by a pulse control circuit, with a sound pressure level of no less than 90 decibels to ensure a clear warning effect even in noisy construction site environments, and a light brightness of no less than 200 lumens to ensure sufficient visibility under different lighting conditions. The voice playback device calls pre-recorded safety warning statements, enhancing personnel alertness through real-time broadcasting, thereby achieving multimodal and immediate risk alerts.

[0051] At the same time, the local alarm module integrates the recognition results with the corresponding image stream segments to construct an alarm message. Its formal definition is:

[0052] in This represents the recognition result at the current moment. Indicates time window The preprocessed image sequence within is used to provide alarm context. This refers to the continuous frame length of the image segment. The alarm message is output to the transport layer after standardized encapsulation to achieve real-time interaction with the cloud platform. This ensures that the alarm signal not only takes effect immediately on-site but can also be remotely stored and analyzed, thus providing a basis and data support for safety supervision.

[0053] The transport layer receives the preprocessed image stream from the edge. With alarm message First, both types of data are encrypted using the AES-256 symmetric encryption algorithm with a 256-bit key, ensuring the data is protected from eavesdropping and tampering during transmission. The encryption process is represented as follows:

[0054] That Let M be the encryption key, M represent the original input data unit, and C be the encrypted ciphertext. The encrypted data maintains confidentiality while preserving transmission efficiency. Subsequently, the transport layer compresses the image stream using H.265 video encoding to remove redundant information in the temporal and spatial domains, reducing the data size. The compressed image data stream is represented as... This step significantly reduces bandwidth usage while maintaining image clarity. The compressed data is further encapsulated in real-time using the RTSP protocol to ensure the transmission format conforms to streaming media standards and utilizes the 5G communication network for low-latency transmission. To ensure reliable delivery of critical alarm information in complex network environments, the transport layer applies a Quality of Service (QoS) mechanism, assigning the highest priority to alarm-related data packets, thereby guaranteeing end-to-end transmission latency of less than 100 milliseconds. Finally, the transport layer defines the encrypted and compressed complete data stream as follows:

[0055] Where C is the encrypted alarm message and related metadata. The compressed video stream and the two components are combined into a transmission unit, which is then uploaded to the cloud image storage module via a 5G network to provide input for subsequent deep inference and visualization management.

[0056] The image storage module receives data streams from the transport layer. First, the ciphertext C of the alarm message is decrypted using AES-256. The decrypted alarm message is used to index the image range related to the alarm. Simultaneously, the image compressed data stream... The H.265 decoder recovers the images into consecutive frames, thus obtaining the reconstructed image data. ,in The image stream generated at the edge maintains consistency in spatial resolution and temporal synchronization with the preprocessed image stream generated at the edge. To ensure the long-term traceability of construction site data, a 512GB solid-state drive is deployed locally at the edge. All image data is stored using a cyclic overwrite method with a 30-day time window. This means that when the storage capacity reaches its limit, older data is overwritten in a first-in, first-out manner, thus maintaining a constant storage cycle. To reduce storage overhead and improve retrieval efficiency, the cloud platform only saves image segments associated with alarm events. The segment duration is defined as 10 minutes before and after the alarm trigger time, formally represented as:

[0057] The time index interval [t-600s, t+600s] corresponds to 600 seconds before and after the alarm time, ensuring complete event context information is included. The final stored image data... The data is output to the deep inference module, providing a data foundation for subsequent target trajectory tracking and behavior analysis.

[0058] The deep inference module receives consecutive image frames output from the image storage module. The system utilizes a spatiotemporally context-aware behavior reasoning engine (STAR-Engine) to perform target trajectory tracking and deep behavior analysis. STAR-Engine represents a comprehensive upgrade from the traditional SORT algorithm, introducing a Re-ID module to address trajectory breakage issues caused by target occlusion or disappearance, and combining graph neural networks to model inter-target interaction behaviors.

[0059] The core of the inference engine is the construction and analysis of the spacetime graph: - A node is defined as each detected target (person, equipment, etc.) in each frame. Edges include two types: spatial edges (the Euclidean distance between targets within the same frame is less than a threshold, indicating a "spatial adjacency" relationship) and temporal edges (the connection of the same target between consecutive frames, indicating a "trajectory continuation" relationship).

[0060] The constructed spatiotemporal graph is input into a spatiotemporal graph convolutional network (ST-GCN). The network learns the feature evolution of nodes and edges, thereby identifying complex group behaviors and high-risk scenarios, such as: - "Personnel entering equipment blind spots" (based on spatiotemporal intersection analysis of personnel and equipment trajectories); - "Group Aggregation Risk" (Sudden Convergence Pattern Identification Based on Multi-Personnel Node Characteristics); - "Illegal cross-operation" (based on the detection of abnormal connection patterns of different types of target nodes in the spatiotemporal graph).

[0061] Furthermore, STAR-Engine, based on learned time-series patterns, can probabilistically predict risks within the next few seconds, shifting from reactive alerts to proactive warnings. The deep inference module ultimately outputs a set of target trajectories. Fine-grained behavioral risk labels and predicted risk information are transmitted to the visualization management module and database, providing more comprehensive and forward-looking support for safety management decisions.

[0062] The visualization management module receives target trajectory data output from the deep inference module. The system displays persistent behavioral risk tags and provides real-time updates via a web page. Page rendering is based on Canvas and WebGL technologies, using a construction site plan as the background layer and overlaying dynamically updated personnel and equipment location trajectories. The trajectory points are composed of a set... The temporal coordinates are mapped to a two-dimensional point set, and WebGL is used to accelerate rendering to ensure smooth display at high frame rates. Device operating status and alarm statistics are dynamically refreshed through a chart component, and the change in the number of alarms over time can be represented as a function. Where t is the time index, This represents the cumulative number of alarms at time t. User interaction features include region labeling, allowing users to label user-defined polygonal regions. The data is stored as spatial constraints to limit the retrieval range of trajectories and alarms; the threshold setting function allows users to configure the attitude matching threshold θ, which the system uses to calculate the matching degree. Risk assessment is performed using user-defined thresholds to meet differentiated management needs; the historical query function supports retrieval based on trajectory sets and alarm events, allowing users to input time intervals. Retrieve the corresponding trajectory and alarm data. The database uses a MySQL relational database, and the table structure includes an alarm record table, a device status table, and a trajectory data table. The alarm record table is... The system stores the unique identifier, trigger time, hazard type, and region of each alarm in a four-tuple format, supporting multi-dimensional retrieval based on time, region, and hazard type. Through database index optimization and SQL multi-condition queries, the system can return analysis results within millisecond response time, thus ensuring real-time performance and traceability in complex construction site environments.

[0063] A method for a construction site AI safety monitoring system includes the following steps: S1: Acquire image streams from the construction site via the edge and preprocess the image streams; S2: Use a lightweight artificial intelligence model to perform real-time target detection and personnel posture recognition on the pre-processed image stream to determine whether there are any safety hazards; S3: When a safety hazard is identified, a local audible and visual alarm is triggered and an alarm message is generated. At the same time, the image stream and alarm message are encrypted and compressed and then sent to the cloud platform through the transport layer. S4: The cloud platform stores the received image streams and uses deep inference models to track target trajectories and perform continuous behavior analysis on the image data to further identify security risks; S5: Displays real-time monitoring screens, alarm statistics and analysis results through a visual management interface, and stores relevant alarm records and device status in the database, supporting multi-dimensional queries and analysis.

[0064] Example: This embodiment uses a construction site of a key provincial building project as the application scenario. Based on the construction site AI safety monitoring system of the present invention, a 30-day safety monitoring practice was carried out to verify the actual operation effect of the system.

[0065] At eight key monitoring points on the construction site, including the top of the tower crane, the entrance to the scaffolding, the operating area of ​​the concrete mixing plant, and the personnel passage, one 4K high-definition camera was deployed at each location to capture raw image streams in real time. .

[0066] The collected raw images cover typical scenarios such as personnel working together, equipment operation, and high-risk environment. Typical raw images include... Figures 3 to 6 As shown, where Figure 3 This is a raw scene showing workers collaborating below the scaffolding; three workers and one small handling device can be clearly observed. Figure 4 A close-up of the original image of the scaffolding area, focusing on two workers climbing the scaffolding. Figure 5 This is an original image of the concrete mixing plant's operating area, showing one mixing machine and two operators. Figure 6 The original image of the personnel passageway under the "China Construction" logo, recording the flow of personnel entering and exiting the work area; The image acquisition module performs the image acquisition according to the preset procedure. Preprocessing is performed first, based on the intrinsic parameter matrix obtained from camera calibration. With distortion coefficient Through mapping function To correct optical distortion and eliminate image edge "stretching," a Gaussian filter kernel with a standard deviation σ = 1.2 is then applied. right Perform convolution operation The system filters out random noise caused by dust and light fluctuations at the construction site, and finally improves performance through adaptive histogram equalization (CLAHE). Improve local contrast, resolve target blurring issues in backlit and shadowed areas, and output a pre-processed image stream. ; The lightweight inference modules at each monitoring point at the edge are loaded with a progressive multi-granularity visual understanding network (PG-Net) specifically trained for construction site scenarios. Real-time target detection is performed, and the PG-Net model is integrated to complete personnel posture recognition. The PG-Net model successfully identifies four core target categories: "helmet," "reflective vest," "workers," and "construction equipment," and outputs the probability of each target category. The bounding box coordinates b=(x,y,w,h) and risk labels are shown in the typical detection output. Figures 7 to 10 As shown, where Figure 7 for Figure 3 The pre-processed test results image shows three personnel and one piece of equipment marked with bounding boxes. Two personnel are labeled "helmet" and "reflectivevest," while the other is labeled "no helmet" (risk label). Figure 8 for Figure 4 The preprocessed test results image clearly shows "helmet: 0.92" (92% confidence level). Figure 9 and Figure 10 They are respectively Figure 5 and Figure 6 The preprocessed detection results images all accurately labeled the target location and category, with no missed or false detections, and the data analysis charts ( Figures 11 to 15 ) Verify the effectiveness of the model, Figure 11 The F1 score for safety helmets, reflective vests, and all target categories showed a stable trend with varying confidence thresholds. The best F1 score for all categories reached 0.93, with the highest F1 score for safety helmets reaching 0.95 and the highest F1 score for reflective vests reaching 0.94. Figure 12 The accuracy rate for classifying safety helmets reached 96%, reflective vests reached 94%, and the false positive rate for background categories was only 4%-6%. Figure 15 The (absolute confusion matrix) shows only 245 misclassifications out of 2839 safety helmet samples and only 299 misclassifications out of 2182 reflective vest samples. Figure 12 , Figure 14 The number of inspection cases for safety helmets and reflective vests covers more than 95% of the workers on the construction site, and the dimensions of the inspection targets match the actual dimensions. The PG-Net model extracted the coordinates of 18 key joints of the detected personnel. This corresponds to the pre-set templates for three types of dangerous postures: "not wearing a safety rope," "leaning towards an edge," and "climbing without protection." Calculate matching degree When the posture of a person in the scaffolding area in the image is detected to match the "not wearing a safety rope" template... When a dangerous posture is identified, the recognition result is output. Posture risk label: Safety leash not fastened, simultaneously sent to the local alarm module and transmission layer; the local alarm module receives. Upon receiving the "dangerous posture" label, a multimodal alarm mechanism is immediately triggered. A 95-decibel sound alarm is generated via the pulse control circuit, a 220-lumen red warning light illuminates, and a pre-recorded message, "Personnel on the east side of the scaffold are not wearing safety ropes. Please stop work immediately and rectify the situation!" is played repeatedly through the on-site loudspeakers. Encapsulate the corresponding preprocessed image stream segment into an alarm message The data is sent to the transport layer; the transport layer receives the data sent from the edge. and Then, the AES-256 symmetric encryption algorithm is used to... and The metadata is encrypted using the H.265 video encoding standard. Compress and encapsulate using the RTSP protocol. With encryption Utilizing the 5G private network deployed at the construction site, and simultaneously enabling QoS mechanisms... Marked as the highest priority, ensuring that the end-to-end transmission latency of alarm messages is only 85ms.

[0067] Ultimately, the encrypted and compressed data stream will be... Send to the cloud platform; receive by the cloud image storage module. Then, it is decrypted using the AES-256 algorithm. Recover the preprocessed image using the H.265 decoder It also implements tiered storage, with 512GB solid-state drives deployed locally at the edge to store the full amount of data in a "first-in, first-out" cyclic overwrite manner. Retention period is 30 days; cloud storage only includes data from [other sources]. Associated image fragments; the cloud-based deep inference module loads STAR-Engine to process stored consecutive image frames. The system performs target trajectory tracking and in-depth behavior analysis, using ST-GCN to identify complex risk scenarios such as "personnel entering equipment blind spots" and "group gatherings." Specifically, based on time-series patterns, the engine predicts that a worker's continuous work in a scaffolding area without a safety harness could escalate into a high-risk event, providing a 45-second early warning. (Analysis of trajectory sets is also included.) The system can reliably track targets, maintaining trajectory continuity even under severe occlusion through the Re-ID module, reducing trajectory breakage rate by 62% compared to the traditional SORT algorithm; the visualization management module displays monitoring results in real time via a web page, embedding... Figures 7 to 10 The detection result frames are overlaid with target bounding boxes, confidence levels, and risk labels, and the real-time images of 8 points are dynamically updated. The trajectories of personnel "not wearing safety ropes" are drawn in red on the background of the construction site plan. It visually presents the movement path of climbing from the bottom of the scaffolding to the third-floor platform, combined with Figure 9The F1 curve data displays the number of alarms for three types of hazards—"missing safety helmet," "not wearing a safety rope," and "not wearing reflective clothing"—within 30 days, and generates a weekly alarm trend chart. The database stores all alarm records, equipment status, and trajectory data, supporting multi-dimensional queries. Entering "2024-05-10 14:20-14:30" retrieves alarm records and associated image clips for the aforementioned high-risk behaviors within 300ms. Filtering for "not wearing a safety rope" hazards allows viewing the trajectory set and processing results of 8 similar alarms. Selecting "scaffolding area" allows statistical analysis of the hazard occurrence rate in that area over 30 days. This embodiment covers a 30-day operation period. During the inspection, the system identified 25 safety hazards, with a 96% accuracy rate in manual verification. 18 hazards were rectified immediately through local alarms at the edge, and 7 persistent hazards were promptly addressed by management personnel through cloud-based trajectory analysis. Compared to traditional manual inspections, the system's missed detection rate was reduced to below 4%, and labor costs were reduced by 87.5%. Compared to traditional monitoring systems that rely on lidar and wearable devices, this system only requires the deployment of 4K cameras and edge computing units, reducing costs by 60%. This fully demonstrates the advantages of this invention in "low cost, high precision, and real-time performance," and can meet the safety monitoring needs of various construction sites.

Claims

1. A construction site AI-based safety monitoring system, characterized in that, include: Edge, transport layer, and cloud platform; The edge device includes an image acquisition module, a lightweight inference module, and a local alarm module. The image acquisition module acquires the original image stream and generates a preprocessed image stream after preprocessing. The lightweight inference module is equipped with a lightweight AI model trained on the construction site scene. It performs real-time target detection and posture recognition on the preprocessed image stream and outputs the safety hazard identification results to the local alarm module and the transmission layer. The transmission layer encrypts and compresses the pre-processed image stream and alarm information output from the edge and transmits them to the cloud platform. The cloud platform includes an image storage module, a deep inference module, a visualization management module, and a database; the image storage module stores pre-processed image streams transmitted via the transport layer; The deep reasoning module is equipped with a target trajectory tracking algorithm to perform behavioral analysis on the preprocessed image stream in the image storage module, identify persistent security risks, and output the behavioral analysis results to the visualization management module and the database.

2. The construction site AI-based safety monitoring system according to claim 1, characterized in that, The image acquisition module acquires raw images at a rate of 25 frames per second using a 4K high-definition camera. The original image data is represented as a matrix sequence. ,in The time index is used, H and W are the image height and width respectively, and C is the number of channels; Image acquisition module for Distortion removal, Gaussian noise reduction, and contrast enhancement are performed sequentially, and the preprocessed image stream is output. ; The distortion correction is based on the camera intrinsic parameter matrix. and distortion coefficient vector This is accomplished through a mapping function, making ; Indicates the original image The distortion-corrected image obtained after distortion correction operation. This represents the original image based on the camera intrinsic parameter matrix K and the distortion coefficient vector d. The mapping function used for distortion correction; The Gaussian noise reduction uses a Gaussian filter kernel. Perform convolution operations: ,in The standard deviation of the filter. Indicates the distortion-reduced image Gaussian filter kernel The denoised image obtained after performing a convolution operation; The contrast enhancement process is achieved through adaptive histogram equalization, improving... Contrast to obtain preprocessed image stream .

3. The construction site AI-based safety monitoring system according to claim 2, characterized in that, The lightweight inference module receives the preprocessed image stream. And load the progressive multi-granularity visual understanding network PG-Net, which has been specifically trained for construction site scenarios, to perform target detection and pose recognition; The PG-Net uses a lightweight Hybrid-SwinTransformer as its backbone network, employing CNN to extract detailed features in shallow layers and SwinTransformer to capture global dependencies in deep layers. The PG-Net network structure contains a two-stage progressive decoder: the first stage outputs the coarse bounding box and basic class of the target, and the second stage adaptively fuses the candidate region features with the multi-scale features of the backbone network to output the accurate target bounding box, fine-grained class, and heat map of 18 key points of the human body in parallel. The network is trained using a multi-task collaborative loss function: in The consistency constraint loss is used to penalize unreasonable situations such as missing key points within the detection box or key point groups located outside the detection box. Indicates the total loss. The weighting coefficients represent the detection loss. Indicates detection loss, The weighting coefficients represent the attitude estimation loss. This represents the attitude estimation loss. The weighting coefficients representing the consistency constraint loss; Recognition results It includes target category, location, and attitude risk labels, and outputs them to the local alarm module and transmission layer.

4. The construction site AI-based safety monitoring system according to claim 3, characterized in that, The local alarm module receives the recognition results output by the lightweight inference module. If the result contains safety hazards, the sound and light alarm device and the voice playback device will be triggered. The audible and visual alarm signals are generated by a pulse control circuit, with a sound pressure level of not less than 90 decibels and a light signal brightness of not less than 200 lumens. The audio playback content consists of pre-recorded safety warning statements; Finally, the local alarm module will... Encapsulate the corresponding image stream segment into an alarm message Send to the transport layer.

5. The construction site AI-based safety monitoring system according to claim 4, characterized in that, The transport layer receives the image stream sent from the edge. and alarm messages First, the data is encrypted using the AES-256 algorithm, and then the image stream is compressed using H.265 encoding. The compressed data stream is represented as follows: The compressed data stream is encapsulated using the RTSP protocol and transmitted to the cloud platform via the 5G network; the transport layer uses QoS mechanisms to distinguish data priorities, assigning the highest priority to warning data; the transport layer outputs the encrypted and compressed data stream. Image storage module in the cloud.

6. The construction site AI-based safety monitoring system according to claim 5, characterized in that, The image storage module receives the data stream sent by the transport layer. Decrypt and decode it to restore it to image data. ; Locally deployed hard drives at the edge store all image data in a cyclic overwrite manner, with a retention period of 30 days. The cloud only stores image clips associated with the alarm, with the clip length being 10 minutes before and after the alarm time. Storing image data The output is sent to the deep inference module for further analysis.

7. A construction site AI-based safety monitoring system according to claim 6, characterized in that, The deep inference module receives the output from the image storage module. It also employs the STAR-Engine, a spatiotemporal context-aware behavioral reasoning engine, for target trajectory tracking and behavior analysis. The STAR-Engine introduces a re-identification module based on the SORT algorithm to solve the target occlusion problem, and constructs a spatiotemporal graph to model the interaction behavior between targets; The spatiotemporal graph is input into the spatiotemporal graph convolutional network ST-GCN to learn the feature evolution of nodes and edges, and to identify group behavior and high-risk scenarios, including personnel entering equipment blind spots, group gathering risks, and illegal cross-operations. Nodes are the detection targets in each frame, and edges include spatial edges and temporal edges. The engine makes probabilistic predictions about risks in the next few seconds based on learned time-series patterns; The deep inference module outputs a set of target trajectories. Behavioral risk labels and predicted risk information are transmitted to the visualization management module and database.

8. The construction site AI-based safety monitoring system according to claim 7, characterized in that, The visualization management module receives the target trajectory set output by the deep inference module. The behavioral analysis results are displayed in real time through a web page, dynamically updating the construction site layout, equipment status, and alarm statistics.

9. A monitoring method applied to the safety monitoring system as described in claim 1, characterized in that, Includes the following steps: Step 1: Acquire image streams from the construction site via the edge and preprocess the image streams; Step 2: Use the lightweight inference module to perform real-time target detection and personnel posture recognition on the preprocessed image stream to determine whether there are any safety hazards; Step 3: When a safety hazard is identified, a local audible and visual alarm is triggered and an alarm message is generated. At the same time, the image stream and alarm message are encrypted and compressed and then sent to the cloud platform through the transport layer. Step 4: The cloud platform stores the received image stream and uses a deep inference model to track the target trajectory and analyze continuous behavior of the image data to further identify security risks; Step 5: Display real-time monitoring screens, alarm statistics and analysis results through a visual management interface, and store relevant alarm records and device status in the database, supporting multi-dimensional queries and analysis.

Images