A site personnel behavior intelligent identification and active early warning system

Abstract

The application provides a construction site personnel behavior intelligent identification and active early warning system, and relates to the field of building engineering safety management. The system comprises a physical layer, a twin layer, a decision layer and an application layer. The physical layer collects multi-source data through UWB positioning equipment, high-definition cameras and environmental sensors. The twin layer constructs a three-dimensional scene by using BIM and laser point clouds, and realizes dynamic mapping through LOD lightweight and incremental update mechanism. The decision layer adopts a CNN-Transformer model for behavior identification, and autonomously generates an early warning strategy based on an improved DQN reinforcement learning algorithm, and balances risks and costs through a multi-objective reward function. The application layer executes early warning through a visual platform, an audible and light alarm, a UAV and other terminals, and links an emergency command system. The application solves the problem of data planarization of traditional monitoring systems, realizes three-dimensional dynamic monitoring and intelligent active early warning, and significantly improves the level of construction site safety management.

Description

Technical Field

[0001] This invention relates to the field of construction safety management, specifically to an intelligent identification and proactive early warning system for construction site personnel behavior. Background Technology

[0002] With the rapid development of intelligent technology, intelligent identification and proactive early warning systems for construction site personnel behavior based on multimodal fusion have demonstrated significant advantages in the safety management of deep foundation pit projects. Deep foundation pit projects involve complex and variable environments, multi-trade collaborative operations, and high-risk scenarios such as falls from heights, collapses, and open flames. Traditional manual supervision methods struggle to achieve 24 / 7 uninterrupted, high-precision safety monitoring, especially under adverse weather conditions and complex construction conditions, where the problems of delayed detection and slow response to safety hazards are particularly prominent.

[0003] Traditional construction site safety management relies primarily on manual inspections and simple video surveillance systems. Manual inspections are inefficient and struggle to cover all high-risk areas. Traditional video surveillance systems only record the scene, lacking real-time analysis and early warning capabilities, making it difficult to detect safety hazards promptly. For high-risk work scenarios such as working at heights and in hazardous areas, safety supervision is challenging, and hazard detection is delayed. Once violations are detected, manual intervention is often required, resulting in slow response times and difficulty in timely stopping dangerous behavior. Existing early warning methods are simplistic and lack multi-level response mechanisms, failing to provide differentiated responses to violations of varying severity. While some intelligent monitoring systems exist, most are limited to image analysis from a single camera, lacking multimodal fusion technology, exhibiting poor adaptability to complex construction site environments, and having incomplete early warning mechanisms, failing to achieve automated closed-loop management from identification to processing.

[0004] Therefore, this invention provides an intelligent identification and proactive early warning system for construction site personnel behavior. This system constructs a dynamic twin of the physical construction site and a virtual model to achieve full-dimensional perception and spatiotemporal reconstruction of personnel behavior. It also utilizes reinforcement learning algorithms to autonomously optimize early warning strategies. This system is applicable to complex construction scenarios such as large-scale infrastructure, building construction, and municipal engineering, and is particularly effective for risk prediction and proactive intervention for personnel in high-risk work areas. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is to provide an intelligent identification and proactive early warning system for construction site personnel behavior. This system solves the problems of single early warning methods and slow response speed in traditional monitoring systems. By using reinforcement learning to autonomously learn early warning strategies for different scenarios, the system's generalization ability is enhanced.

[0006] The technical solution adopted in this invention is to provide an intelligent recognition and proactive early warning system for construction site personnel behavior, characterized by comprising a physical layer module, a twin layer module, a decision layer module, and an application layer module: The physical layer module collects sensor data, device status, and personnel location information; The twin layer module provides fused twin scene data; The decision-making module undergoes reinforcement learning training, and once training is complete, it sends early warning decisions and recommended actions. The application layer module sends control commands to the decision layer in response to the early warning decisions and recommended operations.

[0007] Preferably, the physical layer module includes a spatial positioning device, a behavior recognition device, and environmental sensors. The spatial positioning device employs a centimeter-level UWB positioning system, with a positioning tag embedded in the personnel's safety helmet and a positioning terminal installed on the construction equipment. This is used to collect the real-time coordinates and attitude angles of personnel and equipment, and to generate a 3D point cloud by scanning the construction work area using LiDAR. The behavior recognition device uses a high-definition network camera with a resolution of 8 megapixels or higher, integrating an edge computing chip to identify behaviors such as "not wearing a safety helmet," "unauthorized entry into a dangerous area," and "personnel falling" in real time on the device. The environmental sensors utilize distributed deployment of temperature, humidity, and wind speed sensors, transmitting data wirelessly to the twin layer module using LoRa.

[0008] Preferably, the twin layer module constructs a 3D digital model by combining BIM and laser point clouds. Dynamic mapping from the physical scene to the virtual space is achieved through LOD lightweighting, incremental updates, and multi-source data fusion. LOD lightweighting uses an edge-folding algorithm to convert the design-stage BIM model into a lightweight operational model to meet different detail requirements. An incremental simplification update mechanism is used to locally update environmental factors such as equipment movement and material stacking, reducing data transmission volume. Multi-source data fusion and 3D reconstruction fuse positioning, vision, radar, and environmental data through a federated learning framework to generate a 3D twin scene. After fusion, the coordinate mapping between the virtual model and the physical space is achieved through the ICP point cloud registration algorithm.

[0009] Preferably, the behavior recognition model in the decision-making layer module adopts a CNN-Transformer cross-modal fusion network, integrating visual and radar data. The CNN extracts appearance features and outputs feature vectors, while the radar branch inputs millimeter-wave radar point clouds and extracts micro-motion features such as vertical displacement changes when a person falls and limb swing frequency when climbing through PointNet. The output feature vectors are aligned and weighted using an attention mechanism, and finally, the fused features are input into the Transformer classification head to obtain the output person behavior recognition type and confidence level.

[0010] Preferably, the state space of the reinforcement learning decision model in the decision layer module is defined as a high-dimensional vector containing personnel, equipment, environment, and historical behavior; the action space is a set of discretized early warning actions, including local alarms, broadcasts, drone warnings, and equipment shutdown; and the reward function is a multi-objective optimization function capable of balancing response and cost. The reinforcement learning decision model trains the agent, and the training process includes: In the twin-layer virtual simulation environment, a large number of historical violation scenarios are generated, and the experience replay pool is initialized; Obtain the state from the environment, select actions using a greedy strategy, obtain the next moment and reward after executing the action, and store the transition sample into the experience replay pool; Randomly collect batches of samples from the experience replay pool and update the Q network parameters using the following loss function.

[0011] Preferably, the early warning strategy library in the decision-making layer module includes static strategies with predefined basic rules and dynamic strategies that store the optimal actions generated by the reinforcement learning agent. The basic rule library consists of static strategies, which are hard rules defined based on industry standards and historical accident cases. The dynamic strategy library consists of learning strategies, which store the optimal actions generated by the reinforcement learning agent, represented by state, action, and value. The value is a long-term cumulative reward that can quickly match similar scenarios.

[0012] Preferably, the digital twin visualization platform in the application layer module overlays the 3D scene of the twin layer with the early warning information of the decision layer in real time, supporting access from multiple terminals including PCs, mobile devices, and large screens.

[0013] The application layer module is configured to make a decision based on the warning. Trigger the corresponding execution warning terminal: like This will trigger the camera's audio and visual alarm; like Then the IP broadcast system is invoked to broadcast messages based on the location of the person in the designated area; like Then, the drone is dispatched to plan its path using the RRT obstacle avoidance algorithm and fly to the warning area; like Then, a shutdown command is sent to the device via the Modbus protocol.

[0014] Preferably, the 3D scene of the twin layer module is overlaid and rendered in real time with the early warning information of the decision layer module. When a high-risk early warning is issued, the emergency command system automatically retrieves the emergency plan, displays the resource distribution in the GIS map, and initiates a video conference to assist command.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By combining BIM with laser point cloud to construct a three-dimensional digital model, dynamic mapping from physical scene to virtual space is realized. The ICP point cloud registration algorithm is adopted to achieve a coordinate mapping accuracy of ±3cm between virtual model and physical space. Through LOD lightweighting and incremental update mechanism, the system can adapt to environmental changes such as equipment movement and material stacking in real time. 2. Employing a CNN-Transformer cross-modal fusion network, integrating visual and radar data, it accurately identifies behaviors such as "not wearing a safety helmet," "unauthorized entry into a dangerous area," and "person falls." 3. Based on the improved DQN reinforcement learning algorithm, a multi-objective reward function is constructed for intelligent decision optimization. The multi-objective reward function balances risk and cost and avoids resource waste caused by excessive warnings. 4. The system automatically selects the optimal early warning strategy based on the risk level. It can automatically link with the emergency command system to quickly retrieve contingency plans, locate resources, and activate the command channel when a high-risk warning is issued, greatly improving the speed and efficiency of emergency response. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a system structure diagram of the present invention. Detailed Implementation

[0017] To better understand the purpose, system architecture, and functional implementation of this embodiment, the embodiments and features in the embodiments of this application can be combined with each other without conflict. The exemplary embodiments disclosed in this application will be described below with reference to the accompanying drawings, which include specific technical details disclosed in this embodiment to aid understanding; however, these details should be considered exemplary rather than restrictive. Therefore, those skilled in the art should understand that various improvements and adjustments can be made to the embodiments described herein without departing from the scope and core ideas of the invention. Similarly, for clarity, detailed descriptions of well-known technologies, functions, and structures (such as standard image processing algorithms and common communication protocols) are omitted in the following description.

[0018] Example 1 Figure 1 This is a system structure diagram of the present invention.

[0019] like Figure 1 As shown, a construction site personnel behavior intelligent recognition and proactive early warning system consists of steps S110 to S140.

[0020] In step S110, the physical layer module includes a spatial positioning device, a behavior recognition module, and environmental sensors, which collect sensor data, device status, and personnel location information.

[0021] In step S120, the twin layer module includes 3D model construction, real-time data mapping, multi-source data mapping and virtual simulation, providing fused twin scene data, including personnel behavior status, equipment operation status and environmental data.

[0022] In step S130, the decision layer module includes a behavior recognition model, a reinforcement learning decision model, and an early warning strategy library. Reinforcement learning training is performed, and after training is completed, early warning decisions and recommendation operations are sent.

[0023] In step S140, the application layer module includes a digital twin visualization platform, an execution early warning terminal, and an emergency command system, which sends control commands to the early warning decisions and recommended operations provided by the decision-making layer.

[0024] According to an embodiment of the present invention, in operation S110, the spatial positioning device uses a centimeter-level UWB positioning system to locate personnel positions and equipment operating status. Personnel safety helmets have built-in positioning tags, and construction equipment is equipped with positioning terminals to collect real-time coordinates and attitude angles. The construction work area is scanned using LiDAR to generate a 3D point cloud of the construction area for subsequent geometric reconstruction in the virtual simulation of the twin layer. The behavior recognition device uses a high-definition network camera with at least 8 megapixels, supporting clear imaging in low-light environments. It has 360° panoramic stitching and 30x optical zoom capabilities, covering an area with a radius of 200 meters, providing a high-resolution video source for behavior recognition. Behavior recognition, through the integration of an edge computing chip, performs preliminary identification of various behaviors such as "not wearing a safety helmet," "unauthorized entry into a dangerous area," and "personnel falling" in real time at the device end. Environmental sensors, such as temperature, humidity, and wind speed sensors, are deployed in a distributed manner and transmitted wirelessly to the twin layer via LoRa for environmental risk assessment.

[0025] According to an embodiment of the present invention, in operation S120, a three-dimensional digital model is constructed by combining BIM and laser point cloud, including details such as building structure, equipment layout, and pipeline routing. Data information collected in the physical layer is transmitted via a 5G network. Simultaneously, the video stream, sensor data, and 3D model collected in the physical layer are spatiotemporally aligned. Time synchronization of all devices is achieved via the NTP protocol, establishing a global coordinate system with the site reference point as the origin, unifying the spatial coordinate reference of each device, thereby realizing dynamic mapping of personnel, equipment positions, and status. The twin layer, based on multi-dimensional data, realizes dynamic mapping from the physical scene to the virtual space, mainly through LOD lightweighting, incremental updates, and multi-source data fusion.

[0026] By converting the BIM model in the design phase into a lightweight model for runtime through an edge-folding algorithm, different detailed requirements can be met, as shown in equation (1) below: (1) in, The original BIM model, For the first Simplification rate of Level LOD This is a function for simplifying the grid.

[0027] The incremental simplified update mechanism is used to locally update environmental factors such as equipment movement and material stacking, thereby reducing the amount of data transmission, as shown in equation (2) below: (2) in, For the updated model, For the old model, For the change, For incremental model fusion operations, when the device moves a distance >1m or personnel density >5 people / m 3 When this occurs, the scene is redrawn.

[0028] Multi-source data fusion and 3D reconstruction: By fusing localization, vision, radar and environmental data through a federated learning framework, a 3D twin scene is generated, as shown in equation (3) below: (3) in, For the federated average fusion algorithm, For location data, For visual data, For point cloud data, For environmental data, after fusion, the ICP point cloud registration algorithm is used to realize the coordinate mapping between the virtual model and the physical space, and the position accuracy can be improved to ±3cm.

[0029] According to an embodiment of the present invention, in operation S130, the behavior recognition model is the core perception component of the decision layer. Based on multimodal data, it realizes real-time recognition of dangerous behaviors such as construction workers not wearing safety belts, illegally climbing, and entering restricted areas, providing accurate behavioral state input for reinforcement learning. The behavior recognition model adopts a CNN-Transformer cross-modal fusion architecture, integrating the advantages of visual and radar data. The visual branch extracts the target bounding box of the person through the YOLOv8 model by inputting high-definition camera video frames, and then extracts the appearance features of safety helmet, clothing, and posture contour through CNN, outputting a feature vector. The radar branch takes millimeter-wave radar point cloud as input and extracts micro-motion features such as vertical displacement changes when a person falls and limb swing frequency when climbing using PointNet, and outputs feature vectors. Using attention mechanisms and Feature alignment and weighted fusion are performed as shown in equation (4): (4) in, The number of radar point clouds, and the final fused features The input Transformer classification head ultimately yields the output of the human behavior recognition type and confidence level.

[0030] When the decision-making layer uses data fused from the twin layer, it trains the agent through deep reinforcement learning to enable it to autonomously select the optimal early warning action.

[0031] state space It includes a high-dimensional vector of personnel, equipment, environment, and historical behavior, as shown in equation (5) below: (5) in, Personnel location and movement vector , For the device position and motion parameter vector , The environmental parameter vector includes wind speed, temperature, and humidity. The data represents the characteristics of violations recorded within the last 5 minutes.

[0032] Action space The discrete set of early warning actions includes local alarms, broadcasts, drone warnings, and equipment shutdown, as shown in equation (6) below: (6) in, This is the action vector for local audible and visual alarms. For directional broadcast action vectors, For drone warning action vectors, Equipment shutdown operation space.

[0033] reward function A multi-objective optimization function that can balance response and cost, as shown in equation (7): (7) in, To reduce risk, the degree of risk reduction increases as people move away from danger after a warning, and decreases as they move away. To ensure timely response, the response time is increased if personnel respond within 10 seconds, and decreased if it exceeds 10 seconds. To determine the cost of early warning, deploying drones costs -10, while broadcasting costs -2. , and The weighting is dynamic, increasing according to the degree of danger at each construction stage. When the weight of one increases, the weights of others decrease accordingly.

[0034] The agent is trained using an improved DQN algorithm. First, a large number of historical violation scenarios are generated in a twin-layer virtual simulation environment to initialize the experience replay pool. Secondly, obtain the state from the environment. ,by Greedy strategy selects action Obtain the next moment after performing the action. With rewards Transfer samples deposit Finally from Randomly collect batches of samples and update the Q-network parameters using the following loss function, as shown in equation (8): (8) in, To replay from the experience pool The state obtained by mid-sampling ,action ,award and the next state Expected value For the next action, As a discount factor, These are the current Q-network parameters. For the target Q network parameters, For the next state according to The maximum Q value obtained.

[0035] The early warning strategy library is a combination of rules and learning, containing predefined basic rules and dynamic reinforcement learning strategies. It implements a two-layer decision-making mechanism combining safety net rules and intelligent optimization to avoid decision-making errors in extreme scenarios. The basic rule library consists of static strategies, based on industry standards and historical accident cases, defining rigid rules such as: the distance between personnel and the load on the tower crane... Then trigger action Equipment shutdown; if wind speed And the action is triggered if the workers at height are not evacuated. Broadcasts and drones were used simultaneously to evacuate construction workers. A dynamic policy library, representing the learned policies, stores the optimal actions generated by the reinforcement learning agent, using state, action, and value as a long-term cumulative reward to quickly match similar scenarios.

[0036] According to an embodiment of the present invention, in operation S140, the application layer executes the warning and links emergency command according to the warning action output by the decision layer. In the digital twin visualization platform, the three-dimensional scene of the twin layer and the warning information of the decision layer are superimposed in real time through WebGL technology.

[0037] According to the action Trigger the corresponding terminal: like This will trigger the camera's audio and visual alarm; like Then the IP broadcast system is invoked to broadcast messages based on the location of the person in the designated area; like Then, the drone is dispatched to plan its path using the RRT obstacle avoidance algorithm and fly to the warning area; like Then, a shutdown command is sent to the device via the Modbus protocol.

[0038] The emergency command system is used for high-risk early warning. When necessary, the system automatically retrieves the emergency plan, displays the resource distribution on the GIS map, and initiates a video conference to assist command.

[0039] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0040] The specific embodiments described above do not constitute a limitation on the scope of protection disclosed in this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A construction site worker behavior intelligent recognition and proactive early warning system, characterized in that, It includes a physical layer module, a twin layer module, a decision layer module, and an application layer module: The physical layer module collects sensor data, device status, and personnel location information; The twin layer module provides fused twin scene data; The decision-making module undergoes reinforcement learning training, and once training is complete, it sends early warning decisions and recommended actions. The application layer module sends control commands to the decision layer in response to the early warning decisions and recommended operations.

2. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The physical layer module includes spatial positioning devices, behavior recognition devices, and environmental sensors; The spatial positioning equipment adopts a centimeter-level UWB positioning system, with positioning tags built into personnel safety helmets and positioning terminals installed on construction equipment to collect real-time coordinates and attitude angles of personnel and equipment. The LiDAR is used to scan the construction work area to generate a three-dimensional point cloud. The behavior recognition device uses a high-definition network camera with a resolution of 8 megapixels or higher and integrates an edge computing chip to identify behaviors such as "not wearing a safety helmet", "unauthorized entry into a dangerous area" and "person falling" in real time on the device. The environmental sensors transmit data to the twin layer module via LoRa wireless transmission through a distributed deployment of temperature, humidity, and wind speed sensors.

3. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The twin layer module constructs a three-dimensional digital model by combining BIM and laser point cloud, and realizes dynamic mapping from physical scene to virtual space through LOD lightweighting, incremental update and multi-source data fusion. LOD lightweighting uses an edge-folding algorithm to convert the BIM model in the design phase into a lightweight model in the operational phase to meet different detailed requirements. The incremental simplified update mechanism is used to locally update environmental factors such as equipment movement and material stacking, thereby reducing the amount of data transmission. Multi-source data fusion and 3D reconstruction: By fusing localization, vision, radar and environmental data through a federated learning framework, a 3D twin scene is generated. After fusion, the coordinate mapping between the virtual model and the physical space is realized through the ICP point cloud registration algorithm.

4. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The behavior recognition model in the decision-making layer module adopts a CNN-Transformer cross-modal fusion network to integrate visual and radar data; CNN extracts appearance features and outputs feature vectors. The radar branch inputs millimeter-wave radar point cloud and uses PointNet to extract micro-motion features such as vertical displacement changes when a person falls and limb swing frequency when climbing. The output feature vectors are aligned and weighted using an attention mechanism. Finally, the fused features are input into the Transformer classification head to obtain the output person behavior recognition type and confidence level.

5. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The state space of the reinforcement learning decision model in the decision layer module is defined as a high-dimensional vector containing personnel, equipment, environment, and historical behavior. The action space is a set of discrete early warning actions, including local alarms, broadcasts, drone warnings, and equipment shutdown; The reward function is a multi-objective optimization function that can balance response and cost.

6. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The reinforcement learning decision-making model in the decision layer module trains the agent, and the training process includes: In the twin-layer virtual simulation environment, a large number of historical violation scenarios are generated, and the experience replay pool is initialized; Obtain the state from the environment, select actions using a greedy strategy, obtain the next moment and reward after executing the action, and store the transition sample into the experience replay pool; Randomly collect batches of samples from the experience replay pool and update the Q network parameters using the following loss function.

7. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The early warning strategy library in the decision layer module includes static strategies with predefined basic rules and dynamic strategies that store the optimal actions generated by the reinforcement learning agent. The basic rule base is a static policy, consisting of hard rules defined based on industry standards and historical accident cases; The dynamic policy library is the learning policy, which stores the optimal actions generated by the reinforcement learning agent. It is represented by state, action and value. The value is the long-term cumulative reward that can quickly match similar scenarios.

8. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The digital twin visualization platform in the application layer module overlays the 3D twin scene with the early warning information of the decision layer in real time, supporting access from multiple terminals including PCs, mobile devices, and large screens.

9. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The application layer module is configured to make early warning decision actions. Trigger the corresponding execution warning terminal: like This will trigger the camera's audio and visual alarm; like Then the IP broadcast system is invoked to broadcast messages based on the location of the person in the designated area; like Then, the drone is dispatched to plan its path using the RRT obstacle avoidance algorithm and fly to the warning area; like Then, a shutdown command is sent to the device via the Modbus protocol.

10. The intelligent identification and proactive early warning system for construction site personnel behavior according to claim 1, characterized in that, The 3D scene of the twin layer module is overlaid and rendered in real time with the early warning information of the decision layer module. When a high-risk warning is issued, the emergency command system automatically retrieves the emergency plan, displays the resource distribution on the GIS map, and initiates a video conference to assist command.

Images