A construction project construction safety hazard tracking management system and method

Abstract

The present invention discloses a tracking and management system and method for potential safety hazards in construction engineering, which relates to the technical field of safety management. It includes: a three-dimensional perception module: fusing the information data set to establish a point cloud model of the dynamic construction scene; a posture analysis module: enhancing the images in the point cloud model of the dynamic construction scene, and at the same time reconstructing the actions of the occluded parts in the enhanced point cloud model of the dynamic construction scene to determine the violation actions; an interactive verification module: comparing the violation actions with the BIM construction progress model, and when the violation actions do not start the protection measures as planned, generating a virtual safety protection layer and sending it to the AR patrol terminal. The present invention can capture the three-dimensional structure changes on the construction site in real time, and at the same time uses dynamic multi-exposure fusion and concentration grading processing technologies to effectively improve the image blurring problem caused by poor lighting conditions or air pollution, and improve the image recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of safety management, and particularly to a system and method for tracking and managing potential safety hazards in construction engineering Background Art

[0002] With the acceleration of the urbanization process and the booming development of the construction industry, construction projects are becoming increasingly complex and expanding in scale, which poses higher requirements for safety management at the construction site. However, traditional safety management methods mostly rely on manual inspections, paper records, and experience-based judgments. This method is not only inefficient but also difficult to cope with complex construction environments and frequently occurring potential safety hazards.

[0003] The construction industry has always been one of the high-risk areas for safety accidents. According to statistical data, the accident incidence rate in the construction industry is higher than that in many other industries, especially safety accidents such as falls from heights, object strikes, and mechanical injuries, which not only pose a serious threat to the lives of workers but also have a negative impact on the economic benefits and social image of enterprises.

[0004] In recent years, modern information technologies such as the Internet of Things (IoT), big data analysis, and artificial intelligence have developed rapidly and have gradually been applied to construction safety management. For example, IoT technology can achieve real-time monitoring of the construction site; big data analysis helps to identify potential safety risks and predict the likelihood of accidents; and artificial intelligence can improve the scientificity and effectiveness of safety decision-making.

[0005] The Chinese invention patent with the publication number CN116823529A discloses a security intelligent management system based on behavioral big data analysis, including a construction site area division module, a behavioral recognition terminal layout module, an active personnel identity recognition module, a foreign personnel behavior recognition and processing module, a construction worker safety monitoring and analysis module, a display terminal, and an information storage repository. The invention counts the number of workers wearing safety helmets incorrectly and not wearing safety helmets through intelligent high-definition cameras, not only warns the corresponding personnel but also counts the number of workers wearing safety helmets incorrectly and not wearing safety helmets, reflecting the safety awareness intensity of the construction worker group, more clearly demonstrating the security control effect of the construction site, reducing the likelihood of safety accidents, and from another perspective, more highlighting the effectiveness of the construction site security management work, and can accurately provide a reliable decision-making basis for the smooth progress of the construction site security management work.

[0006] In the process of security prevention of the above-mentioned and similar intelligent management systems, although the existing video surveillance systems can cover key areas of the construction site, it is particularly difficult to capture specific violations due to the diverse and irregular actions of construction workers, especially during dynamic operations. For example, at a busy construction site, workers may move quickly, bend down, turn around, or be partially blocked by other objects, increasing the difficulty of identification. That is to say, in the process of automatically identifying violations (such as not wearing a safety helmet) and abnormal situations (such as not setting up a protective net for high-altitude operations), false alarms and missed alarms may occur due to the complex construction site environment. Summary of the Invention

[0007] The purpose of the present invention is to provide a tracking and management system and method for construction safety hazards in building engineering to solve the problems raised in the above background technology.

[0008] To achieve the above purpose, the present invention provides the following technical solution: A tracking and management system for construction safety hazards in building engineering, including:

[0009] Three-dimensional perception module: By acquiring the information data set of the construction site and fusing the information data set, a dynamic construction scene point cloud model is established.

[0010] Posture analysis module: By acquiring influence data, enhancing the images in the dynamic construction scene point cloud model, reconstructing the actions of the occluded parts in the enhanced dynamic construction scene point cloud model, and determining violation actions through a spatio-temporal encoder, including:

[0011] S1: According to the influence parameters, adjust the parameters of the image enhancement algorithm, determine the image recognition strategy, and enhance the blurred image through the image recognition strategy.

[0012] S2: Through the hierarchical feature fusion network and the fused data, perform feature complementarity on the occluded target, reconstruct the human body contour, and obtain the actions of the occluded parts.

[0013] S3: Through the spatio-temporal encoder, perform long-sequence modeling on the human postures and tool operation trajectories in consecutive video frames to identify violation action patterns.

[0014] Interaction verification module: Compare the violation actions with the BIM construction progress model. When the violation actions do not start the protection measures as planned, a virtual safety protection layer is automatically generated and sent to the AR patrol terminal.

[0015] Furthermore, enhancing the blurred image includes:

[0016] S1.1: Dynamic multi-exposure fusion: Combine images taken at different exposure times to obtain an HDR image. Then, based on the HDR image, adjust the working mode, determine the lighting differences in different regions, and obtain the processed image;

[0017] S1.2: Concentration grading processing: Divide the construction site according to the PM10 concentration level, and perform image processing based on the divided construction site. Specifically:

[0018] When the environment where the PM10 concentration is located is a low-concentration environment, extract the high-level features of the image through downsampling, restore the image details through upsampling, and add skip connections between different levels;

[0019] When the environment where the PM10 concentration is located is a high-concentration environment, obtain the distance and speed information of the object through a millimeter-wave radar for object positioning and discrimination.

[0020] Furthermore, obtaining the processed image includes:

[0021] M1: Determine the dynamic area: Obtain different exposure images of the same scene, divide the areas in the scene according to the motion vectors between adjacent frame images, and obtain the finally generated HDR image based on the divided areas;

[0022] M2: Light intensity gradient analysis: Based on the HDR image, obtain the light intensity change rate within a preset time, and determine the working mode according to the light intensity change rate;

[0023] M3: Regionalization processing: Based on the working mode, determine the lighting differences in different regions, and perform image processing based on the lighting differences and specific regions.

[0024] Furthermore, obtaining the finally generated HDR image includes:

[0025] M1.1: Collect triple-exposure images: Obtain three different exposure images of the same scene through three different exposure times;

[0026] M1.2: Determine the motion area: According to the optical flow vector of the pixel points in each exposure image, obtain the corresponding modulus length of the pixel points, and compare the modulus length with a preset threshold. When the modulus length is greater than the preset threshold, the area where the pixel points are located is a high-motion area; otherwise, the area where the pixel points are located is a low-motion area or a stationary area. The specific calculation formula for the modulus length is:

[0027] ,

[0028] Where: is the optical flow vector corresponding to the pixel at position (x, y). is the abscissa of the pixel. is the ordinate of the pixel;

[0029] M1.3: Determine the exposure combination: According to the region category corresponding to the region where the pixel is located, weight the three-exposure images corresponding to the pixel, specifically:

[0030] When the region where the pixel is located is a high-motion region, the weight distribution of the three-exposure images corresponding to the pixel is: the weight of the short-exposure image is set to 1, the weight of the medium-exposure image is set to 0, and the weight of the long-exposure image is set to 0;

[0031] When the region where the pixel is located is a low-motion region or a static region, the weight distribution of the three-exposure images corresponding to the pixel is: the weight of the short-exposure image is set to 0.3, the weight of the medium-exposure image is set to 0.5, and the weight of the long-exposure image is set to 0.2;

[0032] M1.4: Determine the HDR value: According to the weight distribution of the three-exposure images corresponding to the pixel, obtain the brightness value of the pixel in the finally generated HDR image, and according to the brightness value, obtain the finally generated HDR image. The brightness value of the pixel in the finally generated HDR image is specifically:

[0033] ,

[0034] where: is the brightness value of the pixel (x, y) in the finally generated HDR image, is the weight of the i-th exposure image at the pixel (x, y), is the brightness value of the pixel (x, y) in the i-th exposure image, is the type index of the exposure image.

[0035] Furthermore, determine the working mode, including:

[0036] M2.1: Obtain the average light intensity change ratio: According to the light intensity change between two adjacent HDR images, determine the average light intensity change rate, specifically:

[0037] ,

[0038] where: is the average light intensity change rate, is the light intensity difference between two adjacent HDR image frames, is the total number of pixels in the image, is the time difference between two measurements of the HDR brightness value, is the abscissa of the pixel point, is the ordinate of the pixel point;

[0039] M2.2: Determine the working mode: Compare the average light intensity change rate with a preset change threshold. When the average light intensity change rate is greater than the preset change threshold, the working mode is adjusted to the short exposure mode. When the average light intensity change rate is less than the preset change threshold, the working mode is adjusted to the long exposure mode. Otherwise, the working mode remains unchanged.

[0040] Furthermore, image processing is performed, including:

[0041] M3.1: Construction scene zoning: According to the working mode, obtain HDR images with different exposure levels. At the same time, quantify the light differences between different regions in the HDR image through a histogram, determine the upper limit threshold of the low-light region and the lower limit threshold of the high-light region, and determine the zoning area according to the upper limit threshold of the low-light region and the lower limit threshold of the high-light region. Specifically:

[0042] Compare the light brightness of different regions with the upper limit threshold of the low-light region and the lower limit threshold of the high-light region. When the light brightness is less than the upper limit threshold of the low-light region, the region where the light brightness is located is the low-light region. When the light brightness is greater than the lower limit threshold of the high-light region, the region where the light brightness is located is the high-light region. Otherwise, the region where the light brightness is located is the medium-light region;

[0043] M3.2: Identify specific regions: Identify shadow regions and reflective regions in each zoning area to obtain specific regions;

[0044] M3.3: Image processing: According to the zoning area where the specific region is located, perform image processing. Specifically:

[0045] When the zoning area is the low-light region, perform image processing through image enhancement technology;

[0046] When the zoning area is the medium-light region, perform image processing through local tone mapping technology;

[0047] When the zoning area is the high-light region, perform image processing through a shadow removal algorithm.

[0048] Furthermore, obtain the actions of the occluded parts, including:

[0049] S2.1: Obtain feature information: Process the enhanced image through a hierarchical feature fusion network to obtain multi-level feature information;

[0050] S2.2: Determine the occlusion action: Combine the feature information with the fused data to complement the features of the occlusion target and obtain the angles of the actions occurring in adjacent time periods.

[0051] Furthermore, obtaining the angles of the actions occurring in adjacent time periods includes:

[0052] N1: Feature complementation: Complement the data of different sensors to obtain the position of the occluded part. Specifically:

[0053] ,

[0054] Where: is the finally reconstructed complete human body model, is the partial human body model directly obtained from the original image or sensor data, is the estimated value of the occluded part;

[0055] N2: Human body contour reconstruction: According to the position of the occluded part, reconstruct the three-dimensional pose of the human body and obtain the pose parameters that minimize the prediction. Specifically:

[0056] ,

[0057] Where: is the finally optimized best pose parameter, is the feature vector extracted from the multi-modal data, is the feature vector of the human body model generated according to the pose parameter θ, is the pose parameter before optimization;

[0058] N3: Determine the action: According to the pose parameters that minimize the prediction, obtain the angles of the actions occurring at the occluded part in the adjacent time period. Specifically:

[0059] ,

[0060] Where: is the change of the pose parameter in the adjacent time period, is the finally optimized best pose parameter corresponding to time t + 1, is the finally optimized best pose parameter corresponding to time t.

[0061] Furthermore, sending the virtual safety protection layer to the AR inspection terminal includes:

[0062] W1: Determine the construction stage: According to the time when the violation action occurs, determine the construction stage corresponding to the time when the violation action occurs from the BIM construction progress model;

[0063] W2: Determine the construction location: Based on the location where the violation occurred and the area involved in the construction stage, obtain the relationship between the location where the violation occurred and the area involved in the construction stage;

[0064] W3: Safety measure matching: Compare the safety measures specified in the construction stage with the violation. When the violation does not match the safety measures, proceed to the next step; otherwise, delete the violation.

[0065] W4: Send to the AR inspection terminal: When the violation does not start the protection measures as planned, automatically generate the virtual safety protection layer, convert the data of the virtual safety protection layer into three-dimensional model data, and transmit the three-dimensional model data to the AR device through the wireless communication protocol.

[0066] A method for tracking and managing construction safety hazards in construction projects uses a system for tracking and managing construction safety hazards in construction projects as described in any one of the above.

[0067] Compared with the prior art, the beneficial effects of the present invention are:

[0068] First: The present invention establishes a dynamic construction scene point cloud model by integrating multiple information data sets, enabling the system to capture the three-dimensional structural changes at the construction site in real time. At the same time, using dynamic multi-exposure fusion and concentration grading processing technologies, it effectively improves the problem of image blurring caused by poor lighting conditions or air pollution and improves the image recognition accuracy.

[0069] Second: The present invention uses a hierarchical feature fusion network to perform feature complementation on occluded targets and reconstruct the human body contour, enabling accurate recognition of actions even when part of the body is occluded. And through a spatio-temporal encoder, long-sequence modeling of the human body posture and tool operation trajectory in consecutive video frames is carried out, which helps to accurately identify the violation action pattern and provides a basis for subsequent safety management.

[0070] Third: The present invention compares the violation with the BIM construction progress model to confirm whether it complies with safety specifications. When it is found that the protection measures are not executed as planned, it can automatically create a virtual safety protection layer and send it to the AR inspection terminal to help on-site management personnel respond quickly. Description of the Drawings

[0071] Figure 1 It is the system block diagram of the system for tracking and managing construction safety hazards in construction projects of the present invention;

[0072] Figure 2 It is the flow schematic diagram of obtaining the finally generated HDR image of the present invention;

[0073] Figure 3Schematic diagram of the image processing process in the present invention;

[0074] Figure 4 Brightness value histogram in the present invention. Specific implementation manner

[0075] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0076] In the process of security prevention of existing intelligent management systems, although existing video surveillance systems can cover key areas of the construction site, due to the diverse and irregular actions of construction workers, especially during dynamic operations, it becomes particularly difficult to capture specific violations. For example, at a busy construction site, workers may move quickly, bend down, turn around, or be partially blocked by other objects, increasing the difficulty of recognition. That is to say, in the process of automatically identifying violations (such as not wearing a safety helmet) and abnormal situations (such as not setting up a protective net for high-altitude operations), false alarms and missed alarms may occur due to the complex construction site environment. The technical solution of this application establishes a dynamic construction scene point cloud model and real-time data collection, reconstructs the occluded human actions using a hierarchical feature fusion network, identifies the violation action patterns in continuous video frames using a spatio-temporal encoder, and compares the detected violation actions with the BIM construction progress model, thereby ensuring the implementation of safety measures. At the same time, a virtual safety protection layer can be automatically generated when necessary and sent to the AR patrol terminal, thus realizing the safety monitoring and instant warning of the construction site.

[0077] Refer to Figures 1 - 4 , this embodiment provides a building engineering construction safety hazard tracking and management system. The building engineering construction safety hazard tracking and management system includes a three-dimensional perception module, a posture analysis module, and an interaction verification module. Among them, the three-dimensional perception module obtains the information data set of the construction site, fuses the information data set, and establishes a dynamic construction scene point cloud model. The posture analysis module enhances the images in the dynamic construction scene point cloud model established in the three-dimensional perception module by obtaining influence data, reconstructs the actions of the occluded parts in the enhanced dynamic construction scene point cloud model, and determines the violation actions through a spatio-temporal encoder. The interaction verification module compares the violation actions identified in the posture analysis module with the BIM construction progress model. When the violation actions do not start the protection measures as planned, a virtual safety protection layer is automatically generated and sent to the AR patrol terminal.

[0078] In this embodiment, the 3D perception module is used to obtain an information dataset of the color, shape, texture, distance, and object category of the construction site through a visible light camera, a millimeter-wave radar, and an infrared thermal imaging device, and fuse the obtained information dataset to obtain a fused information data. At the same time, a dynamic construction scene point cloud model is established according to the fused information data. Specifically, the color, shape, and texture of an object are identified through a visible light camera, the distance information is provided by a millimeter-wave radar, and different objects are distinguished by an infrared thermal imager.

[0079] In the process of specific implementation, a visible light camera, a millimeter-wave radar, and an infrared thermal imager are installed at the construction site. The visible light camera outputs an RGB image with a resolution of 1920*1080, the millimeter-wave radar outputs point cloud data, and the infrared thermal imager outputs a thermal map.

[0080] It should be noted that, in order to make the output data of the visible light camera, the millimeter-wave radar, and the infrared thermal imaging device consistent, in this embodiment, the obtained data (RGB image, point cloud data, and thermal map) is processed through spatio-temporal synchronization calibration, which includes timestamp synchronization and calculation of the spatial transformation matrix. Specifically, in the process of timestamp synchronization, with the GPS time as a reference, the timestamp of the data is corrected by minimizing the error function to make its timestamp consistent. In the process of calculating the spatial transformation matrix, the rotation and translation matrices corresponding to different data are determined through a calibration board, so as to convert the coordinate systems of different sensors into a unified world coordinate system. Further, the data after spatio-temporal synchronization calibration can be fused through feature-level or object-level fusion for corresponding data fusion. It should be noted that both the spatio-temporal synchronization calibration and data fusion involved in this embodiment belong to existing conventional technical means, so they will not be specifically elaborated in this embodiment.

[0081] In this embodiment, the posture analysis module is used to obtain the illumination data and dust concentration data of the construction site through the set illumination intensity sensor and dust concentration detector. And according to the obtained illumination data and dust concentration data, the image in the dynamic construction scene point cloud model established in the 3D perception module is enhanced. At the same time, in the dynamic construction scene point cloud model after image enhancement, the actions of the occluded parts are identified and reconstructed, and the violation actions are determined through a spatio-temporal encoder. The specific steps are as follows:

[0082] Step S1: By setting up a light intensity sensor and a dust concentration detector (in this embodiment, the model of the light intensity sensor is set as the TSL2561 sensor, and the model of the dust concentration detector is set as the Thermo Scientific ADR-1500 dust monitor), obtain the light data and dust concentration data at the construction site, and dynamically adjust the image enhancement algorithm parameters (dehazing coefficient and contrast gain) according to the obtained light data and dust concentration data to obtain the corresponding image recognition strategy. And according to this image recognition strategy, enhance the blurred images in the dynamic construction scene point cloud model to obtain clear images. Specifically as follows:

[0083] Step S1.1: Dynamic multi-exposure fusion. That is, combine the images under different exposure times (short exposure, medium exposure, and long exposure) to obtain an image with a high dynamic range and reduce the ghosting problem caused by moving objects. Specifically as follows:

[0084] Step M1: Determine the dynamic area. That is, by obtaining the images of the same scene under different exposure conditions and calculating the motion vectors between adjacent frame images, identify the dynamic area in the scene. At the same time, according to the identified dynamic area, select the corresponding exposure combination. Specifically as follows:

[0085] Step M1.1: Collect three-exposure images. That is, when shooting the same scene, use three exposure times (short exposure: 1 / 1000 s, medium exposure: 1 / 100 s, long exposure: 1 / 10 s) to shoot respectively to obtain three different exposure images of the same scene.

[0086] Step M1.2: Determine the motion area. That is, according to the optical flow vector of the pixel points in the exposure images obtained in Step M1.1, obtain the modulus length corresponding to the pixel point, specifically:

[0087] ,

[0088] Where: is the optical flow vector corresponding to the pixel point at position (x,y), is the abscissa of the pixel point, is the ordinate of the pixel point.

[0089] Furthermore, compare the obtained optical flow vector with a preset threshold (specifically set according to specific data, so the number is not specifically elaborated in this embodiment). When the obtained optical flow vector is greater than the preset threshold, the area where the pixel point corresponding to the optical flow vector is located is a high-motion area. Otherwise, the area where the pixel point corresponding to the optical flow vector is located is a low-motion area or a stationary area.

[0090] In the process of specific implementation, if the optical flow vector of a pixel is (5, 3), then its corresponding modulus is 5.83. Further, in this embodiment, the preset threshold is set to 4. That is to say, if the modulus of this pixel is greater than the preset threshold, the area where this pixel is located is a high-motion area.

[0091] Step M1.3: Determine the exposure combination. That is, according to the category of the motion area corresponding to the area where the pixel is located determined in step M1.2 (high-motion area, low-motion area, and static area), weight distribution is performed on the three-exposure images (short-exposure image, medium-exposure image, and long-exposure image) corresponding to each pixel.

[0092] Specifically, when the motion area corresponding to the area where the pixel is located is a high-motion area, the weight distribution corresponding to this pixel is as follows: the weight of the short-exposure image is set to 1, the weight of the medium-exposure image is set to 0, and the weight of the long-exposure image is set to 0. Specifically:

[0093] ,

[0094] where: is the weight of the short-exposure image, is the weight of the medium-exposure image, is the weight of the long-exposure image.

[0095] Further, when the motion area corresponding to the area where the pixel is located is a low-motion area and a static area, the weight distribution corresponding to this pixel is as follows: the weight of the short-exposure image is set to 0.3, the weight of the medium-exposure image is set to 0.5, and the weight of the long-exposure image is set to 0.2.

[0096] Step M1.4: Determine the HDR value. That is, according to the exposure combination set in step M1.3, obtain the brightness value of the pixel in the finally generated HDR image. Specifically:

[0097] ,

[0098] where: is the brightness value of the pixel (x, y) in the finally generated HDR image, is the weight of the i-th exposure image at the pixel (x, y), is the brightness value of the pixel (x, y) in the i-th exposure image, is the type index of the exposure image.

[0099] In the process of specific implementation, the exposure weight corresponding to the pixel and its corresponding brightness value are set as follows: the weight of the short-exposure image is set to 1, the brightness value of the short-exposure image is set to 50, the weight of the medium-exposure image is set to 0, the brightness value of the medium-exposure image is set to 100, and the weight of the long-exposure image is set to 0, the brightness value of the long-exposure image is set to 150. Then the brightness value of this pixel in the finally generated HDR image is 50.

[0100] Furthermore, according to the brightness value of the pixel in the finally generated HDR image, its corresponding HDR image can be obtained.

[0101] Step M2: Light intensity gradient analysis. That is, according to the HDR image obtained in step M1.4, the change in light intensity within a preset time is obtained, and at the same time, according to the obtained change in light intensity, the working mode is adjusted to adapt to the change in light conditions. Specifically as follows:

[0102] Step M2.1: Obtain the average light intensity change ratio. That is, according to the HDR image obtained in step M1.4, the change in light intensity between two adjacent HDR image frames is obtained, and according to the change in light intensity between two adjacent HDR image frames, the average light intensity change rate is determined. Specifically:

[0103] ,

[0104] Where: is the average light intensity change rate, is the difference in light intensity between two adjacent HDR image frames, is the total number of pixels in the image, is the time difference between two measurements of the HDR brightness value, is the abscissa of the pixel, is the ordinate of the pixel.

[0105] Step M2.2: Determine the working mode. That is, according to the average light intensity change rate obtained in step W1, the adjusted exposure mode is determined. Specifically, the average light intensity change rate obtained in step W1 is compared with a preset change threshold (which is specifically set according to specific data, so the number is not specifically elaborated in this embodiment). When the obtained average light intensity change rate is greater than the preset change threshold, the exposure mode is adjusted to the short-exposure mode. When the obtained average light intensity change rate is less than the preset change threshold, the exposure mode is adjusted to the long-exposure mode. Otherwise, the exposure mode remains unchanged.

[0106] Step M3: Zonal processing. That is, according to the exposure mode determined in step W2, the light differences in different regions are determined. That is to say, according to the light differences, the construction scene is partitioned, specific regions are identified therefrom, and corresponding image processing is performed on the identified specific regions. Specifically as follows:

[0107] Step M3.1: Partitioning of the construction scene. That is, according to the exposure mode determined in step W2, multiple images with different exposure levels are obtained and converted into HDR images. At the same time, the light differences between different regions in the HDR image are quantified through a histogram to obtain the average brightness of each region. Further, according to the quantified light differences, the light brightness corresponding to the 5th percentile is used as the upper threshold of the low-light region, and the light brightness corresponding to the 95th percentile is used as the lower threshold of the high-light region.

[0108] That is to say, the light brightness of different regions is compared with the upper threshold of the low-light region and the lower threshold of the high-light region. When the light brightness is less than the upper threshold of the low-light region, the region where the light brightness is located is the low-light region. When the light brightness is greater than the lower threshold of the high-light region, the region where the light brightness is located is the high-light region. Otherwise, the region where the light brightness is located is the medium-light region.

[0109] During the specific implementation process, a brightness value histogram as shown in Figure 4 is set. Referring to Figure 4 it can be known that: the brightness value corresponding to the 5th percentile is 40 nits, that is, the upper threshold of the low-light region is set to 40 nits. The brightness value corresponding to the 95th percentile is 220 nits, that is, the lower threshold of the high-light region is set to 220 nits. That is to say, the region where the light brightness is less than 40 nits is the low-light region, the region where the light brightness is greater than 220 nits is the high-light region, and the remaining region is the medium-light region.

[0110] Step M3.2: Identifying specific regions. That is, according to the partitioned scene obtained in step M3.1, each partition in the partitioned construction scene is identified, and the regions with shadows and reflections are identified therefrom, that is, this region is the specific region.

[0111] Step M3.3: Image processing. That is, according to the partitioned scene where the specific region is located, corresponding image processing is performed. Specifically:

[0112] When the specific region is located in the low-light region, through image enhancement techniques (such as histogram equalization or adaptive gamma correction), the brightness and contrast are improved.

[0113] When the specific region is located in the medium-light region, through local tone mapping techniques, overexposure phenomena are reduced and details are retained.

[0114] When a specific area is located in the high-brightness area, the overall brightness uniformity is improved through a shadow removal algorithm.

[0115] Step S1.2: Concentration classification processing. That is, according to the PM10 concentration obtained in the construction scene, the concentration is classified. Specifically, when the PM10 concentration is less than 150 μg / m³, the environment where the PM10 concentration is located is a low concentration; otherwise, the environment where the PM10 concentration is located is a high concentration.

[0116] Furthermore, according to the classified concentration environment, the features of the image are classified or repaired at the pixel level, specifically as follows:

[0117] When the environment where the PM10 concentration is located is a low concentration, high-level features of the image are extracted through downsampling, and the image details are restored through upsampling. At the same time, skip connections are added between different levels to retain more original information.

[0118] When the environment where the PM10 concentration is located is a high concentration, the distance and speed information of the object are obtained through a millimeter-wave radar for object positioning and discrimination.

[0119] Step S2: The enhanced image in Step S1 is recognized through a hierarchical feature fusion network and combined with the fused data in the three-dimensional perception module to perform feature complementation on the occluded target, reconstruct the human body contour, and obtain the actions of the occluded part. Specifically as follows:

[0120] Step S2.1: Obtain feature information. That is, the enhanced image in Step S1 is processed through a hierarchical feature fusion network to obtain multi-level feature information. Specifically, different-level features of the image are obtained through multiple convolutional layers. The first convolutional layer extracts edge and texture information, and the remaining convolutional layers extract shape and object structure. The spatial size of the feature map is reduced through max pooling or average pooling to retain the required feature information.

[0121] Step S2.2: Determine the occluded action. That is, according to the feature information obtained in Step S2.1, it is combined with the fused data in the three-dimensional perception module to perform feature complementation on the occluded target, reconstruct the human body contour, and determine the angle of the action that occurs in the adjacent time period. Specifically as follows:

[0122] Step N1: Feature complementation. That is, the data of different sensors are complemented in information to obtain the position of the occluded part, specifically as follows:

[0123] ,

[0124] Among them: is the final reconstructed complete human body model, is a partial human body model directly obtained from the original image or sensor data, is the estimated value of the occluded part.

[0125] Step N2: Human body contour reconstruction. That is, according to the position of the occluded part obtained in Step N1 and the enhanced image in Step S1, reconstruct the three-dimensional pose of the human body. That is to say, by minimizing the error between the predicted pose parameters and the actual observed data, optimize the position and shape of the human body model. In this embodiment, the acquisition formula of the pose parameters is specifically:

[0126] ,

[0127] where: is the final optimized best pose parameter, is the feature vector extracted from the multi-modal data, is the feature vector of the human body model generated according to the pose parameter θ, is the pose parameter before optimization.

[0128] Step N3: Determine the action. That is, according to the finally optimized pose parameters obtained in Step N2, through the pose parameters between two adjacent frames, the angle of the action occurring at the occluded part during the adjacent time period can be obtained, specifically:

[0129] ,

[0130] where: is the change of the pose parameter within the adjacent time period, is the finally optimized best pose parameter corresponding to time t + 1, is the finally optimized best pose parameter corresponding to time t.

[0131] Step S3: Through a spatio-temporal encoder (such as Transformer or LSTM), perform long-sequence modeling on the human body poses and tool operation trajectories in consecutive video frames, and identify the illegal action patterns through the output of the spatio-temporal encoder. Further, a fully connected layer is added after the spatio-temporal encoder as a classifier to map the features to different action categories, so as to identify the illegal actions.

[0132] In this embodiment, the interactive verification module is used to compare the violation actions identified in step S3 with the BIM construction progress model, and generate a virtual safety protection layer according to the comparison result. Specifically, when the corresponding protection measures for the violation actions do not start as planned, a virtual safety protection layer is automatically generated and sent to the AR inspection terminal so that on-site management personnel can view it in real time and take corresponding measures. Specifically, during the process of comparing the violation actions with the BIM construction progress model, it can be matched according to time, location, and safety measures. Specifically as follows:

[0133] Step W1: Determine the construction stage. That is, according to the time when the violation action occurs, determine the construction stage corresponding to the time when the violation action occurs from the BIM construction progress model.

[0134] Step W2: Determine the construction location. That is, according to the location where the violation action occurs and the area involved in the construction stage determined in step W1, match the involved area with the location where the violation action occurs to determine the relationship between the location where the violation action occurs and the area involved in the construction stage. That is to say, determine whether the location where the violation action occurs is within the area involved in the construction stage.

[0135] Step W3: Safety measure matching. That is, compare the violation actions with the safety measures according to the safety measures specified in the construction stage determined in step W1. Specifically:

[0136] When the violation action does not match the safety measure, then execute the next step W4. Otherwise, delete the violation action, that is, the violation action does not conform.

[0137] Furthermore, when the violation action does not match the safety measure, the corresponding protection measures for the violation action do not start as planned.

[0138] Step W4: Send to the AR inspection terminal. That is, according to the fact that the corresponding protection measures for the violation action determined in step W3 do not start as planned, a virtual safety protection layer will be automatically generated and sent to the AR inspection terminal so that on-site management personnel can view it in real time and take corresponding measures.

[0139] Specifically, convert the data of the virtual protection layer into a format suitable for AR devices. That is to say, the AR device receives three-dimensional model data containing information such as location, shape, and color, and at the same time transmits the data to the AR device through wireless communication protocols such as Wi-Fi and Bluetooth.

[0140] This embodiment also provides a method for tracking and managing construction safety hazards in construction projects. This method for tracking and managing construction safety hazards in construction projects uses the above-mentioned system for tracking and managing construction safety hazards in construction projects.

[0141] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended embodiments and their equivalents.

Claims

1. A construction engineering construction safety hazard tracking and management system, characterized in that: Included are: 3D perception module: acquires information data sets from the construction site and fuses the data sets to establish a dynamic construction scene point cloud model; Posture analysis module: by acquiring the impact data, the image in the point cloud model of the dynamic construction scene is enhanced, and the actions of the occluded parts in the enhanced point cloud model of the dynamic construction scene are reconstructed, and the illegal actions are determined through the spatiotemporal encoder, including: S1: According to the influencing parameters, adjusting the image enhancement algorithm parameters, determining the image recognition strategy, and performing image enhancement on the blurred image through the image recognition strategy, including: S1.1: Dynamic multi-exposure fusion: Combine images with different exposure times to obtain an HDR image, and adjust the working mode through the HDR image to determine the illumination difference in different areas and obtain a processed image; S1.2: Concentration classification processing: According to the PM10 concentration, the construction scene is divided into concentrations, and image processing is performed based on the divided construction scenes, specifically: When the PM10 concentration is low, high-level features of the image are extracted by downsampling, image details are restored by upsampling, and jump connections are added between different levels. When the PM10 concentration is high, the millimeter wave radar is used to obtain the distance and speed information of the object to locate and distinguish the object; S2: Through the hierarchical feature fusion network and the fused data, the features of the occluded target are complemented, the human body contour is reconstructed, and the action of the occluded part is obtained, including: S2.1: Obtain feature information: Process the enhanced image through a hierarchical feature fusion network to obtain multi-level feature information; S2.2: Determine the occluding action: Combine the feature information with the fused data, perform feature complementation on the occluding target, and obtain the angle of the action occurring in the adjacent time period, including: N1: Feature complementation: The data from different sensors are complemented to obtain the location of the occluded part, specifically: ， in: To finally reconstruct the complete human body model, is a partial human body model directly obtained from the original image or sensor data, is the estimated value of the occluded part; N2: Human body contour reconstruction: Reconstruct the three-dimensional posture of the human body according to the position of the occluded part, and obtain the posture parameters that minimize the prediction, specifically: ， in: is the optimal posture parameter after final optimization, is the feature vector extracted from multimodal data, is the feature vector of the human body model generated according to the posture parameter θ, is the posture parameter before optimization; N3: Determine the action: According to the minimized predicted posture parameters, obtain the angle of the action occurring in the adjacent time period at the occluded part, specifically: ， in: is the change of posture parameters in adjacent time periods, is the optimal posture parameter after optimization at time t+1. is the optimal posture parameter after final optimization corresponding to time t; S3: Through the spatiotemporal encoder, long-sequence modeling of human postures and tool operation trajectories in continuous video frames is performed to identify illegal action patterns; Interactive verification module: compares the illegal action with the BIM construction progress model. When the illegal action does not start the protective measures as planned, a virtual safety protection layer is automatically generated and sent to the AR inspection terminal.

2. A construction engineering construction safety hazard tracking and management system according to claim 1, characterized in that: Get the processed image, including: M1: Determine the dynamic area: by obtaining different exposure images of the same scene, and dividing the area in the scene according to the motion vector between adjacent frame images, and obtaining the final generated HDR image according to the divided area; M2: Light intensity gradient analysis: According to the HDR image, the light intensity change rate within a preset time is obtained, and the working mode is determined according to the light intensity change rate; M3: Regional processing: According to the working mode, the illumination difference in different regions is determined, and image processing is performed based on the illumination difference and the specific region.

3. A construction engineering construction safety hazard tracking and management system according to claim 2, characterized in that: Get the final HDR image, including: M1.1: Collect three exposure images: obtain three different exposure images of the same scene through three different exposure times; M1.2: Determine the motion area: According to the optical flow vector of each pixel in the exposure image, obtain the modulus length corresponding to the pixel, and compare the modulus length with a preset threshold. When the modulus length is greater than the preset threshold, the area where the pixel is located is a high motion area. Otherwise, the area where the pixel is located is a low motion area or a static area. The specific calculation formula of the modulus length is: ， in: is the optical flow vector corresponding to the pixel at position (x, y), is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel; M1.3: Determine exposure combination: According to the area category corresponding to the area where the pixel point is located, the three exposure images corresponding to the pixel point are weighted, specifically: When the area where the pixel point is located is a high motion area, the weight distribution of the three exposure images corresponding to the pixel point is: the weight of the short exposure image is set to 1, the weight of the medium exposure image is set to 0, and the weight of the long exposure image is set to 0; When the area where the pixel point is located is a low-motion area or a static area, the weight distribution of the three exposure images corresponding to the pixel point is: the weight of the short exposure image is set to 0.3, the weight of the medium exposure image is set to 0.5, and the weight of the long exposure image is set to 0.2; M1.4: Determine HDR value: According to the weight distribution of the three exposure images corresponding to the pixel point, obtain the brightness value of the pixel point in the finally generated HDR image, and obtain the finally generated HDR image according to the brightness value. The brightness value of the pixel point in the finally generated HDR image is specifically: ， in: is the brightness value of the pixel (x, y) in the final generated HDR image, is the weight of the i-th exposure image at the pixel (x, y), is the brightness value of the pixel (x, y) in the i-th exposure image, The type index of the exposure image.

4. A construction engineering construction safety hazard tracking and management system according to claim 2, characterized in that: Determine the working mode, including: M2.1: Obtaining the average illumination intensity change ratio: According to the illumination intensity change between two adjacent HDR images, the average illumination intensity change ratio is determined, specifically: ， in: is the average light intensity change rate, is the light intensity difference between two adjacent HDR image frames, is the total number of pixels in the image, is the time difference between two measurements of HDR brightness values, is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel; M2.2: Determine the working mode: compare the average light intensity change rate with the preset change threshold. When the average light intensity change rate is greater than the preset change threshold, the working mode is adjusted to the short exposure mode. When the average light intensity change rate is less than the preset change threshold, the working mode is adjusted to the long exposure mode. Otherwise, the working mode remains unchanged.

5. A construction engineering construction safety hazard tracking and management system according to claim 2, characterized in that: Perform image processing, including: M3.1: Construction scene partitioning: According to the working mode, HDR images with different exposure levels are obtained, and the illumination differences between different areas in the HDR image are quantified through the histogram to determine the upper threshold of the low light area and the lower threshold of the high light area. The partition area is determined according to the upper threshold of the low light area and the lower threshold of the high light area, specifically: Compare the light brightness of different areas with the upper threshold of the low light area and the lower threshold of the high light area. When the light brightness is less than the upper threshold of the low light area, the area where the light brightness is located is the low light area. When the light brightness is greater than the lower threshold of the high light area, the area where the light brightness is located is the high light area. Otherwise, the area where the light brightness is located is the medium light area. M3.2: Identify specific areas: Identify shadow areas and reflective areas in each partition area to obtain specific areas; M3.3: Image processing: Perform image processing according to the partition area where the specific area is located, specifically: When the partitioned area is a low-light area, image processing is performed using image enhancement technology; When the partitioned area is a medium light area, image processing is performed by using a local tone mapping technique; When the partitioned area is a highlight area, image processing is performed using a shadow removal algorithm.

6. A construction engineering construction safety hazard tracking and management system according to claim 1, characterized in that: Sending the virtual safety protection layer to the AR inspection terminal includes: W1: Determine the construction stage: According to the time when the illegal action occurs, determine the construction stage corresponding to the time when the illegal action occurs from the BIM construction progress model; W2: Determine the construction location: According to the location where the illegal action occurred and the area involved in the construction phase, obtain the relationship between the location where the illegal action occurred and the area involved in the construction phase; W3: Safety measures matching: Compare the safety measures specified in the construction phase with the illegal actions. When the illegal actions do not match the safety measures, execute the next step. Otherwise, delete the illegal actions. W4: Send AR inspection terminal: When the illegal action does not initiate protective measures as planned, the virtual safety protection layer is automatically generated, and the data of the virtual safety protection layer is converted into three-dimensional model data, and the three-dimensional model data is transmitted to the AR device through the wireless communication protocol.

7. A method for tracking and managing safety hazards in construction projects, characterized in that: A construction project construction safety hazard tracking and management system as described in any one of claims 1-6 is used.

Images