Building engineering construction potential safety hazard tracking management system and method

Abstract

The invention discloses a building engineering construction potential safety hazard tracking management system and method, and relates to the technical field of safety management. The three-dimensional sensing module is used for fusing information data sets and establishing a dynamic construction scene point cloud model; the posture analysis module is used for enhancing the image in the dynamic construction scene point cloud model, reconstructing the action of the shielded part in the enhanced dynamic construction scene point cloud model, and determining the illegal action; and the interactive verification module is used for comparing the violation action with a BIM construction progress model, generating a virtual safety protection layer when the violation action does not start a protection measure according to a plan, and sending the virtual safety protection layer to an AR inspection terminal. According to the invention, the three-dimensional structure change of the construction site can be captured in real time, the problem of image blurring caused by poor illumination conditions or air pollution is effectively improved by using dynamic multi-exposure fusion and concentration grading processing technologies, and the image recognition precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of safety management, and in particular to a management system and method for tracking hidden dangers in construction safety of a building project. Background Art

[0002] With the acceleration of urbanization and the booming construction industry, construction projects are becoming increasingly complex and their scale is expanding, which puts higher demands on the safety management of construction sites. However, traditional safety management methods mostly rely on manual inspections, paper records and experience judgments, which is not only inefficient, but also difficult to cope with complex construction environments and frequent safety hazards.

[0003] The construction industry has always been one of the fields with a high incidence of safety accidents. According to statistics, the accident rate in the construction industry is higher than that in many other industries, especially safety accidents such as falling from heights, being hit by objects, and mechanical injuries, which not only pose a serious threat to the safety of workers' lives, but also have a negative impact on the economic benefits and social image of enterprises.

[0004] In recent years, modern information technologies such as the Internet of Things (IoT), big data analysis, and artificial intelligence have developed rapidly and have been gradually applied to construction safety management. For example, IoT technology can achieve real-time monitoring of construction sites; big data analysis can help identify potential safety risks and predict the possibility of accidents; and artificial intelligence can improve the scientificity and effectiveness of safety decisions.

[0005] The Chinese invention patent with publication number CN116823529A discloses a security intelligent management system based on behavioral big data analysis, including a construction site area division module, a behavior recognition terminal deployment module, an active personnel identity recognition module, an outsider behavior recognition processing module, a construction worker safety monitoring and analysis module, a display terminal and an information repository. The invention uses an intelligent high-definition camera to count the number of workers who wear safety helmets incorrectly or do not wear safety helmets. It not only warns the corresponding personnel, but also counts the number of workers who wear safety helmets incorrectly or do not wear safety helmets, reflecting the strength of safety awareness of the construction worker group, and more clearly demonstrates the security control effect of the construction site, reducing the possibility of safety accidents. From another perspective, it further demonstrates the effectiveness of security management work at the construction site, and can accurately provide reliable decision-making basis for the smooth implementation of security management work at the construction site.

[0006] In the process of security prevention for the above-mentioned and similar intelligent management systems, although the existing video surveillance system can cover the key areas of the construction site, it is particularly difficult to capture specific violations due to the variety and lack of regularity of the actions of construction workers, especially when performing dynamic operations. For example, in a busy construction site, workers may move quickly, bend over, turn around, or be partially blocked by other objects, making identification more difficult. In other words, in the process of automatically identifying violations (such as not wearing a safety helmet) and abnormal situations (such as not setting up a protective net for high-altitude operations), there will be false alarms and missed alarms due to the complex construction site environment. Summary of the invention

[0007] The purpose of the present invention is to provide a construction project safety hazard tracking management system and method to solve the problems raised in the above background technology.

[0008] To achieve the above purpose, the present invention provides the following technical solution: a construction engineering construction safety hazard tracking and management system, comprising: 3D perception module: acquires information data sets from the construction site and fuses the data sets to establish a dynamic construction scene point cloud model; Posture analysis module: by acquiring the impact data, the image in the point cloud model of the dynamic construction scene is enhanced, and the actions of the occluded parts in the enhanced point cloud model of the dynamic construction scene are reconstructed, and the illegal actions are determined through the spatiotemporal encoder, including: S1: adjusting image enhancement algorithm parameters according to influencing parameters, determining image recognition strategy, and performing image enhancement on the blurred image through the image recognition strategy; S2: Through the hierarchical feature fusion network and the fused data, the features of the occluded target are complemented, the human body contour is reconstructed, and the action of the occluded part is obtained; S3: Through the spatiotemporal encoder, long-sequence modeling of human postures and tool operation trajectories in continuous video frames is performed to identify illegal action patterns; Interactive verification module: compares the illegal action with the BIM construction progress model. When the illegal action does not start the protective measures as planned, a virtual safety protection layer is automatically generated and sent to the AR inspection terminal.

[0009] Furthermore, the blurred image is enhanced, including: S1.1: Dynamic multi-exposure fusion: Combine images with different exposure times to obtain an HDR image, and adjust the working mode through the HDR image to determine the illumination difference in different areas and obtain a processed image; S1.2: Concentration classification processing: According to the PM10 concentration, the construction scene is divided into concentrations, and image processing is performed based on the divided construction scenes, specifically: When the PM10 concentration is low, high-level features of the image are extracted by downsampling, image details are restored by upsampling, and jump connections are added between different levels. When the PM10 concentration in the environment is high, the millimeter wave radar is used to obtain the distance and speed information of the object to locate and distinguish the object.

[0010] Furthermore, the processed image is obtained, including: M1: Determine the dynamic area: by obtaining different exposure images of the same scene, and dividing the area in the scene according to the motion vector between adjacent frame images, and obtaining the final generated HDR image according to the divided area; M2: Light intensity gradient analysis: According to the HDR image, the light intensity change rate within a preset time is obtained, and the working mode is determined according to the light intensity change rate; M3: Regional processing: According to the working mode, the illumination difference in different regions is determined, and image processing is performed based on the illumination difference and the specific region.

[0011] Furthermore, the final HDR image is obtained, including: M1.1: Collect three exposure images: obtain three different exposure images of the same scene through three different exposure times; M1.2: Determine the motion area: According to the optical flow vector of each pixel in the exposure image, obtain the modulus length corresponding to the pixel, and compare the modulus length with a preset threshold. When the modulus length is greater than the preset threshold, the area where the pixel is located is a high motion area. Otherwise, the area where the pixel is located is a low motion area or a static area. The specific calculation formula of the modulus length is: ,

[0012] in: is the optical flow vector corresponding to the pixel at position (x, y), is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel; M1.3: Determine exposure combination: According to the area category corresponding to the area where the pixel point is located, the three exposure images corresponding to the pixel point are weighted, specifically: When the area where the pixel point is located is a high motion area, the weight distribution of the three exposure images corresponding to the pixel point is: the weight of the short exposure image is set to 1, the weight of the medium exposure image is set to 0, and the weight of the long exposure image is set to 0; When the area where the pixel point is located is a low-motion area or a static area, the weight distribution of the three exposure images corresponding to the pixel point is: the weight of the short exposure image is set to 0.3, the weight of the medium exposure image is set to 0.5, and the weight of the long exposure image is set to 0.2; M1.4: Determine HDR value: According to the weight distribution of the three exposure images corresponding to the pixel point, obtain the brightness value of the pixel point in the finally generated HDR image, and obtain the finally generated HDR image according to the brightness value. The brightness value of the pixel point in the finally generated HDR image is specifically: ,

[0013] in: is the brightness value of the pixel (x, y) in the final generated HDR image, is the weight of the i-th exposure image at the pixel (x, y), is the brightness value of the pixel (x, y) in the i-th exposure image, The type index of the exposure image.

[0014] Furthermore, the working mode is determined, including: M2.1: Obtaining the average illumination intensity change ratio: According to the illumination intensity change between two adjacent HDR images, the average illumination intensity change ratio is determined, specifically: ,

[0015] in: is the average light intensity change rate, is the light intensity difference between two adjacent HDR image frames, is the total number of pixels in the image, is the time difference between two measurements of HDR brightness values, is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel; M2.2: Determine the working mode: compare the average light intensity change rate with the preset change threshold. When the average light intensity change rate is greater than the preset change threshold, the working mode is adjusted to the short exposure mode. When the average light intensity change rate is less than the preset change threshold, the working mode is adjusted to the long exposure mode. Otherwise, the working mode remains unchanged.

[0016] Furthermore, image processing is performed, including: M3.1: Construction scene partitioning: According to the working mode, HDR images with different exposure levels are obtained, and the illumination differences between different areas in the HDR image are quantified through the histogram to determine the upper threshold of the low light area and the lower threshold of the high light area. The partition area is determined according to the upper threshold of the low light area and the lower threshold of the high light area, specifically: Compare the light brightness of different areas with the upper threshold of the low light area and the lower threshold of the high light area. When the light brightness is less than the upper threshold of the low light area, the area where the light brightness is located is the low light area. When the light brightness is greater than the lower threshold of the high light area, the area where the light brightness is located is the high light area. Otherwise, the area where the light brightness is located is the medium light area. M3.2: Identify specific areas: Identify shadow areas and reflective areas in each partition area to obtain specific areas; M3.3: Image processing: Perform image processing according to the partition area where the specific area is located, specifically: When the partitioned area is a low-light area, image processing is performed using image enhancement technology; When the partitioned area is a medium light area, image processing is performed by using a local tone mapping technique; When the partitioned area is a highlight area, image processing is performed using a shadow removal algorithm.

[0017] Furthermore, the actions of the occluded parts are obtained, including: S2.1: Obtain feature information: Process the enhanced image through a hierarchical feature fusion network to obtain multi-level feature information; S2.2: Determine the occluding action: combine the feature information with the fused data, perform feature complementation on the occluding target, and obtain the angle of the action occurring in adjacent time periods.

[0018] Furthermore, the angles of actions occurring in adjacent time periods are obtained, including: N1: Feature complementation: The data from different sensors are complemented to obtain the location of the occluded part, specifically: ,

[0019] in: To finally reconstruct the complete human body model, is a partial human body model directly obtained from the original image or sensor data, is the estimated value of the occluded part; N2: Human body contour reconstruction: Reconstruct the three-dimensional posture of the human body according to the position of the occluded part, and obtain the posture parameters that minimize the prediction, specifically: ,

[0020] in: is the optimal posture parameter after final optimization, is the feature vector extracted from multimodal data, is the feature vector of the human body model generated according to the posture parameter θ, is the posture parameter before optimization; N3: Determine the action: According to the minimized predicted posture parameters, obtain the angle of the action occurring in the adjacent time period at the occluded part, specifically: ,

[0021] in: is the change of posture parameters in adjacent time periods, is the optimal posture parameter after optimization at time t+1. is the optimal posture parameter after final optimization corresponding to time t.

[0022] Furthermore, the virtual safety protection layer is sent to the AR inspection terminal, including: W1: Determine the construction stage: According to the time when the illegal action occurs, determine the construction stage corresponding to the time when the illegal action occurs from the BIM construction progress model; W2: Determine the construction location: According to the location where the illegal action occurred and the area involved in the construction phase, obtain the relationship between the location where the illegal action occurred and the area involved in the construction phase; W3: Safety measures matching: Compare the safety measures specified in the construction phase with the illegal actions. When the illegal actions do not match the safety measures, execute the next step. Otherwise, delete the illegal actions. W4: Send AR inspection terminal: When the illegal action does not initiate protective measures as planned, the virtual safety protection layer is automatically generated, and the data of the virtual safety protection layer is converted into three-dimensional model data, and the three-dimensional model data is transmitted to the AR device through the wireless communication protocol.

[0023] A construction project construction safety hazard tracking and management method uses any of the construction project construction safety hazard tracking and management systems described above.

[0024] Compared with the prior art, the present invention has the following beneficial effects: First, the present invention establishes a dynamic construction scene point cloud model by fusing multiple information data sets, so that the system can capture the three-dimensional structural changes of the construction site in real time. At the same time, the dynamic multi-exposure fusion and concentration grading processing technology are used to effectively improve the image blur problem caused by poor lighting conditions or air pollution, and improve the image recognition accuracy; Second, the present invention uses a hierarchical feature fusion network to complement the features of the occluded target and reconstruct the human body contour, so that it can accurately identify the action even when part of the body is occluded. It also uses a spatiotemporal encoder to model the human posture and tool operation trajectory in continuous video frames in a long sequence, which helps to accurately identify illegal action patterns and provide a basis for subsequent safety management. Third: The present invention compares the illegal actions with the BIM construction progress model to confirm whether they comply with safety regulations. When it is found that the protective measures are not implemented as planned, it can automatically create a virtual safety protection layer and send it to the AR inspection terminal to help on-site managers respond quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a system block diagram of the construction safety hazard tracking management system of the present invention; Figure 2 A schematic diagram of the process of obtaining the final generated HDR image in the present invention; Figure 3 A schematic diagram of the process of image processing in the present invention; Figure 4 It is the brightness value histogram in the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] In the process of security prevention in the existing intelligent management system, although the existing video surveillance system can cover the key areas of the construction site, it is particularly difficult to capture specific violations due to the variety and lack of regularity of the construction workers' movements, especially when performing dynamic operations. For example, in a busy construction site, workers may move quickly, bend, turn around, or be partially blocked by other objects, which increases the difficulty of identification. In other words, in the process of automatic identification of violations (such as not wearing a safety helmet) and abnormal situations (such as no protective net for high-altitude operations), there will be false alarms and missed reports due to the complex construction site environment. The technical solution of this application establishes a point cloud model of a dynamic construction scene and collects real-time data, reconstructs the obscured human body movements using a hierarchical feature fusion network, uses a spatiotemporal encoder to identify illegal action patterns in continuous video frames, and compares the detected illegal actions with the BIM construction progress model, thereby ensuring the implementation of safety measures. At the same time, when necessary, a virtual safety protection layer can be automatically generated and sent to the AR inspection terminal, thereby realizing safety monitoring and immediate warning of the construction site.

[0028] refer to Figure 1-Figure 4 , this embodiment provides a construction project construction safety hazard tracking and management system, which includes a three-dimensional perception module, a posture analysis module and an interactive verification module. The three-dimensional perception module acquires the information data set of the construction site and fuses the information data set to establish a dynamic construction scene point cloud model. The posture analysis module enhances the image in the dynamic construction scene point cloud model established in the three-dimensional perception module by acquiring the impact data, and reconstructs the occluded part action in the enhanced dynamic construction scene point cloud model, and determines the illegal action through the space-time encoder. The interactive verification module compares the illegal action identified in the posture analysis module with the BIM construction progress model. When the illegal action does not start the protective measures as planned, a virtual safety protection layer is automatically generated and sent to the AR inspection terminal.

[0029] In this embodiment, the three-dimensional perception module is used to obtain information data sets of color, shape, texture, distance and object category of the construction site through visible light cameras, millimeter wave radars and infrared thermal imaging equipment, and fuse the obtained information data sets to obtain fused information data. At the same time, a dynamic construction scene point cloud model is established based on the fused information data. Specifically, the color, shape and texture of the object are identified by the visible light camera, the distance information is provided by the millimeter wave radar, and different objects are distinguished by the infrared thermal imager.

[0030] During the implementation process, visible light cameras, millimeter-wave radars and infrared thermal imagers were installed at the construction site. The visible light camera outputs RGB images with a resolution of 1920*1080, the millimeter-wave radar outputs point cloud data, and the infrared thermal imager outputs heat maps.

[0031] It is worth noting that in order to make the output data of the visible light camera, millimeter wave radar and infrared thermal imaging device consistent, the acquired data (RGB image, point cloud data and heat map) is processed by spatiotemporal synchronous calibration in this embodiment, which includes timestamp synchronization and calculation of spatial transformation matrix. Specifically, in the process of timestamp synchronization, the timestamp of the data is corrected by minimizing the error function with reference to GPS time to keep its timestamp consistent. In the process of calculating the spatial transformation matrix, the rotation and translation matrices corresponding to different data are determined by the calibration plate, so that the coordinate systems of different sensors are converted into a unified world coordinate system. Further, the data after spatiotemporal synchronous calibration can be fused by fusion based on feature level or target level. It is worth noting that the spatiotemporal synchronous calibration and data fusion involved in this embodiment belong to existing conventional technical means, so they will not be specifically explained in this embodiment.

[0032] In this embodiment, the posture analysis module is used to obtain the illumination data and dust concentration data of the construction site through the set illumination intensity sensor and dust concentration detector. And based on the obtained illumination data and dust concentration data, the image in the dynamic construction scene point cloud model established in the three-dimensional perception module is enhanced. At the same time, in the dynamic construction scene point cloud model after image enhancement, the actions of the obscured parts are identified and reconstructed, and the illegal actions are determined through the spatiotemporal encoder. The details are as follows: Step S1: By setting a light intensity sensor and a dust concentration detector (in this embodiment, the light intensity sensor model is set to the TSL2561 sensor, and the dust concentration detector model is set to the Thermo Scientific ADR-1500 dust monitor), obtain the light data and dust concentration data of the construction site, and dynamically adjust the image enhancement algorithm parameters (defogging coefficient and contrast gain) according to the obtained light data and dust concentration data to obtain the corresponding image recognition strategy. And according to the image recognition strategy, enhance the blurred image in the dynamic construction scene point cloud model to obtain a clear image. The details are as follows: Step S1.1: Dynamic multi-exposure fusion. That is, combine images with different exposure times (short exposure, medium exposure and long exposure) to obtain images with high dynamic range and reduce the problem of smear caused by moving objects. The details are as follows: Step M1: Determine the dynamic area. That is, by acquiring images of the same scene under different exposure conditions and calculating the motion vectors between adjacent frame images, the dynamic area in the scene is identified. At the same time, according to the identified dynamic area, the corresponding exposure combination is selected. The details are as follows: Step M1.1: Collect three exposure images. That is, when shooting the same scene, use three exposure times (short exposure: 1 / 1000 second, medium exposure: 1 / 100 second, long exposure: 1 / 10 second) to obtain three different exposure images of the same scene.

[0033] Step M1.2: Determine the motion area. That is, according to the optical flow vector of the pixel point in the exposure image obtained in step M1.1, obtain the modulus length corresponding to the pixel point, specifically: ,

[0034] in: is the optical flow vector corresponding to the pixel at position (x, y), is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel.

[0035] Furthermore, the obtained optical flow vector is compared with a preset threshold value (specifically set according to specific data, so the number is not specifically described in this embodiment). When the obtained optical flow vector is greater than the preset threshold value, the area where the pixel point corresponding to the optical flow vector is located is a high-motion area. Otherwise, the area where the pixel point corresponding to the optical flow vector is located is a low-motion area or a static area.

[0036] In the specific implementation process, the optical flow vector of the pixel point is (5,3), and its corresponding modulus is 5.83. Further, in this embodiment, the preset threshold is set to 4. That is, if the modulus of the pixel point is greater than the preset threshold, the area where the pixel point is located is a high motion area.

[0037] Step M1.3: Determine the exposure combination. That is, according to the category of the motion area corresponding to the area where the pixel point is located determined in step M1.2 (high motion area, low motion area and static area), weights are assigned to the three exposure images (short exposure image, medium exposure image and long exposure image) corresponding to each pixel point.

[0038] Specifically, when the motion area corresponding to the pixel point is a high motion area, the weight distribution corresponding to the pixel point is: the weight of the short exposure image is set to 1, the weight of the medium exposure image is set to 0, and the weight of the long exposure image is set to 0. Specifically: ,

[0039] in: is the weight of the short exposure image, is the weight of the medium exposure image, is the weight of the long exposure image.

[0040] Furthermore, when the motion area corresponding to the area where the pixel is located is a low motion area and a static area, the weight distribution corresponding to the pixel is: the weight of the short exposure image is set to 0.3, the weight of the medium exposure image is set to 0.5, and the weight of the long exposure image is set to 0.2.

[0041] Step M1.4: Determine the HDR value. That is, according to the exposure combination set in step M1.3, obtain the brightness value of the pixel in the final generated HDR image, specifically: ,

[0042] in: is the brightness value of the pixel (x, y) in the final generated HDR image, is the weight of the i-th exposure image at the pixel (x, y), is the brightness value of the pixel (x, y) in the i-th exposure image, The type index of the exposure image.

[0043] In the specific implementation process, the exposure weights corresponding to the pixels and their corresponding brightness values ​​are set as follows: the weight of the short exposure image is set to 1, the brightness value of the short exposure image is set to 50, the weight of the medium exposure image is set to 0, the brightness value of the medium exposure image is set to 100, the weight of the long exposure image is set to 0, and the brightness value of the long exposure image is set to 150. Then the brightness value of the pixel in the final generated HDR image is 50.

[0044] Furthermore, according to the brightness value of the pixel in the finally generated HDR image, the corresponding HDR image can be obtained.

[0045] Step M2: Light intensity gradient analysis. That is, based on the HDR image obtained in step M1.4, obtain the change of light intensity within a preset time, and adjust the working mode according to the obtained change of light intensity to adapt to the change of lighting conditions. The details are as follows: Step M2.1: Obtain the average illumination intensity change ratio. That is, according to the HDR image obtained in step M1.4, obtain the illumination intensity change between two adjacent HDR image frames, and determine the average illumination intensity change ratio according to the illumination intensity change between two adjacent HDR image frames, specifically: ,

[0046] in: is the average light intensity change rate, is the light intensity difference between two adjacent HDR image frames, is the total number of pixels in the image, is the time difference between two measurements of HDR brightness values, is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel.

[0047] Step M2.2: Determine the working mode. That is, determine the adjusted exposure mode according to the average illumination intensity change rate obtained in step W1. Specifically, the average illumination intensity change rate obtained in step W1 is compared with a preset change threshold (specifically set according to specific data, so the number is not specifically described in this embodiment). When the average illumination intensity change rate obtained is greater than the preset change threshold, the exposure mode is adjusted to the short exposure mode. When the average illumination intensity change rate obtained is less than the preset change threshold, the exposure mode is adjusted to the long exposure mode. Otherwise, the exposure mode remains unchanged.

[0048] Step M3: Regionalization processing. That is, according to the exposure mode determined in step W2, the illumination difference in different regions is determined. In other words, the construction scene is partitioned according to the illumination difference, and specific regions are identified therefrom, and corresponding image processing is performed on the identified specific regions. The details are as follows: Step M3.1: Construction scene partitioning. That is, according to the exposure mode determined in step W2, multiple images with different exposure levels are obtained and converted into HDR images. At the same time, the illumination differences between different areas in the HDR image are quantified through the histogram to obtain the average brightness of each area. Furthermore, according to the quantified illumination differences, the light brightness corresponding to the 5th percentile is used as the upper threshold of the low light area, and the light brightness corresponding to the 95th percentile is used as the lower threshold of the high light area.

[0049] That is to say, the light brightness of different areas is compared with the upper threshold of the low light area and the lower threshold of the high light area. When the light brightness is less than the upper threshold of the low light area, the area where the light brightness is located is the low light area. When the light brightness is greater than the lower threshold of the high light area, the area where the light brightness is located is the high light area. Otherwise, the area where the light brightness is located is the medium light area.

[0050] In the process of specific implementation, the following settings are made: Figure 4 The brightness value histogram shown is shown in Figure 4It can be seen that the brightness value corresponding to the 5th percentile is 40 nits, that is, the upper threshold of the low light area is set to 40 nits. The brightness value corresponding to the 95th percentile is 220 nits, that is, the lower threshold of the highlight area is set to 220 nits. In other words, the area with a light brightness of less than 40 nits is a low light area, the area with a light brightness of more than 220 nits is a highlight area, and the remaining area is a medium light area.

[0051] Step M3.2: Identify specific areas. That is, according to the partitioned scene obtained in step M3.1, in the partitioned construction scene, each partition is identified, and the area with shadows and reflections is identified, that is, the area is a specific area.

[0052] Step M3.3: Image processing. That is, perform corresponding image processing according to the partition scene where the specific area is located, specifically: When specific areas are located in low-light areas, image enhancement techniques (such as histogram equalization or adaptive gamma correction) are used to increase brightness and contrast.

[0053] When specific areas are in medium light, local tone mapping technology is used to reduce overexposure and preserve details.

[0054] When a specific area is located in the highlight area, the shadow removal algorithm is used to improve the overall brightness uniformity.

[0055] Step S1.2: Concentration classification processing. That is, the concentration is classified according to the PM10 concentration obtained in the construction scene. Specifically, when the PM10 concentration is less than 150μg / m³, the environment where the PM10 concentration is located is a low concentration, otherwise, the environment where the PM10 concentration is located is a high concentration.

[0056] Furthermore, according to the graded concentration environment, the image features are classified or repaired at the pixel level, specifically: When the PM10 concentration is low, high-level features of the image are extracted through downsampling, and image details are restored through upsampling. At the same time, jump connections are added between different levels to retain more original information.

[0057] When the PM10 concentration is high, the millimeter-wave radar is used to obtain the distance and speed information of the object to locate and distinguish the object.

[0058] Step S2: The enhanced image in step S1 is recognized through a hierarchical feature fusion network, and combined with the fused data in the three-dimensional perception module to complement the features of the occluded target, reconstruct the human body contour, and obtain the action of the occluded part. The details are as follows: Step S2.1: Obtain feature information. That is, the enhanced image in step S1 is processed through a hierarchical feature fusion network to obtain multi-level feature information. Specifically, different levels of features of the image are obtained through multiple convolutional layers, where the first convolution layer extracts edge and texture information, and the remaining convolutions extract shape and object structure. The spatial size of the feature map is reduced through maximum pooling or average pooling to retain the required feature information.

[0059] Step S2.2: Determine the occluding action. That is, based on the feature information obtained in step S2.1, combine it with the fused data in the three-dimensional perception module to complement the features of the occluding target, reconstruct the human body contour, and determine the angle of the action in the adjacent time period. The details are as follows: Step N1: Feature complementation. The data from different sensors are complemented to obtain the location of the blocked part. Specifically: ,

[0060] in: To finally reconstruct the complete human body model, is a partial human body model directly obtained from the original image or sensor data, is the estimated value of the occluded part.

[0061] Step N2: Human body contour reconstruction. That is, based on the position of the occluded part obtained in step N1 and the enhanced image in step S1, the three-dimensional posture of the human body is reconstructed. In other words, the position and shape of the human body model are optimized by minimizing the error between the predicted posture parameters and the actual observed data. In this embodiment, the posture parameter acquisition formula is specifically: ,

[0062] in: is the optimal posture parameter after final optimization, is the feature vector extracted from multimodal data, is the feature vector of the human body model generated according to the posture parameter θ, are the posture parameters before optimization.

[0063] Step N3: Determine the action. That is, according to the finally optimized posture parameters obtained in step N2, the angle of the action occurring at the occluded part in the adjacent time period can be obtained through the posture parameters between two adjacent frames, specifically: ,

[0064] in: is the change of posture parameters in adjacent time periods, is the optimal posture parameter after optimization at time t+1. is the optimal posture parameter after final optimization corresponding to time t.

[0065] Step S3: Use a spatiotemporal encoder (such as Transformer or LSTM) to model the human posture and tool operation trajectory in continuous video frames, and identify illegal action patterns through the output of the spatiotemporal encoder. Furthermore, a fully connected layer is added after the spatiotemporal encoder as a classifier to map features to different action categories, thereby identifying illegal actions.

[0066] In this embodiment, the interactive verification module is used to compare the illegal actions identified in step S3 with the BIM construction progress model, and generate a virtual safety protection layer based on the comparison results. Specifically, when the illegal action does not start the corresponding protective measures as planned, a virtual safety protection layer is automatically generated, and the generated virtual safety protection layer is sent to the AR inspection terminal so that on-site managers can view and take corresponding measures in real time. Specifically, in the process of comparing the illegal action with the BIM construction progress model, it can be matched according to time, location and safety measures. The details are as follows: Step W1: Determine the construction phase. That is, according to the time when the illegal action occurred, determine the construction phase corresponding to the time when the illegal action occurred from the BIM construction progress model.

[0067] Step W2: Determine the construction location. That is, according to the location where the illegal action occurred, the area involved in the construction phase determined in step W1 is matched with the location where the illegal action occurred, and the relationship between the location where the illegal action occurred and the area involved in the construction phase is determined. In other words, determine whether the location where the illegal action occurred is located in the area involved in the construction phase.

[0068] Step W3: Safety measures matching. That is, according to the safety measures specified in the construction stage determined in step W1, the illegal actions are compared with the safety measures, specifically: When the illegal action does not comply with the safety measures, the next step W4 is executed. Otherwise, the illegal action is deleted, that is, the illegal action does not comply.

[0069] Furthermore, when the illegal action is inconsistent with the safety measures, the illegal action does not initiate the corresponding protective measures as planned.

[0070] Step W4: Send to AR inspection terminal. That is, according to the violation action determined in step W3, the corresponding protective measures are not initiated as planned, and a virtual safety protection layer will be automatically generated and sent to the AR inspection terminal so that the on-site management personnel can view it in real time and take corresponding measures.

[0071] Specifically, the data of the virtual protection layer is converted into a format suitable for use by the AR device. In other words, the AR device receives the three-dimensional model data containing information such as position, shape, color, etc., and transmits the data to the AR device through wireless communication protocols such as Wi-Fi and Bluetooth.

[0072] This embodiment also provides a construction project construction safety hazard tracking and management method, and the construction project construction safety hazard tracking and management method uses the above-mentioned construction project construction safety hazard tracking and management system.

[0073] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is limited by the attached embodiments and their equivalents.

Claims

1. A construction engineering construction safety hazard tracking and management system, characterized in that: Included are: 3D perception module: acquires information data sets from the construction site and fuses the data sets to establish a dynamic construction scene point cloud model; Posture analysis module: by acquiring the impact data, the image in the point cloud model of the dynamic construction scene is enhanced, and the actions of the occluded parts in the enhanced point cloud model of the dynamic construction scene are reconstructed, and the illegal actions are determined through the spatiotemporal encoder, including: S1: adjusting image enhancement algorithm parameters according to influencing parameters, determining image recognition strategy, and performing image enhancement on the blurred image through the image recognition strategy; S2: Through the hierarchical feature fusion network and the fused data, the features of the occluded target are complemented, the human body contour is reconstructed, and the action of the occluded part is obtained; S3: Through the spatiotemporal encoder, long-sequence modeling of human postures and tool operation trajectories in continuous video frames is performed to identify illegal action patterns; Interactive verification module: compares the illegal action with the BIM construction progress model. When the illegal action does not start the protective measures as planned, a virtual safety protection layer is automatically generated and sent to the AR inspection terminal.

2. A construction engineering construction safety hazard tracking and management system according to claim 1, characterized in that: Image enhancement for blurred images, including: S1.1: Dynamic multi-exposure fusion: Combine images with different exposure times to obtain an HDR image, and adjust the working mode through the HDR image to determine the illumination difference in different areas and obtain a processed image; S1.2: Concentration classification processing: According to the PM10 concentration, the construction scene is divided into concentrations, and image processing is performed based on the divided construction scenes, specifically: When the PM10 concentration is low, high-level features of the image are extracted by downsampling, image details are restored by upsampling, and jump connections are added between different levels. When the PM10 concentration in the environment is high, the millimeter wave radar is used to obtain the distance and speed information of the object to locate and distinguish the object.

3. A construction engineering construction safety hazard tracking and management system according to claim 2, characterized in that: Get the processed image, including: M1: Determine the dynamic area: by obtaining different exposure images of the same scene, and dividing the area in the scene according to the motion vector between adjacent frame images, and obtaining the final generated HDR image according to the divided area; M2: Light intensity gradient analysis: According to the HDR image, the light intensity change rate within a preset time is obtained, and the working mode is determined according to the light intensity change rate; M3: Regional processing: According to the working mode, the illumination difference in different regions is determined, and image processing is performed based on the illumination difference and the specific region.

4. A construction engineering construction safety hazard tracking and management system according to claim 3, characterized in that: Get the final HDR image, including: M1.1: Collect three exposure images: obtain three different exposure images of the same scene through three different exposure times; M1.2: Determine the motion area: According to the optical flow vector of each pixel in the exposure image, obtain the modulus length corresponding to the pixel, and compare the modulus length with a preset threshold. When the modulus length is greater than the preset threshold, the area where the pixel is located is a high motion area. Otherwise, the area where the pixel is located is a low motion area or a static area. The specific calculation formula of the modulus length is: ， in: is the optical flow vector corresponding to the pixel at position (x, y), is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel; M1.3: Determine exposure combination: According to the area category corresponding to the area where the pixel point is located, the three exposure images corresponding to the pixel point are weighted, specifically: When the area where the pixel point is located is a high motion area, the weight distribution of the three exposure images corresponding to the pixel point is: the weight of the short exposure image is set to 1, the weight of the medium exposure image is set to 0, and the weight of the long exposure image is set to 0; When the area where the pixel point is located is a low-motion area or a static area, the weight distribution of the three exposure images corresponding to the pixel point is: the weight of the short exposure image is set to 0.3, the weight of the medium exposure image is set to 0.5, and the weight of the long exposure image is set to 0.2; M1.4: Determine HDR value: According to the weight distribution of the three exposure images corresponding to the pixel point, obtain the brightness value of the pixel point in the finally generated HDR image, and obtain the finally generated HDR image according to the brightness value. The brightness value of the pixel point in the finally generated HDR image is specifically: ， in: is the brightness value of the pixel (x, y) in the final generated HDR image, is the weight of the i-th exposure image at the pixel (x, y), is the brightness value of the pixel (x, y) in the i-th exposure image, The type index of the exposure image.

5. A construction engineering construction safety hazard tracking and management system according to claim 3, characterized in that: Determine the working mode, including: M2.1: Obtaining the average illumination intensity change ratio: According to the illumination intensity change between two adjacent HDR images, the average illumination intensity change ratio is determined, specifically: ， in: is the average light intensity change rate, is the light intensity difference between two adjacent HDR image frames, is the total number of pixels in the image, is the time difference between two measurements of HDR brightness values, is the horizontal coordinate of the pixel point, is the vertical coordinate of the pixel; M2.2: Determine the working mode: compare the average light intensity change rate with the preset change threshold. When the average light intensity change rate is greater than the preset change threshold, the working mode is adjusted to the short exposure mode. When the average light intensity change rate is less than the preset change threshold, the working mode is adjusted to the long exposure mode. Otherwise, the working mode remains unchanged.

6. A construction engineering construction safety hazard tracking and management system according to claim 3, characterized in that: Perform image processing, including: M3.1: Construction scene partitioning: According to the working mode, HDR images with different exposure levels are obtained, and the illumination differences between different areas in the HDR image are quantified through the histogram to determine the upper threshold of the low light area and the lower threshold of the high light area. The partition area is determined according to the upper threshold of the low light area and the lower threshold of the high light area, specifically: Compare the light brightness of different areas with the upper threshold of the low light area and the lower threshold of the high light area. When the light brightness is less than the upper threshold of the low light area, the area where the light brightness is located is the low light area. When the light brightness is greater than the lower threshold of the high light area, the area where the light brightness is located is the high light area. Otherwise, the area where the light brightness is located is the medium light area. M3.2: Identify specific areas: Identify shadow areas and reflective areas in each partition area to obtain specific areas; M3.3: Image processing: Perform image processing according to the partition area where the specific area is located, specifically: When the partitioned area is a low-light area, image processing is performed using image enhancement technology; When the partitioned area is a medium light area, image processing is performed by using a local tone mapping technique; When the partitioned area is a highlight area, image processing is performed using a shadow removal algorithm.

7. A construction engineering construction safety hazard tracking and management system according to claim 1, characterized in that: Get the actions of the blocked parts, including: S2.1: Obtain feature information: Process the enhanced image through a hierarchical feature fusion network to obtain multi-level feature information; S2.2: Determine the occluding action: combine the feature information with the fused data, perform feature complementation on the occluding target, and obtain the angle of the action occurring in adjacent time periods.

8. A construction engineering construction safety hazard tracking and management system according to claim 7, characterized in that: Get the angles of actions that occurred in adjacent time periods, including: N1: Feature complementation: The data from different sensors are complemented to obtain the location of the occluded part, specifically: ， in: To finally reconstruct the complete human body model, is a partial human body model directly obtained from the original image or sensor data, is the estimated value of the occluded part; N2: Human body contour reconstruction: Reconstruct the three-dimensional posture of the human body according to the position of the occluded part, and obtain the posture parameters that minimize the prediction, specifically: ， in: is the optimal posture parameter after final optimization, is the feature vector extracted from multimodal data, is the feature vector of the human body model generated according to the posture parameter θ, is the posture parameter before optimization; N3: Determine the action: According to the minimized predicted posture parameters, obtain the angle of the action occurring in the adjacent time period at the occluded part, specifically: ， in: is the change of posture parameters in adjacent time periods, is the optimal posture parameter after optimization at time t+1. is the optimal posture parameter after final optimization corresponding to time t.

9. A construction engineering construction safety hazard tracking and management system according to claim 1, characterized in that: Sending the virtual safety protection layer to the AR inspection terminal includes: W1: Determine the construction stage: According to the time when the illegal action occurs, determine the construction stage corresponding to the time when the illegal action occurs from the BIM construction progress model; W2: Determine the construction location: According to the location where the illegal action occurred and the area involved in the construction phase, obtain the relationship between the location where the illegal action occurred and the area involved in the construction phase; W3: Safety measures matching: Compare the safety measures specified in the construction phase with the illegal actions. When the illegal actions do not match the safety measures, execute the next step. Otherwise, delete the illegal actions. W4: Send AR inspection terminal: When the illegal action does not initiate protective measures as planned, the virtual safety protection layer is automatically generated, and the data of the virtual safety protection layer is converted into three-dimensional model data, and the three-dimensional model data is transmitted to the AR device through the wireless communication protocol.

10. A method for tracking and managing safety hazards in construction projects, characterized in that: A construction project construction safety hazard tracking and management system as described in any one of claims 1-9 is used.

Images