Construction scene risk hidden danger identification method based on multi-model layered detection

Abstract

The invention provides a construction scene risk hidden danger identification method based on multi-model layered detection, and aims to solve the problems that traditional manual inspection is low in efficiency and single-model detection is insufficient in precision and real-time performance in a complex construction environment. The method comprises the following steps: firstly, constructing a high-altitude long-range data set, introducing a multi-scale feature pyramid network based on an improved YOLOv5l architecture, and training a long-range target positioning model; secondly, constructing a fine close-range data set, and training a high-precision close-range target detection model based on a YOLOv5s architecture in combination with a channel-space attention mechanism; thirdly, building a hierarchical cooperation framework of coarse detection of a long-shot model and fine recognition of a close-shot model, scanning an image through the long-shot model to output a detection frame, dynamically calculating a cutting range according to the size of the detection frame and the image resolution, and generating a cutting image focusing a high-risk area; and finally, inputting the cut image into a close-range model to identify key risk features, outputting a refined detection result, and simultaneously jointly training the two models to optimize the performance.

Description

Technical Field

[0001] This invention relates to the field of construction scenario risk identification, and in particular to a method for identifying construction scenario risks and hazards based on multi-model hierarchical detection. Background Technology

[0002] With the continuous acceleration of urbanization and the ongoing expansion of infrastructure construction, the issue of safety management at construction sites is receiving increasing attention.

[0003] The construction environment is complex and changeable, involving a large number of high-altitude operations, large machinery operations, and dense personnel activities, posing potential risks and hazards such as falls from heights and mechanical injuries.

[0004] Traditional construction site safety management relies primarily on manual inspections and video surveillance playback, which is not only inefficient but also makes it difficult to detect and proactively warn of potential risks in real time, easily leading to safety accidents. In recent years, the rapid development of artificial intelligence and computer vision technologies has provided new solutions for construction site safety supervision. In particular, the widespread application of target detection and image recognition technologies in industrial settings has made it possible to automatically identify safety hazards during construction through intelligent video analysis.

[0005] However, existing single-model-based target detection methods still face many challenges when dealing with complex and ever-changing construction environments. For example, in large-scale, long-distance monitoring scenarios, due to factors such as large differences in target scale, severe occlusion, and frequent changes in lighting, a single model often struggles to balance detection accuracy and real-time performance, resulting in a high rate of missed detections for small targets (such as distant construction workers, hooks, scaffolding, etc.) and inaccurate identification of key risk behaviors. On the other hand, most intelligent recognition systems in construction scenarios lack an effective hierarchical processing mechanism for scene information and fail to fully utilize the complementary relationships between features of different scales and levels. At the same time, the systems also have significant shortcomings in dealing with dynamically changing construction conditions, complex background interference, and controlling equipment deployment costs.

[0006] Therefore, there is an urgent need to propose a new method for identifying risks and hidden dangers in construction scenarios in order to solve the existing technical problems. Summary of the Invention

[0007] The main objective of this invention is to provide a construction scenario risk and hazard identification method based on multi-model hierarchical detection. This method solves the problems of low efficiency and difficulty in real-time early warning of traditional manual inspection and video monitoring playback in the background technology, as well as the shortcomings of existing single-model target detection methods in complex construction environments, such as difficulty in balancing detection accuracy and real-time performance, high false negative rate of small targets, lack of scene information hierarchical processing mechanism, and insufficient ability to cope with dynamic working conditions.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for identifying risks and hidden dangers in construction scenarios based on multi-model hierarchical detection, the method comprising: S1. Construct a high-altitude distant view dataset, and train a distant target localization model based on the improved YOLOv5l architecture and the introduction of a multi-scale feature pyramid network. S2. Construct a detailed near-field dataset, and train a high-precision near-field target detection model based on the YOLOv5s architecture and combined with the channel space attention mechanism. S3. Construct a multi-model hierarchical collaborative framework of "coarse detection by distant model and fine recognition by close-up model". The distant model performs a preliminary scan of the construction scene image and outputs the target detection box. The cropping range is dynamically calculated based on the size of the detection box and the image resolution to generate a cropped image that focuses on the high-risk area. S4. Input the cropped image into the near-field target detection model to identify key risk features and output refined risk detection results. At the same time, train the far-field and near-field models together to optimize the recognition performance.

[0009] In the preferred embodiment, step S1, which involves constructing a high-altitude distant view dataset and training a distant target localization model, specifically includes: S11. Analyze historical monitoring data of construction scenarios, extract the spatiotemporal distribution characteristics of common risks and hazards, screen key risk types and determine risk levels; S12. Collect high-altitude monitoring images covering basic construction operations, high-altitude operations, and densely populated areas. Perform detailed annotations on personnel, hooks, planks, and mechanical equipment targets in the images to generate an annotated dataset containing target category and location information. S13. The labeled dataset is processed using data augmentation strategies such as random cropping, color dithering, and Gaussian noise. The processed dataset is then randomly divided into training and testing sets in an 8:2 ratio. S14. Introduce a multi-scale feature pyramid network into the YOLOv5l architecture, train the model using the training set, and obtain a distant target localization model with small target perception capability.

[0010] In the preferred scheme, the cross-layer feature fusion method of the multi-scale feature pyramid network in step S14 is as follows: S1401. First, calculate the information entropy of the feature map of the k-th layer. Information entropy is a classic indicator for measuring information richness. In a construction scenario, the richer the feature map of the target details, the greater the difference in pixel value distribution, and the higher the information entropy. Therefore, the formula... Calculate, where M and N are the height and width of the feature map of the k-th layer, respectively. The coordinates in the feature map of the k-th layer The normalized probability of a pixel value is calculated by weighting the logarithm of the pixel value probabilities, thus quantifying the information richness of the feature map. The larger the value, the more detailed information about small targets the feature map contains; S1402, Next, calculate the target response value of the feature map of the k-th layer. In construction scenarios, the response features of small targets on the feature map differ significantly from those of the background. Therefore, the formula is used to determine the response features. Calculation, where For the target region in the feature map of the kth layer The total number of pixels, Coordinates within the target area eigenvalues ​​at that location Let be the global average eigenvalue of the k-th layer feature map. By calculating the mean deviation between the target region feature value and the global average eigenvalue, the response strength of the feature map to small targets is quantified. The larger the value, the more pronounced the response of the feature map to small targets; S1403. Finally, calculate the fusion weights of the feature map of the k-th layer. To ensure that feature maps rich in information and strong in response to small targets account for a higher proportion during fusion, based on the above... and The calculation results are obtained through the formula. The calculation is performed, where K is the number of feature layers participating in the fusion, by normalizing the feature maps of each layer. and The product is used to obtain the fusion weights of the feature maps of each layer, so as to allocate the fusion weights according to the contribution of small target detection and improve the perception ability of the far-field model to small targets.

[0011] In the preferred embodiment, step S2 involves constructing a detailed close-up dataset and training a high-precision close-up target detection model, including: S21. Using the high-risk area images cropped from the high-altitude distant view dataset in step S1, and combining them with the open-source close-up dataset as the data source, high-resolution close-up images are collected. S22. Perform pixel-level annotation on risk subjects and environmental factors in close-up images to generate an annotated dataset containing risk category and location information. S23. Use data augmentation strategies such as random cropping, color dithering, and Gaussian noise to simulate dynamic construction scenarios. Divide the labeled dataset into training and testing sets in an 8:2 ratio and share the same labeling standard with the prospective dataset in step S1. S24. Introduce a channel-space attention mechanism into the YOLOv5s architecture, train the model using a training set to enhance the feature extraction capability of key risk areas, and obtain a high-precision near-field target detection model. In the preferred embodiment, in step S24: S2401. First, calculate the local contrast matrix C of the feature map. In the construction scene, the local features of the key risk areas differ significantly from the background. Therefore, the formula is used... Calculation, where coordinates Local contrast value at that location coordinates Feature values ​​of pixels within a 3×3 neighborhood. coordinates The feature value at a location is calculated by taking the average deviation of the feature values ​​of the pixel from those of its neighboring pixels to quantify the difference between the local area and the background. The larger the C value, the more critical the risk area is. S2402. Next, calculate the global saliency matrix S of the feature map. In construction scenarios, key risk areas are often concentrated in the image center. Therefore, combining local contrast and center region preference, the formula... Calculation, where The maximum value of the local contrast matrix C. The center coordinates of the feature map The standard deviation of the Gaussian kernel; S2403, Finally, generate the spatial attention weight matrix. To fuse the aforementioned local contrast and global saliency information into weights that can be directly applied to the feature map, the formula is used. Calculation, where This represents a matrix concatenation operation. This represents a 1×1 convolution operation. This is the Sigmoid activation function.

[0012] In the preferred embodiment, in step S3: S31. Input the real-time monitoring image of the construction scene into the distant target localization model trained in step S1, perform a preliminary scan of the entire image with dimensions [W,H], and output a square detection box containing targets such as personnel and hooks, with the coordinates of the detection box being... ; S32. Calculate the dynamic offset distance. Determine the dynamic offset distance based on the detection box size and image resolution. Calculate the cropping box coordinates based on the offset distance and detection box coordinates. Generate a cropped image focusing on high-risk areas according to the cropping box coordinates to reduce redundant background interference.

[0013] In the preferred embodiment, the calculation of the dynamic offset distance in step S32 further includes: S3201. First, calculate the target scale corresponding to the detection box. The target scale directly reflects the size of the target in the image, therefore, it is determined by the formula... Calculation, where , These represent the width and height of the detection box, respectively. By calculating the square root of the area of ​​the detection box, the actual scale of the target in the image is quantified, providing a basis for adjusting the target size for the offset distance. S3202, Next, calculate the resolution coefficient of the image. Images with different resolutions have different pixel densities, resulting in different requirements for the cropping area. Therefore, the formula is used to determine the appropriate cropping area. Calculation, where The resolution of the current image. To establish a preset baseline resolution, the ratio of the current resolution to the baseline resolution is used to quantify the resolution difference, providing an image resolution basis for adjusting the offset distance. S3203. Next, calculate the adaptive offset coefficient k. To ensure that the offset coefficient adapts to both the target scale and resolution, based on the above... and The calculation results are obtained through the formula. Calculation, where Based on the offset coefficient, To adjust the coefficient, The target scale attenuation coefficient is used to make small targets correspond to larger offset coefficients through the target scale attenuation term, and to make low-resolution images correspond to larger offset coefficients through the resolution compensation term, thereby achieving adaptive adjustment of the offset coefficient; In the preferred embodiment, step S4 includes: S41. Input the cropped image generated in step S3 into the near-field target detection model trained in step S2, identify key risk features in the image such as the wearing status of the safety helmet and the fixing status of the hook, and output refined detection results containing risk type, location and level. S42: Jointly train the distant target localization model and the near target detection model to optimize the collaboration logic between the models; use test set data and real construction project scene data to verify the model performance, and adjust the feature fusion weights of the multi-scale feature pyramid network and the attention coefficient parameters of the channel space attention mechanism according to the verification results to ensure the stability and accuracy of the model in complex construction scenarios.

[0014] In the preferred embodiment, step S42 includes: S4201. First, calculate the positioning loss of the distant target positioning model. The localization loss needs to reflect the positional deviation between the detection box and the ground truth box, therefore the CIoU loss formula is used. Calculation, where For the detection box b and the ground truth box The intersection and union ratio, Let c be the squared Euclidean distance between the center of the detection box and the center of the ground truth box, and let c be the squared length of the diagonal of the smallest bounding rectangle enclosing the detection box and the ground truth box. For balance coefficient, w and h are the width and height of the detection frame. , To accurately measure the true frame width and height, we comprehensively consider the intersection-over-union ratio, center distance, and aspect ratio to precisely quantify the positioning deviation and provide a basis for optimizing the positioning of the distant model. S4202, Next, calculate the classification loss of the near-field target detection model. To address the imbalance between risk categories, a modified Focal Loss formula is used. Calculate, where C is the total number of risk categories. To predict the probability of the i-th type of risk in the model, For tags, The difficulty coefficient is assigned a higher weight to difficult samples to balance the training contribution of different categories of samples and provide a basis for the classification optimization of the near-field model. S4203, Next, calculate the model's cooperative error. The collaborative error needs to reflect the consistency of the detection results of the two models, therefore, it is determined by the formula... Calculation, where Output detection boxes for the distant view model Output detection boxes of the close-up model The intersection and union ratio, To predict risk probabilities for the long-term model, To predict risk probabilities for near-field models, this formula quantifies the deviation between the results of the two models by using the crossover ratio and probability difference, providing a basis for collaborative optimization of models; S4204, Finally, calculate the combined loss. To coordinate positioning, classification, and collaborative optimization objectives, a formula is used. Calculation, where , , The formula uses weighted coefficients to weight and fuse the three loss components, enabling joint training to simultaneously optimize the independent and collaborative performance of the two models and improve the overall recognition effect.

[0015] The preferred solution also includes steps for outputting risk and hazard results and linking early warnings: Results output: The detailed detection results output by the close-up model include risk category, risk location and risk level. The risk location is converted from the image pixel coordinates to the actual coordinates of the construction site through the calibration parameters of the monitoring camera. The risk level is divided according to the probability of occurrence and the degree of harm. Early warning linkage: Risk results are transmitted to the construction site safety management platform. The platform triggers different early warning mechanisms according to the risk level. High-level risks trigger audible and visual alarms and push emergency messages to the mobile terminals of safety management personnel. Medium-level risks only push messages to mobile terminals, and low-level risks are only recorded in the platform log. At the same time, the platform automatically stores risk images and identification results to form a risk hazard ledger for easy traceability and analysis later.

[0016] This invention provides a method for identifying risks and hazards in construction scenarios based on multi-model hierarchical detection. This method constructs a multi-model hierarchical collaborative framework of "coarse detection by distant model and fine identification by near model", and combines advanced data processing technology and deep learning algorithms to effectively improve the problem of false alarms and missed alarms in long-distance construction scenarios by single target detection models, thereby enhancing the algorithm's comprehensive understanding ability.

[0017] In terms of distant scene detection, key risk types were screened by analyzing historical monitoring data, and a high-altitude distant scene dataset covering typical construction scenarios was constructed. An improved YOLOv5l architecture was adopted, and a multi-scale feature pyramid network (FPN) was introduced to enhance the perception of small targets and improve the model's adaptability under complex lighting conditions. Moreover, this dataset supports multi-task training, providing a general foundation for model transfer learning. In terms of close-range scene recognition, a refined close-range scene dataset was constructed based on high-risk area images cropped from the distant scene dataset and open-source datasets. The YOLOv5s architecture combined with the channel-space attention mechanism (CBAM) was used to enhance the feature extraction of key risk areas, improving the stability of dynamic target detection. At the same time, the use of open-source datasets reduced annotation costs and shortened the training cycle.

[0018] In terms of multi-model collaboration and dynamic cropping, after the distant model initially scans the image and outputs the detection box, it dynamically calculates the cropping range based on the detection box size and image resolution, reducing redundant background interference and improving the inference efficiency of a single image. Through the collaboration of distant and near models, the overall detection accuracy is improved, the ability to identify occluded targets is enhanced, and the false detection rate is reduced.

[0019] Furthermore, this method supports joint detection of multiple risk types. After deployment in construction projects, it increases the number of risk events identified daily, shortens the accident warning response time, and can efficiently identify potential risks in construction scenarios and provide safety accident warnings. This provides intelligent and efficient technical support for ensuring the safety of construction personnel and the stable operation of construction projects. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1This is a flowchart of the construction scenario risk and hazard identification method of the present invention. Detailed Implementation

[0021] Example 1 like Figure 1 As shown, a method for identifying risks and hazards in construction scenarios based on multi-model hierarchical detection is proposed. This method includes: S1. Construct a high-altitude distant view dataset, and train a distant target localization model based on the improved YOLOv5l architecture and the introduction of a multi-scale feature pyramid network. S2. Construct a detailed near-field dataset, and train a high-precision near-field target detection model based on the YOLOv5s architecture and combined with the channel space attention mechanism. S3. Construct a multi-model hierarchical collaborative framework of "coarse detection by distant model and fine recognition by close-up model". The distant model performs a preliminary scan of the construction scene image and outputs the target detection box. The cropping range is dynamically calculated based on the size of the detection box and the image resolution to generate a cropped image that focuses on the high-risk area. S4. Input the cropped image into the near-field target detection model to identify key risk features and output refined risk detection results. At the same time, train the far-field and near-field models together to optimize the recognition performance.

[0022] By employing a multi-model hierarchical collaboration mechanism, combining the advantages of long-range localization and detailed close-range recognition, this approach addresses the issues of low recognition accuracy and poor real-time performance of traditional single-model approaches in complex construction scenarios. It achieves efficient and accurate identification of construction risks and hazards, providing intelligent technical support for construction site safety management. The actual deployment steps are as follows: First, high-altitude long-range and detailed close-range datasets are labeled using annotation software (such as LabelImg). Then, improved YOLOv5l and YOLOv5s models are built using the PyTorch deep learning framework. YOLOv5l introduces a multi-scale feature pyramid network, while YOLOv5s incorporates a channel-spatial attention mechanism. The model is trained using a training module, and then the OpenCV module is used to acquire and preprocess images of the construction scene. A multi-model hierarchical collaborative framework is deployed on the edge computing device. The image processing and model inference module written in Python completes the entire process from image input, coarse detection of distant scenes, dynamic cropping to fine recognition of close-up scenes. Finally, the recognition results are transmitted to the safety management platform. The software and modules used include the labeling software LabelImg, the deep learning framework PyTorch, the image processing library OpenCV, the model inference module TorchScript, the edge computing device system, and the data transmission module of the safety management platform.

[0023] In the preferred embodiment, step S1, which involves constructing a high-altitude distant view dataset and training a distant target localization model, specifically includes: S11. Analyze historical monitoring data of construction scenarios, extract the spatiotemporal distribution characteristics of common risks and hazards, screen key risk types and determine risk levels; S12. Collect high-altitude monitoring images covering basic construction operations, high-altitude operations, and densely populated areas. Perform detailed annotations on personnel, hooks, planks, and mechanical equipment targets in the images to generate an annotated dataset containing target category and location information. S13. The labeled dataset is processed using data augmentation strategies such as random cropping, color dithering, and Gaussian noise. The processed dataset is then randomly divided into training and testing sets in an 8:2 ratio. S14. Introduce a multi-scale feature pyramid network into the YOLOv5l architecture, train the model using the training set, and obtain a distant target localization model with small target perception capability.

[0024] By analyzing historical data to screen key risk types and ensure dataset relevance, data augmentation strategies are employed to enhance the model's adaptability to complex construction environments. A multi-scale feature pyramid network is introduced to enhance the long-range model's ability to perceive small targets, reducing the false negative rate for small targets. The actual deployment steps are as follows: Python data analysis libraries (Pandas, NumPy) are used to analyze the spatiotemporal distribution characteristics of historical monitoring data to determine key risk types. High-altitude long-range images of different construction scenarios are collected using monitoring cameras, and target information is labeled using LabelImg. The dataset is processed and divided into training and test sets using PyTorch's DataLoader module and data augmentation functions (RandomCrop and ColorJitter in torchvision.transforms). An improved YOLOv5l model with a multi-scale feature pyramid network is built within the PyTorch framework. Model training is completed using the training module, and model performance is verified using the test set. The software and modules used in the deployment include Python. The system includes data analysis libraries (Pandas, NumPy), image acquisition software, labeling software LabelImg, the deep learning framework PyTorch, and model training and validation modules. The PyTorch framework includes the torchvision.transforms data augmentation module and the DataLoader data loading module.

[0025] In the preferred scheme, the cross-layer feature fusion method of the multi-scale feature pyramid network in step S14 is as follows: S1401. First, calculate the information entropy of the feature map of the k-th layer. Information entropy is a classic indicator for measuring information richness. In a construction scenario, the richer the feature map of the target details, the greater the difference in pixel value distribution, and the higher the information entropy. Therefore, the formula... Calculate, where M and N are the height and width of the feature map of the k-th layer, respectively. The coordinates in the feature map of the k-th layer The normalized probability of a pixel value is calculated by weighting the logarithm of the pixel value probabilities, thus quantifying the information richness of the feature map. The larger the value, the more detailed the small target information contained in the feature map; when using it, first count the frequency of each pixel value in the k-th layer feature map, and then calculate the normalized probability. Then substitute into the formula and sum to get ; S1402, Next, calculate the target response value of the feature map of the k-th layer. In construction scenarios, the response features of small targets on the feature map differ significantly from those of the background. Therefore, the formula is used to determine the response features. Calculation, where For the target region in the feature map of the kth layer The total number of pixels, Coordinates within the target area eigenvalues ​​at that location Let be the global average eigenvalue of the k-th layer feature map. By calculating the mean deviation between the target region feature value and the global average eigenvalue, the response strength of the feature map to small targets is quantified. The larger the value, the more pronounced the response of the feature map to small targets; when using it, first determine the target region. Calculate its total number of pixels and the characteristic values ​​of each point in the region Then calculate the global average eigenvalue. Substituting into the formula and summing to obtain the mean value. ; S1403. Finally, calculate the fusion weights of the feature map of the k-th layer. To ensure that feature maps rich in information and strong in response to small targets account for a higher proportion during fusion, based on the above... and The calculation results are obtained through the formula. The calculation is performed, where K is the number of feature layers participating in the fusion, by normalizing the feature maps of each layer. and The product yields the fusion weights of the feature maps from each layer, allowing for the allocation of fusion weights based on the contribution of small object detection, thus improving the ability of the distant model to perceive small objects. When using this method, the weights for each layer are first calculated. and Multiply the products, sum them to get the denominator, and then divide each product by the denominator to get the result. Features from each layer are integrated according to their weights.

[0026] By calculating information entropy, target response value, and fusion weights, the multi-scale feature pyramid network can allocate fusion weights based on the contribution of feature maps to small target detection. This significantly improves the ability of the distant model to perceive small targets, reduces the false negative rate of distant small targets, and enhances the model's adaptability to complex construction environments. The actual deployment steps are as follows: Write a function to calculate the information entropy of the feature map in the PyTorch framework, taking the feature map size and pixel values ​​as input, and outputting... Write a function to calculate the target response value, taking the target region coordinates, eigenvalues, and global average eigenvalues ​​as input, and outputting... Write a function to calculate the fusion weights, taking input from each layer. and Output These functions are integrated into the feature fusion module of the multi-scale feature pyramid network, which automatically performs cross-layer feature fusion calculations during model training. After training, the model is converted to ONNX format and deployed to an edge computing device. The fusion calculation function is called through the model inference module. The software and modules used for deployment include the deep learning framework PyTorch (including a custom feature calculation function module), the model format conversion tool (ONNXRuntime), and the model inference module (ONNX Runtime inference engine) on the edge computing device.

[0027] In the preferred embodiment, step S2 involves constructing a detailed close-up dataset and training a high-precision close-up target detection model, including: S21. Using the high-risk area images cropped from the high-altitude distant view dataset in step S1, and combining them with the open-source close-up dataset as the data source, high-resolution close-up images are collected. S22. Perform pixel-level annotation on risk subjects and environmental factors in close-up images to generate an annotated dataset containing risk category and location information. S23. Use data augmentation strategies such as random cropping, color dithering, and Gaussian noise to simulate dynamic construction scenarios. Divide the labeled dataset into training and testing sets in an 8:2 ratio and share the same labeling standard with the prospective dataset in step S1. S24. Introduce a channel-space attention mechanism into the YOLOv5s architecture, train the model using a training set to enhance the feature extraction capability of key risk areas, and obtain a high-precision near-field target detection model. This study utilizes high-risk area images from a distant-view dataset and open-source datasets to construct a close-view dataset, reducing data acquisition costs. Pixel-level annotation improves dataset accuracy, and a unified annotation standard ensures multi-model collaboration. A channel-spatial attention mechanism is introduced to enhance feature extraction of key risk areas, improving the accuracy of the close-view model. The actual deployment steps are as follows: High-risk area images are extracted from distant-view images using OpenCV image cropping functions. Combined with the open-source close-view dataset, pixel-level annotations are performed on the close-view images using LabelMe annotation software. The dataset is processed and divided into training and test sets using PyTorch's DataLoader module and data augmentation functions. A YOLOv5s model with a channel-spatial attention mechanism is built within the PyTorch framework. Model training is completed through the training module (setting the optimizer and loss function), and the model performance is verified using the test set. The software and modules used include the image processing library OpenCV (including the image cropping module), the annotation software LabelMe, open-source dataset management tools (such as TensorFlow Datasets), and the deep learning framework PyTorch (including torchvision.transforms). The module includes a data augmentation module, a DataLoader module, and a model training and validation module.

[0028] In the preferred embodiment, in step S24: S2401. First, calculate the local contrast matrix C of the feature map. In the construction scene, the local features of the key risk areas differ significantly from the background. Therefore, the formula is used... Calculation, where coordinates Local contrast value at that location coordinates Feature values ​​of pixels within a 3×3 neighborhood. coordinates The feature value at a location is calculated by taking the average deviation of the feature values ​​of the pixel from those of its neighboring pixels to quantify the difference between the local area and the background. The larger the C value, the more critical the risk area is. First determine the coordinates when using it. Given a 3×3 neighborhood, obtain the feature values ​​within the neighborhood and at the given point, calculate the absolute value of the deviation, sum them, and take the mean value to obtain the result. ; S2402. Next, calculate the global saliency matrix S of the feature map. In construction scenarios, key risk areas are often concentrated in the image center. Therefore, combining local contrast and center region preference, the formula... Calculation, where The maximum value of the local contrast matrix C. The center coordinates of the feature map The standard deviation is the Gaussian kernel. When using it, first find the maximum value of C. Calculate coordinates With center coordinates Substitute the squared distance into the exponent term, and then... Multiply to get ; S2403, Finally, generate the spatial attention weight matrix. To fuse the aforementioned local contrast and global saliency information into weights that can be directly applied to the feature map, the formula is used. Calculation, where This represents a matrix concatenation operation. This represents a 1×1 convolution operation. This is the Sigmoid activation function.

[0029] This formula integrates local contrast and global saliency information into spatial attention weights, enabling the model to focus on key risk areas. In practice, matrices C and S are first concatenated, then their dimensions are compressed using a 1×1 convolution, and finally mapped to the 0-1 interval using a Sigmoid activation function. .

[0030] By calculating the local contrast matrix, global saliency matrix, and spatial attention weight matrix, the channel-spatial attention mechanism can accurately locate key risk areas, improve the feature extraction accuracy of the near-field model for risk subjects, reduce the false detection rate, and enhance the model's adaptability to dynamic construction scenarios. The actual deployment steps are as follows: Write a function to calculate the local contrast matrix in the PyTorch framework, taking the feature map as input and outputting C; write a function to calculate the global saliency matrix, taking C, the center coordinates of the feature map, and... Write a function to generate the spatial attention weight matrix, taking C and S as inputs, and using PyTorch's nn.Concat module, nn.Conv2d module (with a 1×1 convolution kernel), and nn.Sigmoid module to output S. These functions are integrated into the channel-spatial attention mechanism module and embedded between the neck and head networks of the YOLOv5s architecture. Spatial attention weights are automatically calculated during model training. After training, the model is converted to TensorRT format to accelerate inference and deployed to edge computing devices. The software and modules used for deployment include the deep learning framework PyTorch (including nn.Concat, nn.Conv2d, nn.Sigmoid modules and custom calculation functions), the model format conversion tool (TensorRT Converter), and the TensorRT inference engine on the edge computing device.

[0031] In the preferred embodiment, in step S3: S31. Input the real-time monitoring image of the construction scene into the distant target localization model trained in step S1, perform a preliminary scan of the entire image with dimensions [W,H], and output a square detection box containing targets such as personnel and hooks, with the coordinates of the detection box being... ; S32. Calculate the dynamic offset distance. Determine the dynamic offset distance based on the detection box size and image resolution. Calculate the cropping box coordinates based on the offset distance and detection box coordinates. Generate a cropped image focusing on high-risk areas according to the cropping box coordinates to reduce redundant background interference.

[0032] The real-time monitoring image of the construction scene is input into the distant target localization model trained in step S1. A preliminary scan of the entire image of size [W,H] is performed, outputting square detection boxes containing targets such as personnel and hooks. The coordinates of the detection boxes are... W is the image width, and H is the image height. , The coordinates of the top left corner of the detection box. , The coordinates of the bottom right corner of the detection box; The dynamic offset distance is calculated based on the detection box size and image resolution. The cropping box coordinates are calculated based on the offset distance and detection box coordinates. A cropped image focusing on high-risk areas is generated according to the cropping box coordinates to reduce redundant background interference.

[0033] Through a multi-model hierarchical collaboration framework and dynamic cropping, a transition from global scanning of distant scenes to fine-grained recognition of close-up details is achieved. Dynamic cropping focuses on high-risk areas, reduces redundant background interference, improves model inference efficiency, and ensures that the cropped image completely contains the risky target, providing high-quality input for fine-grained close-up recognition. The actual deployment steps are as follows: On an edge computing device, the OpenCV VideoCapture module is used to read the monitoring image in real time, obtain the image size [W,H], and input the image into the deployed distant target localization model to obtain the detection box coordinates. The system uses a custom Python function to calculate the dynamic offset distance based on the detection box size and image resolution, and then combines the detection box coordinates to calculate the cropping box coordinates. The OpenCV crop function is used to generate the cropped image, which is then temporarily stored in shared memory. The software and modules used include the OpenCV image processing library (including VideoCapture and crop modules), the far-view model inference module on the edge computing device, and a custom Python computation and data storage module.

[0034] In the preferred embodiment, the calculation of the dynamic offset distance in step S32 further includes: S3201. First, calculate the target scale corresponding to the detection box. The target scale directly reflects the size of the target in the image, therefore, it is determined by the formula... Calculation, where , These represent the width and height of the detection box, respectively. By calculating the square root of the area of ​​the detection box, the actual scale of the target in the image is quantified, providing a basis for adjusting the target size for the offset distance. When using it, first calculate the width and height of the detection frame, multiply them, and then take the square root to get the result. ; S3202, Next, calculate the resolution coefficient of the image. Images with different resolutions have different pixel densities, resulting in different requirements for the cropping area. Therefore, the formula is used to determine the appropriate cropping area. Calculation, where The resolution of the current image. To establish a preset baseline resolution, the ratio of the current resolution to the baseline resolution is used to quantify the resolution difference, providing an image resolution basis for adjusting the offset distance. When using this method, first calculate the square root of the resolution of the current image and the reference image, then calculate the ratio to obtain the final resolution. ; S3203. Next, calculate the adaptive offset coefficient k. To ensure that the offset coefficient adapts to both the target scale and resolution, based on the above... and The calculation results are obtained through the formula. Calculation, where Based on the offset coefficient, To adjust the coefficient, The target scale attenuation coefficient is used to make small targets correspond to larger offset coefficients through the target scale attenuation term, and to make low-resolution images correspond to larger offset coefficients through the resolution compensation term, thereby achieving adaptive adjustment of the offset coefficient; Small targets and low-resolution images correspond to larger offset coefficients; when using them... W and preset , , Substituting into the formula, we obtain k; By calculating the target scale, resolution coefficient, adaptive offset coefficient, and dynamic offset distance, the dynamic offset distance is adaptively adjusted to ensure that the cropped region of small target, low-resolution images can completely contain the risk target, while reducing redundant background in the cropped region of large target, high-resolution images. This improves the quality of cropped images and the efficiency of model inference, while also enhancing the adaptability of the multi-model hierarchical collaboration framework to different construction scenarios. The actual deployment steps are as follows: Write a target scale calculation function in Python, input the detection box coordinates, and output... Write a function to calculate the resolution coefficient, taking the current image resolution and a reference resolution as input, and outputting... Write functions to calculate the adaptive offset coefficient and dynamic offset distance, and input... W and preset coefficients, output k and These functions are integrated into the dynamic cropping module and linked with the vision model inference module. After obtaining the coordinates of the detection box, the dynamic offset distance calculation and cropping operation are automatically completed and deployed in the multi-threaded processing framework of the edge computing device. In the preferred embodiment, step S4 includes: S41. Input the cropped image generated in step S3 into the near-field target detection model trained in step S2, identify key risk features in the image such as the wearing status of the safety helmet and the fixing status of the hook, and output refined detection results containing risk type, location and level. S42: Jointly train the distant target localization model and the near target detection model to optimize the collaboration logic between the models; use test set data and real construction project scene data to verify the model performance, and adjust the feature fusion weights of the multi-scale feature pyramid network and the attention coefficient parameters of the channel space attention mechanism according to the verification results to ensure the stability and accuracy of the model in complex construction scenarios.

[0035] The beneficial effects are as follows: the near-field model accurately identifies and outputs detailed risk information, providing precise decision-making basis for safety management; joint training optimizes the collaboration logic between models, improving overall recognition performance; and combined with real construction data verification and parameter adjustment, it enhances the stability and accuracy of the model in complex scenarios, reducing the probability of accidents. The actual deployment and usage steps are as follows: on the edge computing device, the cropped image in shared memory is input into the near-field target detection model through a sub-thread of the multi-threaded processing framework to obtain detection results such as risk type, location, and level. The results are formatted using the Python data processing module and transmitted to the safety management platform via the MQTT protocol module. During model joint training, the far-field and near-field model parameters are loaded in the PyTorch framework, a joint training dataset (including labeled far-field and near-field images) is used, a joint loss function is set, joint training is completed through the training module, and the model is verified using the test set and real construction data. Based on the verification results, parameters such as feature fusion weights and attention coefficients are adjusted. In the preferred embodiment, step S42 includes: S4201. First, calculate the positioning loss of the distant target positioning model. The localization loss needs to reflect the positional deviation between the detection box and the ground truth box, therefore the CIoU loss formula is used. Calculation, where For the detection box b and the ground truth box The intersection and union ratio, Let c be the squared Euclidean distance between the center of the detection box and the center of the ground truth box, and let c be the squared length of the diagonal of the smallest bounding rectangle enclosing the detection box and the ground truth box. For balance coefficient, w and h are the width and height of the detection frame. , To accurately measure the true frame width and height, we comprehensively consider the intersection-over-union ratio, center distance, and aspect ratio to precisely quantify the positioning deviation and provide a basis for optimizing the positioning of the distant model. Calculate before use , , Substituting v into the formula, we get ; S4202, Next, calculate the classification loss of the near-field target detection model. To address the imbalance between risk categories, a modified Focal Loss formula is used. Calculate, where C is the total number of risk categories. To predict the probability of the i-th type of risk in the model, For tags, The difficulty coefficient is assigned a higher weight to difficult samples to balance the training contribution of different categories of samples and provide a basis for the classification optimization of the near-field model. When using it, first obtain the predicted probability of each risk category. With tags Substituting into the formula and summing, we get ; S4203, Next, calculate the model's cooperative error. The collaborative error needs to reflect the consistency of the detection results of the two models, therefore, it is determined by the formula... Calculation, where Output detection boxes for the distant view model Output detection boxes of the close-up model The intersection and union ratio, To predict risk probabilities for the long-term model, To predict risk probabilities for near-field models, this formula quantifies the deviation between the results of the two models by using the crossover ratio and probability difference, providing a basis for collaborative optimization of models; When using this method, calculate the intersection-union ratio and the absolute value of the probability difference between the two detection boxes, and substitute them into the formula to obtain... ; S4204, Finally, calculate the combined loss. To coordinate positioning, classification, and collaborative optimization objectives, a formula is used. Calculation, where , , The formula uses weighted coefficients to weight and fuse the three loss components, enabling joint training to simultaneously optimize the independent and collaborative performance of the two models and improve the overall recognition effect.

[0036] When using , , Multiply by the corresponding weight coefficients and then sum them up to obtain .

[0037] By jointly calculating localization loss, classification loss, and collaborative error, joint training can simultaneously optimize the localization accuracy of the far-view model, the classification accuracy of the near-view model, and the collaboration between the two models, thereby improving the overall recognition accuracy, reducing the false positive and false negative rates, ensuring stable operation of the model in complex construction scenarios, and providing reliable support for safety early warning. The actual deployment steps are as follows: Write a CIoU loss calculation function in the PyTorch framework, input the detection box and ground truth box parameters, and output... Write an improved Focal Loss calculation function, taking the predicted probability and label of the risk category as input, and outputting... Write a function to calculate the collaborative error, taking the bounding boxes and predicted probabilities of the two models as input, and outputting... Write a function to calculate the joint loss, and input... , , and weighting coefficients, output These functions are integrated into a joint training module, which optimizes model parameters through backpropagation during model training, saves the optimal model weights after training, and deploys them to edge computing devices. The preferred solution also includes steps for outputting risk and hazard results and linking early warnings: Results output: The detailed detection results output by the close-up model include risk category, risk location and risk level. The risk location is converted from the image pixel coordinates to the actual coordinates of the construction site through the calibration parameters of the monitoring camera. The risk level is divided according to the probability of occurrence and the degree of harm. Early warning linkage: Risk results are transmitted to the construction site safety management platform. The platform triggers different early warning mechanisms according to the risk level. High-level risks trigger audible and visual alarms and push emergency messages to the mobile terminals of safety management personnel. Medium-level risks only push messages to mobile terminals, and low-level risks are only recorded in the platform log. At the same time, the platform automatically stores risk images and identification results to form a risk hazard ledger for easy traceability and analysis later.

[0038] By accurately outputting risk results and linking tiered early warnings, real-time visual management of construction risks is achieved. Rapid response to high-risk incidents reduces hazard, and ledger storage facilitates subsequent analysis and management optimization, improving the efficiency and level of on-site safety management. The actual deployment steps are as follows: On an edge computing device, a custom Python function is used to convert the pixel coordinates output from the near-view model into actual coordinates using camera calibration parameters. The risk level is determined based on the probability and severity of the risk. The risk results are transmitted to the safety management platform via the MQTT protocol module. The platform triggers corresponding early warning mechanisms through integrated audible and visual alarm control and message push modules, and simultaneously stores risk images and results through a database module. Establish a ledger; deploy the software and modules used, including the Python coordinate transformation module, MQTT protocol module, security management platform, database management system, and image storage module.

[0039] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A method for identifying risks and hidden dangers in construction scenarios based on multi-model hierarchical detection, characterized by: The method includes: S1. Construct a high-altitude distant view dataset, and train a distant target localization model based on the improved YOLOv5l architecture and the introduction of a multi-scale feature pyramid network. S2. Construct a detailed near-field dataset, and train a high-precision near-field target detection model based on the YOLOv5s architecture and combined with the channel space attention mechanism. S3. Construct a multi-model hierarchical collaborative framework of "coarse detection by distant model and fine recognition by near model". The distant model performs a preliminary scan of the construction scene image and outputs the target detection box. The cropping range is dynamically calculated according to the size of the detection box and the image resolution to generate a cropped image that focuses on the high-risk area. S4. Input the cropped image into the near-field target detection model to identify key risk features and output refined risk detection results. At the same time, train the far-field and near-field models together to optimize the recognition performance.

2. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 1, characterized in that: Step S1 involves constructing a high-altitude distant view dataset and training a distant target localization model, specifically including: S11. Analyze historical monitoring data of construction scenarios, extract the spatiotemporal distribution characteristics of common risks and hazards, screen key risk types and determine risk levels; S12. Collect high-altitude monitoring images covering basic construction operations, high-altitude operations, and densely populated areas. Perform detailed annotations on personnel, hooks, planks, and mechanical equipment targets in the images to generate an annotated dataset containing target category and location information. S13. The labeled dataset is processed using data augmentation strategies such as random cropping, color dithering, and Gaussian noise. The processed dataset is then randomly divided into training and testing sets in an 8:2 ratio. S14. Introduce a multi-scale feature pyramid network into the YOLOv5l architecture, train the model using the training set, and obtain a distant target localization model with small target perception capability.

3. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 2, characterized in that: The cross-layer feature fusion method for the multi-scale feature pyramid network in step S14 is as follows: S1401. First, calculate the information entropy of the feature map of the k-th layer. Information entropy is a classic indicator for measuring information richness. In a construction scenario, the richer the feature map of the target details, the greater the difference in pixel value distribution, and the higher the information entropy. Therefore, the formula... Calculate, where M and N are the height and width of the feature map of the k-th layer, respectively. The coordinates in the feature map of the k-th layer The normalized probability of a pixel value is calculated by weighting the logarithm of the pixel value probabilities, thus quantifying the information richness of the feature map. The larger the value, the more detailed information about small targets the feature map contains; S1402, Next, calculate the target response value of the feature map of the k-th layer. In construction scenarios, the response features of small targets on the feature map differ significantly from those of the background. Therefore, the formula is used to determine the response features. Calculation, where For the target region in the feature map of the kth layer The total number of pixels, Coordinates within the target area eigenvalues ​​at that location Let be the global average eigenvalue of the k-th layer feature map. By calculating the mean deviation between the target region feature value and the global average eigenvalue, the response strength of the feature map to small targets is quantified. The larger the value, the more pronounced the response of the feature map to small targets; S1403. Finally, calculate the fusion weights of the feature map of the k-th layer. To ensure that feature maps rich in information and strong in response to small targets account for a higher proportion during fusion, based on the above... and The calculation results are obtained through the formula. The calculation is performed, where K is the number of feature layers participating in the fusion, by normalizing the feature maps of each layer. and The product is used to obtain the fusion weights of the feature maps of each layer, so as to allocate the fusion weights according to the contribution of small target detection and improve the perception ability of the far-field model to small targets.

4. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 2, characterized in that: Step S2 involves constructing a detailed close-up dataset and training a high-precision close-up target detection model, including: S21. Using the high-risk area images cropped from the high-altitude distant view dataset in step S1, and combining them with the open-source close view dataset as the data source, high-resolution close view images are collected. S22. Perform pixel-level annotation on risk subjects and environmental factors in close-up images to generate an annotated dataset containing risk category and location information. S23. Use data augmentation strategies such as random cropping, color dithering, and Gaussian noise to simulate dynamic construction scenarios. Divide the labeled dataset into training and testing sets in an 8:2 ratio and share the same labeling standard with the prospective dataset in step S1. S24. Introduce a channel-space attention mechanism in the YOLOv5s architecture, train the model using a training set to enhance the feature extraction capability of key risk areas, and obtain a high-precision near-field target detection model.

5. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 4, characterized in that: In step S24: S2401. First, calculate the local contrast matrix C of the feature map. In the construction scene, the local features of the key risk areas differ significantly from the background. Therefore, the formula is used... Calculation, where coordinates Local contrast value at that location coordinates Feature values ​​of pixels within a 3×3 neighborhood. coordinates The feature value at a location is calculated by taking the average deviation of the feature values ​​of the pixel from those of its neighboring pixels to quantify the difference between the local area and the background. The larger the C value, the more critical the risk area is. S2402. Next, calculate the global saliency matrix S of the feature map. In construction scenarios, key risk areas are often concentrated in the image center. Therefore, combining local contrast and center region preference, the formula... Calculation, where The maximum value of the local contrast matrix C. The center coordinates of the feature map The standard deviation is the Gaussian kernel. S2403, Finally, generate the spatial attention weight matrix. To fuse the aforementioned local contrast and global saliency information into weights that can be directly applied to the feature map, the formula is used. Calculation, where This represents a matrix concatenation operation. This represents a 1×1 convolution operation. This is the Sigmoid activation function.

6. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 1, characterized in that: In step S3: S31. Input the real-time monitoring image of the construction scene into the distant target localization model trained in step S1, perform a preliminary scan of the entire image with dimensions [W,H], and output a square detection box containing targets such as personnel and hooks, with the coordinates of the detection box being... ; S32. Calculate the dynamic offset distance. Determine the dynamic offset distance based on the detection box size and image resolution. Calculate the cropping box coordinates based on the offset distance and detection box coordinates. Generate a cropped image focusing on high-risk areas according to the cropping box coordinates to reduce redundant background interference.

7. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 6, characterized in that: The calculation of the dynamic offset distance in step S32 also includes: S3201. First, calculate the target scale corresponding to the detection box. The target scale directly reflects the size of the target in the image, therefore, it is determined by the formula... Calculation, where , These represent the width and height of the detection box, respectively. By calculating the square root of the area of ​​the detection box, the actual scale of the target in the image is quantified, providing a basis for adjusting the target size for the offset distance. S3202, Next, calculate the resolution coefficient of the image. Images with different resolutions have different pixel densities, resulting in different requirements for the cropping area. Therefore, the formula is used to determine the appropriate cropping area. Calculation, where The resolution of the current image. To establish a preset baseline resolution, the ratio of the current resolution to the baseline resolution is used to quantify the resolution difference, providing an image resolution basis for adjusting the offset distance. S3203. Next, calculate the adaptive offset coefficient k. To ensure that the offset coefficient adapts to both the target scale and resolution, based on the above... and The calculation results are obtained through the formula. Calculation, where Based on the offset coefficient, To adjust the coefficient, The target scale attenuation coefficient is used to make smaller targets correspond to larger offset coefficients through the target scale attenuation term, and to make low-resolution images correspond to larger offset coefficients through the resolution compensation term, thereby achieving adaptive adjustment of the offset coefficient.

8. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 1, characterized in that: Step S4 includes: S41. Input the cropped image generated in step S3 into the near-field target detection model trained in step S2, identify key risk features in the image such as the wearing status of the safety helmet and the fixing status of the hook, and output refined detection results containing risk type, location and level. S42. Jointly train the distant target localization model and the near target detection model, and optimize the collaboration logic between the models; use test set data and real construction project scene data to verify the model performance, and adjust the feature fusion weight of the multi-scale feature pyramid network and the attention coefficient parameters of the channel space attention mechanism according to the verification results to ensure the stability and accuracy of the model in complex construction scenarios.

9. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 8, characterized in that: Step S42 includes: S4201. First, calculate the positioning loss of the distant target positioning model. The localization loss needs to reflect the positional deviation between the detection box and the ground truth box, therefore the CIoU loss formula is used. Calculation, where For the detection box b and the ground truth box The intersection and union ratio, Let c be the squared Euclidean distance between the center of the detection box and the center of the ground truth box, and let c be the squared length of the diagonal of the smallest bounding rectangle enclosing the detection box and the ground truth box. For balance coefficient, w and h are the width and height of the detection frame. , To accurately measure the true frame width and height, we comprehensively consider the intersection-over-union ratio, center distance, and aspect ratio to precisely quantify the positioning deviation and provide a basis for optimizing the positioning of the distant model. S4202, Next, calculate the classification loss of the near-field target detection model. To address the imbalance between risk categories, a modified Focal Loss formula is used. Calculate, where C is the total number of risk categories, To predict the probability of the i-th type of risk in the model, For tags, where This indicates the existence of type i risk. Conversely, The difficulty coefficient is assigned a higher weight to difficult samples to balance the training contribution of different categories of samples and provide a basis for the classification optimization of the near-field model. S4203, Next, calculate the model's cooperative error. The collaborative error needs to reflect the consistency of the detection results of the two models, therefore, it is determined by the formula... Calculation, where Output detection boxes for the distant view model Output detection boxes of the close-up model The intersection and union ratio, To predict risk probabilities for the long-term model, To predict risk probabilities for near-field models, this formula quantifies the deviation between the results of the two models by using the crossover ratio and probability difference, providing a basis for collaborative optimization of models; S4204, Finally, calculate the combined loss. To coordinate positioning, classification, and collaborative optimization objectives, a formula is used. Calculation, where , , The formula uses weighted coefficients to weight and fuse the three parts of the loss, enabling joint training to simultaneously optimize the independent and collaborative performance of the two models and improve the overall recognition effect.

10. The method for identifying construction scene risks and hidden dangers based on multi-model hierarchical detection according to claim 1, characterized in that: It also includes steps for outputting risk and hazard results and linking early warnings: Results output: The detailed detection results output by the close-up model include risk category, risk location and risk level. The risk location is converted from the image pixel coordinates to the actual coordinates of the construction site through the calibration parameters of the monitoring camera. The risk level is divided according to the probability of occurrence and the degree of harm. Early warning linkage: Risk results are transmitted to the construction site safety management platform. The platform triggers different early warning mechanisms according to the risk level. High-level risks trigger audible and visual alarms and push emergency messages to the mobile terminals of safety management personnel. Medium-level risks only push messages to mobile terminals, and low-level risks are only recorded in the platform log. At the same time, the platform automatically stores risk images and identification results to form a risk hazard ledger for easy traceability and analysis later.

Images