Unsafe action detection method and system

Abstract

The invention provides an unsafe action detection method and system. The method comprises the following steps: acquiring an original PPE wearing image and dividing the original PPE wearing image into training, verification and test set images according to a preset proportion; obtaining original feature target data and original associated feature data based on the multi-target detection sub-model, the training set image and the personnel unsafe action detection sub-model; performing small target supplementation based on the small target detection sub-model to obtain a target enhancement detection result; carrying out fusion to obtain fusion associated feature data so as to update the model and obtain a training model; and based on the data and the model, obtaining a performance improvement value, and if a condition is satisfied, importing the to-be-detected image into the YOLO model to obtain an unsafe action detection result. According to the unsafe action detection method provided by the invention, the problem of missed detection of the unsafe actions of the personnel caused by small target feature loss under the conditions of complex environment interference and large target scale difference is avoided, and real-time detection of the unsafe actions of the personnel is realized.

Description

Technical Field

[0001] This invention relates to the field of deep learning-based target detection technology, and in particular to an unsafe action detection method and system. Background Technology

[0002] In high-risk workplaces in industries such as telecommunications, power, and manufacturing, real-time monitoring of personal protective equipment (PPE) usage and issuing alerts helps prevent potential accidents, reduce injuries, and improve occupational safety standards. Currently, mainstream target detection algorithms fall into two main categories: one is a two-stage model based on region proposal, represented by Faster R-CNN and Mask R-CNN. While these algorithms offer good detection accuracy, their limited computational speed makes them unsuitable for real-time detection in scenarios such as human-machine collaborative assembly. The other category is a one-stage detection model, which simplifies the detection problem into a regression problem. Through convolutional neural networks, the class probability and location coordinates of the target can be directly obtained. Representative algorithms include YOLO, SSD, and RetinaNet. This type of model, with its smaller network size and faster computational speed, has significant advantages in applications requiring real-time identification and rapid decision-making. The YOLO algorithm, in particular, effectively detects unsafe behaviors such as employees not wearing safety helmets or smoking in industrial settings, while also balancing high accuracy and fast inference speed. It has driven the development of deep learning-based target detection and has become one of the most widely used models in industry. With technological iteration, the YOLO series has been updated to YOLOv8, which has shown excellent performance on the MSCOCO dataset, surpassing previous versions in both speed and accuracy. It can be called a State of the Art (SOTA) model available in the industry, successfully changing the status quo that deep learning networks have a huge number of parameters and are difficult to apply to edge devices.

[0003] Despite significant progress in first-stage detection models, especially the YOLO series algorithms, under the current technological background, several technical shortcomings remain in practical applications in high-risk workplaces. First, existing models struggle to achieve efficient and accurate detection in complex and variable environments, with large differences in target size and scale, and high target similarity. For example, complex environmental backgrounds severely interfere with the accuracy of part detection, affecting feature extraction from targets of different shapes and sizes. Second, in abnormal personnel movement detection tasks, interference information in complex environments, low lighting, and mechanical occlusion prevent existing algorithms from fully extracting accurate target location information and stronger semantic information from feature maps. They are also affected by background interference, noise, and low image quality. Furthermore, the large size and high computational requirements of deep learning models further restrict the speed and accuracy of detection. Finally, even the high-performance YOLOv8 has significant shortcomings in small object detection scenarios: on the one hand, its detection head used for loss calculation relies on deep-level features, which may lose some local details due to oversampling by the network, resulting in missing small object information; on the other hand, the PAN architecture used in YOLOv8 causes information loss due to multiple upsampling and downsampling operations during feature transfer and fusion; in addition, there is a common phenomenon of extremely unbalanced distribution of foreground and complex background information in small object detection, and existing mechanisms are unable to effectively balance the relationship between the two, making it difficult to guarantee the accuracy and completeness of small object detection. Summary of the Invention

[0004] The present invention aims to provide a method and system for detecting unsafe actions, in order to solve the above-mentioned technical problems, avoid the problem of missed detection of unsafe actions of personnel due to the loss of small target features in the case of complex environmental interference and large differences in target size, and realize the real-time detection of unsafe actions of personnel in high-risk workplaces.

[0005] To address the aforementioned technical problems, this invention provides a method for detecting unsafe actions, comprising: Acquire the original PPE wearing images and divide the original PPE wearing images into training set images, validation set images and test set images according to a preset ratio; Construct an initial YOLO model, which includes a YOLOv7 multi-object detection sub-model, a MAC-YOLO unsafe human action detection sub-model, and an ML-YOLO small object detection sub-model. Based on the YOLOv7 multi-object detection sub-model, multi-dimensional feature optimization and object recognition are performed on the training set images to obtain the original feature object data; Based on the MAC-YOLO unsafe action detection sub-model, the PPE state of personnel in the training set images is bound to the original associated feature data. Based on the ML-YOLO small target detection sub-model and the original associated feature data, small targets are added to the training set images to obtain target enhancement detection results; The original correlation feature data is fused with the target enhancement detection results to obtain fused correlation feature data, which is then used to update the initial YOLO model and obtain the YOLO training model. Based on the validation set images, test set images, the initial YOLO model, and the trained YOLO model, the YOLO model and its performance improvement value are obtained. If the performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

[0006] In the above scheme, the YOLOv7 multi-object detection sub-model optimizes multi-dimensional features and identifies targets in the training set images, accurately extracting key target information such as personnel and PPE equipment, generating raw feature target data, and providing reliable basic target input for subsequent action detection and small target recognition. Next, the MAC-YOLO unsafe action detection sub-model binds the PPE status of personnel in the training set images, associating PPE wearing status with unsafe actions, generating raw associated feature data, and achieving linked detection of unsafe actions and PPE status. Then, the ML-YOLO small target detection sub-model supplements the raw associated feature data with small targets, repairing lost small target features, identifying missed small-sized PPE and long-distance unsafe actions, avoiding the problem of missed detection of unsafe actions due to lost small target features in complex environments and with large differences in target scale, resulting in enhanced target detection results and improving unsafe action detection across all scenarios. Subsequently, fused correlation feature data is obtained by integrating the original correlation feature data with the target augmentation detection results. The initial YOLO model is then updated based on this fused correlation feature data, integrating the detection capabilities of each sub-model to generate a YOLO training model with full-scene detection capabilities. Finally, the performance of the YOLO training model is optimized using validation set images, and the performance improvement value is calculated by combining it with test set images. If the model performance improvement value meets a preset second condition, it can be determined that the YOLO model has achieved the required improvement in detection accuracy. At this point, the PPE-wearing images to be detected are imported, enabling real-time detection of unsafe actions by personnel in high-risk workplaces.

[0007] Furthermore, the YOLOv7 multi-object detection sub-model performs multi-dimensional feature optimization and object recognition on the training set images to obtain original feature object data, including: The YOLOv7 multi-target detection sub-model includes the ODConv module, the SimAM attention module, the neck PAN module, and the YOLOv7 detection head module; Based on the ODConv module, multidimensional attention adaptive feature extraction and convolution kernel optimization are performed on the training set images to obtain the initial PPE wearing feature map; The SimAM attention module is used to calculate the pixel importance score of the initial PPE wearing feature map, and attention weights are assigned to the pixels of the initial PPE wearing feature map based on the pixel importance score to obtain the enhanced PPE wearing feature map. The feature map of enhanced PPE wearing is extracted based on the neck PAN module to obtain shallow and deep features; The shallow and deep features are fused based on the neck PAN module to obtain a fused PPE wearing feature map; The YOLOv7 detection head module is used to confirm the fused PPE wearing feature map and obtain the feature target category, feature bounding box coordinates and feature target confidence. The original feature target data consists of the feature target category, feature bounding box coordinates, and feature target confidence score.

[0008] In the above scheme, the ODConv module performs multi-dimensional attention-adaptive feature extraction and convolution kernel optimization on the training set images, which can adaptively capture the low-level features of targets with different shapes and sizes to generate an initial PPE wearing feature map. Next, the SimAM attention module calculates the pixel importance score of the initial PPE wearing feature map and assigns attention weights to the pixels based on the pixel importance score. This combines spatial attention and channel attention without increasing computational cost, focusing on the target region and suppressing complex background interference to generate an enhanced PPE wearing feature map. Then, the neck PAN module performs layered extraction on the enhanced PPE wearing feature map, obtaining shallow and deep features respectively, laying the foundation for subsequent feature fusion. Subsequently, the neck PAN module fuses the shallow and deep features, solving the problem of large target scale differences and generating a fused PPE wearing feature map, improving the comprehensiveness of target recognition. Finally, the YOLOv7 detection head module confirms the target in the fused PPE wearing feature map, obtaining the feature target category, feature bounding box coordinates, and feature target confidence to reduce the recognition error of similar targets. Finally, the feature target category, feature bounding box coordinates, and feature target confidence are integrated into the original feature target data to form a structured output, providing reliable input for the subsequent action detection of the MAC-YOLO unsafe action detection sub-model and the small target supplementary detection of the ML-YOLO small target detection sub-model.

[0009] Furthermore, the step of performing multidimensional attention-adaptive feature extraction and convolution kernel optimization on the training set images based on the ODConv module to obtain the initial PPE wearing feature map includes: The ODConv module includes a GAP submodule, a total FC submodule, a four-branch activation submodule, and a four-branch FC submodule; Based on the GAP submodule, feature extraction is performed on the training set images to obtain the input features of the PPE wearing images; The GAP submodule performs a global averaging operation on the channels of the PPE wearing image input features to obtain a low-dimensional global feature vector. Based on the total FC submodule and the four-branch activation submodule, the low-dimensional global feature vector is subjected to dimensionality reduction transformation and ReLU activation to obtain spatial attention branch, channel attention branch, filter attention branch and kernel attention branch; Based on the four-branch FC submodule, the spatial attention branch, channel attention branch, filter attention branch and kernel attention branch are adjusted in dimension to obtain the spatial attention weight, channel attention dimension weight, filter attention dimension weight and kernel attention dimension weight; The original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules are integrated to obtain the aggregated convolutional kernel; The aggregated convolution kernel and the PPE wearing image input features are convolved to obtain the initial PPE wearing feature map.

[0010] In the above scheme, the GAP submodule extracts features from the training set images, initially capturing basic visual features related to PPE wearing and generating PPE wearing image input features, laying the initial feature foundation for subsequent multi-dimensional attention optimization. Next, the GAP submodule performs a global averaging operation on the channels of the PPE wearing image input features, transforming the high-dimensional spatial feature map into a low-dimensional global feature vector. This reduces the number of model parameters and the risk of overfitting, while also summarizing global feature information, providing a global basis for the calculation of attention dimension weights. Then, the total FC submodule and the four-branch activation submodule perform dimensionality reduction transformation on the low-dimensional global feature vector, improving the expressive power and computational efficiency of the features. ReLU activation then splits the feature vector into spatial attention branches, channel attention branches, filter attention branches, and kernel attention branches, providing structural support for multi-dimensional attention weight learning. Subsequently, the four-branch FC submodule adjusts the dimensions of the four attention branches respectively, generating spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights, achieving precise control over the four key dimensions of the convolutional kernel. Next, by weighting and integrating the original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules dimension, an aggregated convolutional kernel adapted to different target shapes, sizes, and scene backgrounds can be generated, solving the problems of fixed parameters and insufficient adaptability of traditional convolutional kernels. Finally, since the aggregated convolutional kernel can adaptively focus on the key features of targets at different scales, by performing a convolution operation between the aggregated convolutional kernel and the input features of the PPE-wearing image, an initial PPE-wearing feature map that fuses global and local information can be output.

[0011] Furthermore, the MAC-YOLO-based unsafe action detection sub-model binds the PPE state of personnel to the training set images to obtain the original associated feature data, including: Obtain raw feature target data based on training set images; The original feature target data is filtered based on a preset first confidence threshold to obtain the first feature target data; The training set images are cropped based on the first feature target data to obtain the original human area images; The MAC-YOLO unsafe human action detection sub-model includes the RFAConv convolution module, the EMA multi-scale attention module, the GSbottleneck module, the GSConv module, the VoV-GSCSP module, and the MAC-YOLO detection head module; Based on the RFAConv convolution module, feature extraction is performed on the original personnel region image to obtain a preliminary feature map of personnel actions; The preliminary feature map of human actions is decomposed into multiple scales based on the EMA multi-scale attention module to extract human action features at several scales. Based on the EMA multi-scale attention module, cross-dimensional dependencies are established on the human action features at several scales to enhance the preliminary human action feature map through cross-dimensional dependencies and obtain the enhanced human action feature map. Based on the GSConv module, GSbottleneck module and VoV-GSCSP module, feature purification is performed on the enhanced feature map of human actions to obtain the purified feature map of human actions. Based on the MAC-YOLO detection head module, the preset LMPDIoU loss function, and the purified feature map of personnel actions, the category of unsafe personnel actions, the coordinates of the bounding box of the action area, and the confidence level of personnel actions are determined. The categories of unsafe actions performed by personnel are bound to the original feature target data to obtain the PPE association status and association confidence of personnel actions; The original association feature data consists of the PPE association status of the personnel action, association confidence, coordinates of the action area bounding box, category of unsafe personnel action, and personnel action confidence.

[0012] In the above scheme, by obtaining raw feature target data from the training set images, the MAC-YOLO unsafe action detection sub-model is provided with key target information containing both personnel and PPE equipment, laying the foundation for binding unsafe actions with PPE wearing status. Next, by pre-setting a first confidence threshold and feature target confidence, the raw feature target data is filtered to remove invalid targets and obtain more accurate first feature target data, ensuring accurate location of the personnel's location in subsequent steps. Then, the training set images are cropped based on the first feature target data to obtain the original personnel region image, focusing on the personnel's location, reducing computational load on irrelevant background areas, and improving model computational efficiency. Subsequently, the RFAConv convolutional module extracts features from the original personnel region image, capturing key features of personnel actions and generating a preliminary personnel action feature map. Then, the EMA multi-scale attention module decomposes this preliminary feature map into multiple scales, extracting personnel action features at several scales. Without channel dimensionality reduction, the EMA multi-scale attention module establishes cross-dimensional dependencies for these features, strengthening the feature representation of unsafe actions and enhancing the preliminary feature map to obtain an enhanced personnel action feature map. Next, a slim-neck design paradigm consisting of the GSConv, GSbottleneck, and VoV-GSCSP modules is used to refine the enhanced feature map, reducing network computational complexity and improving feature utilization efficiency, generating a refined personnel action feature map. Finally, using the MAC-YOLO detection head module and a pre-defined LMPDIoU loss function, the unsafe action category, action region bounding box coordinates, and personnel action confidence are determined based on the refined feature map, improving the accuracy of action localization and classification. Finally, by binding the category of unsafe actions of personnel with the PPE wearing status in the original feature target data, the PPE association status and association confidence of personnel actions are obtained, and the structured original association feature data are formed by combining the coordinates of the action area bounding box, the category of unsafe actions of personnel, and the confidence of personnel actions.

[0013] Furthermore, the step of supplementing the training set images with small targets based on the ML-YOLO small target detection sub-model and the original associated feature data to obtain target enhancement detection results includes: Based on a preset first confidence threshold, a preset second confidence threshold, original associated feature data, and training set images, the suspected PPE wearing target area is obtained; Based on the suspected PPE wearing target area and training set images, obtain human motion reconstruction feature maps; Construct an ML-YOLO small target detection model; the ML-YOLO small target detection model includes an ML-YOLO backbone network module, a preset first detection head module, a preset second detection head module, and an MCSTA module; Residual connections are performed on the feature maps of human motion reconstruction based on the ML-YOLO backbone network module to obtain shallow feature data; Deep semantic feature extraction is performed on the feature map of human action reconstruction based on the ML-YOLO backbone network module to obtain deep feature data; Cross-layer fusion of shallow and deep feature data is performed to obtain multi-level fused feature maps; Based on the preset first detection head module and the preset second detection head module, target detection is performed on the multi-level fused feature map to obtain the fused target detection result; Attention enhancement is applied to the fused target detection results using the MCSTA module to obtain initial enhanced target detection results. Based on the preset third confidence threshold and the initial target enhancement detection results, the target enhancement detection results are obtained.

[0014] In the above scheme, by combining a preset first confidence threshold and a preset second confidence threshold, the original associated feature data and training set images are filtered to pinpoint suspected PPE-wearing target regions with confidence levels between the preset first and second confidence thresholds, thus supplementing the detection of small targets and locking in key areas. Next, feature reconstruction is performed on the training set images based on the suspected PPE-wearing target regions to generate a human action reconstruction feature map. This retains relevant feature information of small targets, providing a high-quality feature foundation for subsequent small target feature extraction and detection. Then, the ML-YOLO backbone network module performs residual connection processing on the human action reconstruction feature map to compensate for the loss of small target features due to upsampling and downsampling during feature transfer, obtaining shallow feature data containing local details of small targets. Subsequently, the ML-YOLO backbone network module performs deep semantic feature extraction on the human action reconstruction feature map to capture high-order semantic information of small targets, obtaining deep feature data and providing semantic support for accurate small target recognition. Next, by fusing shallow and deep feature data across layers, a multi-level fused feature map is generated that combines detailed and semantic information of small targets, solving the problems of small target scale and insufficient feature representation. Then, target detection is performed on the multi-level fused feature map using a preset first detection head module and a preset second detection head module, respectively, enabling comprehensive coverage detection of small targets of different sizes and obtaining fused target detection results. Finally, the MCSTA module is used to enhance the attention of the fused target detection results, which can strengthen the distinction between small target features and complex backgrounds, alleviate the problem of imbalance in the distribution of foreground and background information, and ultimately obtain enhanced target detection results for small targets.

[0015] Further, based on the validation set images, test set images, the initial YOLO model, and the trained YOLO model, the YOLO model and its performance improvement value are obtained. If the performance improvement value meets a preset second condition, the PPE-wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result, including: The performance of the YOLO trained model is evaluated based on the validation set images until a preset first condition is met, at which point the YOLO model is obtained. The YOLO initial model and the YOLO model are tested based on the test set images to obtain the model performance improvement value. If the model performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

[0016] In the above scheme, the performance of the YOLO training model is evaluated using validation set images to adjust the parameters of the YOLO training model until a preset first condition is met, at which point training stops and the YOLO model is obtained. Next, the initial YOLO model and the current YOLO model are tested using test set images, and the performance improvement of the current YOLO model compared to the initial YOLO model is calculated. If the performance improvement meets a preset second condition, the YOLO model is considered to have met the standard, and the PPE-wearing images to be detected are imported into the YOLO model, ultimately achieving accurate detection of unsafe actions in high-risk workplaces.

[0017] This invention provides an unsafe action detection system, comprising a data acquisition and partitioning unit, an initial model construction unit, a multi-dimensional feature extraction and recognition unit, a personnel PPE status binding unit, a small target supplementary detection unit, a model update unit, and a detection execution unit, specifically: The data acquisition and division unit is used to acquire the original PPE wearing images and divide the original PPE wearing images into training set images, validation set images and test set images according to a preset ratio. The initial model building unit is used to build the YOLO initial model, which includes the YOLOv7 multi-object detection sub-model, the MAC-YOLO unsafe human action detection sub-model, and the ML-YOLO small object detection sub-model. The multi-dimensional feature extraction and recognition unit is used to perform multi-dimensional feature optimization and target recognition on the training set images based on the YOLOv7 multi-target detection sub-model to obtain the original feature target data. The personnel PPE state binding unit is used to bind personnel PPE state to training set images based on the MAC-YOLO personnel unsafe action detection sub-model to obtain original associated feature data. The small target supplementation detection unit is used to supplement small targets in the training set images based on the ML-YOLO small target detection sub-model and the original associated feature data, and obtain the target enhancement detection result. The model update unit is used to fuse the original correlation feature data with the target enhancement detection result to obtain fused correlation feature data, and update the YOLO initial model based on the fused correlation feature data to obtain the YOLO training model. The detection execution unit is used to obtain the YOLO model and the model performance improvement value based on the validation set images, test set images, the initial YOLO model and the trained YOLO model. If the model performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

[0018] This invention provides an unsafe action detection system. In practical applications, it only requires a multi-dimensional feature extraction and recognition unit. Using the YOLOv7 multi-object detection sub-model, it optimizes multi-dimensional features and identifies targets in the training set images, accurately extracting key target information such as personnel and PPE (Personnel Protection Equipment) from the images, generating raw feature target data, and providing reliable basic target input for subsequent action detection and small target recognition. Next, a personnel PPE state binding unit is used. Through the MAC-YOLO personnel unsafe action detection sub-model, it binds the personnel PPE state in the training set images, associating PPE wearing status with unsafe actions, generating raw associated feature data, and achieving linked detection of unsafe actions and PPE state. Then, a small target supplementation detection unit is used. Through the ML-YOLO small target detection sub-model, it supplements the raw associated feature data with small targets, repairing lost small target features, identifying missed small-sized PPE and long-distance unsafe actions, avoiding the problem of missed detection of unsafe actions due to lost small target features in complex environments and with large differences in target scale. This results in enhanced target detection results, improving unsafe action detection across all scenarios. Subsequently, a model update unit is used to obtain fused correlation feature data by integrating the original correlation feature data and the target augmentation detection results. This fused correlation feature data is then used to update the initial YOLO model, integrating the detection capabilities of each sub-model to generate a YOLO training model with full-scene detection capabilities. Finally, a detection execution unit is used to optimize the performance of the YOLO training model using validation set images and calculate the model performance improvement value using test set images. If the model performance improvement value meets a preset second condition, it can be determined that the YOLO model has achieved the required improvement in detection accuracy. At this point, the PPE-wearing image to be detected is imported, enabling real-time detection of unsafe actions by personnel in high-risk workplaces.

[0019] Furthermore, the multi-dimensional feature extraction and recognition unit is used to perform multi-dimensional feature optimization and target recognition on the training set images based on the YOLOv7 multi-object detection sub-model to obtain original feature target data, including: The YOLOv7 multi-target detection sub-model includes the ODConv module, the SimAM attention module, the neck PAN module, and the YOLOv7 detection head module; Based on the ODConv module, multidimensional attention adaptive feature extraction and convolution kernel optimization are performed on the training set images to obtain the initial PPE wearing feature map; The SimAM attention module is used to calculate the pixel importance score of the initial PPE wearing feature map, and attention weights are assigned to the pixels of the initial PPE wearing feature map based on the pixel importance score to obtain the enhanced PPE wearing feature map. The feature map of enhanced PPE wearing is extracted based on the neck PAN module to obtain shallow and deep features; The shallow and deep features are fused based on the neck PAN module to obtain a fused PPE wearing feature map; The YOLOv7 detection head module is used to confirm the fused PPE wearing feature map and obtain the feature target category, feature bounding box coordinates and feature target confidence. The original feature target data consists of the feature target category, feature bounding box coordinates, and feature target confidence score.

[0020] In the above scheme, the ODConv module performs multi-dimensional attention-adaptive feature extraction and convolution kernel optimization on the training set images, which can adaptively capture the low-level features of targets with different shapes and sizes to generate an initial PPE wearing feature map. Next, the SimAM attention module calculates the pixel importance score of the initial PPE wearing feature map and assigns attention weights to the pixels based on the pixel importance score. This combines spatial attention and channel attention without increasing computational cost, focusing on the target region and suppressing complex background interference to generate an enhanced PPE wearing feature map. Then, the neck PAN module performs layered extraction on the enhanced PPE wearing feature map, obtaining shallow and deep features respectively, laying the foundation for subsequent feature fusion. Subsequently, the neck PAN module fuses the shallow and deep features, solving the problem of large target scale differences and generating a fused PPE wearing feature map, improving the comprehensiveness of target recognition. Finally, the YOLOv7 detection head module confirms the target in the fused PPE wearing feature map, obtaining the feature target category, feature bounding box coordinates, and feature target confidence to reduce the recognition error of similar targets. Finally, the feature target category, feature bounding box coordinates, and feature target confidence are integrated into the original feature target data to form a structured output, providing reliable input for the subsequent action detection of the MAC-YOLO unsafe action detection sub-model and the small target supplementary detection of the ML-YOLO small target detection sub-model.

[0021] Furthermore, the step of performing multidimensional attention-adaptive feature extraction and convolution kernel optimization on the training set images based on the ODConv module to obtain the initial PPE wearing feature map includes: The ODConv module includes a GAP submodule, a total FC submodule, a four-branch activation submodule, and a four-branch FC submodule; Based on the GAP submodule, feature extraction is performed on the training set images to obtain the input features of the PPE wearing images; The GAP submodule performs a global averaging operation on the channels of the PPE wearing image input features to obtain a low-dimensional global feature vector. Based on the total FC submodule and the four-branch activation submodule, the low-dimensional global feature vector is subjected to dimensionality reduction transformation and ReLU activation to obtain spatial attention branch, channel attention branch, filter attention branch and kernel attention branch; Based on the four-branch FC submodule, the spatial attention branch, channel attention branch, filter attention branch and kernel attention branch are adjusted in dimension to obtain the spatial attention weight, channel attention dimension weight, filter attention dimension weight and kernel attention dimension weight; The original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules are integrated to obtain the aggregated convolutional kernel; The aggregated convolution kernel and the PPE wearing image input features are convolved to obtain the initial PPE wearing feature map.

[0022] In the above scheme, the GAP submodule extracts features from the training set images, initially capturing basic visual features related to PPE wearing and generating PPE wearing image input features, laying the initial feature foundation for subsequent multi-dimensional attention optimization. Next, the GAP submodule performs a global averaging operation on the channels of the PPE wearing image input features, transforming the high-dimensional spatial feature map into a low-dimensional global feature vector. This reduces the number of model parameters and the risk of overfitting, while also summarizing global feature information, providing a global basis for the calculation of attention dimension weights. Then, the total FC submodule and the four-branch activation submodule perform dimensionality reduction transformation on the low-dimensional global feature vector, improving the expressive power and computational efficiency of the features. ReLU activation then splits the feature vector into spatial attention branches, channel attention branches, filter attention branches, and kernel attention branches, providing structural support for multi-dimensional attention weight learning. Subsequently, the four-branch FC submodule adjusts the dimensions of the four attention branches respectively, generating spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights, achieving precise control over the four key dimensions of the convolutional kernel. Next, by weighting and integrating the original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules dimension, an aggregated convolutional kernel adapted to different target shapes, sizes, and scene backgrounds can be generated, solving the problems of fixed parameters and insufficient adaptability of traditional convolutional kernels. Finally, since the aggregated convolutional kernel can adaptively focus on the key features of targets at different scales, by performing a convolution operation between the aggregated convolutional kernel and the input features of the PPE-wearing image, an initial PPE-wearing feature map that fuses global and local information can be output.

[0023] Furthermore, the personnel PPE state binding unit is used to bind personnel PPE states to training set images based on the MAC-YOLO unsafe action detection sub-model to obtain original associated feature data, including: Obtain raw feature target data based on training set images; The original feature target data is filtered based on a preset first confidence threshold to obtain the first feature target data; The training set images are cropped based on the first feature target data to obtain the original human area images; The MAC-YOLO unsafe human action detection sub-model includes the RFAConv convolution module, the EMA multi-scale attention module, the GSbottleneck module, the GSConv module, the VoV-GSCSP module, and the MAC-YOLO detection head module; Based on the RFAConv convolution module, feature extraction is performed on the original personnel region image to obtain a preliminary feature map of personnel actions; The preliminary feature map of human actions is decomposed into multiple scales based on the EMA multi-scale attention module to extract human action features at several scales. Based on the EMA multi-scale attention module, cross-dimensional dependencies are established on the human action features at several scales to enhance the preliminary human action feature map through cross-dimensional dependencies and obtain the enhanced human action feature map. Based on the GSConv module, GSbottleneck module and VoV-GSCSP module, feature purification is performed on the enhanced feature map of human actions to obtain the purified feature map of human actions. Based on the MAC-YOLO detection head module, the preset LMPDIoU loss function, and the purified feature map of personnel actions, the category of unsafe personnel actions, the coordinates of the bounding box of the action area, and the confidence level of personnel actions are determined. The categories of unsafe actions performed by personnel are bound to the original feature target data to obtain the PPE association status and association confidence of personnel actions; The original association feature data consists of the PPE association status of the personnel action, association confidence, coordinates of the action area bounding box, category of unsafe personnel action, and personnel action confidence.

[0024] In the above scheme, by obtaining raw feature target data from the training set images, the MAC-YOLO unsafe action detection sub-model is provided with key target information containing both personnel and PPE equipment, laying the foundation for binding unsafe actions with PPE wearing status. Next, by filtering the raw feature target data using a preset first confidence threshold, invalid targets are filtered out, resulting in more accurate first feature target data to ensure accurate location of the personnel's location. Then, the training set images are cropped based on the first feature target data to obtain the original personnel region image, focusing on the personnel's location, reducing computational load on irrelevant background areas, and improving model computational efficiency. Subsequently, the RFAConv convolutional module extracts features from the original personnel region image, capturing key features of personnel actions and generating a preliminary personnel action feature map. Then, the EMA multi-scale attention module decomposes this preliminary feature map into multiple scales, extracting personnel action features at several scales. Without channel dimensionality reduction, the EMA multi-scale attention module establishes cross-dimensional dependencies for these features, strengthening the feature representation of unsafe actions and enhancing the preliminary feature map to obtain an enhanced personnel action feature map. Next, a slim-neck design paradigm consisting of the GSConv, GSbottleneck, and VoV-GSCSP modules is used to refine the enhanced feature map, reducing network computational complexity and improving feature utilization efficiency, generating a refined personnel action feature map. Finally, using the MAC-YOLO detection head module and a pre-defined LMPDIoU loss function, the unsafe action category, action region bounding box coordinates, and personnel action confidence are determined based on the refined feature map, improving the accuracy of action localization and classification. Finally, by binding the category of unsafe actions of personnel with the PPE wearing status in the original feature target data, the PPE association status and association confidence of personnel actions are obtained, and the structured original association feature data are formed by combining the coordinates of the action area bounding box, the category of unsafe actions of personnel, and the confidence of personnel actions. Attached Figure Description

[0025] Figure 1 A flowchart of an unsafe action detection method provided in an embodiment of the present invention; Figure 2 This is an architecture diagram of a YOLOv7 multi-object detection sub-model provided in an embodiment of the present invention; Figure 3 An architecture diagram of an ODConv module provided in an embodiment of the present invention; Figure 4This is an architecture diagram of a MAC-YOLO sub-model for detecting unsafe human actions, provided in an embodiment of the present invention. Figure 5 This is an architecture diagram of an ML-YOLO small target detection sub-model provided in an embodiment of the present invention; Figure 6 This is an architecture diagram of an unsafe action detection system provided in an embodiment of the present invention; in: X, PPE wearing image input features; y, initial PPE wearing feature map. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] This embodiment provides a method for detecting unsafe actions; please refer to its flowchart. Figure 1 ,include: Step S1: Obtain the original PPE wearing images and divide the original PPE wearing images into training set images, validation set images and test set images according to a preset ratio; Step S2: Construct the initial YOLO model, which includes the YOLOv7 multi-object detection sub-model, the MAC-YOLO unsafe human action detection sub-model, and the ML-YOLO small object detection sub-model; Step S3: Based on the YOLOv7 multi-object detection sub-model, perform multi-dimensional feature optimization and object recognition on the training set images to obtain the original feature object data; Step S4: Bind the PPE state of personnel to the training set images based on the MAC-YOLO unsafe action detection sub-model to obtain the original associated feature data; Step S5: Based on the ML-YOLO small target detection sub-model and the original associated feature data, small targets are added to the training set images to obtain the target enhancement detection results; Step S6: Fuse the original association feature data with the target enhancement detection results to obtain fused association feature data, and update the YOLO initial model based on the fused association feature data to obtain the YOLO training model; Step S7: Based on the validation set images, test set images, the initial YOLO model, and the trained YOLO model, obtain the YOLO model and the model performance improvement value. If the model performance improvement value meets the preset second condition, import the PPE wearing image to be detected into the YOLO model to obtain the unsafe action detection result.

[0028] In this embodiment, the acquisition scope of the original PPE wearing images covers typical high-risk scenarios in industries such as communications, power, and manufacturing, with at least three representative scenarios selected for each industry, such as communication base station construction or maintenance, power pole inspection or substation operation, and automotive parts assembly or chemical workshop operation. When acquiring image and video data, facial features are blurred, retaining only the PPE wearing area, and sensitive information such as name tags and faces are removed. Data storage uses AES encryption and is accessible only to authorized R&D personnel. The acquisition process complies with relevant national laws, regulations, and regulatory requirements regarding cybersecurity and personal information protection. The original PPE wearing images are updated regularly to adapt to changes in scenario processes. The initial data for the original PPE wearing images includes image data, which is a single JPG / PNG image with a resolution ≥1920×1080, containing images from different angles and distances (1-10 meters), with no fewer than 5000 images collected in each scene. The initial data also includes video data, which is in MP4 format, with a frame rate of 25fps, and a single segment length of 30 seconds to 5 minutes, containing dynamic scenes such as personnel movement or work actions (e.g., bending over, climbing), with no fewer than 100 segments collected in each scene. The collected image and video data are labeled using LabelImg or LabelMe, with a labeling accuracy ≥98%, and a labeling deviation of ≤2 pixels for small PPE targets (e.g., insulating gloves). This results in image and video data containing PPE category, PPE wearing status, and PPE target coordinates. The PPE categories include safety helmets, insulating gloves, safety belts, protective clothing, and goggles; the PPE wearing status includes correctly worn, not worn, and improperly worn; the PPE target coordinates are obtained using bounding box annotation. A data annotation review mechanism was established to ensure data accuracy through cross-validation of annotation results. Next, image and video data containing PPE categories, PPE wearing status, and PPE target coordinates were cleaned, removing blurry, overexposed, underexposed, or invalid data without valid PPE targets, retaining a valid sample ratio of ≥85%. Then, the cleaned image and video data underwent illumination enhancement, specifically by adjusting brightness and performing gamma correction on cleaned image and video data with low or strong light, generating 10 derivative samples with different light intensities. Next, occlusion simulation was performed, using image synthesis technology to add scene-specific occlusions, such as cables, equipment parts, and dust effects, to the illuminated image and video data, generating samples with occlusion rates of 10%-50%.Next, small target enhancement is performed. Small-sized PPE samples such as insulating gloves and safety belts in the image and video data after occlusion simulation are cropped and scaled to increase the proportion of small targets in the cleaned image and video data (≥30%). This targeted enhancement, combined with scene characteristics, solves the problem of interference in actual working conditions and obtains the original PPE-wearing images. The original PPE-wearing images are divided into training set images, validation set images, and test set images according to a preset ratio of 7:2:1. The YOLOv7 multi-target detection sub-model performs multi-dimensional feature optimization and target recognition on the training set images, accurately extracting key target information such as personnel and PPE (Personal Protective Equipment) from the images, generating original feature target data, providing reliable basic target input for subsequent action detection and small target recognition. Then, the MAC-YOLO unsafe action detection sub-model binds the personnel PPE status in the training set images, associating the PPE wearing status with unsafe actions, generating original associated feature data, and achieving linked detection of unsafe actions and PPE status. Then, the ML-YOLO small target detection sub-model supplements the original associated feature data with small targets, which can repair lost small target features, identify missed small-sized PPEs and unsafe actions at long distances, and avoid the problem of missed detection of unsafe actions caused by the loss of small target features in complex environmental interference and large differences in target scale. This results in target enhancement detection results, which improves the detection of unsafe actions in all scenarios. Subsequently, the original associated feature data and the target enhancement detection results are fused to obtain fused associated feature data, and the initial YOLO model is updated based on the fused associated feature data. The detection capabilities of each sub-model are integrated to generate a YOLO training model with full-scene detection capabilities. Finally, the performance of the YOLO training model is optimized using validation set images, and the model performance improvement value is calculated by combining the test set images. If the model performance improvement value meets the preset second condition, it can be determined that the improvement in detection accuracy of the YOLO model has met the standard. At this time, the PPE wearing image to be detected is imported to realize the real-time detection of unsafe actions of personnel in high-risk workplaces. The YOLO model obtained through this technical solution improves the accuracy and real-time performance of unsafe behavior detection, enabling timely identification and correction of unsafe actions by personnel, thereby effectively reducing direct economic losses caused by accidents, such as equipment damage and downtime losses. The YOLO model in this technical solution can reduce production interruptions and downtime caused by unsafe behaviors, improving production efficiency and increasing the company's production capacity and output. Furthermore, the YOLO model in this technical solution can more accurately identify personnel's operational status, thereby optimizing resource allocation, improving resource utilization, and reducing production costs.

[0029] This embodiment addresses the core requirements for PPE (Personal Protective Equipment) inspection in high-risk workplaces in industries such as telecommunications, power, and manufacturing. It collects original PPE-wearing images to address technical issues in different scenarios, as shown in Table 1. These scenarios include: image blurring due to strong outdoor light / rainy weather in telecommunications base station construction scenarios; scaffolding and cables obscuring workers' heads / torso levels; helmet colors blending with the environment; detecting small targets at heights in power line inspection scenarios; interference from metal reflections on equipment; low-light environments during nighttime inspections; high-density operations in manufacturing assembly lines; image noise caused by workshop dust; and the need for simultaneous inspection of multiple PPE items such as protective suits and goggles. A scenario-based input data system is built based on the technical problems in different scenarios, enabling the optimized YOLO model to directly interface with real-world application data for practical implementation.

[0030] Table 1. Correspondence between technical problems, required data types, and data functions in different application scenarios. Further, please see Figure 2 The YOLOv7 multi-object detection sub-model performs multi-dimensional feature optimization and object recognition on the training set images to obtain original feature object data, including: The YOLOv7 multi-target detection sub-model includes the ODConv module, the SimAM attention module, the neck PAN module, and the YOLOv7 detection head module; Based on the ODConv module, multidimensional attention adaptive feature extraction and convolution kernel optimization are performed on the training set images to obtain the initial PPE wearing feature map; The SimAM attention module is used to calculate the pixel importance score of the initial PPE wearing feature map, and attention weights are assigned to the pixels of the initial PPE wearing feature map based on the pixel importance score to obtain the enhanced PPE wearing feature map. The feature map of enhanced PPE wearing is extracted based on the neck PAN module to obtain shallow and deep features; The shallow and deep features are fused based on the neck PAN module to obtain a fused PPE wearing feature map; The YOLOv7 detection head module is used to confirm the fused PPE wearing feature map and obtain the feature target category, feature bounding box coordinates and feature target confidence. The original feature target data consists of the feature target category, feature bounding box coordinates, and feature target confidence score.

[0031] In this embodiment, before inputting the YOLOv7 multi-object detection sub-model, the training set images are adaptively scaled and normalized to reduce black borders and eliminate the impact of pixel value differences on the model. Then, the CSPDarknet module in the YOLOv7 multi-object detection sub-model is replaced with the ODConv module (Omni-Dimensional Dynamic Convolution). The ODConv module performs multi-dimensional attention adaptive feature extraction and convolution kernel optimization on the training set images, which can adaptively capture the low-level features of targets with different shapes and sizes, generating an initial PPE wearing feature map. Next, the SimAM attention module calculates the pixel importance score of the initial PPE wearing feature map, and attention weights are assigned to the pixels of the initial PPE wearing feature map based on the pixel importance score. This combines spatial attention and channel attention without increasing the amount of additional computation, focusing on the target region, suppressing complex background interference, and generating an enhanced PPE wearing feature map. By adding the SimAM attention module to the YOLOv7 multi-object detection sub-model, the system can focus on information more critical to the current task from a large amount of input information, effectively solving the problem of information overload and improving the efficiency and accuracy of task processing, even with limited computing power. Then, the neck PAN module extracts layered features from the enhanced PPE wearing feature map, obtaining shallow and deep features separately, laying the foundation for subsequent feature fusion. Subsequently, the neck PAN module fuses the shallow and deep features, addressing the issue of large differences in target scale and generating a fused PPE wearing feature map, improving the comprehensiveness of target recognition. Next, the YOLOv7 detection head module uses its preset E-IoU loss function to confirm the target in the fused PPE wearing feature map, obtaining the feature target category, feature bounding box coordinates, and feature target confidence score to reduce the recognition error of similar targets. The feature target categories include personnel, safety helmets, protective clothing, and equipment. Finally, the feature target category, feature bounding box coordinates, and feature target confidence are integrated into the original feature target data. Specifically, this original feature target data is actually a list of multi-target recognition results. Each item in the list contains the feature target ID, feature target category (e.g., 0 = personnel, 1 = safety helmet, 2 = protective clothing, 3 = equipment), feature bounding box coordinates, and feature target confidence, thus forming a structured output. This provides reliable input for the subsequent action detection of the MAC-YOLO unsafe action detection sub-model and the small target supplementary detection of the ML-YOLO small target detection sub-model.The feature bounding box coordinates are (x1, y1, x2, y2), where x1 represents the horizontal pixel coordinate of the top-left corner vertex of the feature bounding box, y1 represents the vertical pixel coordinate of the top-left corner vertex of the feature bounding box, x2 represents the horizontal pixel coordinate of the bottom-right corner vertex of the feature bounding box, and y2 represents the vertical pixel coordinate of the bottom-right corner vertex of the feature bounding box.

[0032] Further, please see Figure 3 The step of performing multidimensional attention adaptive feature extraction and convolution kernel optimization on the training set images based on the ODConv module to obtain the initial PPE wearing feature map y includes: The ODConv module includes a GAP submodule, a total FC submodule, a four-branch activation submodule, and a four-branch FC submodule; Based on the GAP submodule, feature extraction is performed on the training set images to obtain the input feature X of the PPE wearing image; The GAP submodule performs a global averaging operation on the channels of the PPE-wearing image input feature X to obtain a low-dimensional global feature vector. Based on the total FC submodule and the four-branch activation submodule, the low-dimensional global feature vector is subjected to dimensionality reduction transformation and ReLU activation to obtain spatial attention branch, channel attention branch, filter attention branch and kernel attention branch; Based on the four-branch FC submodule, the spatial attention branch, channel attention branch, filter attention branch and kernel attention branch are adjusted in dimension to obtain the spatial attention weight, channel attention dimension weight, filter attention dimension weight and kernel attention dimension weight; The original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules are integrated to obtain the aggregated convolutional kernel; The convolutional kernel and the PPE wearing image are input to the feature X and convolved to obtain the initial PPE wearing feature map y.

[0033] In this embodiment, the GAP submodule extracts features from the training set images, initially capturing basic visual features related to PPE wearing and generating PPE-wearing image input features X, laying the initial feature foundation for subsequent multidimensional attention optimization. Next, the GAP submodule performs a global averaging operation on the channels of the PPE-wearing image input features X, transforming the high-dimensional spatial feature map into a low-dimensional global feature vector. The length of this low-dimensional global feature vector is equal to the number of input channels, reducing the number of model parameters and mitigating the risk of overfitting. Simultaneously, it summarizes global feature information, providing a global basis for calculating attention dimension weights. Then, the total FC submodule and the four-branch activation submodule perform dimensionality reduction on the low-dimensional global feature vector, improving feature expressiveness and computational efficiency. ReLU activation then splits the feature vector into spatial attention branches, channel attention branches, filter attention branches, and kernel attention branches, providing structural support for multidimensional attention weight learning. Subsequently, the four-branch FC submodule adjusts the dimensions of each of the four attention branches, generating spatial attention weights. Channel attention dimension weights Filter attention dimension weights And kernel attention dimension weights For i=1…n, specifically: the spatial attention branch is dimensionally adjusted through a four-branch FC submodule and activated using a Sigmoid function to generate spatial attention weights. The weights of the k×k spatial dimension of the convolution kernel are adjusted accordingly; the channel attention branches are dimensionally adjusted through a four-branch FC submodule and activated using a Sigmoid function to generate the channel attention dimension weights. The weights of the input channel dimension of the convolution kernel (cin×1) are adjusted accordingly; the dimension of the filter attention branch is adjusted through a four-branch FC submodule and activated by Sigmoid to generate the filter attention dimension weights. The weights of the convolution kernel's cout×1 output channel dimension are adjusted accordingly; the kernel attention branch is dimension-adjusted through a four-branch FC submodule and activated using Softmax to generate the kernel attention dimension weights. This allows for precise control of the four key dimensions of the convolution kernel by adjusting the weights of the n×1 number dimension of the kernel. Next, the original convolution kernels of the four-branch FC submodules are... Spatial attention weights Channel attention dimension weights Filter attention dimension weights And kernel attention dimension weights Perform a dimension-wise weighted integration, specifically: adjust the spatial attention weights. The k×k spatial location parameters of the original convolution kernel are multiplied point-by-point to enhance the feature response of the spatial region where large PPE devices or small PPE helmets are located; the channel attention dimension weights are also increased. Multiply by the input channel parameters of the convolution kernel to highlight the input channel features for recognizing the color channel of a PPE helmet or the contour channel of a PPE device; weight the filter attention dimension. Multiplying the output channel (filter) parameters of the convolution kernel optimizes the output dimension weight allocation of the feature map; it also optimizes the kernel attention dimension weights. Multiplying with the entire convolutional kernel matrix allows for dynamic filtering of the contributions of multiple convolutional kernels, suppressing redundant kernels and enhancing effective kernels. This ultimately generates aggregated convolutional kernels that adapt to different target shapes, sizes, and scene backgrounds, solving the problems of fixed parameters and insufficient adaptability in traditional convolutional kernels. Finally, because the aggregated convolutional kernel can adaptively focus on key features of targets at different scales, such as the overall contour features of large PPE devices or the local detail features of small PPE helmets, performing a convolution operation between the aggregated convolutional kernel and the input features X of the PPE wearing image can output an initial PPE wearing feature map y that fuses global and local information.

[0034] Further, please see Figure 4 The method for binding personnel PPE status to training set images based on the MAC-YOLO unsafe action detection sub-model to obtain raw associated feature data includes: Obtain raw feature target data based on training set images; The original feature target data is filtered based on a preset first confidence threshold to obtain the first feature target data; The training set images are cropped based on the first feature target data to obtain the original human area images; The MAC-YOLO unsafe human action detection sub-model includes the RFAConv convolution module, the EMA multi-scale attention module, the GSbottleneck module, the GSConv module, the VoV-GSCSP module, and the MAC-YOLO detection head module; Based on the RFAConv convolution module, feature extraction is performed on the original personnel region image to obtain a preliminary feature map of personnel actions; The preliminary feature map of human actions is decomposed into multiple scales based on the EMA multi-scale attention module to extract human action features at several scales. Based on the EMA multi-scale attention module, cross-dimensional dependencies are established on the human action features at several scales to enhance the preliminary human action feature map through cross-dimensional dependencies and obtain the enhanced human action feature map. Based on the GSConv module, GSbottleneck module and VoV-GSCSP module, feature purification is performed on the enhanced feature map of human actions to obtain the purified feature map of human actions. Based on the MAC-YOLO detection head module, the preset LMPDIoU loss function, and the purified feature map of personnel actions, the category of unsafe personnel actions, the coordinates of the bounding box of the action area, and the confidence level of personnel actions are determined. The categories of unsafe actions performed by personnel are bound to the original feature target data to obtain the PPE association status and association confidence of personnel actions; The original association feature data consists of the PPE association status of the personnel action, association confidence, coordinates of the action area bounding box, category of unsafe personnel action, and personnel action confidence.

[0035] In this embodiment, the MAC-YOLO unsafe action detection sub-model is used to accurately identify whether actions in training set images are safe and associate them with PPE wearing status. By obtaining raw feature target data from the training set images, the MAC-YOLO unsafe action detection sub-model provides key target information containing personnel and PPE equipment, laying the raw data foundation for binding unsafe actions with PPE wearing status. Next, the raw feature target data is filtered by a preset first confidence threshold, which filters out invalid targets with a feature target confidence score <0.7, obtaining more accurate first feature target data to ensure accurate location of the personnel's area. The preset first confidence threshold is 0.7. Then, the training set images are cropped based on the first feature target data to obtain the original personnel area image, which focuses on the personnel's area, reduces the computational load on irrelevant background areas, and improves model computational efficiency. Before inputting the original personnel area image into the MAC-YOLO unsafe action detection sub-model, it undergoes illumination compensation and noise filtering. Subsequently, the RFAConv convolutional module extracts features from the original personnel region image, capturing key regional features of personnel actions (such as the head and torso) to generate a preliminary personnel action feature map. This solves the problem of inaccurate feature capture caused by parameter sharing in traditional convolution. Next, the EMA multi-scale attention module decomposes the preliminary personnel action feature map into multiple scales, extracting personnel action features at several scales. Without channel dimensionality reduction, the EMA multi-scale attention module establishes cross-dimensional dependencies for these features, enabling the large local receptive fields of neurons in the MAC-YOLO unsafe action detection sub-model to efficiently obtain clearer multi-scale personnel action features. This helps reduce the influence of interference factors in the image, resulting in better pixel-level attention to the preliminary personnel action feature map. By strengthening the feature representation of unsafe actions through cross-dimensional dependencies, the preliminary personnel action feature map is enhanced, resulting in an enhanced personnel action feature map. Then, the neck of the MAC-YOLO unsafe human action detection sub-model is modified by using the slim-neck design paradigm. The learning ability of the MAC-YOLO unsafe human action detection sub-model is improved by stacking GSConv modules, the GSbottleneck module enhances the ability to process features, and the VoV-GSCSP module improves feature utilization efficiency and network performance. Thus, the GSConv module, GSbottleneck module, and VoV-GSCSP module can purify the enhanced human action feature map, reduce the computational complexity of the network, improve the feature utilization efficiency, and generate purified human action feature maps.Since the MAC-YOLO unsafe action detection sub-model relies on an IoU-based bounding box regression loss function to determine the target's location, it typically uses the coordinates of the top-left and bottom-right corners to define a unique rectangular box. However, this type of loss function becomes ineffective when the aspect ratios of the predicted and ground truth boxes are the same (but different). Therefore, in this embodiment, a preset LMPDIoU loss function (minimum point distance bounding box regression loss function) is introduced. By using the MAC-YOLO detection head module and the preset LMPDIoU loss function to perform action bounding box regression, the distance between the top-left and bottom-right points of the predicted and ground truth boxes during action localization can be directly minimized. This solves the problem that the original loss function cannot effectively optimize when the aspect ratios of the predicted and ground truth boxes are the same, thus accelerating the model's convergence speed. As a result, it can determine the category of unsafe actions (e.g., setting 0 = not wearing a safety helmet, 1 = not wearing protective clothing, 2 = smoking, 3 = working at height without protection), the bounding box coordinates of the action region, and the confidence level of the action based on the purified feature map of the personnel action, thereby improving the accuracy of action localization and action classification. Finally, by judging the correlation between the category of unsafe actions and the PPE wearing status, the category of unsafe actions is bound to the PPE wearing status in the original feature target data. For example, climbing actions need to be associated with the seat belt wearing status. The PPE association status and association confidence of the personnel actions are obtained, and the structured original association feature data are formed by combining the bounding box coordinates of the action area, the category of unsafe actions, and the confidence of the personnel actions.

[0036] Further, please see Figure 5 The step of supplementing training set images with small targets based on the ML-YOLO small target detection sub-model and original associated feature data to obtain target enhancement detection results includes: Based on a preset first confidence threshold, a preset second confidence threshold, original associated feature data, and training set images, the suspected PPE wearing target area is obtained; Based on the suspected PPE wearing target area and training set images, obtain human motion reconstruction feature maps; Construct an ML-YOLO small target detection model; the ML-YOLO small target detection model includes an ML-YOLO backbone network module, a preset first detection head module, a preset second detection head module, and an MCSTA module; Residual connections are performed on the feature maps of human motion reconstruction based on the ML-YOLO backbone network module to obtain shallow feature data; Deep semantic feature extraction is performed on the feature map of human action reconstruction based on the ML-YOLO backbone network module to obtain deep feature data; Cross-layer fusion of shallow and deep feature data is performed to obtain multi-level fused feature maps; Based on the preset first detection head module and the preset second detection head module, target detection is performed on the multi-level fused feature map to obtain the fused target detection result; Attention enhancement is applied to the fused target detection results using the MCSTA module to obtain initial enhanced target detection results. Based on the preset third confidence threshold and the initial target enhancement detection results, the target enhancement detection results are obtained.

[0037] In this embodiment, the ML-YOLO small target detection model is used to supplement the identification of small-sized targets (such as long-distance insulating gloves and small seat belt buckles) and long-distance unsafe actions missed in the training set images, thus improving the detection coverage of the entire scene. By combining a preset first confidence threshold and a preset second confidence threshold, the original associated feature data and training set images are filtered to locate suspected PPE-wearing target areas (such as insulating gloves of long-distance personnel) with association confidence between the preset first confidence threshold and the preset second confidence threshold, thus supplementing the detection of small targets and locking in the key area. The preset second confidence threshold is set to 0.5. Next, the training set images are locally magnified based on the suspected PPE-wearing target areas, and feature reconstruction is performed based on super-resolution to generate a personnel action reconstruction feature map, which can retain the relevant feature information of small targets and provide a high-quality feature foundation for subsequent small target feature extraction and detection. Then, the personnel action reconstruction feature map is processed by residual connection through the ML-YOLO backbone network module to compensate for the loss of small target features due to upsampling and downsampling during feature transfer, and to obtain shallow feature data containing local details of small targets. Subsequently, the ML-YOLO backbone network module is used to extract deep semantic features from the reconstructed feature map of human actions to capture high-order semantic information of small targets and obtain deep feature data, providing semantic support for accurate identification of small targets. Next, by fusing shallow and deep feature data across layers, a multi-level fused feature map containing both detailed and semantic information of small targets is generated, solving the problems of small target scale and insufficient feature representation. This embodiment introduces a very small target detection head as a preset second detection head module, and the original detection head of the model as a preset first detection head module. Then, the preset first and preset second detection head modules are used to perform target detection on the multi-level fused feature map, enabling comprehensive coverage detection of small targets of different sizes, ≤50×50 pixels, such as seatbelt buckles and goggles, thereby obtaining fused target detection results. Next, the MCSTA module enhances the attention of the fused target detection results. By utilizing the pre-stored block attention patterns and remote pixel interactions within the MCSTA module, the distinguishability between small target features and complex backgrounds is strengthened, alleviating the imbalance in foreground and background information distribution and obtaining initial target enhancement detection results for the small targets. Finally, the initial target enhancement detection results are filtered according to a preset third confidence threshold. Results with a confidence level greater than or equal to the preset third confidence threshold are retained, resulting in the target enhancement detection final. The preset third confidence threshold is set to 0.8. The MCSTA module is plug-and-play, capable of being quickly embedded into any natural image target detection convolutional neural network, making it suitable for detecting extremely small targets. Furthermore, the MCSTA module is suitable for scenarios with multiple small targets and is easily deployed on edge devices.

[0038] Further, based on the validation set images, test set images, the initial YOLO model, and the trained YOLO model, the YOLO model and its performance improvement value are obtained. If the performance improvement value meets a preset second condition, the PPE-wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result, including: The performance of the YOLO trained model is evaluated based on the validation set images until a preset first condition is met, at which point the YOLO model is obtained. The YOLO initial model and the YOLO model are tested based on the test set images to obtain the model performance improvement value. If the model performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

[0039] In this embodiment, the performance of the YOLO training model is evaluated using validation set images to adjust the parameters of the YOLO training model until a preset first condition is met, at which point training stops and the YOLO model is obtained. Next, cross-scenario tests are performed on the initial YOLO model and the YOLO model using test set images. For example, a power line inspection test set is used to verify the model's generalization ability in communication scenarios. The performance improvement of the YOLO model compared to the initial YOLO model is calculated. If the performance improvement meets a preset second condition, the YOLO model is confirmed to be up to standard. The PPE wearing image to be detected is then imported into the YOLO model. This allows for the determination of the safety risk level of the PPE wearing image and the generation of a final safety detection report for the entire scenario, ultimately achieving accurate detection of unsafe actions in high-risk workplaces. When the model performance is the accuracy rate in occluded scenarios, the preset second condition can be set to 15%, meaning that if the accuracy rate in occluded scenarios improves by ≥15%, the YOLO model is confirmed to be up to standard. The safety risk level can be set as: low risk (no unsafe behavior); medium risk (minor improper wearing); high risk (not wearing critical PPE or engaging in high-risk actions). The final safety inspection report for the entire scenario includes {corresponding scenario ID, inspection time, categories of all targets in the scenario (including regular and small targets), target location, target confidence level, details of unsafe actions by personnel, PPE wearing compliance rate, safety risk level, and preset warning suggestions}, achieving a complete closed loop of data from input to output. The preset warning suggestions are as follows: Personnel A is not wearing a safety helmet, high risk, immediate rectification recommended.

[0040] This embodiment provides an unsafe action detection system; please refer to [link / reference]. Figure 6 It includes a data acquisition and partitioning unit, an initial model building unit, a multi-dimensional feature extraction and recognition unit, a personnel PPE status binding unit, a small target supplementary detection unit, a model update unit, and a detection execution unit, specifically: The data acquisition and division unit is used to acquire the original PPE wearing images and divide the original PPE wearing images into training set images, validation set images and test set images according to a preset ratio. The initial model building unit is used to build the YOLO initial model, which includes the YOLOv7 multi-object detection sub-model, the MAC-YOLO unsafe human action detection sub-model, and the ML-YOLO small object detection sub-model. The multi-dimensional feature extraction and recognition unit is used to perform multi-dimensional feature optimization and target recognition on the training set images based on the YOLOv7 multi-target detection sub-model to obtain the original feature target data. The personnel PPE state binding unit is used to bind personnel PPE state to training set images based on the MAC-YOLO personnel unsafe action detection sub-model to obtain original associated feature data. The small target supplementation detection unit is used to supplement small targets in the training set images based on the ML-YOLO small target detection sub-model and the original associated feature data, and obtain the target enhancement detection result. The model update unit is used to fuse the original correlation feature data with the target enhancement detection result to obtain fused correlation feature data, and update the YOLO initial model based on the fused correlation feature data to obtain the YOLO training model. The detection execution unit is used to obtain the YOLO model and the model performance improvement value based on the validation set images, test set images, the initial YOLO model and the trained YOLO model. If the model performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

[0041] In this embodiment, data acquisition is divided into units. The acquisition range of the original PPE wearing images covers typical high-risk scenarios in industries such as communications, power, and manufacturing. At least three representative scenarios are selected in each industry, such as communication base station construction or maintenance, power pole inspection or substation operation, and automotive parts assembly or chemical workshop operation. When acquiring image and video data, facial features are blurred, retaining only the PPE wearing area, and sensitive information such as name tags and faces are removed. Data storage uses AES encryption and is accessible only to authorized R&D personnel. The acquisition process complies with relevant national laws, regulations, and regulatory requirements for cybersecurity and personal information protection. The original PPE wearing images are updated regularly to adapt to changes in scenario processes. The initial data for the original PPE wearing images includes image data, which is a single JPG / PNG image with a resolution ≥1920×1080, containing images from different angles and distances (1-10 meters), with no fewer than 5000 images collected in each scene. The initial data also includes video data, which is in MP4 format, with a frame rate of 25fps, and a single segment length of 30 seconds to 5 minutes, containing dynamic scenes such as personnel movement or work actions (e.g., bending over, climbing), with no fewer than 100 segments collected in each scene. The collected image and video data are labeled using LabelImg or LabelMe, with a labeling accuracy ≥98%, and a labeling deviation of ≤2 pixels for small PPE targets (e.g., insulating gloves). This results in image and video data containing PPE category, PPE wearing status, and PPE target coordinates. The PPE categories include safety helmets, insulating gloves, safety belts, protective clothing, and goggles; the PPE wearing status includes correctly worn, not worn, and improperly worn; the PPE target coordinates are obtained using bounding box annotation. A data annotation review mechanism was established to ensure data accuracy through cross-validation of annotation results. Next, image and video data containing PPE categories, PPE wearing status, and PPE target coordinates were cleaned, removing blurry, overexposed, underexposed, or invalid data without valid PPE targets, retaining a valid sample ratio of ≥85%. Then, the cleaned image and video data underwent illumination enhancement, specifically by adjusting brightness and performing gamma correction on cleaned image and video data with low or strong light, generating 10 derivative samples with different light intensities. Next, occlusion simulation was performed, using image synthesis technology to add scene-specific occlusions, such as cables, equipment parts, and dust effects, to the illuminated image and video data, generating samples with occlusion rates of 10%-50%.Then, small target enhancement is performed. Small-sized PPE samples such as insulating gloves and safety belts in the image and video data after occlusion simulation are cropped and scaled to increase the proportion of small targets in the cleaned image and video data (≥30%). This allows for targeted enhancement based on scene characteristics, addressing interference issues in actual working conditions and obtaining original PPE-wearing images. The original PPE-wearing images are divided into training set images, validation set images, and test set images according to a preset ratio of 7:2:1. A multi-dimensional feature extraction and recognition unit uses the YOLOv7 multi-target detection sub-model to perform multi-dimensional feature optimization and target recognition on the training set images. This accurately extracts key target information such as personnel and PPE (Personal Protective Equipment) from the images, generating original feature target data and providing reliable basic target input for subsequent action detection and small target recognition. Next, a PPE (Personnel Physical Equipment) state binding unit is employed. Using the MAC-YOLO unsafe action detection sub-model, the training set images are bound to the PPE state, associating PPE wearing status with unsafe actions to generate original associated feature data. This enables the linked detection of unsafe actions and PPE state. Then, a small target supplementation detection unit is used. Using the ML-YOLO small target detection sub-model, the original associated feature data is supplemented with small target features. This repairs lost small target features, identifies missed small-sized PPEs and unsafe actions at long distances, and avoids missed detections of unsafe actions due to lost small target features in complex environments and with large differences in target scale. This results in enhanced target detection results, improving unsafe action detection across all scenarios. Finally, a model update unit is used. By fusing the original associated feature data and the enhanced target detection results, fused associated feature data is obtained. The initial YOLO model is then updated based on this fused associated feature data, integrating the detection capabilities of each sub-model to generate a YOLO training model with full-scene detection capabilities. Finally, a detection execution unit is employed to optimize the performance of the YOLO training model using validation set images and calculate the model performance improvement value by combining it with test set images. If the model performance improvement value meets a preset second condition, it can be determined that the improvement in detection accuracy of the YOLO model has met the standard. At this point, the PPE-wearing image to be detected is imported to achieve real-time detection of unsafe actions by personnel in high-risk workplaces. The YOLO model obtained through this technical solution can improve the accuracy and real-time performance of unsafe behavior detection, promptly identify and correct unsafe behaviors, thereby effectively reducing direct economic losses caused by accidents, such as equipment damage and downtime losses. The YOLO model of this technical solution can reduce production interruptions and downtime caused by unsafe behaviors, thereby improving production efficiency and increasing the enterprise's production capacity and output.This technical solution utilizes the YOLO model to more accurately identify personnel's operational status, thereby optimizing resource allocation, improving resource utilization, and reducing production costs.

[0042] This embodiment addresses the core requirements for PPE (Personal Protective Equipment) inspection in high-risk workplaces in industries such as telecommunications, power, and manufacturing. It collects original PPE-wearing images to address technical issues in various scenarios, including: image blurring due to strong outdoor light / rainy weather in telecommunications base station construction; head / torso obstruction by scaffolding and cables; helmet color blending with ambient color; small target detection at heights in power line inspections; interference from metal reflections on equipment; low-light environments during nighttime inspections; high-density work in manufacturing assembly lines; image noise caused by workshop dust; and the need for simultaneous inspection of multiple PPEs such as protective suits and goggles. A scenario-based input data system is built based on the technical problems in different scenarios, enabling the optimized YOLO model to directly interface with real-world application data for practical implementation.

[0043] Furthermore, the multi-dimensional feature extraction and recognition unit is used to perform multi-dimensional feature optimization and target recognition on the training set images based on the YOLOv7 multi-object detection sub-model to obtain original feature target data, including: The YOLOv7 multi-target detection sub-model includes the ODConv module, the SimAM attention module, the neck PAN module, and the YOLOv7 detection head module; Based on the ODConv module, multidimensional attention adaptive feature extraction and convolution kernel optimization are performed on the training set images to obtain the initial PPE wearing feature map y; The SimAM attention module is used to calculate the pixel importance score of the initial PPE wearing feature map y, and attention weights are assigned to the pixels of the initial PPE wearing feature map y based on the pixel importance score to obtain the enhanced PPE wearing feature map. The feature map of enhanced PPE wearing is extracted based on the neck PAN module to obtain shallow and deep features; The shallow and deep features are fused based on the neck PAN module to obtain a fused PPE wearing feature map; The YOLOv7 detection head module is used to confirm the fused PPE wearing feature map and obtain the feature target category, feature bounding box coordinates and feature target confidence. The original feature target data consists of the feature target category, feature bounding box coordinates, and feature target confidence score.

[0044] In this embodiment, before inputting the YOLOv7 multi-object detection sub-model, the training set images are adaptively scaled and normalized to reduce black borders and eliminate the impact of pixel value differences on the model. Then, the CSPDarknet module in the YOLOv7 multi-object detection sub-model is replaced with the ODConv module (Omni-Dimensional Dynamic Convolution). The ODConv module performs multi-dimensional attention adaptive feature extraction and convolution kernel optimization on the training set images, which can adaptively capture the low-level features of targets with different shapes and sizes, generating an initial PPE wearing feature map. Next, the SimAM attention module calculates the pixel importance score of the initial PPE wearing feature map, and attention weights are assigned to the pixels of the initial PPE wearing feature map based on the pixel importance score. This combines spatial attention and channel attention without increasing the amount of additional computation, focusing on the target region, suppressing complex background interference, and generating an enhanced PPE wearing feature map. By adding the SimAM attention module to the YOLOv7 multi-object detection sub-model, the system can focus on information more critical to the current task from a large amount of input information, effectively solving the problem of information overload and improving the efficiency and accuracy of task processing, even with limited computing power. Then, the neck PAN module extracts layered features from the enhanced PPE wearing feature map, obtaining shallow and deep features separately, laying the foundation for subsequent feature fusion. Subsequently, the neck PAN module fuses the shallow and deep features, addressing the issue of large differences in target scale and generating a fused PPE wearing feature map, improving the comprehensiveness of target recognition. Next, the YOLOv7 detection head module uses its preset E-IoU loss function to confirm the target in the fused PPE wearing feature map, obtaining the feature target category, feature bounding box coordinates, and feature target confidence score to reduce the recognition error of similar targets. The feature target categories include personnel, safety helmets, protective clothing, and equipment. Finally, the feature target category, feature bounding box coordinates, and feature target confidence are integrated into the original feature target data. Specifically, this original feature target data is actually a list of multi-target recognition results. Each item in the list contains the feature target ID, feature target category (e.g., 0 = personnel, 1 = safety helmet, 2 = protective clothing, 3 = equipment), feature bounding box coordinates, and feature target confidence, thus forming a structured output. This provides reliable input for the subsequent action detection of the MAC-YOLO unsafe action detection sub-model and the small target supplementary detection of the ML-YOLO small target detection sub-model.The feature bounding box coordinates are (x1, y1, x2, y2), where x1 represents the horizontal pixel coordinate of the top-left corner vertex of the feature bounding box, y1 represents the vertical pixel coordinate of the top-left corner vertex of the feature bounding box, x2 represents the horizontal pixel coordinate of the bottom-right corner vertex of the feature bounding box, and y2 represents the vertical pixel coordinate of the bottom-right corner vertex of the feature bounding box.

[0045] Furthermore, the step of performing multidimensional attention-adaptive feature extraction and convolution kernel optimization on the training set images based on the ODConv module to obtain the initial PPE wearing feature map y includes: The ODConv module includes a GAP submodule, a total FC submodule, a four-branch activation submodule, and a four-branch FC submodule; Based on the GAP submodule, feature extraction is performed on the training set images to obtain the input feature X of the PPE wearing image; The GAP submodule performs a global averaging operation on the channels of the PPE-wearing image input feature X to obtain a low-dimensional global feature vector. Based on the total FC submodule and the four-branch activation submodule, the low-dimensional global feature vector is subjected to dimensionality reduction transformation and ReLU activation to obtain spatial attention branch, channel attention branch, filter attention branch and kernel attention branch; Based on the four-branch FC submodule, the spatial attention branch, channel attention branch, filter attention branch and kernel attention branch are adjusted in dimension to obtain the spatial attention weight, channel attention dimension weight, filter attention dimension weight and kernel attention dimension weight; The original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules are integrated to obtain the aggregated convolutional kernel; The convolutional kernel and the PPE wearing image are input to the feature X and convolved to obtain the initial PPE wearing feature map y.

[0046] In this embodiment, the GAP submodule extracts features from the training set images, initially capturing basic visual features related to PPE wearing and generating PPE-wearing image input features X, laying the initial feature foundation for subsequent multidimensional attention optimization. Next, the GAP submodule performs a global averaging operation on the channels of the PPE-wearing image input features X, transforming the high-dimensional spatial feature map into a low-dimensional global feature vector. The length of this low-dimensional global feature vector is equal to the number of input channels, reducing the number of model parameters and mitigating the risk of overfitting. Simultaneously, it summarizes global feature information, providing a global basis for calculating attention dimension weights. Then, the total FC submodule and the four-branch activation submodule perform dimensionality reduction on the low-dimensional global feature vector, improving feature expressiveness and computational efficiency. ReLU activation then splits the feature vector into spatial attention branches, channel attention branches, filter attention branches, and kernel attention branches, providing structural support for multidimensional attention weight learning. Subsequently, the four-branch FC submodule adjusts the dimensions of each of the four attention branches, generating spatial attention weights. Channel attention dimension weights Filter attention dimension weights And kernel attention dimension weights For i=1…n, specifically: the spatial attention branch is dimensionally adjusted through a four-branch FC submodule and activated using a Sigmoid function to generate spatial attention weights. The weights of the k×k spatial dimension of the convolution kernel are adjusted accordingly; the channel attention branches are dimensionally adjusted through a four-branch FC submodule and activated using a Sigmoid function to generate the channel attention dimension weights. The weights of the input channel dimension of the convolution kernel (cin×1) are adjusted accordingly; the dimension of the filter attention branch is adjusted through a four-branch FC submodule and activated by Sigmoid to generate the filter attention dimension weights. The weights of the convolution kernel's cout×1 output channel dimension are adjusted accordingly; the kernel attention branch is dimension-adjusted through a four-branch FC submodule and activated using Softmax to generate the kernel attention dimension weights. This allows for precise control of the four key dimensions of the convolution kernel by adjusting the weights of the n×1 number dimension of the kernel. Next, the original convolution kernels of the four-branch FC submodules are... Spatial attention weights Channel attention dimension weights Filter attention dimension weights And kernel attention dimension weights Perform a dimension-wise weighted integration, specifically: adjust the spatial attention weights. The k×k spatial location parameters of the original convolution kernel are multiplied point-by-point to enhance the feature response of the spatial region where large PPE devices or small PPE helmets are located; the channel attention dimension weights are also increased. Multiply by the input channel parameters of the convolution kernel to highlight the input channel features for recognizing the color channel of a PPE helmet or the contour channel of a PPE device; weight the filter attention dimension. Multiplying the output channel (filter) parameters of the convolution kernel optimizes the output dimension weight allocation of the feature map; it also optimizes the kernel attention dimension weights. Multiplying with the entire convolutional kernel matrix allows for dynamic filtering of the contributions of multiple convolutional kernels, suppressing redundant kernels and enhancing effective kernels. This ultimately generates aggregated convolutional kernels that adapt to different target shapes, sizes, and scene backgrounds, solving the problems of fixed parameters and insufficient adaptability in traditional convolutional kernels. Finally, because the aggregated convolutional kernel can adaptively focus on key features of targets at different scales, such as the overall contour features of large PPE devices or the local detail features of small PPE helmets, performing a convolution operation between the aggregated convolutional kernel and the input features X of the PPE wearing image can output an initial PPE wearing feature map y that fuses global and local information.

[0047] Furthermore, the personnel PPE state binding unit is used to bind personnel PPE states to training set images based on the MAC-YOLO unsafe action detection sub-model to obtain original associated feature data, including: Obtain raw feature target data based on training set images; The original feature target data is filtered based on a preset first confidence threshold to obtain the first feature target data; The training set images are cropped based on the first feature target data to obtain the original human area images; The MAC-YOLO unsafe human action detection sub-model includes the RFAConv convolution module, the EMA multi-scale attention module, the GSbottleneck module, the GSConv module, the VoV-GSCSP module, and the MAC-YOLO detection head module; Based on the RFAConv convolution module, feature extraction is performed on the original personnel region image to obtain a preliminary feature map of personnel actions; The preliminary feature map of human actions is decomposed into multiple scales based on the EMA multi-scale attention module to extract human action features at several scales. Based on the EMA multi-scale attention module, cross-dimensional dependencies are established on the human action features at several scales to enhance the preliminary human action feature map through cross-dimensional dependencies and obtain the enhanced human action feature map. Based on the GSConv module, GSbottleneck module and VoV-GSCSP module, feature purification is performed on the enhanced feature map of human actions to obtain the purified feature map of human actions. Based on the MAC-YOLO detection head module, the preset LMPDIoU loss function, and the purified feature map of personnel actions, the category of unsafe personnel actions, the coordinates of the bounding box of the action area, and the confidence level of personnel actions are determined. The categories of unsafe actions performed by personnel are bound to the original feature target data to obtain the PPE association status and association confidence of personnel actions; The original association feature data consists of the PPE association status of the personnel action, association confidence, coordinates of the action area bounding box, category of unsafe personnel action, and personnel action confidence.

[0048] In this embodiment, the MAC-YOLO unsafe action detection sub-model is used to accurately identify whether actions in training set images are safe and associate them with PPE wearing status. By obtaining raw feature target data from the training set images, the MAC-YOLO unsafe action detection sub-model provides key target information containing personnel and PPE equipment, laying the raw data foundation for binding unsafe actions with PPE wearing status. Next, the raw feature target data is filtered by a preset first confidence threshold, which filters out invalid targets with a feature target confidence score <0.7, obtaining more accurate first feature target data to ensure accurate location of the personnel's area. The preset first confidence threshold is 0.7. Then, the training set images are cropped based on the first feature target data to obtain the original personnel area image, which focuses on the personnel's area, reduces the computational load on irrelevant background areas, and improves model computational efficiency. Before inputting the original personnel area image into the MAC-YOLO unsafe action detection sub-model, it undergoes illumination compensation and noise filtering. Subsequently, the RFAConv convolutional module extracts features from the original personnel region image, capturing key regional features of personnel actions (such as the head and torso) to generate a preliminary personnel action feature map. This solves the problem of inaccurate feature capture caused by parameter sharing in traditional convolution. Next, the EMA multi-scale attention module decomposes the preliminary personnel action feature map into multiple scales, extracting personnel action features at several scales. Without channel dimensionality reduction, the EMA multi-scale attention module establishes cross-dimensional dependencies for these features, enabling the large local receptive fields of neurons in the MAC-YOLO unsafe action detection sub-model to efficiently obtain clearer multi-scale personnel action features. This helps reduce the influence of interference factors in the image, resulting in better pixel-level attention to the preliminary personnel action feature map. By strengthening the feature representation of unsafe actions through cross-dimensional dependencies, the preliminary personnel action feature map is enhanced, resulting in an enhanced personnel action feature map. Then, the neck of the MAC-YOLO unsafe human action detection sub-model is modified by using the slim-neck design paradigm. The learning ability of the MAC-YOLO unsafe human action detection sub-model is improved by stacking GSConv modules, the GSbottleneck module enhances the ability to process features, and the VoV-GSCSP module improves feature utilization efficiency and network performance. Thus, the GSConv module, GSbottleneck module, and VoV-GSCSP module can purify the enhanced human action feature map, reduce the computational complexity of the network, improve the feature utilization efficiency, and generate purified human action feature maps.Since the MAC-YOLO unsafe action detection sub-model relies on an IoU-based bounding box regression loss function to determine the target's location, it typically uses the coordinates of the top-left and bottom-right corners to define a unique rectangular box. However, this type of loss function becomes ineffective when the aspect ratios of the predicted and ground truth boxes are the same (but different). Therefore, in this embodiment, a preset LMPDIoU loss function (minimum point distance bounding box regression loss function) is introduced. By using the MAC-YOLO detection head module and the preset LMPDIoU loss function to perform action bounding box regression, the distance between the top-left and bottom-right points of the predicted and ground truth boxes during action localization can be directly minimized. This solves the problem that the original loss function cannot effectively optimize when the aspect ratios of the predicted and ground truth boxes are the same, thus accelerating the model's convergence speed. As a result, it can determine the category of unsafe actions (e.g., setting 0 = not wearing a safety helmet, 1 = not wearing protective clothing, 2 = smoking, 3 = working at height without protection), the bounding box coordinates of the action region, and the confidence level of the action based on the purified feature map of the personnel action, thereby improving the accuracy of action localization and action classification. Finally, by judging the correlation between the category of unsafe actions and the PPE wearing status, the category of unsafe actions is bound to the PPE wearing status in the original feature target data. For example, climbing actions need to be associated with the seat belt wearing status. The PPE association status and association confidence of the personnel actions are obtained, and the structured original association feature data are formed by combining the bounding box coordinates of the action area, the category of unsafe actions, and the confidence of the personnel actions.

[0049] To verify the performance of the trained YOLO model in this technical solution and avoid relying solely on general metrics, four verification metrics were designed: anti-interference capability, small target detection capability, multi-target synchronous detection, and real-time performance. These metrics are used to evaluate the YOLO training model. The core metrics and target values ​​are shown in Table 2 below. When all four verification metrics meet the target values, i.e., the preset first condition is met, the YOLO model is obtained.

[0050] Table 2 Validation Dimensions of YOLO Training Models The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting unsafe actions, characterized in that, include: Acquire the original PPE wearing images and divide the original PPE wearing images into training set images, validation set images and test set images according to a preset ratio; Construct an initial YOLO model, which includes a YOLOv7 multi-object detection sub-model, a MAC-YOLO unsafe human action detection sub-model, and an ML-YOLO small object detection sub-model. Based on the YOLOv7 multi-object detection sub-model, multi-dimensional feature optimization and object recognition are performed on the training set images to obtain the original feature object data; Based on the MAC-YOLO unsafe action detection sub-model, the PPE state of personnel in the training set images is bound to the original associated feature data. Based on the ML-YOLO small target detection sub-model and the original associated feature data, small targets are added to the training set images to obtain target enhancement detection results; The original correlation feature data is fused with the target enhancement detection results to obtain fused correlation feature data, which is then used to update the initial YOLO model and obtain the YOLO training model. Based on the validation set images, test set images, the initial YOLO model, and the trained YOLO model, the YOLO model and its performance improvement value are obtained. If the performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

2. The method for detecting unsafe actions according to claim 1, characterized in that, The YOLOv7 multi-object detection sub-model performs multi-dimensional feature optimization and object recognition on the training set images to obtain original feature object data, including: The YOLOv7 multi-target detection sub-model includes the ODConv module, the SimAM attention module, the neck PAN module, and the YOLOv7 detection head module; Based on the ODConv module, multidimensional attention adaptive feature extraction and convolution kernel optimization are performed on the training set images to obtain the initial PPE wearing feature map; The SimAM attention module is used to calculate the pixel importance score of the initial PPE wearing feature map, and attention weights are assigned to the pixels of the initial PPE wearing feature map based on the pixel importance score to obtain the enhanced PPE wearing feature map. The feature map of enhanced PPE wearing is extracted based on the neck PAN module to obtain shallow and deep features; The shallow and deep features are fused based on the neck PAN module to obtain a fused PPE wearing feature map; The YOLOv7 detection head module is used to confirm the fused PPE wearing feature map and obtain the feature target category, feature bounding box coordinates and feature target confidence. The original feature target data consists of the feature target category, feature bounding box coordinates, and feature target confidence score.

3. The method for detecting unsafe actions according to claim 2, characterized in that, The process of performing multidimensional attention-adaptive feature extraction and convolution kernel optimization on the training set images based on the ODConv module to obtain the initial PPE wearing feature map includes: The ODConv module includes a GAP submodule, a total FC submodule, a four-branch activation submodule, and a four-branch FC submodule; Based on the GAP submodule, feature extraction is performed on the training set images to obtain the input features of the PPE wearing images; The GAP submodule performs a global averaging operation on the channels of the PPE wearing image input features to obtain a low-dimensional global feature vector. Based on the total FC submodule and the four-branch activation submodule, the low-dimensional global feature vector is subjected to dimensionality reduction transformation and ReLU activation to obtain spatial attention branch, channel attention branch, filter attention branch and kernel attention branch; Based on the four-branch FC submodule, the spatial attention branch, channel attention branch, filter attention branch and kernel attention branch are adjusted in dimension to obtain the spatial attention weight, channel attention dimension weight, filter attention dimension weight and kernel attention dimension weight; The original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules are integrated to obtain the aggregated convolutional kernel; The aggregated convolution kernel and the PPE wearing image input features are convolved to obtain the initial PPE wearing feature map.

4. The method for detecting unsafe actions according to claim 1, characterized in that, The MAC-YOLO-based unsafe action detection sub-model binds personnel PPE states to the training set images to obtain raw associated feature data, including: Obtain raw feature target data based on training set images; The original feature target data is filtered based on a preset first confidence threshold to obtain the first feature target data; The training set images are cropped based on the first feature target data to obtain the original human area images; The MAC-YOLO unsafe human action detection sub-model includes the RFAConv convolution module, the EMA multi-scale attention module, the GSbottleneck module, the GSConv module, the VoV-GSCSP module, and the MAC-YOLO detection head module; Based on the RFAConv convolution module, feature extraction is performed on the original personnel region image to obtain a preliminary feature map of personnel actions; The preliminary feature map of human actions is decomposed into multiple scales based on the EMA multi-scale attention module to extract human action features at several scales. Based on the EMA multi-scale attention module, cross-dimensional dependencies are established on the human action features at several scales to enhance the preliminary human action feature map through cross-dimensional dependencies and obtain the enhanced human action feature map. Based on the GSConv module, GSbottleneck module and VoV-GSCSP module, feature purification is performed on the enhanced feature map of human actions to obtain the purified feature map of human actions. Based on the MAC-YOLO detection head module, the preset LMPDIoU loss function, and the purified feature map of personnel actions, the category of unsafe personnel actions, the coordinates of the bounding box of the action area, and the confidence level of personnel actions are determined. The categories of unsafe actions performed by personnel are bound to the original feature target data to obtain the PPE association status and association confidence of personnel actions; The original association feature data consists of the PPE association status of the personnel action, association confidence, coordinates of the action area bounding box, category of unsafe personnel action, and personnel action confidence.

5. The method for detecting unsafe actions according to claim 4, characterized in that, The ML-YOLO small object detection sub-model and original associated feature data are used to supplement the training set images with small objects to obtain object enhancement detection results, including: Based on a preset first confidence threshold, a preset second confidence threshold, original associated feature data, and training set images, the suspected PPE wearing target area is obtained; Based on the suspected PPE wearing target area and training set images, obtain human motion reconstruction feature maps; Construct an ML-YOLO small target detection model; the ML-YOLO small target detection model includes an ML-YOLO backbone network module, a preset first detection head module, a preset second detection head module, and an MCSTA module; Residual connections are performed on the feature maps of human motion reconstruction based on the ML-YOLO backbone network module to obtain shallow feature data; Deep semantic feature extraction is performed on the feature map of human action reconstruction based on the ML-YOLO backbone network module to obtain deep feature data; Cross-layer fusion of shallow and deep feature data is performed to obtain multi-level fused feature maps; Based on the preset first detection head module and the preset second detection head module, target detection is performed on the multi-level fused feature map to obtain the fused target detection result; Attention enhancement is applied to the fused target detection results using the MCSTA module to obtain initial enhanced target detection results. Based on the preset third confidence threshold and the initial target enhancement detection results, the target enhancement detection results are obtained.

6. The method for detecting unsafe actions according to claim 1, characterized in that, The process involves obtaining the YOLO model and its performance improvement value based on validation set images, test set images, the initial YOLO model, and the trained YOLO model. If the performance improvement value meets a preset second condition, the PPE-wearing image to be detected is imported into the YOLO model to obtain unsafe action detection results, including: The performance of the YOLO trained model is evaluated based on the validation set images until a preset first condition is met, at which point the YOLO model is obtained. The YOLO initial model and the YOLO model are tested based on the test set images to obtain the model performance improvement value. If the model performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

7. An unsafe action detection system, characterized in that, It includes a data acquisition and partitioning unit, an initial model building unit, a multi-dimensional feature extraction and recognition unit, a personnel PPE status binding unit, a small target supplementary detection unit, a model update unit, and a detection execution unit, specifically: The data acquisition and division unit is used to acquire the original PPE wearing images and divide the original PPE wearing images into training set images, validation set images and test set images according to a preset ratio. The initial model building unit is used to build the YOLO initial model, which includes the YOLOv7 multi-object detection sub-model, the MAC-YOLO unsafe human action detection sub-model, and the ML-YOLO small object detection sub-model. The multi-dimensional feature extraction and recognition unit is used to perform multi-dimensional feature optimization and target recognition on the training set images based on the YOLOv7 multi-target detection sub-model to obtain the original feature target data. The personnel PPE state binding unit is used to bind personnel PPE state to training set images based on the MAC-YOLO personnel unsafe action detection sub-model to obtain original associated feature data. The small target supplementation detection unit is used to supplement small targets in the training set images based on the ML-YOLO small target detection sub-model and the original associated feature data, and obtain the target enhancement detection result. The model update unit is used to fuse the original correlation feature data with the target enhancement detection result to obtain fused correlation feature data, and update the YOLO initial model based on the fused correlation feature data to obtain the YOLO training model. The detection execution unit is used to obtain the YOLO model and the model performance improvement value based on the validation set images, test set images, the initial YOLO model and the trained YOLO model. If the model performance improvement value meets the preset second condition, the PPE wearing image to be detected is imported into the YOLO model to obtain the unsafe action detection result.

8. The unsafe action detection system according to claim 7, characterized in that, The multi-dimensional feature extraction and recognition unit is used to perform multi-dimensional feature optimization and target recognition on the training set images based on the YOLOv7 multi-object detection sub-model, and to obtain the original feature target data, including: The YOLOv7 multi-target detection sub-model includes the ODConv module, the SimAM attention module, the neck PAN module, and the YOLOv7 detection head module; Based on the ODConv module, multidimensional attention adaptive feature extraction and convolution kernel optimization are performed on the training set images to obtain the initial PPE wearing feature map; The SimAM attention module is used to calculate the pixel importance score of the initial PPE wearing feature map, and attention weights are assigned to the pixels of the initial PPE wearing feature map based on the pixel importance score to obtain the enhanced PPE wearing feature map. The feature map of enhanced PPE wearing is extracted based on the neck PAN module to obtain shallow and deep features; The shallow and deep features are fused based on the neck PAN module to obtain a fused PPE wearing feature map; The YOLOv7 detection head module is used to confirm the fused PPE wearing feature map and obtain the feature target category, feature bounding box coordinates and feature target confidence. The original feature target data consists of the feature target category, feature bounding box coordinates, and feature target confidence score.

9. The unsafe action detection system according to claim 8, characterized in that, The process of performing multidimensional attention-adaptive feature extraction and convolution kernel optimization on the training set images based on the ODConv module to obtain the initial PPE wearing feature map includes: The ODConv module includes a GAP submodule, a total FC submodule, a four-branch activation submodule, and a four-branch FC submodule; Based on the GAP submodule, feature extraction is performed on the training set images to obtain the input features of the PPE wearing images; The GAP submodule performs a global averaging operation on the channels of the PPE wearing image input features to obtain a low-dimensional global feature vector. Based on the total FC submodule and the four-branch activation submodule, the low-dimensional global feature vector is subjected to dimensionality reduction transformation and ReLU activation to obtain spatial attention branch, channel attention branch, filter attention branch and kernel attention branch; Based on the four-branch FC submodule, the spatial attention branch, channel attention branch, filter attention branch and kernel attention branch are adjusted in dimension to obtain the spatial attention weight, channel attention dimension weight, filter attention dimension weight and kernel attention dimension weight; The original convolutional kernels, spatial attention weights, channel attention dimension weights, filter attention dimension weights, and kernel attention dimension weights of the four-branch FC submodules are integrated to obtain the aggregated convolutional kernel; The aggregated convolution kernel and the PPE wearing image input features are convolved to obtain the initial PPE wearing feature map.

10. An unsafe action detection system according to claim 7, characterized in that, The personnel PPE state binding unit is used to bind personnel PPE states to training set images based on the MAC-YOLO unsafe action detection sub-model, and obtain original associated feature data, including: Obtain raw feature target data based on training set images; The original feature target data is filtered based on a preset first confidence threshold to obtain the first feature target data; The training set images are cropped based on the first feature target data to obtain the original human area images; The MAC-YOLO unsafe human action detection sub-model includes the RFAConv convolution module, the EMA multi-scale attention module, the GSbottleneck module, the GSConv module, the VoV-GSCSP module, and the MAC-YOLO detection head module; Based on the RFAConv convolution module, feature extraction is performed on the original personnel region image to obtain a preliminary feature map of personnel actions; The preliminary feature map of human actions is decomposed into multiple scales based on the EMA multi-scale attention module to extract human action features at several scales. Based on the EMA multi-scale attention module, cross-dimensional dependencies are established on the human action features at several scales to enhance the preliminary human action feature map through cross-dimensional dependencies and obtain the enhanced human action feature map. Based on the GSConv module, GSbottleneck module and VoV-GSCSP module, feature purification is performed on the enhanced feature map of human actions to obtain the purified feature map of human actions. Based on the MAC-YOLO detection head module, the preset LMPDIoU loss function, and the purified feature map of personnel actions, the category of unsafe personnel actions, the coordinates of the bounding box of the action area, and the confidence level of personnel actions are determined. The categories of unsafe actions performed by personnel are bound to the original feature target data to obtain the PPE association status and association confidence of personnel actions; The original association feature data consists of the PPE association status of the personnel action, association confidence, coordinates of the action area bounding box, category of unsafe personnel action, and personnel action confidence.

Images