Intelligent Behavior Detection Methods for Miner Safety

Abstract

This application relates to the field of behavior detection technology, and more particularly to an intelligent behavior detection method for miner safety. The method includes: segmenting and sampling video data, labeling the sampled video data into categories, and dividing it into training, validation, and test sets; constructing a target detection model based on the YOLOv8 model, the target detection model including an input layer, a backbone network, a neck network, and a head network; simplifying the feature fusion layer of the Bidirectional Feature Pyramid Network (BiFPN) in the neck network, and introducing an improved MPDIoU* loss function in the head network for feature recognition; training the improved model using the training and validation sets to obtain the trained model; and using the trained model to identify unsafe behaviors of coal miners. This application can identify and analyze unsafe behaviors in complex underground mining environments without relying on complex equipment, and the model exhibits high stability and adaptability.

Description

Technical Field

[0001] This application relates to the field of behavior detection technology, and in particular to an intelligent behavior detection method for miner safety. Background Technology

[0002] Coal mines, especially underground coal mines, are unique industrial production environments. Their workspaces are filled with dust, the work sites are far from the ground, there are multiple potential disaster-causing factors in the work area, and workers are concentrated. Therefore, in the coal mine production environment, personnel safety is of paramount importance, and the standardization and monitoring of personnel safety behavior is also particularly important.

[0003] While image dynamic recognition has broad application prospects, existing recognition technologies still face significant challenges due to the low visibility, complex environment, and poor communication conditions in underground mine tunnels. Although existing computational models perform well in terms of accuracy and real-time performance, their high complexity and computational requirements make them prone to performance degradation in environments with limited hardware resources, especially when running on low-performance devices, where they may fail to meet real-time and response speed requirements. Furthermore, existing behavior recognition methods rely heavily on high-quality video data and accurate object tracking, which may make it difficult to effectively capture key information in extremely complex behaviors or situations with many interfering factors, thus affecting the accuracy of recognition. Summary of the Invention

[0004] The technical problem this invention aims to solve is that existing target detection models often suffer from poor stability, insufficient adaptability, and low computational efficiency when applied in complex coal mine environments. This results in weak recognition capabilities for irregular miner behaviors, complex equipment, and obstructions during the detection process. In particular, the detection accuracy for abnormal behaviors is low in dusty, dimly lit environments with background interference, the model convergence speed is slow, and the training efficiency is not high.

[0005] Therefore, the present invention provides an intelligent behavior detection method for miner safety.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A smart behavior detection method for miner safety includes,

[0008] S1, the video data is sampled in segments, and the sampled video data is labeled with categories, dividing it into training set, validation set and test set;

[0009] S2, an object detection model is constructed based on the YOLOv8 model. The object detection model includes an input layer, a backbone network, a neck network, and a head network. In the neck network, the number of feature fusion layers of the bidirectional feature pyramid network (BiFPN structure) is simplified, and an improved MPDIoU* loss function is introduced into the head network for feature recognition.

[0010] S3 uses the training set and validation set to train the improved model, resulting in a trained model, which is then used to identify unsafe behaviors of coal miners.

[0011] Furthermore, in step S1, the input video is evenly divided into k time segments, each time segment covering different parts of the video, and then a short segment is randomly extracted from each time segment to obtain a segment sequence.

[0012] Furthermore, in step S1, the aggregated segment sequence is normalized using Softmax to output the final action category probability of the video. , among which, T k It is a segment relative to all categories, W is the network parameter, F is the segment returned by the function, and g is the normalization function.

[0013] Furthermore, in step S2, the backbone network integrates the SPPE architecture and the DCNv4 module. The DCNv4 module is added to the bottom layer of the SPPE architecture to propose a new SPPE_DCNv4 structure. After inputting the image, two overlapping convolution operations are performed in the SPPE architecture. Then, the C2f module is used to further process the feature map. The convolution and C2f modules are then used together to perform multiple downsampling operations on the feature map. Finally, the feature map is extracted through the SPPF_DCNv4 module.

[0014] Further, in step S2, the neck network extracts multi-scale feature maps {Fi} from the input data. The BiFPN structure includes three fusion feature layers. In the BiFPN structure, a top-down path and a bottom-up path are used to perform bidirectional feature fusion on the feature maps. Then, the feature maps at multiple scales are weighted and fused. Finally, the feature map Y is output through a multi-path residual connection for the abnormal behavior recognition task.

[0015] Furthermore, the top-down path is represented as follows: The bottom-up path is represented as follows: .

[0016] Furthermore, in the head network, an improved MPDIoU* loss function is used to locate and define the target, thereby improving the bounding box regression accuracy, optimizing the model's boundary prediction capability, and introducing multi-point distance metrics and considering changes in object shape to further enhance the bounding box regression accuracy.

[0017] Furthermore, the head network specifically includes:

[0018] Calculate the prediction range IoU;

[0019] By introducing a nonlinear transformation to improve the gradient, the improved loss function can be defined as: ,in, It is a non-linear function used to amplify gradients;

[0020] Calculate the point distance penalty term, which includes calculating the square of the distance between the top-left corner of the predicted bounding box and the top-left corner of the ground truth bounding box. The squared distance between the bottom right corner of the predicted bounding box and the bottom right corner of the ground truth bounding box. The sum of the squared width and squared height of the predicted bounding box, and the squared width and squared height of the actual bounding box: ;

[0021] Calculate MPDIoU: ;

[0022] The final loss function is: .

[0023] A computer device, comprising:

[0024] processor;

[0025] Memory, used to store executable instructions;

[0026] The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the intelligent behavior detection method for miner safety as described above.

[0027] A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the intelligent behavior detection method for miner safety as described above.

[0028] The beneficial effects of this invention are that it aims to solve problems such as interference, low lighting, and mechanical obstruction in the complex environment of underground coal mines, especially the challenges faced by existing behavior recognition methods in terms of algorithm speed and accuracy when performing tasks involving the identification of abnormal human behavior. This invention proposes an improved YOLOv8 method:

[0029] 1. Import SPPE and DCNv4 modules:

[0030] The backbone network innovatively integrates the SPPE architecture and the DCNv4 module. Addressing the specific needs of detecting unsafe miner behavior, SPPE enhances the detection capability for targets of varying sizes through a multi-scale spatial pyramid, adapting to the diverse actions and positional changes of miners within the mine. DCNv4, through deformable convolution, strengthens its adaptability to complex backgrounds and occluded scenes, sensitively capturing subtle changes in miner behavior.

[0031] 2. Optimize BiFPN and add a residual connection module:

[0032] The Bidirectional Feature Pyramid Network (BiFPN) has been innovatively simplified and optimized. The feature fusion process of BiFPN has been simplified, improving the feature extraction efficiency; a multi-channel residual connection module has been added to enhance the accuracy of identifying abnormal miner behavior and effectively suppress interference from complex background noise.

[0033] 3. Employ an optimized MPDIoU loss function:

[0034] The MPDIoU (Modified Partial DIoU) loss function is optimized and introduced into the YOLOv8 framework. Through optimization strategies, it is specifically applied to the accurate detection of unsafe miner behaviors. By calculating multi-point internal distances, the shortcomings of traditional IoU in handling irregular shapes and occlusions are addressed, improving bounding box localization accuracy, especially suitable for recognizing complex poses of miners in mining environments.

[0035] 4. Optimize the YOLOv8 network structure:

[0036] The YOLOv8 network structure is simplified, the number of downsampling steps is reduced, and the image resolution is improved; more details of miner behavior are preserved, the detection capability of small and distant targets is enhanced, feature extraction is optimized, background influence is reduced, and the focus is on miner behavior features.

[0037] These improvements significantly enhance the model's stability and adaptability, particularly in the identification and analysis of unsafe behaviors in the complex environment of underground coal mines, thereby improving the model's detection performance. Attached Figure Description

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] Figure 1 This is a schematic diagram of the implementation process of the target detection model network in this invention.

[0040] Figure 2 This is a schematic diagram of the overall structure of the target detection model network in this invention.

[0041] Figure 3 This is a schematic diagram of the SPPE_DCNv4 structure in this invention.

[0042] Figure 4 This is a comparison chart of the BiFPN network before and after simplification.

[0043] Figure 5 This is a schematic diagram of the overall architecture of MPDIoU in this invention.

[0044] Figure 6 This is a schematic diagram of the target detection model in this invention for recognizing different behaviors.

[0045] Figure 7 This is a comparison chart showing the results of YOLOv8 network before and after the improvement in recognizing the behavior of people in the mine. Detailed Implementation

[0046] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0047] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, features defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0048] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0049] Example 1

[0050] Reference Figure 1 ,2 A smart behavior detection method for miner safety, comprising:

[0051] S1, Behavioral Data Collection

[0052] Using the KBA127 explosion-proof mining camera, behavioral video is collected in different scenarios. The process mainly includes signal acquisition, conversion and transmission, processing and analysis, and subsequent applications. First, the camera is installed in different locations underground and activated. Its photosensitive device acquires image signals and converts them into optical signals, which are then transmitted to the ground monitoring center via optical fiber in TCP / IP full-duplex mode. At the monitoring center, the video signal undergoes preliminary processing and intelligent analysis to identify personnel behavior patterns and store the data. Finally, monitoring personnel can view the video in real time, and the system issues alarms for abnormal behavior, achieving behavioral supervision.

[0053] S1.1, Video Segment Sampling

[0054] The obtained video data is processed using video sampling technology, and the input video v is evenly divided into k time segments, each covering a different part of the video:

[0055] ,

[0056] Then, a short segment is randomly selected from each time period to obtain the segment sequence:

[0057] ,

[0058] Where T k From S k Fragments obtained by random sampling.

[0059] S1.2, Softmax normalization, classifies the acquired video data into categories.

[0060] Softmax normalization is applied to the aggregated segment sequence to output the final action class probabilities of the video:

[0061] ,

[0062] Where W is the network parameter, F is the function return segment, and T is the function return value. k , where g is the normalization function relative to all categories of segments. Ultimately, the category with the highest probability is determined as the category to which the video belongs.

[0063] This segmented sampling method can effectively avoid redundancy in consecutive frames while reducing computational costs, making it suitable for behavior recognition tasks in long videos.

[0064] Prior to training, XML annotations for five unsafe behaviors were created using the LabelImg tool, and a network model was built based on the mapping between samples and labels. To expand the dataset, data augmentation techniques such as Mosaic, HSV, translation, scaling, and horizontal flipping (Fliplr) were applied, increasing the dataset to 25,000 images. Finally, a 7:2:1 cross-validation strategy was employed, dividing the dataset into training (70%), validation (20%), and test (10%) sets to ensure the accuracy and reliability of the model.

[0065] S2, Constructing the object detection model

[0066] The target detection model in this application is an improvement upon the YOLOv8 model. The model consists of four parts: input, backbone network, neck, and head. This application simplifies the YOLOv8 model network structure, reduces the number of downsampling operations, and improves image resolution; it preserves more details of miner behavior, enhances the detection capability for small and distant targets, optimizes feature extraction, reduces background influence, and focuses on miner behavior features. Specifically:

[0067] The backbone network innovatively integrates the SPPE architecture and the DCNv4 module. Addressing the specific needs of detecting unsafe miner behavior, the SPPE architecture enhances the detection capability for targets of varying sizes through a multi-scale spatial pyramid, adapting to the diverse actions and positional changes of miners within the mine. The DCNv4 module, through deformable convolution, strengthens its adaptability to complex backgrounds and occluded scenes, sensitively capturing subtle changes in miner behavior.

[0068] In the neck network, the Bidirectional Feature Pyramid Network (BiFPN) was innovatively simplified and optimized. The BiFPN feature fusion process was simplified, improving feature extraction efficiency; a multi-channel residual connection module was added to enhance the accuracy of identifying abnormal miner behavior and effectively suppress interference from complex background noise.

[0069] In the head network, the MPDIoU (Modified Partial DIoU) loss function is optimized and improved, and the YOLOv8 framework is introduced. Through optimization strategies, it is specifically applied to the task of accurately detecting unsafe behaviors of miners. By calculating the internal distance between multiple points, the shortcomings of traditional IoU in handling irregular shapes and occlusions are addressed, improving the bounding box localization accuracy, which is especially suitable for recognizing complex poses of miners in mining environments.

[0070] These improvements significantly enhance the model's stability and adaptability, particularly in the identification and analysis of unsafe behaviors in the complex environment of underground coal mines, thereby improving the model's detection performance.

[0071] To address the complex environment (insufficient lighting, cluttered backgrounds) and diverse unsafe behaviors of miners in coal mines, this study constructed the MBD-1500 dataset for detecting unsafe miner behaviors based on the "Coal Mine Safety Regulations" (GB16423-2004). This dataset, collected in real underground scenes using professional camera equipment, contains 1500 high-quality video samples covering various working conditions, including different lighting conditions, working angles, shooting distances, and field of view. It systematically records typical unsafe behavior categories underground, effectively solving the applicability problem of existing public datasets in the special scenarios of coal mines, and providing important data support for developing behavior detection models with strong generalization capabilities. The dataset includes five types of unsafe behaviors: worker falls, not wearing safety helmets, illegal riding, crossing equipment, and entering dangerous warning zones. To capture long-term behavioral correlations in the coal mine environment, segmented video sampling technology was used to process the acquired video data. Specifically:

[0072] S2.1, Input layer inputs segmented video data

[0073] S2.2, Backbone Network

[0074] This application innovatively integrates the SPPE architecture and the DCNv4 module into the backbone network, proposing an efficient and robust detection method to address the specific needs of detecting unsafe mining behavior. The Self-Perceived Pyramid Fusion (SPPF) structure is used as the output of the backbone network for image fusion, but max pooling only extracts the maximum value, ignoring global information. This application proposes a new SPPE_DCNv4 structure by adding a DCNv4 module to the underlying SPPE architecture, as shown below. Figure 3As shown, by integrating efficient Spatial Pyramid Pooling (SPPE) and Deformable Convolution (DCNv4) modules, this paper addresses two key challenges of traditional unsafe behavior detection methods in the complex environment of underground mines: insufficient multi-scale feature extraction capabilities and weak adaptability to target deformation and occlusion. Specifically, the SPPE architecture achieves high efficiency and accuracy in unsafe behavior detection through lightweight design and efficient multi-scale feature processing capabilities. Its lightweight design is reflected in the simplified network structure, the use of a lightweight feature pyramid network, and parallel processing of multi-scale features, significantly reducing computational complexity. Efficient multi-scale feature processing is achieved through multi-scale feature extraction, feature pyramid networks, and parallel processing, improving the model's ability to capture targets at different scales and its detection efficiency. Optimized allocation of computational resources enables more efficient capture of multi-scale features in miner behavior, especially key subtle movements (such as hand operations) and diverse postures (such as falls and crossing equipment), thereby improving the accuracy of miner behavior identification.

[0075] Meanwhile, the DCNv4 module significantly enhances the model's adaptability to target deformations (such as miner postures and equipment operation) and occlusions (such as equipment obstructing the miner's body) by dynamically adjusting the sampling position of the convolution kernel, effectively overcoming the limitations of traditional convolution in fixed sampling modes. Furthermore, DCNv4 can reduce the impact of complex background interference in the mining environment (such as mine roadways and machinery), focusing more on the extraction of miner behavioral features. Through these two core innovations, the model in this study achieves higher accuracy and stronger robustness in the task of identifying unsafe behaviors in mines, providing more reliable technical support for mine safety monitoring.

[0076] The backbone network specifically includes:

[0077] 1. Input image. The size of the input image is 640×640×3, that is, 640 pixels wide, 640 pixels high, and 3 color channels (RGB).

[0078] 2. Convolution operation

[0079] By using two overlapping convolution operations, the input image size is reduced from 640×640×3 to 320×320×32. With a kernel size of k×k and a stride of s, the output size is calculated using the following formula:

[0080] ,

[0081] The specific formula is as follows: , , .

[0082] 3. C2f module processing

[0083] The C2f module is used to further process the feature map, reducing its size from 320×320×32 to 160×160×64. The C2f module contains convolution and pooling operations, and the size change formula is similar to that of the convolution operation described above.

[0084] 4. After one convolution and six C2f modules, the feature map size is reduced from 160×160×64 to 80×80×120. The size change at each step follows the convolution formula described above.

[0085] 5. Further downsampling

[0086] After one convolution and three C2f modules, the feature map size is reduced from 80×80×120 to 40×40×256. The convolution formula is also followed.

[0087] 6. SPPF_DCNv4 module processing

[0088] Finally, the SPPF_DCNv4 module was used to reduce the feature map size from 40×40×256 to 20×20×512. The SPPF module is a spatial pyramid pooling module used to enhance the global receptive field of features. The spatial aggregation formula of DCNv4 is as follows:

[0089] ,

[0090] Among them, y g This is the output feature of the g-th group; m gk It is the spatial aggregation weight of the k-th sampling point in the g-th group; x g p is the input feature of the g-th group; p0 is the position of the current sampling point; p k These are predefined grid sampling points; Δp gk It corresponds to the grid sampling position p in the g-th group. k The offset.

[0091] The 640×640×3 image, after the above processing, is input into the backbone network. Through overlapping operations and two convolutions, a 320×320×32 image is obtained. Through the c2f module, a 160×160×64 image is obtained. Then, through one convolution and six c2f modules, an 80×80×120 image is obtained. Then, through one convolution and three c2f modules, a 40×40×256 image is obtained. Finally, through the SPPE_DCNv4 module, a 20×20×512 image is obtained.

[0092] The optimized YOLOv8 backbone network improves image resolution by simplifying the network structure and reducing the number of downsampling operations, further enriching the miner behavior feature information. At the same time, it reduces the interference of background information, making the network more focused on the detection of key behaviors, thereby improving the overall recognition effect.

[0093] The results show that the collaborative design of the SPPE architecture and DCNv4 significantly improves the accuracy and robustness of feature extraction, demonstrating excellent performance in complex scenarios such as the detection of unsafe mining behavior. This combined architecture not only overcomes the limitations of traditional methods but also provides a new technical approach for the detection of unsafe mining behavior. The SPPE architecture, through efficient multi-scale feature extraction, and DCNv4, through dynamic adaptation to target deformation, work together to significantly improve detection performance in complex mining environments, providing reliable technical support for mine safety monitoring.

[0094] S2.3, Neck Network

[0095] This application innovatively simplifies and optimizes the Bidirectional Feature Pyramid Network (BiFPN) and introduces the YOLOv8 network. Based on the core architecture of BiFPN, this study simplifies the network structure and redesigns its key components, significantly reducing computational complexity while further improving the model's detection capabilities. This simplification and optimization is particularly suitable for the task of identifying unsafe behaviors in complex mining environments.

[0096] Bidirectional Feature Pyramid Network (BiFPN) is an efficient feature pyramid network for object detection. It generates high-quality feature maps through bidirectional information transfer and multi-layer feature fusion, significantly improving detection performance. However, its complex structure and high computational cost can pose challenges in practical applications, especially in resource-constrained environments. Therefore, this paper proposes an optimization approach by reducing the number of feature fusion layers in BiFPN: BiFPN typically contains five feature fusion layers, such as... Figure 4 As shown in small figure (a), feature fusion is performed using both top-down and bottom-up paths. By reducing the number of these fusion layers, only three layers are retained, as shown... Figure 4 As shown in small figure (b) in the figure, this reduces computational complexity. The purpose of simplifying BiFPN is to reduce computational complexity, improve operational efficiency, and at the same time maintain its excellent performance as much as possible, making it more suitable for practical deployment and application. However, simplifying BiFPN may lead to a decrease in feature representation ability and limited information transmission, which in turn affects detection accuracy, especially when dealing with complex scenes or small targets.

[0097] To compensate for the functional deficiencies caused by the simplification of BiFPN, this application introduces an improved MPDIoU* loss function, which can better optimize the model's localization accuracy. The MPDIoU* loss function, by combining the calculation of IoU (Intersection over Union), can more accurately measure the degree of matching between the predicted bounding box and the ground truth bounding box, thereby effectively improving the model's performance in object localization. This optimization not only compensates for the shortcomings of BiFPN in feature fusion after simplification but also further enhances the model's detection performance, especially in handling complex scenes or small targets, significantly improving detection accuracy and robustness.

[0098] Combining the above methods, the simplified BiFPN exhibits improved performance, specifically in the following ways: First, computational efficiency is significantly improved. The simplified structure reduces computational load, increasing operational efficiency and making it suitable for deployment in resource-constrained environments. Second, detection accuracy is optimized. By incorporating an improved loss function, the simplified BiFPN surpasses the original structure in detection accuracy. Furthermore, the model's generalization ability is enhanced. Combined with multi-scale feature enhancement and optimized post-processing algorithms, it can better adapt to different scenarios and targets. Finally, resource utilization is optimized. The simplified structure maintains high detection performance while reducing computational resource consumption, making it particularly suitable for applications in resource-constrained environments such as edge devices.

[0099] By simplifying the BiFPN structure and introducing a multi-path residual connection module, the network can maintain the stability of the information flow while fusing more features, thereby improving the accuracy of abnormal behavior recognition. The specific process is as follows:

[0100] 1. Input Feature Extraction: Extracting multi-scale feature maps {F} from the input data. i}

[0101] 2. Bidirectional feature fusion:

[0102] Top-down path: ;

[0103] Bottom-up path:

[0104] ;

[0105] Weighted fusion: , where α and β are learnable fusion weights.

[0106] 3. Multi-path residual connection: .

[0107] 4. Output feature generation: The fused feature map Y is used for the abnormal behavior recognition task.

[0108] The above formulas clearly describe the signal processing of the simplified BiFPN structure and multi-path residual connection modules. These modules significantly improve the accuracy of abnormal behavior recognition by optimizing feature fusion and information flow stability.

[0109] By preserving the bidirectional information flow mechanism of BiFPN, this study achieves efficient cross-scale feature fusion. At the same time, by simplifying network parameters and optimizing feature interaction methods, it significantly enhances the model's adaptability to complex backgrounds, target deformation, and occlusion in the mining environment.

[0110] S2.4, Header Network

[0111] The core innovation of this section lies in optimizing and improving the MPDIoU (Modified Partial DIoU, Mean Positive Distance Intersection over Union) loss function and introducing it into the YOLOv8 framework. MPDIoU is an improved IoU loss function used to optimize the model's localization accuracy. However, the standard IoU loss function becomes very small in gradient when the overlap area between the target box and the predicted box is small, leading to optimization difficulties. To improve this problem, a nonlinear transformation is introduced to adjust the gradient magnitude, ensuring that the gradient remains large even when the overlap area is small, thereby accelerating the optimization process. This research fully utilizes the multi-keypoint distance metric and dynamic adaptability of the improved MPDIoU* loss function, and specifically optimizes it for the complex scenario requirements of miner behavior detection. This innovative design significantly improves the model's localization accuracy and robustness against unsafe miner behaviors, providing a more accurate detection solution for complex situations such as non-rigid deformation and partial occlusion in mining environments.

[0112] An improved MPDIoU* loss function is adopted, which addresses the shortcomings of the traditional YOLOv8 CIoU loss function in handling complex object shapes and occlusion, thereby further improving the bounding box regression accuracy and optimizing the model's boundary prediction capability. The specific calculation formula is as follows:

[0113] 1. Calculate the prediction range IoU: ,

[0114] Among them, Area(B pred ∩B gt ) is the prediction box B pred and the real frame B gt The area of ​​the intersection, Area(B) pred ∪B gt ) is the area of ​​their union.

[0115] The standard IoU loss function is defined as: ,

[0116] in, It predicts the bounding box. It is the actual bounding box.

[0117] When the overlap between the target bounding box and the predicted bounding box is small, the gradient of the IoU becomes very small, making optimization difficult. Specifically, the gradient is proportional to the area of ​​the overlapping region; when the overlapping region is small, the gradient is naturally small.

[0118] 2. Introduce nonlinear transformations to improve gradients.

[0119] To improve gradient calculation, nonlinear transformations, such as exponential functions, can be introduced to adjust the magnitude of the gradient. The improved loss function can be defined as:

[0120] ,

[0121] in, It is a non-linear function used to amplify gradients. It is a constant.

[0122] 3. Calculate the point distance penalty:

[0123] Reference Figure 5 The coordinates of the top left, top right, bottom left, and bottom right corners of the prediction box are respectively The coordinates of the top left, top right, bottom left, and bottom right corners of the true bounding box are respectively... .

[0124] Calculate the squared distance between the top-left and bottom-right corners: ,

[0125] ;

[0126] Calculate the sum of the squares of the width and height: ;

[0127] 4. Calculate MPDIoU: .

[0128] The point distance penalty term is used to penalize the center point distance and size deviation between the predicted bounding box and the ground truth bounding box.

[0129] Loss function:

[0130] ,

[0131] This loss function optimizes the accuracy of bounding box regression by minimizing the value of MPDIoU*.

[0132] The results show that the optimized MPDIoU* loss function effectively addresses the shortcomings of the traditional IoU loss function in handling complex object shapes and occlusions. By introducing multi-point distance metrics and considering changes in object shape, the bounding box regression accuracy is improved. This improvement optimizes the model's robustness in boundary prediction, enabling the model to locate targets more accurately and significantly enhancing the localization accuracy in miner behavior detection tasks. This improvement provides more reliable technical support for behavior recognition in complex industrial scenarios.

[0133] S2.5, Optimize YOLOv8 network structure

[0134] Simplify the network structure, reduce the number of downsampling iterations, and improve image resolution, as shown in the following formula:

[0135] ,

[0136] in, Here, W represents the input features, S represents the step size, and P represents padding. is the activation function. The object detection model in this application is based on the optimized YOLOv8 network structure, which retains more details of miner behavior, enhances the detection capability of small and distant targets, optimizes feature extraction, reduces background influence, and focuses on miner behavior features.

[0137] S3, use the training set and validation set to train the improved model, and obtain the trained model.

[0138] During model training, we employed a series of optimization strategies to improve model performance and generalization ability. First, we performed diverse preprocessing on the training data, including data augmentation, normalization, and cleaning, to enhance data diversity and quality. During training, we selected an appropriate optimizer and learning rate scheduling strategy, and balanced training efficiency and memory usage by adjusting the batch size. Simultaneously, we introduced Dropout to prevent overfitting and used an early stopping mechanism to avoid overfitting during training. Furthermore, we used the validation set to evaluate model performance in real time and selected the optimal hyperparameter combination to optimize model performance. To further improve training efficiency, we adopted multi-GPU distributed training and used tools such as TensorBoard for visual monitoring of the training process. Finally, we further optimized the model's performance and complexity through model fusion and pruning techniques. These comprehensive measures ensured efficient training of the model on both the training and validation sets, laying a solid foundation for subsequent testing and applications.

[0139] S4 uses a trained model to accurately identify unsafe behaviors of coal miners.

[0140] Figure 6The diagram illustrates the accurate identification of unsafe behaviors of coal miners using the model trained in this application. Figure 6 (a) The small image in the diagram is a schematic diagram of detecting the behavior of not wearing a safety helmet; (b) The small image is a schematic diagram of detecting the behavior of crossing the equipment; (c) The small image is a schematic diagram of detecting illegal riding; (d) The small image is a schematic diagram of detecting a worker falling; (e) The small image is a schematic diagram of detecting entering a danger warning zone.

[0141] Figure 7 To compare the detection results before and after the improvement, in (a) the inset image shows that the network before the improvement did not detect the occluded person, while the improved model, as shown in (b) the inset image, can identify the person, fully demonstrating the adaptability of our model. Overall, this improved YOLOv8 network structure, by optimizing the network architecture, not only improves the accuracy of identifying unsafe behaviors in coal mines but also accelerates the network convergence speed, providing strong technical support for ensuring safe production in coal mines.

[0142] Furthermore, in the stage of accurately identifying unsafe behaviors of coal miners using the trained model, we can further optimize and expand it in several aspects: For example, the model was fine-tuned based on the behavioral characteristics of coal miners and actual working scenarios to improve the accuracy of identifying specific unsafe behaviors; data augmentation technology was used to improve image quality and content visibility, especially in complex environments such as low light and high dust, enhancing the robustness of the model; in terms of system deployment, a hardware architecture of "front-end recognition + back-end analysis" was constructed, selecting appropriate cameras and edge devices according to the environmental conditions of different underground locations, and monitoring the behavior of coal miners in real time through an intelligent monitoring system, issuing an alarm and recording the violation immediately upon detection of unsafe behavior; the unsafe behavior identification system can also be combined with the existing safety management system of the coal mine to form an intelligent dual prevention mechanism, ensuring that workers comply with safety procedures by linking the personnel positioning system and the violation assessment mechanism, thereby effectively improving the intelligent level of coal mine safety production.

[0143] This approach improves the detection of unsafe miner behavior by incorporating the SPPE architecture, DCNv4 module, simplified BiFPN structure, multi-path residual connection module, and MPDIoU loss function, combined with the YOLOv8 model. This significantly enhances the accuracy and robustness of the detection. The SPPE architecture effectively captures multi-scale features, the DCNv4 module improves adaptability to deformable targets, the BiFPN structure optimizes multi-scale feature fusion, residual connections ensure stable information transmission, and the MPDIoU loss function provides more accurate bounding box regression in complex scenes, thus enhancing the model's detection capabilities in complex environments.

[0144] In other embodiments, a convolutional neural network (CNN) architecture can be used instead of the SPPE architecture for feature extraction via multi-scale convolutional kernels; dilated convolutions can be used instead of the DCNv4 module to expand the receptive field and adapt to irregular targets; an improved FPN architecture can also replace the simplified BiFPN to ensure multi-scale information fusion; meanwhile, traditional IoU or GIoU loss functions can still effectively perform bounding box regression. Although these alternatives can meet basic requirements, they may not be as robust and accurate as those in the embodiments of this application when dealing with fine features and variable targets in complex scenes, especially in cases of complex object shapes and occlusion, where they cannot provide accurate boundary predictions like MPDIoU.

[0145] Example 2

[0146] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement an intelligent behavior detection method for miner safety as provided in the above method embodiments.

[0147] Example 3

[0148] This application embodiment also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program segment related to implementing the intelligent behavior detection method for miner safety provided in the method embodiment. The at least one instruction or at least one program segment is loaded and executed by the processor to implement the intelligent behavior detection method for miner safety provided in the above method embodiment. Optionally, in this embodiment, the storage medium can be located in at least one of multiple network servers in a computer network. Optionally, in this embodiment, the storage medium can include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0149] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined by the scope of the claims.

Claims

1. A smart behavior detection method for miner safety, characterized in that, include, S1, the video data is sampled in segments, and the sampled video data is labeled with categories, dividing it into training set, validation set and test set; S2, a target detection model is constructed based on the YOLOv8 model. The target detection model includes an input layer, a backbone network, a neck network, and a head network. In the neck network, the number of feature fusion layers of the BiFPN structure is simplified, and an improved MPDIoU* loss function is introduced into the head network for feature recognition. S3. The improved model is trained using the training set and validation set to obtain the trained model. The trained model is then used to identify unsafe behaviors of coal miners. In step S2, the neck network extracts a multi-scale feature map {Fi} from the input data. The BiFPN structure includes three fusion feature layers. In the BiFPN structure, a top-down path and a bottom-up path are used to perform bidirectional feature fusion on the feature map. Then, the feature maps at multiple scales are weighted and fused. Finally, a feature map Y is output through a multi-path residual connection for the abnormal behavior recognition task. The head network specifically includes: Calculate the prediction range IoU; By introducing a nonlinear transformation to improve the gradient, the improved loss function can be defined as: ,in, It is a non-linear function used to amplify gradients. It is a constant; Calculate the point distance penalty term, which includes calculating the square of the distance between the top-left corner of the predicted bounding box and the top-left corner of the ground truth bounding box. The squared distance between the bottom right corner of the predicted bounding box and the bottom right corner of the ground truth bounding box. The sum of the squared width and squared height of the predicted bounding box, and the squared width and squared height of the actual bounding box: ; Calculate MPDIoU: ,in, It predicts the bounding box. It is the true bounding box; The final loss function is: ; In step S1, the aggregated segment sequence is normalized using Softmax, and the final action category probability of the output video is obtained. , among which, T k It is a segment relative to all categories, W is the network parameter, F is the segment returned by the function, and g is the normalization function; In step S2, the backbone network integrates the SPPE architecture and the DCNv4 module. The DCNv4 module is added to the bottom layer of the SPPE architecture to propose a new SPPE_DCNv4 structure. After inputting the image, two overlapping convolution operations are performed in the SPPE architecture. Then, the C2f module is used to further process the feature map. The convolution and C2f modules are then used together to perform multiple downsampling processes on the feature map. Finally, the feature map is extracted through the SPPF_DCNv4 module.

2. The intelligent behavior detection method for miner safety according to claim 1, characterized in that, In step S1, the input video is evenly divided into k time periods, each time period covering different parts of the video. Then, a short segment is randomly selected from each time period to obtain a segment sequence.

3. The intelligent behavior detection method for miner safety according to claim 1, characterized in that, The top-down path is represented as follows: The bottom-up path is represented as follows: .

4. The intelligent behavior detection method for miner safety according to claim 1, characterized in that, In the head network, an improved MPDIoU* loss function is used to locate and define the target, thereby improving the bounding box regression accuracy, optimizing the model's boundary prediction capability, and introducing multi-point distance metrics and considering changes in object shape to further enhance the bounding box regression accuracy.

5. A computer device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the intelligent behavior detection method for miner safety as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the intelligent behavior detection method for miner safety as described in any one of claims 1 to 4.

Images