Artificial intelligence identification method for unsafe behaviors of mine underground personnel

Abstract

The invention discloses an artificial intelligence identification method for unsafe behaviors of mine underground personnel, and the method comprises the steps: constructing a classification system of the unsafe behaviors, and determining the specific expression forms of all types of behaviors; the unsafe behaviors comprise article type unsafe behaviors, action type unsafe behaviors and area type unsafe behaviors; acquiring a to-be-processed target data set according to the processing flow of the optimized data set; wherein the optimized processing flow comprises data acquisition diversification and pertinence processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing and data set labeling processing; constructing a recognition model according to the target data set; and identifying the to-be-identified data according to the identification model, and determining the unsafe behavior of the underground personnel of the mine. According to the embodiment of the invention, unsafe behaviors of underground coal mine personnel can be quickly and effectively identified, the safety production level of a coal mine is improved, accidents are reduced, and the method can be widely applied to the technical field of artificial intelligence.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence recognition method for unsafe behaviors of personnel in underground mines. Background Art

[0002] Coal miners need to perform complex operations in a small space. These operations not only require a high degree of professional knowledge and skills, but also require miners to be vigilant at all times and avoid unsafe behaviors to ensure the safety of themselves and others. However, in actual work, due to the particularity of the coal mine environment and the high pressure of work, unsafe behaviors of personnel occur from time to time, which may lead to accidents, causing serious injuries or even loss of life. At present, the identification of unsafe behaviors of personnel in coal mines mainly relies on manual inspections and surveillance video playback. These methods have problems such as low efficiency, strong subjectivity, poor accuracy, high cost, and difficulty in covering the entire operating area. Therefore, how to effectively identify and prevent unsafe behaviors of underground personnel and improve the level of coal mine safety production is a key issue that needs to be solved urgently.

[0003] Traditional means of identifying unsafe behaviors of underground coal mine personnel mainly rely on manual inspection and surveillance video playback. Manual inspection requires a lot of manpower and material resources. Inspectors work for a long time in a complex underground environment. Not only is the labor intensity high, but also it is limited by people's attention and energy, which is prone to omissions and difficult to achieve comprehensive and continuous monitoring. At the same time, manual judgment is highly subjective, and different inspectors may have different judgment criteria for the same behavior, which makes it difficult to ensure the accuracy and consistency of the recognition results. Although surveillance video playback can assist post-analysis to a certain extent, this method is inefficient and requires a lot of time to check video records one by one, making it difficult to detect and stop unsafe behaviors in progress in a timely manner. In addition, traditional methods have limitations in coverage and cannot monitor various operating areas underground in real time and in an all-round manner, making it difficult for some potential safety hazards to be detected in a timely manner.

[0004] With the rapid development of modern science and technology, machine vision and computer technology have been widely used in various fields. These technologies have also been gradually introduced into the field of coal mine safety to solve the problem of identifying unsafe behaviors of personnel. However, the existing recognition methods based on machine vision still face many challenges in the complex underground environment of coal mines. The underground environment of coal mines is characterized by low illumination, high dust content, and frequent equipment vibration. These factors lead to the decline of monitoring video image quality, noise interference, blurring, and other problems, which seriously affect the accuracy of behavior recognition. For example, low illumination noise will cause granular noise in the image, reduce image clarity, and make it difficult to accurately identify the outlines of people and objects; dust particle interference will cause image loss and distortion, resulting in non-uniform spots, which interfere with the judgment of behavior; dynamic blur and distortion caused by equipment vibration will make it difficult to capture the details of personnel actions, affecting the extraction of behavior features.

[0005] At the algorithm level, although deep learning algorithms have made some progress in the field of behavior recognition, they are still insufficient in their application in coal mines. Although some traditional machine learning algorithms (such as SVM, KNN, decision tree and random forest) are simple and highly interpretable, they have limited capabilities in processing complex behavior patterns and multidimensional data, and cannot meet the needs of accurate identification of unsafe behaviors of personnel in complex environments in coal mines. Deep learning convolutional neural networks (such as YOLO, Faster R-CNN, etc.) show high efficiency in image feature extraction and behavior recognition, but the demand for large amounts of training data and computing resources has become a bottleneck in their application. Recurrent neural networks (RNN / LSTM) are good at processing time series data and can capture the temporal dependencies in unsafe behaviors, but they have the challenges of gradient vanishing and high training complexity. The spatiotemporal graph convolutional network (ST-GCN) has advantages in human skeleton-based behavior recognition by integrating time and space information, but it has high computational complexity and requires specialized hardware support. The method of combining human posture estimation with behavior recognition (such as OpenPose+LSTM) can use skeleton information to reduce background interference in fine-grained behavior detection, but the accuracy of posture estimation is greatly affected by factors such as occlusion and ambient lighting, and its application in the complex environment of actual coal mines is limited. Although deep reinforcement learning (such as DQN and DDPG) is highly adaptive and can gradually optimize the recognition strategy, the training time is too long and it is difficult to meet the requirements of real-time safety monitoring in coal mines.

[0006] In summary, the existing unsafe behavior identification technology for underground coal mine personnel still has many problems in terms of accuracy, efficiency, and adaptability. Summary of the invention

[0007] The main purpose of the embodiment of the present invention is to propose an artificial intelligence identification method for unsafe behaviors of underground mine personnel, which can quickly and effectively identify unsafe behaviors of underground mine personnel, improve the safety level of coal mine production, and reduce the occurrence of accidents.

[0008] To achieve the above purpose, an embodiment of the present invention provides an artificial intelligence identification method for unsafe behaviors of personnel in mines, comprising the following steps:

[0009] Construct a classification system for unsafe behaviors and determine the specific manifestations of various behaviors; the unsafe behaviors include object-related unsafe behaviors, action-related unsafe behaviors, and area-related unsafe behaviors;

[0010] According to the processing flow of the optimized data set, the target data set to be processed is collected; wherein the optimized processing flow includes data collection diversification and targeted processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing, and data set annotation processing;

[0011] According to the target data set, the model training process is optimized by using the improved YOLOv11 algorithm to build a recognition model;

[0012] The data to be identified is identified according to the identification model to determine the unsafe behavior of personnel in the mine.

[0013] In some embodiments, the classification system for constructing unsafe behaviors is specifically as follows: unsafe behaviors of underground personnel are divided into three categories: object category, action category and area category; among them, object category unsafe behaviors are used to characterize unsafe behaviors such as improper use of miners' personal protective equipment and improper storage of tools and items; action category unsafe behaviors are used to characterize unsafe behaviors corresponding to workers' weak safety awareness and neglect of underground work procedures; area category unsafe behaviors are used to characterize unsafe behaviors corresponding to workers' fluke mentality, lack of sense of responsibility and neglect of safety warnings.

[0014] In some embodiments, before the step of building the recognition model, a step of preprocessing the target data set is also included, and the step includes:

[0015] Video data was collected from underground coal mine monitoring videos and related data sources, and the video data was converted into a unified format, specifically: the resolution width was adjusted to 432 pixels, the height was 368 pixels, the frame rate was 30 Hz, the contrast was 60, the saturation was 60, and the interval frame duration was 0.01;

[0016] Perform augmentation operations on the original downhole data set and the laboratory simulation data set, wherein the augmentation operations include black-and-white processing to change the light intensity and data flipping to simulate different viewing angles; after the augmentation operations, filter out low-quality or non-compliant data to construct a self-made data set;

[0017] To address the low-light noise, dust particle interference and equipment vibration blur in underground monitoring videos, the DnCNN denoising algorithm based on deep learning is used for denoising;

[0018] Based on the real images of underground personnel, a dataset containing miners, safety helmets, and human skeleton data was created by increasing contrast, reducing brightness, and flipping the images.

[0019] In some embodiments, the optimization of the model training process by using the improved YOLOv11 algorithm comprises the following steps:

[0020] Constructing a function enhancement module, wherein the function enhancement module is composed of a channel activation module and a spatial attention module in series, and the function enhancement module is used to filter invalid information features and process the input feature map;

[0021] A K-means++ optimized target anchor box is constructed. The processing process of the K-means++ optimized target anchor box includes: data preparation; extracting the target object bounding box size from the data set and normalizing it; initializing the centroid, randomly selecting a bounding box as the first centroid, and selecting the remaining centroids according to the squared distance probability distribution; updating the anchor box, taking the final cluster centroid as the new anchor box and applying the new anchor box to the model.

[0022] In some embodiments, the constructing of the recognition model comprises the following steps:

[0023] Based on the improved YOLOv11 algorithm, the PyTorch deep learning framework was selected to design a network architecture containing multiple convolutional layers, pooling layers, and activation functions, and CSPDarknet was used as the backbone network;

[0024] Combined with the FPN structure, a 640×640 pixel image is input and standardized preprocessed;

[0025] The output of the detection head includes category prediction and bounding box regression. Three anchor boxes are set. The sizes of the anchor boxes are (10, 13), (16, 30), and (33, 23) according to the statistics of the dataset. The YOLO loss function is adopted, which consists of positioning loss, classification loss, and confidence loss. The Adam optimizer is used, and the initial learning rate is 0.001.

[0026] Prepare a well-labeled dataset, use transfer learning to start training from pre-trained weights, use cross entropy loss function to evaluate classification, and use smooth L1 loss to optimize bounding box regression; apply data augmentation techniques to enhance the dataset; use mini-batch gradient descent or Adam optimizer;

[0027] During the training process, the best hyperparameters were determined through grid search or Bayesian optimization. The batch size was set to 16, the number of training rounds was 50, and each round had about 1000 iterations. The learning rate decay strategy was used to halve the learning rate after every 10 epochs. Based on the early stopping method, the training was stopped when the validation set loss did not improve for 5 consecutive epochs, and the relevant model weight files obtained from the training were saved.

[0028] The performance of the model is measured by mAP and detection rate, and a comprehensive evaluation is performed by calculating the precision, recall and average precision AP to construct the recognition model that meets the evaluation requirements.

[0029] In some embodiments, during the augmentation operation, the number of samples of the self-made data set totals 31,000 images, including 9,700 items, 15,800 actions, and 6,500 areas; the specific distribution of the number of underground and laboratory samples of various unsafe behaviors is as follows: 3,200 underground samples and 2,900 laboratory samples for not wearing a helmet; 2,100 underground samples and 1,500 laboratory samples for improper storage of items and tools; 1,500 underground samples and 1,600 laboratory samples for falling; 3,100 underground samples and 2,500 laboratory samples for riding on belts; 2,600 underground samples and 2,100 laboratory samples for climbing guardrails; 1,300 underground samples and 1,100 laboratory samples for illegal carjacking; 1,500 underground samples and 1,300 laboratory samples for invading dangerous areas; 1,500 underground samples and 1,200 laboratory samples for leaving work posts without permission;

[0030] In the process of data denoising, the DnCNN network has a total of seventeen layers. The first layer consists of a convolutional layer ReLU activation function composed of 64 convolution kernels of size 3×3; the second to sixteenth layers have more batch normalization layers than the first layer; the last layer consists of a convolutional layer, which is responsible for outputting the residual image of the image.

[0031] In some embodiments, the function enhancement module is composed of a channel activation module CAM and a spatial attention module SAM connected in series, and is placed after the concat operation in the YOLOv11 model; the channel activation module is used to assign weights to each channel of the input feature map; the spatial attention module is used to supplement the channel activation module, by performing max-pooling and average-pooling operations on the feature map of size H×H×C, respectively, and then connecting the two feature maps and performing convolution and nonlinear activation to obtain the weight coefficients of each part, and finally multiplying the weight coefficients by the input feature image to obtain a scaled new feature map.

[0032] Another aspect of the embodiment of the present invention further provides an artificial intelligence recognition system for unsafe behaviors of personnel in mines, including:

[0033] The first module is used to construct a classification system for unsafe behaviors and determine the specific manifestations of various behaviors; the unsafe behaviors include object-related unsafe behaviors, action-related unsafe behaviors, and area-related unsafe behaviors;

[0034] The second module is used to collect the target data set to be processed according to the processing flow of the optimized data set; wherein the optimized processing flow includes data collection diversification and targeted processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing, and data set annotation processing;

[0035] The third module is used to optimize the model training process and build a recognition model based on the target data set through an improved YOLOv11 algorithm;

[0036] The fourth module is used to identify the data to be identified according to the recognition model and determine the unsafe behavior of personnel underground in the mine. In order to achieve the above-mentioned purpose, another aspect of an embodiment of the present invention proposes an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the method described above when executing the computer program.

[0037] To achieve the above objective, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method is implemented.

[0038] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0039] The embodiments of the present invention include at least the following beneficial effects: the present invention provides an artificial intelligence identification method for unsafe behaviors of personnel in underground mines, and the scheme determines the specific manifestations of various behaviors by constructing a classification system of unsafe behaviors; the unsafe behaviors include unsafe behaviors of objects, unsafe behaviors of actions, and unsafe behaviors of regions; according to the processing flow of the optimized data set, the target data set to be processed is collected; wherein the optimized processing flow includes diversified and targeted data collection processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing, and data set annotation processing; according to the target data set, the model training process is optimized by the improved YOLOv11 algorithm to construct a recognition model; according to the recognition model, the data to be identified is identified to determine the unsafe behaviors of personnel in underground mines. The embodiments of the present invention can quickly and effectively identify unsafe behaviors of personnel in underground coal mines, improve the safety level of coal mine production, and reduce accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

[0041] Figure 2 is a flow chart of the overall steps provided by an embodiment of the present invention;

[0042] Figure 3 is an example diagram of an augmented downhole data set provided by an embodiment of the present invention;

[0043] Figure 4 It is a DnCNN network model architecture diagram provided by an embodiment of the present invention;

[0044] Figure 5 is an example diagram of data set annotation provided by an embodiment of the present invention;

[0045] Figure 6 It is a YOLOv11 network structure diagram provided by an embodiment of the present invention;

[0046] Figure 7 is a structural diagram of a function enhancement module provided by an embodiment of the present invention;

[0047] Figure 8 It is a flow chart of the unsafe behavior identification model for coal mine personnel provided by an embodiment of the present invention;

[0048] Fig. 9 This is a diagram showing the effect of helmet wearing detection provided by an embodiment of the present invention;

[0049] Fig.10 It is a hotspot map of key parts of the human body and their visual attention provided by an embodiment of the present invention;

[0050] Fig.11 This is a skeleton recognition result diagram of a worker walking and working, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0052] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0053] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0055] The artificial intelligence identification method for unsafe behaviors of underground personnel in mines provided by the embodiment of the present invention relates to the field of artificial intelligence technology. The artificial intelligence identification method for unsafe behaviors of underground personnel in mines provided by the embodiment of the present invention can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a vehicle-mounted terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application of an artificial intelligence identification method for unsafe behaviors of underground personnel in mines, etc., but is not limited to the above forms.

[0056] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0057] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0058] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0059] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0060] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal 102 may also be a vehicle-mounted terminal of various device types described above, but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the present invention.

[0061] Based on the example Figure 1 In the implementation environment shown, an embodiment of the present invention provides an artificial intelligence identification method for unsafe behaviors of personnel underground in a mine. The following is explained using the application of the artificial intelligence identification method for unsafe behaviors of personnel underground in a mine in a server 101 as an example. It can be understood that the method can also be applied to the terminal 102.

[0062] Reference Figure 2 , Figure 2 The flowchart of the artificial intelligence identification method for unsafe behavior of underground mine personnel applied to the server provided in the embodiment of the present invention, the execution subject of the method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 , the method may include the following steps:

[0063] Construct a classification system for unsafe behaviors and determine the specific manifestations of various behaviors; the unsafe behaviors include object-related unsafe behaviors, action-related unsafe behaviors, and area-related unsafe behaviors;

[0064] According to the processing flow of the optimized data set, the target data set to be processed is collected; wherein the optimized processing flow includes data collection diversification and targeted processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing, and data set annotation processing;

[0065] According to the target data set, the model training process is optimized by using the improved YOLOv11 algorithm to build a recognition model;

[0066] The data to be identified is identified according to the identification model to determine the unsafe behavior of personnel in the mine.

[0067] In some embodiments, the classification system for constructing unsafe behaviors is specifically as follows: unsafe behaviors of underground personnel are divided into three categories: object category, action category and area category; among them, object category unsafe behaviors are used to characterize unsafe behaviors such as improper use of miners' personal protective equipment and improper storage of tools and items; action category unsafe behaviors are used to characterize unsafe behaviors corresponding to workers' weak safety awareness and neglect of underground work procedures; area category unsafe behaviors are used to characterize unsafe behaviors corresponding to workers' fluke mentality, lack of sense of responsibility and neglect of safety warnings.

[0068] In some embodiments, before the step of building the recognition model, a step of preprocessing the target data set is also included, and the step includes:

[0069] Video data was collected from underground coal mine monitoring videos and related data sources, and the video data was converted into a unified format, specifically: the resolution width was adjusted to 432 pixels, the height was 368 pixels, the frame rate was 30 Hz, the contrast was 60, the saturation was 60, and the interval frame duration was 0.01;

[0070] Perform augmentation operations on the original downhole data set and the laboratory simulation data set, wherein the augmentation operations include black-and-white processing to change the light intensity and data flipping to simulate different viewing angles; after the augmentation operations, filter out low-quality or non-compliant data to construct a self-made data set;

[0071] To address the low-light noise, dust particle interference and equipment vibration blur in underground monitoring videos, the DnCNN denoising algorithm based on deep learning is used for denoising;

[0072] Based on the real images of underground personnel, a dataset containing miners, safety helmets, and human skeleton data was created by increasing contrast, reducing brightness, and flipping the images.

[0073] In some embodiments, the optimization of the model training process by using the improved YOLOv11 algorithm comprises the following steps:

[0074] Constructing a function enhancement module, wherein the function enhancement module is composed of a channel activation module and a spatial attention module in series, and the function enhancement module is used to filter invalid information features and process the input feature map;

[0075] A K-means++ optimized target anchor box is constructed. The processing process of the K-means++ optimized target anchor box includes: data preparation; extracting the target object bounding box size from the data set and normalizing it; initializing the centroid, randomly selecting a bounding box as the first centroid, and selecting the remaining centroids according to the squared distance probability distribution; updating the anchor box, taking the final cluster centroid as the new anchor box and applying the new anchor box to the model.

[0076] In some embodiments, the constructing of the recognition model comprises the following steps:

[0077] Based on the improved YOLOv11 algorithm, the PyTorch deep learning framework was selected to design a network architecture containing multiple convolutional layers, pooling layers, and activation functions, and CSPDarknet was used as the backbone network;

[0078] Combined with the FPN structure, a 640×640 pixel image is input and standardized preprocessed;

[0079] The output of the detection head includes category prediction and bounding box regression. Three anchor boxes are set. The sizes of the anchor boxes are (10, 13), (16, 30), and (33, 23) according to the statistics of the dataset. The YOLO loss function is adopted, which consists of positioning loss, classification loss, and confidence loss. The Adam optimizer is used, and the initial learning rate is 0.001.

[0080] Prepare a well-labeled dataset, use transfer learning to start training from pre-trained weights, use cross entropy loss function to evaluate classification, and use smooth L1 loss to optimize bounding box regression; apply data augmentation techniques to enhance the dataset; use mini-batch gradient descent or Adam optimizer;

[0081] During the training process, the best hyperparameters were determined through grid search or Bayesian optimization. The batch size was set to 16, the number of training rounds was 50, and each round had about 1000 iterations. The learning rate decay strategy was used to halve the learning rate after every 10 epochs. Based on the early stopping method, the training was stopped when the validation set loss did not improve for 5 consecutive epochs, and the relevant model weight files obtained from the training were saved.

[0082] The performance of the model is measured by mAP and detection rate, and a comprehensive evaluation is performed by calculating the precision, recall and average precision AP to construct the recognition model that meets the evaluation requirements.

[0083] In some embodiments, during the augmentation operation, the number of samples of the self-made data set totals 31,000 images, including 9,700 items, 15,800 actions, and 6,500 areas; the specific distribution of underground and laboratory samples of various unsafe behaviors is as follows: 3,200 underground samples and 2,900 laboratory samples for not wearing a helmet; 2,100 underground samples and 1,500 laboratory samples for improper storage of items and tools; 1,500 underground samples and 1,600 laboratory samples for falling; 3,100 underground samples and 2,500 laboratory samples for riding on belts; 2,600 underground samples and 2,100 laboratory samples for climbing guardrails; 1,300 underground samples and 1,100 laboratory samples for illegal carjacking; 1,500 underground samples and 1,300 laboratory samples for invading dangerous areas; 1,500 underground samples and 1,200 laboratory samples for leaving work posts without permission;

[0084] In the process of data denoising, the DnCNN network has a total of seventeen layers. The first layer consists of a convolutional layer ReLU activation function composed of 64 convolution kernels of size 3×3; the second to sixteenth layers have more batch normalization layers than the first layer; the last layer consists of a convolutional layer, which is responsible for outputting the residual image of the image.

[0085] In some embodiments, the function enhancement module is composed of a channel activation module CAM and a spatial attention module SAM connected in series, and is placed after the concat operation in the YOLOv11 model; the channel activation module is used to assign weights to each channel of the input feature map; the spatial attention module is used to supplement the channel activation module, by performing max-pooling and average-pooling operations on the feature map of size H×H×C, respectively, and then connecting the two feature maps and performing convolution and nonlinear activation to obtain the weight coefficients of each part, and finally multiplying the weight coefficients by the input feature image to obtain a scaled new feature map.

[0086] The following is an example of a specific application scenario. Figure 8 , combined with the flow chart of the unsafe behavior identification model of coal mine personnel, the specific implementation process of the embodiment of the present invention is described in detail:

[0087] 1. Construction of unsafe behavior classification system

[0088] Through in-depth analysis of the work processes, operating specifications and possible risk factors of underground coal mine personnel, combined with a large number of actual cases and safety standards, the unsafe behaviors of underground personnel are systematically divided into three categories: object, action and area.

[0089] Unsafe behaviors related to items focus on the improper use of personal protective equipment by miners and improper storage of tools and items. For example, not wearing a hard hat will greatly increase the possibility of head injury in an environment with falling objects and collision risks underground; improper storage of items and tools may cause safety accidents such as falls and collisions, and it is difficult to quickly obtain the required tools in an emergency, affecting the efficiency of emergency response.

[0090] Unsafe behaviors in the form of movements are mainly caused by the workers’ weak safety awareness and neglect of underground working procedures. Falling is easy to happen when working on unstable or slippery ground, which may cause injuries to miners. Taking informal transportation methods such as belts, climbing guardrails, and illegally riding on vehicles are serious violations of safety regulations, greatly increasing the risk of miners being injured or even endangering their lives.

[0091] Regional unsafe behaviors are usually related to workers' fluke mentality, lack of sense of responsibility, and neglect of safety warnings. Intrusion into dangerous areas means that miners enter areas marked as dangerous or prohibited without permission, and may face unknown safety risks, such as gas leakage areas and unstable roof areas; leaving the work station without authorization may cause the workflow to be interrupted, affecting production efficiency, and also make the area lose safety monitoring, increasing the possibility of safety accidents.

[0092] 2. Optimization of data set processing flow

[0093] (1) Diversification and targeting of data collection

[0094] The surveillance videos of key locations underground in coal mines (such as cutting faces, transport lanes, etc.) are determined as the main data source. These areas have frequent personnel activities and high operational risks. The collected video data can effectively reflect the actual working status and behavioral performance of underground personnel.

[0095] With the help of underground workers, explosion-proof mobile phones are used to record the behavior of people in specific scenarios to supplement the blind spots and missing details that may exist in the surveillance video. At the same time, a wide range of public data sets in non-mine scenarios (such as COCO, SHWD, Hard Workers Dataset, Safety Helmet Detection and Helmet Dataset, etc.) are collected to enhance the model's generalization ability for different scenarios and behavior patterns by using their rich sample diversity. In addition, a laboratory environment is used to simulate the underground mine scene and construct an auxiliary data set to verify and optimize the performance of the model under specific conditions.

[0096] (2) Data format conversion and parameter adjustment

[0097] Using OpenCV, a powerful open source computer vision library, the collected video data in various formats (such as AVI, MP4 or MPEG, etc.) are converted into a format that is easy to process later. During the conversion process, the video resolution width is precisely adjusted to 432 pixels, the height is 368 pixels, and the frame rate is set to 30Hz. By reasonably reducing the resolution and frame rate, the amount of data per frame is significantly reduced without losing key visual information, thereby improving data processing efficiency and reducing storage space. At the same time, according to the characteristics of the underground monitoring environment, the contrast is adjusted to 60 and the saturation is adjusted to 60 to optimize the image visual effect and enhance the recognition of people and objects in the image. The interval frame duration is set to 0.01 to ensure that sufficient feature information can be obtained when extracting images and the increase in workload caused by image duplication can be avoided.

[0098] (3) Dataset augmentation improves diversity

[0099] A variety of data augmentation operations are performed on the original and laboratory simulation data sets. The black-and-white processing technology is used to simulate the change of underground light conditions from color to black-and-white, enhancing the model's ability to recognize personnel behavior under different lighting environments; the data flipping technology is used to rotate the original image 180 degrees, so that the model can recognize unsafe personnel behavior from different perspectives and improve the robustness of the model.

[0100] After the augmentation operation is completed, the data set is strictly screened. Detailed data quality assessment standards are formulated to exclude data with unclear images, missing key information, incorrect annotations, or data irrelevant to the research purpose, to ensure the high quality and reliability of the data set and provide a solid data foundation for subsequent model training.

[0101] (4) Dataset denoising to improve image quality

[0102] In-depth research on the noise problems faced by surveillance video images in underground coal mine environments identified low-light noise, dust particle interference, and blur caused by equipment vibration as the main types of noise. To address these issues, the DnCNN denoising algorithm based on deep learning was used.

[0103] The core principle of the DnCNN algorithm is to build a deep convolutional neural network and learn the relationship between a large number of noisy and noise-free image pairs, so as to accurately predict the noise component in the noisy image (i.e., the residual between the noisy image and the clean image), and then subtract the residual from the noisy image to achieve the purpose of denoising. Its network structure consists of 17 layers. The first layer is composed of 64 convolutional kernels of size 3×3, combined with the ReLU activation function, which is responsible for the preliminary extraction of image features; the second to the sixteenth layers add batch normalization layers on the basis of the first layer, which effectively solves the problem of internal covariate shift, speeds up model training and improves stability; the last layer is a convolutional layer, which is used to output the residual image of the image. When training the DnCNN network, the mean square error (MSE) is selected as the loss function. By calculating the difference between the residual map predicted by the network and the actual residual map, the network parameters are continuously adjusted so that the model can accurately remove the noise in the image, improve the image clarity and contrast, and provide clear and accurate image data for subsequent behavior recognition.

[0104] (5) Dataset annotation to ensure accuracy

[0105] According to the detection objectives and network model requirements, a data set containing miners, safety helmets, human skeleton data, etc. was carefully prepared to ensure that all data sources are real images of underground personnel to ensure the authenticity and representativeness of the data.

[0106] A series of preprocessing operations were performed on the 300 original images collected, including increasing contrast, reducing brightness, flipping, etc., to expand the data set to 780 images, and then divided into training set, validation set and test set according to the ratio of 70% (546 images), 20% (156 images) and 10% (78 images), to ensure the rationality of each data set in terms of sample quantity and distribution, so as to effectively evaluate the performance of the model.

[0107] Label Img, an open source data annotation tool widely used in the field of machine vision, is used for annotation. The tool supports a variety of common image formats (such as JPEG, PNG, etc.) and can export a variety of annotation files (such as XML and TXT, etc.) to meet the needs of different machine learning models. During the annotation process, precise labels are assigned to objects or behaviors in the image according to different types of unsafe behaviors, such as labeling the absence of a helmet as "no-hat", labeling the wearing of a helmet as "hat", and other unsafe behaviors in the same way, ensuring that each label accurately points to the corresponding object or behavior, providing accurate training data for the model, and improving the accuracy and efficiency of model recognition.

[0108] 3. Identify model building and improvement strategies

[0109] (1) Deep Improvement of YOLOv11 Algorithm

[0110] ① Functional enhancement module design and integration

[0111] Considering the impact of complex background interference such as occlusion or long-distance targets in the complex underground environment on recognition accuracy, the function enhancement module (FEM) is innovatively designed and the YOLOv11 model is introduced. Figure 7 ,This module is composed of a channel activation module (CAM) and a spatial attention module (SAM) in series, and is placed after the "concat" operation in the YOLOv11 model. It aims to effectively filter out features containing invalid information and indirectly improve the utilization of effective information, thereby improving the model's ability to recognize important areas in complex backgrounds.

[0112] The channel activation module assigns weights to each channel of the input feature map, highlights important features and suppresses unimportant features, removes channels containing a lot of invalid information, reduces redundant information, and increases the percentage of effective information; the spatial attention module complements the channel activation module by performing max-pooling and average-pooling operations on feature maps of size H×H×C, then connecting the two feature maps and performing convolution and nonlinear activation to obtain the weight coefficients of each part, and finally multiplying the weight coefficients by the input feature image to obtain a scaled new feature map, thereby "focusing on" the most meaningful local features and further improving the model's ability to capture key information in complex backgrounds. Its calculation process accurately processes the input feature map according to a specific formula to ensure the effective implementation of the module function.

[0113] ②K-means++ algorithm optimizes anchor boxes

[0114] In view of the influence of factors such as the posture of personnel in coal mines, the distance and angle from the camera on the detection effect, the K-means++ algorithm is used to optimize the anchor frame in the YOLOv11 algorithm. The K-means++ algorithm improves the initial centroid selection, increases the diversity of initialization, effectively reduces the probability of falling into the local optimal solution, and can better adapt to the shape and size distribution of objects in the coal mine dataset.

[0115] The specific implementation process includes four key steps: first, data preparation is performed to extract the bounding box sizes of all target objects from the coal mine underground data set, and normalization is performed to eliminate the influence of image size on anchor box selection; then the centroid operation is initialized, a bounding box is randomly selected as the first centroid, and then the remaining centroids are selected one by one according to the square probability distribution of the distance from each data point to the selected centroid; then clustering iteration is performed, using Euclidean distance as the metric, the bounding box is assigned to the nearest centroid, and the centroid of each cluster is updated to the mean position of all points in the cluster, and this process is repeated until convergence (that is, the change in the centroid is less than the preset threshold or reaches the predetermined number of iterations); finally, the anchor box is updated, and the final cluster centroid, that is, the average width and height of the bounding box in each cluster, is applied to the YOLOv11 model as a new optimized anchor box, which significantly improves the model's detection accuracy and efficiency for human targets at different postures, distances and angles.

[0116] (2) Refined management of model construction and training

[0117] ①Model building architecture design

[0118] Based on the improved YOLOv11 algorithm, a model for identifying unsafe behaviors of underground personnel in coal mines was built, and PyTorch, which has superior performance and is widely used, was selected as the deep learning framework to ensure the efficiency and stability of model construction and training.

[0119] The network architecture is carefully designed, including multiple convolutional layers, pooling layers and activation functions. The layers work together to effectively extract feature information from the image. CSPDarknet is used as the feature extraction backbone network. Its unique structural design can significantly enhance the feature extraction capability and better capture the key features in the image. Combined with the feature pyramid network (FPN) structure, the features at different levels are fused to effectively improve the detection accuracy of small objects and ensure the accurate recognition of small targets such as small tools worn by underground personnel or local details of the body.

[0120] The model input layer sets the image size to 640×640 pixels, and uses standardized preprocessing operations to make the input data meet the model training requirements and improve the model training effect. The detection head is designed to output category prediction and bounding box regression results. According to the category characteristics of unsafe behaviors of coal mine personnel, three anchor boxes are set. Their sizes are determined to be (10,13), (16,30), and (33,23) after detailed statistical analysis of the data set, which can adapt to the detection needs of targets of different sizes.

[0121] The YOLO loss function is selected, which consists of three parts: positioning loss, classification loss, and confidence loss. It comprehensively evaluates the difference between the model prediction results and the true labels, and provides accurate optimization direction for model training. The Adam optimizer is used, which has an adaptive learning rate adjustment function. It can dynamically adjust the learning rate according to the gradient changes during the training process to accelerate the model convergence speed. The initial learning rate is set to 0.001, which enables the model to quickly learn data features in the early stage of training, and gradually reduce the learning rate in the later stage to ensure the stability and accuracy of model training.

[0122] ②Model training strategy optimization

[0123] Prepare a carefully labeled data set, use transfer learning technology, start training the model from the pre-trained weights, and use the common features learned by the pre-trained model on large-scale data to accelerate the convergence of the model on the coal mine underground personnel behavior data set, while improving the initial accuracy of the model.

[0124] During the training process, the cross entropy loss function is used to evaluate the classification effect, accurately measure the difference between the model predicted category and the true category, and guide the model to optimize the classification decision; the smooth L1 loss is used to optimize the bounding box regression, so that the model can more accurately predict the position and size of the target object.

[0125] A variety of data enhancement techniques are widely used, such as random cropping, rotation, scaling and color transformation, to simulate the complex and changeable environmental conditions underground, increase the diversity of the data set, improve the model's generalization ability for human behavior in different environments, and ensure that the model can maintain good recognition performance in practical applications.

[0126] The training process uses mini-batch gradient descent (SGD) or Adam optimizer, and the best hyperparameters are determined through grid search or Bayesian optimization. For example, the batch size is set to 16 to balance memory consumption and training speed while ensuring the stability of model training; the number of training rounds is set to 50, with about 1,000 iterations per round, so that the model can fully learn the data characteristics. The learning rate adjustment strategy uses the method of halving the learning rate after every 10 epochs, which converges quickly in the early stage of training and gradually reduces the learning rate in the later stage to avoid model overfitting and improve the generalization ability of the model. At the same time, the early stopping method is set to stop training when the validation set loss has not improved for 5 consecutive epochs to prevent the model from overtraining and ensure that the model performance reaches the optimal level. TensorBoard is used for monitoring during the training process, and indicators such as loss and accuracy during the training process are visualized in real time, which is convenient for timely adjustment of training parameters and optimization of the training process.

[0127] 4. Establishment of model effectiveness evaluation system

[0128] A comprehensive model effectiveness evaluation system is built, using mAP (Mean Average Precision) and detection rate as core evaluation indicators, and comprehensively considering the performance of the model in different categories of target detection tasks.

[0129] The mAP indicator measures the average detection performance of the model in each category by calculating the average average precision AP of all target classes, where AP is calculated by combining precision P and recall R. The calculation formula for precision P is: It is used to measure the proportion of samples predicted by the model as positive and actually positive, reflecting the accuracy of the model prediction; the calculation formula of recall rate R is: It measures the proportion of samples that are actually positive and correctly predicted as positive by the model, reflecting the coverage of the model for positive samples. By calculating the precision and recall under different thresholds, drawing curves, calculating the area under the curve to get the AP value, and then calculating mAP, the detection performance of the model in different categories is comprehensively evaluated to ensure that the model has high accuracy and comprehensiveness in identifying various types of unsafe behaviors of underground personnel.

[0130] The detection rate is measured by the number of frames processed per second, which directly reflects the real-time performance of the model in practical applications, ensuring that the model can meet the requirements of real-time safety monitoring in coal mines and promptly detect and handle unsafe behaviors of personnel.

[0131] 5. System implementation and overall application architecture construction

[0132] Build a system early warning model for unsafe behaviors of underground coal mine personnel, design a system network architecture that connects the underground and the ground, and achieve seamless connection between underground data collection, processing, and analysis and ground monitoring and management.

[0133] Deploy the trained model on edge devices (such as built-in chips in underground monitoring cameras or dedicated edge computing servers, etc.), make full use of the advantages of edge computing, reduce data transmission delays, and achieve real-time detection of unsafe behaviors of underground personnel. When the model detects unsafe behavior, it immediately triggers the alarm system, and promptly issues an alarm through sound, light, or remote communication to remind underground workers to correct the unsafe behavior. At the same time, the alarm information is transmitted to the ground monitoring center to notify managers to take corresponding measures, such as dispatching personnel to check on site, adjust the operation process, or organize safety training, etc., to effectively prevent the occurrence of safety accidents and ensure the safe and orderly operation of underground coal mines.

[0134] In summary, compared with the prior art, the present invention has the following advantages:

[0135] 1. The classification method of the present invention helps to quickly analyze and deal with unsafe behaviors of underground personnel and improve management efficiency.

[0136] 2. The dataset processing process improves data quality and diversity, and enhances the adaptability and accuracy of the model.

[0137] 3. The improvement of the YOLOv11 algorithm effectively improves the detection efficiency and accuracy, and can better identify various unsafe behaviors.

[0138] 4. The constructed early warning model and system network architecture realize the automation of underground operation safety monitoring and improve the safety production level of coal mines.

[0139] The following is an experimental verification of the method provided by the embodiment of the present invention in conjunction with the accompanying drawings:

[0140] First, according to the unsafe behavior classification standards, common unsafe behaviors in coal mines are classified and sorted, and the specific manifestations of each behavior are clarified.

[0141] During the data collection phase, surveillance videos were retrieved from key locations such as the cutting face and transport tunnel of a mine in Yulin, and supplementary data was captured using explosion-proof mobile phones. At the same time, public data sets and laboratory simulation data sets in non-mine scenarios were collected.

[0142] Use OpenCV to convert the format of the collected video data and adjust the parameters, such as adjusting the resolution to 432×368, setting the frame rate to 30, and setting the contrast and saturation to 60.

[0143] Perform augmentation operations on the dataset, including black and white processing and data flipping, and then screen to ensure data quality, build a self-made dataset and count the number of samples of each type. Figure 3 , Figure 3 The following are some examples of augmented images of the underground data set, including the original image, the black-and-white processed image, and the flipped image.

[0144] The DnCNN algorithm is used to denoise the data set, and the model weights are adjusted through training to improve image clarity. Figure 4 The DnCNN network model provided by the embodiment of the present invention is Figure 4 It presents a multi-layer structure.

[0145] refer to Figure 5 In this embodiment of the present invention, the Label Img tool is used to annotate the data set and accurately mark various unsafe behaviors in the YOLO format. Figure 5 The equipment annotation and personnel annotation interface shown in the figure, where: Figure 5 (a) and (b) show examples of data annotation for different objectives, respectively.

[0146] refer to Figure 6 , the network architecture of the YOLOv11 network is as follows Figure 6As shown in the figure, a recognition model based on the improved YOLOv11 algorithm is constructed, and the network architecture parameters are set, such as using the CSPDarknet backbone network, FPN structure, and setting a suitable anchor box size; the model parameters are configured, such as selecting the Adam optimizer and the initial learning rate is 0.001.

[0147] Prepare training data, divide it into training set, validation set and test set, use transfer learning to train the model, adopt data enhancement technology and adjust the learning rate strategy, and use TensorBoard to monitor the training process.

[0148] After training is completed, the model performance is evaluated on the test set, and indicators such as mAP, precision and recall are calculated to verify the effectiveness of the model.

[0149] The trained model is deployed on edge devices to realize real-time unsafe behavior detection in the actual underground coal mine environment, and alarms are issued in time for detected unsafe behaviors to ensure the safety of underground operations.

[0150] refer to Fig. 9 , Fig. 9 The effect diagram of helmet wearing detection is displayed to verify the detection ability of the model in different situations.

[0151] refer to Fig.10 , Fig.10 Displays the key parts of the human body and their visual attention hotspots to assist in understanding the estimation of key points of the human skeleton.

[0152] refer to Fig.11 , Fig.11 The skeleton recognition results of workers walking and working are shown to illustrate the model's ability to analyze behavioral states.

[0153] Another aspect of the embodiment of the present invention further provides an artificial intelligence recognition system for unsafe behaviors of personnel in mines, including:

[0154] The first module is used to construct a classification system for unsafe behaviors and determine the specific manifestations of various behaviors; the unsafe behaviors include object-related unsafe behaviors, action-related unsafe behaviors, and area-related unsafe behaviors;

[0155] The second module is used to collect the target data set to be processed according to the processing flow of the optimized data set; wherein the optimized processing flow includes data collection diversification and targeted processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing, and data set annotation processing;

[0156] The third module is used to optimize the model training process and build a recognition model based on the target data set through an improved YOLOv11 algorithm;

[0157] The fourth module is used to identify the data to be identified according to the recognition model and determine the unsafe behavior of personnel underground in the mine. In order to achieve the above-mentioned purpose, another aspect of an embodiment of the present invention proposes an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the aforementioned method when executing the computer program.

[0158] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0159] The embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned artificial intelligence identification method of unsafe behaviors of underground mine personnel when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0160] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0161] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned artificial intelligence identification method for unsafe behaviors of personnel underground in a mine.

[0162] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0163] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0164] It should be noted that in various specific embodiments of the present invention, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present invention needs to obtain the user's sensitive personal information, it will obtain the user's separate permission or consent through a pop-up window or jump to a confirmation page, and after clearly obtaining the user's separate permission or consent, it will obtain the necessary user-related data for the normal operation of the embodiment of the present invention.

[0165] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0166] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0167] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0169] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0170] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0171] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0172] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0174] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store programs.

[0175] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the embodiments of the present invention is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of the rights of the embodiments of the present invention.

Claims

1. An artificial intelligence identification method for unsafe behaviors of personnel in mines, characterized in that: The following steps are involved: Construct a classification system for unsafe behaviors and determine the specific manifestations of each type of behavior; The unsafe behaviors include object-related unsafe behaviors, action-related unsafe behaviors, and area-related unsafe behaviors; According to the processing flow of the optimized data set, the target data set to be processed is collected; wherein the optimized processing flow includes data collection diversification and targeted processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing, and data set annotation processing; According to the target data set, the model training process is optimized by using the improved YOLOv11 algorithm to build a recognition model; The data to be identified is identified according to the identification model to determine the unsafe behavior of personnel in the mine.

2. The artificial intelligence identification method for unsafe behaviors of underground mine personnel according to claim 1 is characterized in that: The unsafe behavior classification system is constructed as follows: the unsafe behaviors of underground personnel are divided into three categories: object category, action category and area category; among them, the object category unsafe behaviors are used to characterize the unsafe behaviors of miners who use personal protective equipment improperly and store tools and items improperly; the action category unsafe behaviors are used to characterize the unsafe behaviors corresponding to the staff's weak safety awareness and neglect of underground work procedures; the area category unsafe behaviors are used to characterize the unsafe behaviors corresponding to the staff's fluke mentality, lack of sense of responsibility and neglect of safety warnings.

3. The artificial intelligence identification method for unsafe behaviors of underground mine personnel according to claim 1 is characterized in that: Before the step of building the recognition model, a step of preprocessing the target data set is also included, which step includes: Video data was collected from underground coal mine monitoring videos and related data sources, and the video data was converted into a unified format, specifically: the resolution width was adjusted to 432 pixels, the height was 368 pixels, the frame rate was 30 Hz, the contrast was 60, the saturation was 60, and the interval frame duration was 0.01; Perform augmentation operations on the original downhole data set and the laboratory simulation data set, wherein the augmentation operations include black-and-white processing to change the light intensity and data flipping to simulate different viewing angles; after the augmentation operations, filter out low-quality or non-compliant data to construct a self-made data set; To address the low-light noise, dust particle interference and equipment vibration blur in underground monitoring videos, the DnCNN denoising algorithm based on deep learning is used for denoising; Based on the real images of underground personnel, a dataset containing miners, safety helmets, and human skeleton data was created by increasing contrast, reducing brightness, and flipping the images.

4. The artificial intelligence identification method for unsafe behaviors of underground mine personnel according to claim 1 is characterized in that: The improved YOLOv11 algorithm is used to optimize the model training process, including the following steps: Constructing a function enhancement module, wherein the function enhancement module is composed of a channel activation module and a spatial attention module in series, and the function enhancement module is used to filter invalid information features and process the input feature map; A K-means++ optimized target anchor box is constructed. The processing process of the K-means++ optimized target anchor box includes: data preparation; extracting the target object bounding box size from the data set and normalizing it; initializing the centroid, randomly selecting a bounding box as the first centroid, and selecting the remaining centroids according to the squared distance probability distribution; updating the anchor box, taking the final cluster centroid as the new anchor box and applying the new anchor box to the model.

5. The artificial intelligence identification method for unsafe behaviors of underground mine personnel according to claim 4 is characterized in that: The construction of the recognition model comprises the following steps: Based on the improved YOLOv11 algorithm, the PyTorch deep learning framework was selected to design a network architecture containing multiple convolutional layers, pooling layers, and activation functions, and CSPDarknet was used as the backbone network; Combined with the FPN structure, a 640×640 pixel image is input and standardized preprocessed; The output of the detection head includes category prediction and bounding box regression. Three anchor boxes are set. The sizes of the anchor boxes are (10, 13), (16, 30), and (33, 23) according to the statistics of the dataset. The YOLO loss function is adopted, which consists of positioning loss, classification loss, and confidence loss. The Adam optimizer is used, and the initial learning rate is 0.

001. Prepare a well-labeled dataset, use transfer learning to start training from pre-trained weights, use cross entropy loss function to evaluate classification, and use smooth L1 loss to optimize bounding box regression; apply data augmentation techniques to enhance the dataset; use mini-batch gradient descent or Adam optimizer; During the training process, the best hyperparameters were determined through grid search or Bayesian optimization. The batch size was set to 16, the number of training rounds was 50, and each round had about 1000 iterations. The learning rate decay strategy was used to halve the learning rate after every 10 epochs. Based on the early stopping method, the training was stopped when the validation set loss did not improve for 5 consecutive epochs, and the relevant model weight files obtained from the training were saved. The performance of the model is measured by mAP and detection rate, and a comprehensive evaluation is performed by calculating the precision, recall and average precision AP to construct the recognition model that meets the evaluation requirements.

6. The artificial intelligence identification method for unsafe behaviors of underground mine personnel according to claim 3 is characterized in that: During the augmentation process, the number of samples in the self-made data set totaled 31,000 images, including 9,700 items, 15,800 actions, and 6,500 areas; the specific distribution of underground and laboratory samples of various unsafe behaviors was as follows: 3,200 samples of not wearing a safety helmet underground and 2,900 samples in the laboratory; 2,100 samples of improper storage of items and tools underground and 1,500 samples in the laboratory; 1,500 samples of falling underground and 1,600 samples in the laboratory; 3,100 samples of riding on belts underground and 2,500 samples in the laboratory; 2,600 samples of climbing guardrails underground and 2,100 samples in the laboratory; 1,300 samples of illegal carjacking underground and 1,100 samples in the laboratory; 1,500 samples of invading dangerous areas underground and 1,300 samples in the laboratory; 1,500 samples of leaving the work station without permission underground and 1,200 samples in the laboratory; In the process of data denoising, the DnCNN network has a total of seventeen layers. The first layer consists of a convolutional layer ReLU activation function composed of 64 convolution kernels of size 3×3; the second to sixteenth layers have more batch normalization layers than the first layer; the last layer consists of a convolutional layer, which is responsible for outputting the residual image of the image.

7. The artificial intelligence identification method for unsafe behaviors of underground mine personnel according to claim 4 is characterized in that: The function enhancement module is composed of a channel activation module CAM and a spatial attention module SAM connected in series, and is placed after the concat operation in the YOLOv11 model; the channel activation module is used to assign weights to each channel of the input feature map; the spatial attention module is used to supplement the channel activation module, by performing max-pooling and average-pooling operations on the feature map of size H×H×C, and then connecting the two feature maps and performing convolution and nonlinear activation to obtain the weight coefficients of each part, and finally multiplying the weight coefficients by the input feature image to obtain a scaled new feature map.

8. An artificial intelligence recognition system for unsafe behaviors of underground mine personnel, characterized in that: include: The first module is used to build a classification system for unsafe behaviors and determine the specific manifestations of each type of behavior; The unsafe behaviors include object-related unsafe behaviors, action-related unsafe behaviors, and area-related unsafe behaviors; The second module is used to collect the target data set to be processed according to the processing flow of the optimized data set; wherein the optimized processing flow includes data collection diversification and targeted processing, data format conversion and parameter adjustment processing, data set augmentation processing, data set denoising processing, and data set annotation processing; The third module is used to optimize the model training process and build a recognition model based on the target data set through an improved YOLOv11 algorithm; The fourth module is used to identify the data to be identified according to the identification model and determine the unsafe behavior of personnel in the mine.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images