Miner well entering wearable equipment identification method and device, medium and equipment

Abstract

The invention discloses a miner well entering wearable equipment identification method and device, a medium and equipment, and relates to the technical field of mine operation safety. In order to solve the technical problem that the identification result of a wearable device is inaccurate due to the fact that an existing target identification model is poor in anti-interference capability, a principal component convolution coupling module parallel to a backbone network is added to the backbone network of an original YOLOv5 network, so that the original YOLOv5 network is improved; therefore, the anti-interference capability of the wearable device identification model performance is improved. The principal component analysis layer removes redundant features irrelevant to the non-wearable device in the to-be-recognized image through principal component extraction operation, and main features in the input image or the features are reserved; the output of the principal component analysis layer and the output of the CBS module in the YOLOv5 backbone network are fused and then principal component extraction is performed, so that the wearable device features in the feature transmission process are further enhanced, and a more accurate wearable device identification result can be obtained.

Description

Technical Field

[0001] The present application relates to the field of mine operation safety technology, and in particular to a method, device, medium and equipment for identifying equipment worn by miners entering a mine. Background Art

[0002] When working in coal mines and other mines, it is crucial to have complete safety equipment to ensure mining safety. However, in actual production, miners may not wear the equipment properly before entering the mine due to poor safety awareness, imperfect management system and other factors, which poses a great threat to life safety and work safety. Therefore, identifying the equipment worn by miners before entering the mine can reduce safety risks during work.

[0003] At present, with the continuous advancement of computer vision technology, especially the rapid development of deep learning algorithms, target recognition methods based on deep learning have been widely used in many fields. In the field of mining safety production, this technology can be used to identify wearable devices before miners enter the mine to ensure the safety of miners.

[0004] However, the existing deep learning-based target recognition model is limited by factors such as poor anti-interference ability. When using wearable devices to identify miners before they enter a coal mine in a complex environment, the recognition results obtained are inaccurate. Summary of the invention

[0005] Based on this, in order to solve the technical problem of inaccurate identification of equipment worn by miners entering the mine in the prior art, the present invention provides a method, device, medium and equipment for identifying equipment worn by miners entering the mine.

[0006] The present invention provides a method for identifying a device worn by a miner entering a mine, comprising:

[0007] An improved backbone network is formed by adding a principal component convolution coupling module in parallel with the backbone network to the backbone network of the original YOLOv5 network; the principal component convolution coupling module includes a plurality of principal component analysis layers and feature fusion layers stacked in sequence; the input end of the feature fusion layer is simultaneously connected to the output end of the CBS module in the backbone network and the output end of the principal component analysis layer; and an improved YOLOv5 network including the improved backbone network, the neck network in the original YOLOv5 network and the head network in the original YOLOv5 network is constructed;

[0008] Collect images of the miners’ wearable devices before entering the mine to build a data set, and use the data set to train the improved YOLOv5 network to obtain a wearable device recognition model;

[0009] The image to be identified, which is collected before the miners enter the mine, is input into the wearable device recognition model. The feature of the image to be identified is extracted by improving the CBS module of the backbone network to obtain a preliminary feature map. The principal component analysis layer of the backbone network is improved to perform principal component extraction on the image to be identified, so as to remove redundant features irrelevant to non-wearable devices in the image to be identified and obtain a principal component feature map. The preliminary feature map and the principal component feature map are fused by improving the feature fusion layer of the backbone network to obtain a fused feature map. The fused feature map is enhanced by the neck network to obtain an enhanced feature map. The enhanced feature map is used to detect wearable devices through the head network, and the category and position of the wearable device in the image to be identified are output.

[0010] Furthermore, the principal component convolution coupling module specifically includes:

[0011] A first principal component analysis layer connected to the output end of the input layer of the backbone network; a first feature fusion layer connected to the output end of the first principal component analysis layer and the output end of the first CBS module of the backbone network;

[0012] A second principal component analysis layer connected to the output end of the first feature fusion layer; a second feature fusion layer connected to the output end of the second principal component analysis layer and the output end of the second CBS module of the backbone network;

[0013] A third principal component analysis layer connected to the output end of the second feature fusion layer; a third feature fusion layer connected to the output end of the third principal component analysis layer and the output end of the third CBS module of the backbone network;

[0014] A fourth principal component analysis layer connected to the output end of the third feature fusion layer; a fourth feature fusion layer connected to the output end of the fourth principal component analysis layer and the output end of the fourth CBS module of the backbone network;

[0015] The first principal component analysis layer is used to perform principal component extraction operations on the image to be identified; the first feature fusion layer to the third feature fusion layer are used to fuse the input principal component feature map and the preliminary feature map; the second principal component analysis layer to the fourth principal component analysis layer perform principal component extraction operations on the fused feature maps output by the first feature fusion layer to the third feature fusion layer.

[0016] Furthermore, the first feature fusion layer, the second feature fusion layer, the third feature fusion layer and the fourth feature fusion layer fuse the input principal component feature map and the preliminary feature map by adding point by point.

[0017] Furthermore, the improved YOLOv5 network further includes:

[0018] An improved neck network is formed by replacing the neck network in the original YOLOv5 network with the FPN-PAN dual-tower network and adding a multi-scale kernel attention mechanism module at the output of the FPN-PAN dual-tower network;

[0019] The input end of the downsampling layer of the FPN network in the FPN-PAN dual-tower network is connected to the output end of the backbone network; the input end of each downsampling layer in the PAN network in the FPN-PAN dual-tower network is connected to the output end of each upsampling layer in the FPN network; the FPN network in the FPN-PAN dual-tower network is used to sequentially perform downsampling operations and upsampling operations on the feature map output by the backbone network to enhance the semantic expression of the wearable device features in the feature map output by the backbone network; the PAN network in the FPN-PAN dual-tower network is used to perform a downsampling operation on the output of the FPN network to enhance the position information of the wearable device features in the feature map output by the FPN network;

[0020] The input end of the multi-scale kernel attention mechanism module is connected to the output end of the PAN network; the multi-scale kernel attention mechanism module is used to capture small-sized target features in the feature map output by the PAN network through a multi-scale convolution kernel attention branch.

[0021] Furthermore, the multi-scale kernel attention mechanism module specifically includes:

[0022] 4 parallel branches with convolution kernels of different sizes to capture features of different scales; each branch includes a convolution operation, batch normalization, and ReLU activation function connected in sequence, and the convolution kernel sizes used in the convolution operation in each branch are 1×1, 3×3, 5×5, and 7×7 respectively;

[0023] A global average pooling layer connected to the output ends of the four branches, wherein the global average pooling layer is used to fuse the feature maps output by the four branches to obtain a fused feature vector;

[0024] A first fully connected layer connected to an output end of the global average pooling layer, wherein the first fully connected layer is used to reduce the dimension of the fused feature vector to obtain a reduced-dimensional feature vector;

[0025] A second fully connected layer connected to the output end of the first fully connected layer, wherein the second fully connected layer is used to increase the dimension of the reduced-dimensional feature vector to obtain an increased-dimensional feature vector;

[0026] Four parallel softmax functions connected to the output of the second fully connected layer, the softmax function is used to convert the increased-dimensional feature vector into an attention score;

[0027] 4 parallel weighted layers connected to the output ends of the 4 parallel softmax functions, the weighted layers are used to weight the input features of the 4 parallel branches with different sizes of convolution kernels using attention scores to generate weighted feature maps;

[0028] A fusion layer connected to the output ends of the four parallel weighted layers, wherein the fusion layer is used to fuse the outputs of the four parallel weighted layers.

[0029] The present invention provides a device for identifying equipment worn by miners entering a mine, comprising:

[0030] A model building module, used for forming an improved backbone network by adding a principal component convolution coupling module in parallel with the backbone network to the backbone network of the original YOLOv5 network; the principal component convolution coupling module includes a plurality of principal component analysis layers and feature fusion layers stacked in sequence; the input end of the feature fusion layer is simultaneously connected to the output end of the CBS module in the backbone network and the output end of the principal component analysis layer; and constructing an improved YOLOv5 network including the improved backbone network, the neck network in the original YOLOv5 network and the head network in the original YOLOv5 network;

[0031] The model training module is used to collect images of the wearable devices of miners before entering the mine to build a data set, and use the data set to train the improved YOLOv5 network to obtain a wearable device recognition model;

[0032] The recognition module is used to input the image to be recognized collected before the miners enter the mine into the wearable device recognition model, and extract features of the image to be recognized by improving the CBS module of the backbone network to obtain a preliminary feature map; the principal component analysis layer of the backbone network is improved to perform principal component extraction on the image to be recognized to remove redundant features in the image to be recognized that are not related to non-wearable devices, and obtain a principal component feature map; the preliminary feature map and the principal component feature map are fused by improving the feature fusion layer of the backbone network to obtain a fused feature map; the fused feature map is enhanced by the neck network to obtain an enhanced feature map; the enhanced feature map is detected by the head network, and the category and position of the wearable device in the image to be recognized are output.

[0033] The present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method for identifying wearable equipment for miners entering a mine is implemented.

[0034] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for identifying the device worn by miners entering a mine is implemented.

[0035] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0036] In the method for identifying wearable devices for miners entering the mine provided by the present invention, the original YOLOv5 is improved by adding a principal component convolution coupling module in parallel with the backbone network to the backbone network of the original YOLOv5 network, thereby improving the performance of the trained wearable device identification model, so that it has stronger anti-interference ability and real-time computing ability when facing complex coal mine operation sites. Specifically, the principal component analysis layer removes redundant features that are not related to non-wearable devices in the image to be identified through principal component extraction operations, retaining the main features in the input image or features; the output of the principal component analysis layer is fused with the output of the CBS module in the YOLOv5 backbone network and then the principal component is extracted, which further enhances the wearable device features in the feature transfer process and improves the anti-interference ability of the model, thereby obtaining more accurate wearable device identification results in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0038] Figure 1 A schematic diagram of a flow chart of a method for identifying a device worn by a miner entering a mine provided by the present invention;

[0039] Figure 2 The main component convolution coupling module structure framework diagram provided by the present invention;

[0040] Figure 3 A schematic diagram of the connection between the FPN-PAN dual-tower network and the multi-scale kernel attention mechanism module provided by the present invention;

[0041] Figure 4 A schematic diagram of the structure of the multi-scale kernel attention mechanism module provided by the present invention;

[0042] Figure 5 A flow chart of the steps for determining when a miner goes down a mine provided by the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of the present invention will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] With the continuous deepening of computer vision research, more and more researchers have begun to use video-based equipment detection methods. Commonly used deep learning target detection algorithms are not suitable for real-time equipment recognition in coal mine operations in complex environments due to factors such as long calculation time, large size, and poor anti-interference ability; and lightweight equipment detection algorithms sacrifice part of the model structure to increase calculation speed, resulting in limited recognition accuracy and difficulty in meeting actual application needs.

[0045] Based on this, the present invention provides an intelligent detection method for the wearing of five kinds of safety equipment, namely, helmet, towel, rubber boots, self-rescuer and mining lamp, before miners go down the mine. The method is used to accurately detect the wearing of safety equipment by miners at the wellhead punching point, and prohibit miners who do not wear safety equipment completely or wear them in the wrong position from entering the mine to work, so as to prevent the occurrence of safety accidents. The method is used to solve the technical problems in the prior art that the equipment detection model is disturbed by factors such as image noise and complex background, resulting in missed detection and false detection, as well as the problem of low accuracy in judging small target objects.

[0046] Embodiment 1:

[0047] Figure 1 The process of the method for identifying the wearable device for miners entering the mine in this embodiment is shown. Figure 1 The method for identifying the equipment worn by miners entering the mine is described in detail, which specifically includes the following steps:

[0048] S1: A principal component convolution coupling module parallel to the backbone network is added to the backbone network of the original YOLOv5 network to form an improved backbone network; the principal component convolution coupling module includes multiple principal component analysis layers and feature fusion layers stacked in sequence; the input end of the feature fusion layer is simultaneously connected to the output end of the CBS (Convolution-BatchNorm-SiLU, convolution-batch normalization-SiLU activation function) module in the backbone network and the output end of the principal component analysis layer; and an improved YOLOv5 network including the improved backbone network, the neck network in the original YOLOv5 network and the head network in the original YOLOv5 network is constructed.

[0049] The principal component convolution coupling module constructed is as follows Figure 2 As shown on the right, Figure 2 The left side of the figure is the backbone network structure of the original YOLOv5 network, including:

[0050] A first principal component analysis layer connected to the output end of the input layer of the backbone network; a first feature fusion layer connected to the output end of the first principal component analysis layer and the output end of the first CBS module of the backbone network; a second principal component analysis layer connected to the output end of the first feature fusion layer; a second feature fusion layer connected to the output end of the second principal component analysis layer and the output end of the second CBS module of the backbone network; a third principal component analysis layer connected to the output end of the second feature fusion layer; a third feature fusion layer connected to the output end of the third principal component analysis layer and the output end of the third CBS module of the backbone network; a fourth principal component analysis layer connected to the output end of the third feature fusion layer; a fourth feature fusion layer connected to the output end of the fourth principal component analysis layer and the output end of the fourth CBS module of the backbone network; the output end of the fourth feature fusion layer is connected to the input end of the spatial pyramid module.

[0051] The first principal component analysis layer is used to perform principal component extraction on the image to be identified; the first feature fusion layer to the third feature fusion layer fuse the input principal component feature map and the preliminary feature map by adding point by point; the second principal component analysis layer to the fourth principal component analysis layer perform principal component extraction on the fusion feature map output by the first feature fusion layer to the third feature fusion layer, so as to gradually remove redundant features irrelevant to non-wearable devices and enhance the features of wearable devices. The spatial pyramid module is connected to the output end of the fourth feature fusion layer to solve the problem that the network structure with a fully connected layer needs to fix the scale of the input image, resulting in unnecessary precision loss.

[0052] The principal component analysis layer reduces the dimensionality of the input features by the principal component analysis method. Applying principal component analysis to image recognition and feature extraction problems can extract the most important features for image classification as needed, so that the essential features of the image can be reflected to the maximum extent, avoiding the model from falling into overfitting in classification prediction and other problems. Here, the principal component analysis image feature reconstruction enhancement algorithm is used to achieve dimensionality reduction processing of image feature data and shallow image feature extraction. The principal component analysis layer is coupled with the CBS module. The features after each CBS module operation and the features after principal component analysis dimensionality reduction are first added and fused point by point, and finally output to the fully connected layer, and then the model recognition results are uniformly tested, thereby enhancing the overall model's ability to express images. The calculation steps of principal component analysis are: standardize the data; calculate the covariance matrix; perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors; sort the eigenvectors according to the size of the eigenvalues; select the matrix composed of the first k eigenvectors as the principal component; project the original data onto the selected principal component to obtain the reduced dimensionality data, specifically:

[0053] Assume that the input image size is w×h, and construct an M-dimensional column vector by column, where M=w×h. Assume that there are N images in total, and N images correspond to N M-dimensional column vectors, X j is the column vector corresponding to the jth image, and X is the image matrix composed of N images. Then the overall covariance matrix is:

[0054]

[0055]

[0056] Where μ is the average image vector of the sample set images.

[0057] Use SVD (Singular Value Decomposition) to solve the eigenvalues ​​and corresponding eigenvectors of the matrix. Let its eigenvalue be λ i , i = (1, 2, ... wh), the corresponding eigenvector after orthogonal normalization is μ i . Select the eigenvectors corresponding to the eigenvalues ​​of L before the contribution rate ranking to form the projection matrix Eig=(μ1,μ2,…μ L ),in:

[0058]

[0059] Here, α∈[90%~99%]. The original high-dimensional (wh×1) image sequence X i The result after projection matrix dimensionality reduction is:

[0060] Feature i =Eig·X i

[0061] Feature i is a column vector of L×1 bits. This achieves the purpose of dimensionality reduction, that is, extracting useful features. While compressing data to simplify calculations, the implicit features in the original data are retained to the greatest extent.

[0062] S2: Collect images of the miners’ wearable devices before entering the mine to build a data set, and use the data set to train the improved YOLOv5 network to obtain a wearable device recognition model.

[0063] By collecting data sets of six objects, including helmets, towels, rubber boots, self-rescuers, mining lamps, and miners, at different angles and in different environments, after labeling, a basic data set is formed. Randomly select 5% of them as test sets and the rest as training sets. Label the original data set, take 5% of them as test sets and the rest as training sets. Expand the training data set, including cutting and pasting small target detection objects to other random images, random cutting, random inversion, random fusion, normalization and other steps, in order to expand the data set and improve the detection accuracy of small target objects.

[0064] After the labeling is completed, the training samples are input into the improved deep neural network model. The training set is input into the improved model, and the training is stopped after the specified number of rounds or after the accuracy converges. The obtained results are used as weight files. After the model converges, the category and location information of the miner and five wearable devices are predicted. The image frames of the real-time video stream obtained from the camera are input into the model to obtain the recognition results. The recognition results include the category information and location information of the miner and the five safety equipment. The category information is the confidence, and the location information is the center point coordinates and length and width of the detection box.

[0065] S3: The image to be identified, which is collected before the miners enter the mine, is input into the wearable device recognition model. The CBS module of the improved backbone network is used to extract features of the image to be identified to obtain a preliminary feature map. The principal component analysis layer of the improved backbone network is used to extract the principal component of the image to be identified, so as to remove redundant features in the image to be identified that are not related to non-wearable devices, and obtain a principal component feature map. The preliminary feature map and the principal component feature map are fused by the feature fusion layer of the improved backbone network to obtain a fused feature map. The fused feature map is enhanced by the neck network to obtain an enhanced feature map. The enhanced feature map is detected by the head network to output the category and position of the wearable device in the image to be identified.

[0066] Embodiment 2:

[0067] This embodiment provides an improved neck network: the neck network in the original YOLOv5 network is replaced by a FPN-PAN (Feature Pyramid Network-Path Aggregation Network) dual-tower network, and a multi-scale kernel attention mechanism module is added after the output of the FPN-PAN dual-tower network is convolved. The connection method between the FPN-PAN dual-tower network and the multi-scale kernel attention mechanism module is as follows: Figure 3 shown.

[0068] Among them, the three downsampling layers of the FPN network in the FPN-PAN dual-tower network receive a total of three feature maps as input, which are the outputs from the third and fourth feature fusion layers and the output of a pyramid layer. The input end of each downsampling layer in the PAN network in the FPN-PAN dual-tower network is connected to the output end of each upsampling layer in the FPN network; the FPN network in the FPN-PAN dual-tower network is used to perform downsampling operations and upsampling operations on the feature maps output by the backbone network in sequence to enhance the semantic expression of the wearable device features in the feature maps output by the backbone network; the PAN network in the FPN-PAN dual-tower network is used to perform downsampling operations on the output of the FPN network to enhance the location information of the wearable device features in the feature maps output by the FPN network. FPN is to transmit the semantic features of the deep layer to the shallow layer, thereby enhancing the semantic expression at multiple scales; while PAN, on the contrary, transmits the positioning information of the shallow layer to the deep layer, enhancing the positioning capability at multiple scales.

[0069] Among them, the outputs of the three downsampling layers of the PAN network are respectively input into the three multi-scale kernel attention mechanism modules after convolution operation. The output ends of the three multi-scale kernel attention mechanism modules are respectively connected to the inputs of the three detection frames included in the head network in the original YOLOv5 network. The sizes of the three detection frames are 19×19×155, 28×28×255 and 72×72×255 respectively. The head network uses the non-maximum suppression algorithm to suppress the redundant wearable device detection frames and output the category information and location information of the detection object. The category information includes six types: safety helmet, towel, rubber boots, self-rescuer, mining lamp and miner's confidence; the location information is the center point coordinates and length and width of the target detection frame.

[0070] The multi-scale kernel attention mechanism module takes the feature map output by the PAN network as input, and captures the features of small and medium-sized targets in the feature map output by the PAN network through the multi-scale convolution kernel attention branch. The multi-scale kernel attention mechanism module introduces the number of channels C and the dimensionality reduction coefficient r to construct an attention branch with multi-scale convolution kernels, which are 1x1, 3x3, 5x5, and 7x7 respectively. It aims to effectively capture the features of small-sized targets through convolution kernels with different receptive fields, and improve the network's perception ability on small targets. The multi-scale kernel attention mechanism module constructs an attention branch of the multi-scale convolution kernel, with the help of 4 convolution kernels of different sizes, to effectively capture the features of small-sized targets and improve the network's perception ability on small targets. The structure of the multi-scale kernel attention mechanism module is shown in the figure below. Figure 4As shown in the figure, it includes 4 parallel branches with convolution kernels of different sizes, a global average pooling layer connected to the output of the 4 branches, a first fully connected layer connected to the output of the global average pooling layer, a second fully connected layer connected to the output of the first fully connected layer, 4 parallel softmax functions connected to the output of the second fully connected layer, 4 parallel weighted layers connected to the output of the 4 parallel softmax functions, and a fusion layer connected to the output of the 4 parallel weighted layers. The 4 branches are composed of convolution, batch normalization and ReLU activation functions. Features are extracted using convolution kernels of different sizes. The sizes of the 4 convolution kernels are 1×1, 3×3, 5×5 and 7×7, thereby obtaining feature maps of the 4 branches. These 4 feature maps are fused through the global average pooling layer. The input of the global average pooling layer is the feature image of the 4 branches, and the output is a feature vector of 1×1×C, where C represents the depth. In order to reduce the dimension, the feature vector is compressed to 1×1×C / r through a fully connected layer. There are two fully connected layers. The first one compresses the feature vector to 1×1×C / r, and the second one upgrades the new feature vector to the original channel dimension. The softmax function is used to convert the upgraded feature value into an attention score, which is weightedly fused with the original feature map to output a new feature map. The new feature map is fused as the output feature map of the multi-scale kernel attention mechanism module.

[0071] Embodiment 3:

[0072] This embodiment provides a process flow for determining the steps for miners to go down the mine. After obtaining the recognition result, the wearing status of the safety equipment is determined. If the person passes, the person is allowed to go through. Otherwise, an alarm is sounded. The process of detecting the safety equipment of miners before going down the mine includes: obtaining an input image, and judging whether the person has entered the detection area based on the input image. If the person has not entered the detection area, the next frame of the image is taken to judge whether the person has entered the detection area; if the person has entered the detection area, subject recognition is performed. Subject recognition is performed. When the person is determined to be a subject, whether the equipment worn by the subject is complete and in specification is detected. If it is complete and in specification, it is displayed as passed and the gate is opened; if the equipment worn by the subject is not complete and in specification, it is displayed as failed, and an audible and visual alarm is sounded. The process of detecting the safety equipment of miners before going down the mine is as follows. Figure 5 As shown, the specific steps include:

[0073] I: Extracting real-time monitoring video stream from wellhead camera.

[0074] II: Take out the latest image frame from the surveillance video stream and input it into the model for feature comparison. According to the output result, determine whether there is anyone entering the detection area of ​​the current image frame, that is, whether there is anyone currently performing on-the-job detection. If so, execute step III; otherwise, extract the next frame of image to continue detection.

[0075] III: If there is miner information in the current image, it is considered that there are miners currently undergoing on-the-job inspection. Then, in order to avoid misjudging the correspondence between people and equipment, it is necessary to determine the current detection subject. The determination algorithm uses the camera monocular ranging technology and the output miner position information to calculate the distance from all miners to the camera. The one with the closest distance is the current detection subject, and all device information in this frame matches this person.

[0076] VI: Equipment completeness judgment, that is, whether all the output detection frame category information contains five types of detection equipment. If not, it will fail and sound and light alarm will be issued; if it is complete, it will enter the standardization judgment.

[0077] V: Equipment compliance judgment, that is, the relative distance is calculated based on the position information of the equipment and the miner's detection frame. If all are within the preset threshold range, the judgment test is passed, that is, the miner is wearing complete and comprehensive safety equipment, it is displayed as passed, and the gate is opened. Otherwise, it fails and an audible and visual alarm is issued.

[0078] based on Figure 1 The method for identifying the wearable equipment for miners entering the mine shown in the figure collects images of miners wearing safety equipment in complex environments. After the annotation is completed, the training samples are input into the improved deep neural network model. After the model converges, the category and location information of the miners and five types of wearable equipment are predicted, and the wearing status of the safety equipment is judged. If it passes, it is released, otherwise an alarm is issued. The multi-layer convolution module of the model feature extraction is coupled with the image feature reconstruction enhancement algorithm, which effectively reduces the dimension of the image while retaining the main features of the image. At the same time, an improved small target enhancement attention mechanism with multi-scale kernel attention is introduced to solve the identification of small-volume safety equipment (such as self-rescuers). The present invention proposes an improved deep neural network model to identify the wearable equipment for miners entering the mine, which not only saves computing power and parameter quantity, but also improves the recognition efficiency and the accuracy of small target detection.

[0079] The above is a method for identifying a miner's wearable device when entering a mine provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding device for identifying a miner's wearable device when entering a mine, including:

[0080] The input module is used to input the to-be-recognized images collected by the miners before entering the mine into the wearable device recognition model. The wearable device recognition model is obtained by training the improved YOLOv5 network. The improved YOLOv5 network is formed by adding a principal component convolution coupling module parallel to the backbone network to the backbone network of the original YOLOv5 network; the principal component convolution coupling module includes a plurality of principal component analysis layers and feature fusion layers stacked in sequence; the input end of the feature fusion layer is simultaneously connected to the output end of the CBS module in the backbone network of the original YOLOv5 network and the output end of the principal component analysis layer.

[0081] The recognition module is used to input the image to be recognized collected before the miners enter the mine into the wearable device recognition model, and extract features of the image to be recognized by improving the CBS module of the backbone network to obtain a preliminary feature map; the principal component analysis layer of the backbone network is improved to perform principal component extraction on the image to be recognized to remove redundant features in the image to be recognized that are not related to non-wearable devices, and obtain a principal component feature map; the preliminary feature map and the principal component feature map are fused by improving the feature fusion layer of the backbone network to obtain a fused feature map; the fused feature map is enhanced by the neck network to obtain an enhanced feature map; the enhanced feature map is detected as a wearable device by the head network, and the category and position of the wearable device in the image to be recognized are output.

[0082] For the specific limitations of the device for identifying the wearable equipment for miners entering the mine, please refer to the limitations of the method for identifying the wearable equipment for miners entering the mine, which will not be repeated here. Each module in the above-mentioned device for identifying the wearable equipment for miners entering the mine can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0083] The present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program can be used to execute the above-mentioned method for identifying the wearable equipment for miners entering the mine.

[0084] The present invention also provides a structure of a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned method for identifying the wearable device for miners entering the mine.

[0085] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0086] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for identifying equipment worn by miners entering a mine, characterized in that: include: An improved backbone network is formed by adding a principal component convolution coupling module in parallel with the backbone network to the backbone network of the original YOLOv5 network; The principal component convolution coupling module includes a plurality of principal component analysis layers and feature fusion layers stacked in sequence; The input end of the feature fusion layer is simultaneously connected to the output end of the CBS module in the backbone network and the output end of the principal component analysis layer; And build an improved YOLOv5 network including an improved backbone network, a neck network in the original YOLOv5 network, and a head network in the original YOLOv5 network; Collect images of the miners’ wearable devices before entering the mine to build a data set, and use the data set to train the improved YOLOv5 network to obtain a wearable device recognition model; The images to be identified collected by the miners before entering the mine are input into the wearable device recognition model, and the features of the images to be identified are extracted by improving the CBS module of the backbone network to obtain a preliminary feature map; the principal component analysis layer of the backbone network is improved to extract the principal components of the images to be identified, so as to remove the redundant features in the images to be identified that are not related to non-wearable devices, and obtain the principal component feature map; By improving the feature fusion layer of the backbone network, the preliminary feature map and the principal component feature map are fused to obtain a fused feature map; Perform feature enhancement on the fused feature map through the neck network to obtain an enhanced feature map; The head network is used to detect wearable devices on the enhanced feature map, and the category and location of the wearable devices in the image to be identified are output.

2. The method for identifying a device worn by a miner entering a mine as claimed in claim 1, characterized in that: The principal component convolution coupling module specifically includes: A first principal component analysis layer connected to the output end of the input layer of the backbone network; a first feature fusion layer connected to the output end of the first principal component analysis layer and the output end of the first CBS module of the backbone network; A second principal component analysis layer connected to the output end of the first feature fusion layer; a second feature fusion layer connected to the output end of the second principal component analysis layer and the output end of the second CBS module of the backbone network; A third principal component analysis layer connected to the output end of the second feature fusion layer; a third feature fusion layer connected to the output end of the third principal component analysis layer and the output end of the third CBS module of the backbone network; A fourth principal component analysis layer connected to the output end of the third feature fusion layer; a fourth feature fusion layer connected to the output end of the fourth principal component analysis layer and the output end of the fourth CBS module of the backbone network; The first principal component analysis layer is used to perform principal component extraction operations on the image to be identified; the first feature fusion layer to the third feature fusion layer are used to fuse the input principal component feature map and the preliminary feature map; the second principal component analysis layer to the fourth principal component analysis layer perform principal component extraction operations on the fused feature maps output by the first feature fusion layer to the third feature fusion layer.

3. The method for identifying a device worn by a miner entering a mine as claimed in claim 2, characterized in that: The first feature fusion layer, the second feature fusion layer, the third feature fusion layer and the fourth feature fusion layer fuse the input principal component feature map and the preliminary feature map by adding point by point.

4. The method for identifying a miner's wearable device when entering a mine as claimed in claim 1, characterized in that: The improved YOLOv5 network further includes: An improved neck network is formed by replacing the neck network in the original YOLOv5 network with the FPN-PAN dual-tower network and adding a multi-scale kernel attention mechanism module at the output of the FPN-PAN dual-tower network; The input end of the downsampling layer of the FPN network in the FPN-PAN dual-tower network is connected to the output end of the backbone network; the input end of each downsampling layer in the PAN network in the FPN-PAN dual-tower network is connected to the output end of each upsampling layer in the FPN network; the FPN network in the FPN-PAN dual-tower network is used to sequentially perform downsampling operations and upsampling operations on the feature map output by the backbone network to enhance the semantic expression of the wearable device features in the feature map output by the backbone network; the PAN network in the FPN-PAN dual-tower network is used to perform a downsampling operation on the output of the FPN network to enhance the position information of the wearable device features in the feature map output by the FPN network; The input end of the multi-scale kernel attention mechanism module is connected to the output end of the PAN network; the multi-scale kernel attention mechanism module is used to capture small-sized target features in the feature map output by the PAN network through a multi-scale convolution kernel attention branch.

5. The method for identifying a miner's wearable device when entering a mine as claimed in claim 4, characterized in that: The multi-scale kernel attention mechanism module specifically includes: 4 parallel branches with convolution kernels of different sizes to capture features of different scales; each branch includes a convolution operation, batch normalization, and ReLU activation function connected in sequence, and the convolution kernel sizes used in the convolution operation in each branch are 1×1, 3×3, 5×5, and 7×7 respectively; A global average pooling layer connected to the output ends of the four branches, wherein the global average pooling layer is used to fuse the feature maps output by the four branches to obtain a fused feature vector; A first fully connected layer connected to an output end of the global average pooling layer, wherein the first fully connected layer is used to reduce the dimension of the fused feature vector to obtain a reduced-dimensional feature vector; A second fully connected layer connected to the output end of the first fully connected layer, wherein the second fully connected layer is used to increase the dimension of the reduced-dimensional feature vector to obtain an increased-dimensional feature vector; Four parallel softmax functions connected to the output of the second fully connected layer, the softmax function is used to convert the increased-dimensional feature vector into an attention score; 4 parallel weighted layers connected to the output ends of the 4 parallel softmax functions, the weighted layers are used to weight the input features of the 4 parallel branches with different sizes of convolution kernels using attention scores to generate weighted feature maps; A fusion layer connected to the output ends of the four parallel weighted layers, wherein the fusion layer is used to fuse the outputs of the four parallel weighted layers.

6. A device for identifying equipment worn by miners entering a mine, characterized in that: include: A model building module, used for forming an improved backbone network by adding a principal component convolution coupling module in parallel with the backbone network to the backbone network of the original YOLOv5 network; the principal component convolution coupling module includes a plurality of principal component analysis layers and feature fusion layers stacked in sequence; The input end of the feature fusion layer is simultaneously connected to the output end of the CBS module in the backbone network and the output end of the principal component analysis layer; And build an improved YOLOv5 network including an improved backbone network, a neck network in the original YOLOv5 network, and a head network in the original YOLOv5 network; The model training module is used to collect images of the wearable devices of miners before entering the mine to build a data set, and use the data set to train the improved YOLOv5 network to obtain a wearable device recognition model; The recognition module is used to input the image to be recognized collected by the miners before entering the mine into the wearable device recognition model, extract features of the image to be recognized by improving the CBS module of the backbone network, and obtain a preliminary feature map; the principal component analysis layer of the backbone network is improved to perform principal component extraction on the image to be recognized, so as to remove redundant features in the image to be recognized that are not related to non-wearable devices, and obtain a principal component feature map; By improving the feature fusion layer of the backbone network, the preliminary feature map and the principal component feature map are fused to obtain a fused feature map; Perform feature enhancement on the fused feature map through the neck network to obtain an enhanced feature map; The head network is used to detect wearable devices on the enhanced feature map, and the category and location of the wearable devices in the image to be identified are output.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 5 is implemented.

Images