A Method, Device, Medium and Equipment for Identifying Wearable Equipment for Miners Entering the Mine

Abstract

The present invention discloses a method, device, medium and equipment for identifying miners' in-well wearing equipment, which relates to the technical field of mine operation safety. In order to solve the technical problem that the inaccurate recognition result of wearing equipment is caused by the poor anti-interference ability of the existing target recognition model, the present invention improves the original YOLOv5 by adding a principal component convolution coupling module parallel to the backbone network in the backbone network of the original YOLOv5, so as to improve the anti-interference ability of the wearing equipment recognition model performance. The principal component analysis layer removes the redundant features irrelevant to non-wearing equipment in the image to be recognized through principal component extraction operations, and retains the main features in the input image or features; after fusing the output of the principal component analysis layer with the output of the CBS module in the YOLOv5 backbone network, principal component extraction is carried out again, which further enhances the features of the wearing equipment in the feature transmission process, so as to obtain a more accurate recognition result of the wearing equipment.

Description

Technical Field

[0001] This application relates to the technical field of mine operation safety, and particularly to a method, device, medium and equipment for identifying the wearing equipment of miners entering the mine. Background Technique

[0002] When working in mines such as coal mines, it is crucial to have complete safety equipment to ensure the safety of miners' operations. However, in actual production, due to various factors such as poor safety awareness and imperfect management systems of miners, they may not properly wear the equipment before entering the mine, posing a great threat to life safety and operation safety. Therefore, identifying the wearing equipment of miners before they enter the mine can reduce safety risks during the work process.

[0003] Currently, with the continuous progress of computer vision technology, especially the rapid development of deep learning algorithms, object recognition methods based on deep learning have been widely applied in many fields. In the field of mine safety production, this technology can be applied to identify the wearing equipment of miners before they enter the mine to ensure the safety of miners.

[0004] However, the existing object recognition models based on deep learning are restricted by factors such as poor anti-interference ability, and when identifying the wearing equipment of miners before they enter the mine in a coal mine operation site with a complex environment, the obtained recognition results are inaccurate. Summary of the Invention

[0005] Based on this, in order to solve the technical problem of inaccurate identification of the wearing equipment of miners entering the mine in the prior art, the present invention provides a method, device, medium and equipment for identifying the wearing equipment of miners entering the mine.

[0006] The present invention provides a method for identifying the wearing equipment of miners entering the mine, including:

[0007] An improved backbone network is formed by adding a principal component convolution coupling module parallel to the backbone network in the original YOLOv5 network; the principal component convolution coupling module includes a plurality of sequentially stacked principal component analysis layers and feature fusion layers; the input end of the feature fusion layer is simultaneously connected to the output end of the CBS module in the backbone network and the output end of the principal component analysis layer; and an improved YOLOv5 network including the improved backbone network, the neck network in the original YOLOv5 network, and the head network in the original YOLOv5 network is constructed;

[0008] Collect the images of the wearing equipment of miners before entering the mine to construct a data set, and use the data set to train the improved YOLOv5 network to obtain a wearing equipment recognition model;

[0009] Input the to-be-recognized image collected before the miner enters the well into the wearable device recognition model. Extract features from the to-be-recognized image through the improved CBS module of the backbone network to obtain a preliminary feature map. Perform principal component extraction operations on the to-be-recognized image through the principal component analysis layer of the improved backbone network to remove redundant features irrelevant to non-wearable devices in the to-be-recognized image and obtain a principal component feature map. Fuse the preliminary feature map and the principal component feature map through the feature fusion layer of the improved backbone network to obtain a fused feature map. Enhance the features of the fused feature map through the neck network to obtain an enhanced feature map. Detect wearable devices on the enhanced feature map through the head network and output the category and location of the wearable devices in the to-be-recognized image.

[0010] Furthermore, the principal component convolution coupling module specifically includes:

[0011] A first principal component analysis layer connected to the output end of the input layer of the backbone network; a first feature fusion layer connected to the output ends of the first principal component analysis layer and the output end of the first CBS module of the backbone network simultaneously;

[0012] A second principal component analysis layer connected to the output end of the first feature fusion layer; a second feature fusion layer connected to the output ends of the second principal component analysis layer and the output end of the second CBS module of the backbone network simultaneously;

[0013] A third principal component analysis layer connected to the output end of the second feature fusion layer; a third feature fusion layer connected to the output ends of the third principal component analysis layer and the output end of the third CBS module of the backbone network simultaneously;

[0014] A fourth principal component analysis layer connected to the output end of the third feature fusion layer; a fourth feature fusion layer connected to the output ends of the fourth principal component analysis layer and the output end of the fourth CBS module of the backbone network simultaneously;

[0015] The first principal component analysis layer is used to perform principal component extraction operations on the to-be-recognized image; the first to third feature fusion layers are used to fuse the input principal component feature map and the preliminary feature map; the second to fourth principal component analysis layers perform principal component extraction operations on the fused feature maps output by the first to third feature fusion layers.

[0016] Furthermore, the first, second, third, and fourth feature fusion layers fuse the input principal component feature map and the preliminary feature map by element-wise addition.

[0017] Furthermore, the improved YOLOv5 network further includes:

[0018] By replacing the neck network in the original YOLOv5 network with an FPN-PAN two-tower network and adding a multi-scale kernel attention mechanism module at the output end of the FPN-PAN two-tower network, an improved neck network is formed;

[0019] The input end of the downsampling layer of the FPN network in the FPN-PAN two-tower network is connected to the output end of the backbone network; the input ends of the downsampling layers in the PAN network of the FPN-PAN two-tower network are connected to the output ends of the upsampling layers in the FPN network; the FPN network in the FPN-PAN two-tower network is used to perform downsampling operations and upsampling operations on the feature map output by the backbone network in sequence to enhance the semantic expression of the wearable device features in the feature map output by the backbone network; the PAN network in the FPN-PAN two-tower network is used to perform downsampling operations on the output of the FPN network to enhance the position information of the wearable device features in the feature map output by the FPN network;

[0020] The input end of the multi-scale kernel attention mechanism module is connected to the output end of the PAN network; the multi-scale kernel attention mechanism module is used to capture small-scale target features in the feature map output by the PAN network through a multi-scale convolutional kernel attention branch.

[0021] Further, the multi-scale kernel attention mechanism module specifically includes:

[0022] 4 parallel branches with convolutional kernels of different sizes to capture features of different scales; among them, each branch includes a convolutional operation, batch normalization, and ReLU activation function connected in sequence, and the sizes of the convolutional kernels used in the convolutional operations in each branch are 1×1, 3×3, 5×5, and 7×7 respectively;

[0023] A global average pooling layer connected to the output ends of the 4 branches, and the global average pooling layer is used to fuse the feature maps output by the 4 branches to obtain a fused feature vector;

[0024] A first fully connected layer connected to the output end of the global average pooling layer, and the first fully connected layer is used to reduce the dimension of the fused feature vector to obtain a dimension-reduced feature vector;

[0025] A second fully connected layer connected to the output end of the first fully connected layer, and the second fully connected layer is used to increase the dimension of the dimension-reduced feature vector to obtain a dimension-increased feature vector;

[0026] 4 parallel softmax functions connected to the output end of the second fully connected layer, and the softmax functions are used to convert the dimension-increased feature vector into attention scores;

[0027] Four parallel weighted layers connected to the output ends of four parallel softmax functions, where the weighted layers are used to weight the input features of four parallel branches with different-sized convolutional kernels using attention scores to generate weighted feature maps;

[0028] A fusion layer connected to the output ends of the four parallel weighted layers, where the fusion layer is used to fuse the outputs of the four parallel weighted layers.

[0029] The present invention provides a device for identifying miners' in-well wearing equipment, including:

[0030] A model construction module, which is used to form an improved backbone network by adding a principal component convolution coupling module parallel to the backbone network in the original YOLOv5 network; the principal component convolution coupling module includes a plurality of sequentially stacked principal component analysis layers and feature fusion layers; the input end of the feature fusion layer is simultaneously connected to the output end of the CBS module in the backbone network and the output end of the principal component analysis layer; and an improved YOLOv5 network including the improved backbone network, the neck network in the original YOLOv5 network, and the head network in the original YOLOv5 network is constructed;

[0031] A model training module, which is used to collect images of miners' wearing equipment before entering the well to construct a dataset, and use the dataset to train the improved YOLOv5 network to obtain a wearing equipment recognition model;

[0032] An identification module, which is used to input the image to be identified collected before the miner enters the well into the wearing equipment recognition model, extract features of the image to be identified through the CBS module of the improved backbone network to obtain a preliminary feature map; perform principal component extraction operations on the image to be identified through the principal component analysis layer of the improved backbone network to remove redundant features irrelevant to non-wearing equipment in the image to be identified to obtain a principal component feature map; fuse the preliminary feature map and the principal component feature map through the feature fusion layer of the improved backbone network to obtain a fused feature map; enhance the features of the fused feature map through the neck network to obtain an enhanced feature map, and detect the wearing equipment in the enhanced feature map through the head network, and output the category and location of the wearing equipment in the image to be identified.

[0033] The present invention provides a computer-readable storage medium, where the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for identifying miners' in-well wearing equipment is implemented.

[0034] The present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the above-mentioned method for identifying miners' in-well wearing equipment is implemented.

[0035] The above at least one technical solution adopted by the present invention can achieve the following beneficial effects:

[0036] In the method for identifying miners' in-well wearing devices provided by the present invention, by adding a principal component convolution coupling module parallel to the backbone network in the original YOLOv5 network to improve the original YOLOv5, the performance of the trained wearing device recognition model is improved, so that it has stronger anti-interference ability and real-time computing ability in the face of complex coal mine operation sites. Specifically, the principal component analysis layer removes redundant features irrelevant to non-wearing devices in the image to be recognized through principal component extraction operations, and retains the main features in the input image or features; after fusing the output of the principal component analysis layer with the output of the CBS module in the YOLOv5 backbone network, principal component extraction is performed again, further enhancing the wearing device features in the feature transfer process, improving the anti-interference ability of the model, and thus obtaining more accurate wearing device recognition results in a complex environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention, and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the drawings:

[0038] Figure 1 is a schematic flow chart of a method for identifying miners' in-well wearing devices provided by the present invention;

[0039] Figure 2 is a structural framework diagram of the principal component convolution coupling module provided by the present invention;

[0040] Figure 3 is a schematic diagram of the connection manner between the FPN-PAN twin tower network and the multi-scale kernel attention mechanism module provided by the present invention;

[0041] Figure 4 is a structural schematic diagram of the multi-scale kernel attention mechanism module provided by the present invention;

[0042] Figure 5 is a flow chart of the determination steps for miners to go down the well provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0044] With the continuous in - depth research of computer vision, more and more researchers have started the method of device detection based on video. The commonly used deep - learning object detection algorithms are not suitable for real - time device recognition in coal - mine operation sites with complex environments due to factors such as long computing time, large volume, and poor anti - interference ability; while lightweight device detection algorithms sacrifice part of the model structure to improve the computing speed, resulting in limited recognition accuracy and being difficult to meet the actual application requirements.

[0045] Based on this, the present invention provides an intelligent detection method for the wearing of five safety devices, namely safety helmets, towels, gumboots, self - rescue devices, and miner's lamps, before miners enter the mine. It is used to accurately detect the wearing situation of miners' safety devices at the well - head card - punching place, and prohibit miners who do not wear safety devices completely or wear them in incorrect positions from entering the mine to work, so as to prevent the occurrence of safety accidents. It is used to solve the technical problems of missed detection and false detection caused by factors such as image noise and complex backgrounds in the existing device detection model, as well as the problem of low determination accuracy for small - target objects.

[0046] Embodiment 1:

[0047] Figure 1 The flow of the method for identifying the equipment worn by miners entering the mine in this embodiment is shown. Specifically, in combination with Figure 1 The method for identifying the equipment worn by miners entering the mine is described in detail as follows, which specifically includes the following steps:

[0048] S1: Add a principal - component convolution coupling module parallel to the backbone network in the original YOLOv5 network to form an improved backbone network; the principal - component convolution coupling module includes a plurality of sequentially stacked principal - component analysis layers and feature fusion layers; the input end of the feature fusion layer is simultaneously connected to the output end of the CBS (Convolution - BatchNorm - SiLU, convolution - batch normalization - SiLU activation function) module in the backbone network and the output end of the principal - component analysis layer; and an improved YOLOv5 network including the improved backbone network, the neck network in the original YOLOv5 network, and the head network in the original YOLOv5 network is constructed.

[0049] The constructed principal - component convolution coupling module is as shown in Figure 2 on the right side, Figure 2 and the left side in

[0050] The first principal component analysis layer connected to the output end of the input layer of the backbone network; the first feature fusion layer connected to the output end of the first principal component analysis layer and the output end of the first CBS module of the backbone network at the same time; the second principal component analysis layer connected to the output end of the first feature fusion layer; the second feature fusion layer connected to the output end of the second principal component analysis layer and the output end of the second CBS module of the backbone network at the same time; the third principal component analysis layer connected to the output end of the second feature fusion layer; the third feature fusion layer connected to the output end of the third principal component analysis layer and the output end of the third CBS module of the backbone network at the same time; the fourth principal component analysis layer connected to the output end of the third feature fusion layer; the fourth feature fusion layer connected to the output end of the fourth principal component analysis layer and the output end of the fourth CBS module of the backbone network at the same time; the output end of the fourth feature fusion layer is connected to the input end of the spatial pyramid module.

[0051] The above-mentioned first principal component analysis layer is used to perform principal component extraction operations on the image to be recognized; the first to third feature fusion layers fuse the input principal component feature maps and preliminary feature maps by element-wise addition; the second to fourth principal component analysis layers perform principal component extraction operations on the fusion feature maps output by the first to third feature fusion layers to gradually remove redundant features unrelated to non-wearable devices and enhance wearable device features. The spatial pyramid module is connected to the output end of the fourth feature fusion layer to solve the problem that the network structure with a fully connected layer requires a fixed scale of the input image, resulting in unnecessary accuracy loss.

[0052] The principal component analysis layer reduces the dimension of the input features through the principal component analysis method. Applying principal component analysis to image recognition and feature extraction problems can extract the most important part of the features for image classification as needed, maximizing the reflection of the essential features of the image and avoiding the model falling into overfitting in problems such as classification prediction. Here, the principal component analysis image feature reconstruction enhancement algorithm is used to implement the dimensionality reduction processing of image feature data and the extraction of shallow image features. The principal component analysis layer is coupled with the CBS module. First, the features after the operation of each CBS module and the features after dimensionality reduction by principal component analysis are fused by element-wise addition, and finally output to the fully connected layer, and then the recognition results of the model are detected uniformly, thereby enhancing the overall model's performance on images. The calculation steps of principal component analysis are as follows: standardize the data; calculate the covariance matrix; perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues and corresponding eigenvectors; sort the eigenvectors according to the size of the eigenvalues; select the matrix composed of the first k eigenvectors as the principal component; project the original data onto the selected principal component to obtain the dimensionality-reduced data. Specifically:

[0053] Assume the size of the input image is \(w\times h\). The input image is formed into an \(M -\)dimensional column vector by columns, where \(M = w\times h\). Suppose there are \(N\) images in total, and the \(N\) images correspond to \(N\) \(M -\)dimensional column vectors, \(X_j\) j is the column vector corresponding to the \(j -\)th image, and \(X\) is the image matrix composed of \(N\) images. Then the overall covariance matrix:

[0054]

[0055]

[0056] where \(\mu\) is the average image vector of the sample set of images.

[0057] Use the SVD (Singular Value Decomposition) method to solve the eigenvalues and corresponding eigenvectors of the matrix. Let its eigenvalues be \(\lambda_i\) i , \(i=(1,2,\cdots,wh)\), and the corresponding orthonormalized eigenvectors be \(\mu_i\) i . Select the eigenvectors corresponding to the top \(L\) eigenvalues in terms of contribution rate to form the projection matrix \(Eig = (\mu_1,\mu_2,\cdots,\mu_L)\) L ), where:

[0058]

[0059] where \(\alpha\in[90\%\sim99\%]\). The original high - dimensional (\(wh\times1\)) image column \(X_j\) i After being dimension - reduced by the projection matrix, the result is:

[0060] Feature i \(= Eig\cdot X_j\) i

[0061] Feature i is an \(L\times1\) - dimensional column vector. In this way, the purpose of dimension reduction is achieved, that is, useful features are extracted. While compressing data and simplifying calculations, the implicit features in the original data are retained to the greatest extent.

[0062] S2: Collect the images of the wearing devices before miners enter the well to construct a dataset, and use the dataset to train the improved YOLOv5 network to obtain a wearing device recognition model.

[0063] By collecting datasets of six objects, namely safety helmets, towels, gumboots, self-rescuers, miner's lamps, and miners, under different angles and in different environments, after annotation is completed, a basic dataset for use is formed. Randomly select 5% of them as the test set, and the others as the training set. Make annotations for the original dataset, take 5% of it as the test set, and the remaining part as the training set. Perform augmentation operations on the training dataset, including steps such as cutting and pasting small target detection objects onto other random images, random cutting, random inversion, random fusion, and normalization, with the aim of expanding the dataset and improving the detection accuracy of small target objects.

[0064] After annotation is completed, input the training samples into the improved deep neural network model. Input the training set into the improved model, stop training after a specified number of rounds or when the accuracy converges, and use the obtained result as the weight file. After the model converges, predict the category and location information of miners and five types of wearable devices. Obtain the image frames of the real-time video stream from the camera and input them into the model to obtain the recognition result. The recognition result includes the category information and location information of miners and five types of safety devices. The category information is the confidence level, and the location information is the center point coordinates and length and width of the detection box.

[0065] S3: Input the image to be recognized collected before the miner enters the well into the wearable device recognition model. Extract features from the image to be recognized through the CBS module that improves the backbone network to obtain a preliminary feature map; perform principal component extraction operations on the image to be recognized through the principal component analysis layer that improves the backbone network to remove redundant features unrelated to non-wearable devices in the image to be recognized and obtain a principal component feature map; fuse the preliminary feature map and the principal component feature map through the feature fusion layer that improves the backbone network to obtain a fused feature map; enhance the features of the fused feature map through the neck network to obtain an enhanced feature map; detect the enhanced feature map through the head network and output the category and location of the wearable devices in the image to be recognized.

[0066] Example 2:

[0067] This embodiment provides an improved neck network: Replace the neck network in the original YOLOv5 network with an FPN-PAN (Feature Pyramid Network-Path Aggregation Network) two-tower network, and after performing a convolution operation on the output of the FPN-PAN two-tower network, add a multi-scale kernel attention mechanism module. The connection method between the FPN-PAN two-tower network and the multi-scale kernel attention mechanism module is as Figure 3 shown.

[0068] Among them, the three downsampling layers of the FPN network in the FPN-PAN double-tower network respectively receive a total of three feature maps, namely the outputs from the third and fourth feature fusion layers and the output of a pyramid layer, as inputs. The input ends of the downsampling layers in the PAN network of the FPN-PAN double-tower network are connected to the output ends of the upsampling layers in the FPN network; the FPN network in the FPN-PAN double-tower network is used to sequentially perform downsampling operations and upsampling operations on the feature maps output by the backbone network to enhance the semantic expression of the wearable device features in the feature maps output by the backbone network; the PAN network in the FPN-PAN double-tower network is used to perform downsampling operations on the output of the FPN network to enhance the position information of the wearable device features in the feature maps output by the FPN network. FPN is to transmit deep semantic features to the shallow layer, thereby enhancing the semantic expression at multiple scales; while PAN, on the contrary, transmits shallow positioning information to the deep layer to enhance the positioning ability at multiple scales.

[0069] Among them, after the outputs of the three downsampling layers of the PAN network pass through convolution operations respectively, they are input into three multi-scale kernel attention mechanism modules. The output ends of the three multi-scale kernel attention mechanism modules are respectively connected to the inputs of the three detection boxes included in the head network in the original YOLOv5 network. The sizes of the three detection boxes are 19×19×155, 28×28×255, and 72×72×255 respectively. The head network uses the non-maximum suppression algorithm to suppress redundant wearable device detection boxes and outputs the category information and position information of the detected objects. The category information includes six types: the confidence levels of safety helmets, towels, gum boots, self-rescuers, miner's lamps, and miners; the position information is the center point coordinates and the length and width of the target detection box.

[0070] The multi-scale kernel attention mechanism module takes the feature maps output by the PAN network as inputs and captures the features of small-sized targets in the feature maps output by the PAN network through the multi-scale convolutional kernel attention branch. The multi-scale kernel attention mechanism module introduces the number of channels C and the dimensionality reduction coefficient r to construct an attention branch with multi-scale convolutional kernels, which are 1x1, 3x3, 5x5, and 7x7 respectively, aiming to effectively capture the features of small-sized targets through convolutional kernels with different receptive fields and improve the network's perception ability for small targets. The multi-scale kernel attention mechanism module effectively captures the features of small-sized targets through the constructed attention branch with multi-scale convolutional kernels and by means of 4 convolutional kernels with different sizes, thereby improving the network's perception ability for small targets. The structure of the multi-scale kernel attention mechanism module is as Figure 4As shown, it includes 4 parallel branches with convolutional kernels of different sizes, a global average pooling layer connected to the output ends of the 4 branches, a first fully connected layer connected to the output end of the global average pooling layer, a second fully connected layer connected to the output end of the first fully connected layer, 4 parallel softmax functions connected to the output end of the second fully connected layer, 4 parallel weighting layers connected to the output ends of the 4 parallel softmax functions, and a fusion layer connected to the output ends of the 4 parallel weighting layers. The 4 branches are composed of convolution, batch normalization, and ReLU activation functions, and use convolutional kernels of different sizes to extract features. The sizes of the 4 convolutional kernels are 1×1, 3×3, 5×5, and 7×7, and thus feature maps of the 4 branches are obtained. These 4 feature maps are fused through the global average pooling layer. The input of the global average pooling layer is the feature images of the 4 branches, and the output is a feature vector of 1×1×C, where C represents the depth. To reduce the dimension, the feature vector is compressed to 1×1×C / r through the fully connected layer. There are 2 fully connected layers. The first one compresses the feature vector to 1×1×C / r, and the second one upsamples the new feature vector to the original channel dimension. The softmax function is used to convert the upsampled feature values into attention scores, which are weighted and fused with the original feature map to output a new feature map. The new feature maps are fused as the output feature map of the multi-scale kernel attention mechanism module.

[0071] Embodiment 3:

[0072] This embodiment provides a determination step process for miners going down the well. After obtaining the recognition result, the wearing status of safety equipment is judged. If it passes, they are released; if it fails, an alarm is given. The process of detecting miners' safety equipment before going down the well includes: obtaining the input image, judging whether a person enters the detection area according to the input image. If the person does not enter the detection area, the next frame of image is taken to judge whether the person enters the detection area; if the person enters the detection area, then the main body recognition is performed. For the main body recognition, when it is determined to be the main body, it is detected whether the equipment worn by the main body is complete and standard. If it is complete and standard, it is shown as passed and the turnstile is opened; if the equipment worn by the main body is not complete and standard, it is shown as failed and an audible and visual alarm is given. The process of detecting miners' safety equipment before going down the well is as Figure 5 shown, and its specific steps include:

[0073] I: Extract the real-time monitoring video stream from the wellhead camera.

[0074] II: Take out the latest image frame from the monitoring video stream and input it into the model for feature comparison. According to the output result, it is judged whether someone enters the detection area in the current image frame, that is, whether someone is currently undergoing the going-to-work detection. If so, step III is executed; otherwise, the next frame of image is extracted and the detection continues.

[0075] III: If there is miner information in the current image, it is considered that there is a miner conducting the going-to-work detection. Next, to avoid misjudging the correspondence between people and devices, it is necessary to determine the current detection subject. The determination algorithm uses the monocular ranging technology of the camera to calculate the distances from all miners to the camera using the output miner position information. The one with the closest distance is the current detection subject, and all device information in this frame is matched with this person.

[0076] VI: Determination of equipment completeness, that is, according to whether the output category information of all detection frames contains five detection devices. If it is not complete, it fails and an audible and visual alarm is given; if it is complete, it enters the standardization determination.

[0077] V: Determination of equipment standardization, that is, calculate the relative distance according to the position information of the equipment and the miner detection frame. If all are within the preset threshold range, the determination test is passed, that is, the miner wears safety equipment completely and completely, shows passed, and the turnstile opens; otherwise, it fails and an audible and visual alarm is given.

[0078] Based on Figure 1 The miner's in-well wearing equipment recognition method shown above, collect images of miners wearing safety equipment in a complex environment. After annotation is completed, input the training samples into an improved deep neural network model. After the model converges, predict the category and position information of miners and five wearing equipment, and judge the wearing status of safety equipment. If it passes, it is released; if it does not pass, an alarm is given. The multi-layer convolution module for feature extraction of this model couples an image feature reconstruction and enhancement algorithm, which effectively reduces the dimension of the image while retaining the main features of the image. At the same time, an improved small target enhancement attention mechanism with multi-scale kernel attention is introduced to solve the identification of small-volume safety equipment (such as self-rescue devices). The present invention proposes an improved deep neural network model to identify miners' in-well wearing equipment, which not only saves computing power and the number of parameters, but also improves the recognition efficiency and the accuracy of small target detection.

[0079] The above is the miner's in-well wearing equipment recognition method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding miner's in-well wearing equipment recognition device, including:

[0080] An input module for inputting the to-be-recognized image collected before the miner enters the well into the wearing equipment recognition model. The wearing equipment recognition model is trained by an improved YOLOv5 network, and the improved YOLOv5 network is formed by adding a principal component convolution coupling module parallel to the backbone network in the original YOLOv5 network backbone network; the principal component convolution coupling module includes a plurality of principal component analysis layers and feature fusion layers stacked in sequence; the input end of the feature fusion layer is connected to the output ends of the CBS module in the backbone network of the original YOLOv5 network and the output end of the principal component analysis layer at the same time.

[0081] An identification module is configured to input the to-be-identified image collected before a miner enters the mine into a wearable device identification model. The to-be-identified image is subjected to feature extraction through an improved CBS module of the backbone network to obtain a preliminary feature map. The to-be-identified image is subjected to principal component extraction operation through the principal component analysis layer of the improved backbone network to remove redundant features irrelevant to non-wearable devices in the to-be-identified image and obtain a principal component feature map. The preliminary feature map and the principal component feature map are fused through the feature fusion layer of the improved backbone network to obtain a fused feature map. The fused feature map is subjected to feature enhancement through a neck network to obtain an enhanced feature map. The enhanced feature map is subjected to wearable device detection through a head network, and the category and location of the wearable device in the to-be-identified image are output.

[0082] For the specific limitations of the miner's in-mine wearable device identification device, reference can be made to the limitations of the miner's in-mine wearable device identification method in the above text, which will not be elaborated here. Each module in the above miner's in-mine wearable device identification device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules.

[0083] The present invention also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above-mentioned miner's in-mine wearable device identification method.

[0084] The present invention also provides the structure of a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, there may also be other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned miner's in-mine wearable device identification method.

[0085] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above various methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0086] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope recorded in the present invention.

Claims

1. A method for identifying miners' in - well wearing devices, characterized in that, Including: Forming an improved backbone network by adding a principal component convolution coupling module parallel to the backbone network in the original YOLOv5 backbone network; The principal component convolution coupling module includes a plurality of sequentially stacked principal component analysis layers and feature fusion layers; specifically including a first principal component analysis layer connected to the output end of the input layer of the backbone network; a first feature fusion layer connected to the output end of the first principal component analysis layer and the output end of the first CBS module of the backbone network at the same time; A second principal component analysis layer connected to the output end of the first feature fusion layer; a second feature fusion layer connected to the output end of the second principal component analysis layer and the output end of the second CBS module of the backbone network at the same time; a third principal component analysis layer connected to the output end of the second feature fusion layer; a third feature fusion layer connected to the output end of the third principal component analysis layer and the output end of the third CBS module of the backbone network at the same time; a fourth principal component analysis layer connected to the output end of the third feature fusion layer; a fourth feature fusion layer connected to the output end of the fourth principal component analysis layer and the output end of the fourth CBS module of the backbone network at the same time; the first principal component analysis layer is used to perform principal component extraction operations on the image to be recognized; the first to third feature fusion layers are used to fuse the input principal component feature maps and preliminary feature maps; the second to fourth principal component analysis layers perform principal component extraction operations on the fused feature maps output by the first to third feature fusion layers; The input end of the feature fusion layer is connected to the output end of the CBS module in the backbone network and the output end of the principal component analysis layer at the same time; and an improved YOLOv5 network including the improved backbone network, the neck network in the original YOLOv5 network, and the head network in the original YOLOv5 network is constructed; Collect the wearable device images of miners before entering the well to construct a dataset, and use the dataset to train the improved YOLOv5 network to obtain a wearable device recognition model; Input the image to be recognized collected before the miner enters the well into the wearable device recognition model, extract features from the image to be recognized through the CBS module of the improved backbone network to obtain a preliminary feature map; perform principal component extraction operations on the image to be recognized through the principal component analysis layer of the improved backbone network to remove redundant features irrelevant to the wearable device in the image to be recognized and obtain a principal component feature map; Fuse the preliminary feature map and the principal component feature map through the feature fusion layer of the improved backbone network to obtain a fused feature map; Enhance the features of the fused feature map through the neck network to obtain an enhanced feature map; Detect the wearable device on the enhanced feature map through the head network and output the category and position of the wearable device in the image to be recognized.

2. The miner's in-well wearing device identification method according to claim 1, wherein, The first, second, third, and fourth feature fusion layers fuse the input principal component feature maps and preliminary feature maps by point-by-point addition.

3. The miner's in-well wearing device identification method according to claim 1, characterized in that, The improved YOLOv5 network further includes: By replacing the neck network in the original YOLOv5 network with an FPN-PAN two-tower network and adding a multi-scale kernel attention mechanism module at the output end of the FPN-PAN two-tower network, an improved neck network is formed; The input end of the downsampling layer of the FPN network in the FPN-PAN two-tower network is connected to the output end of the backbone network; the input ends of the downsampling layers in the PAN network of the FPN-PAN two-tower network are connected to the output ends of the upsampling layers in the FPN network; the FPN network in the FPN-PAN two-tower network is used to perform downsampling operations and upsampling operations on the feature maps output by the backbone network in sequence to enhance the semantic expression of the wearable device features in the feature maps output by the backbone network; the PAN network in the FPN-PAN two-tower network is used to perform downsampling operations on the output of the FPN network to enhance the position information of the wearable device features in the feature maps output by the FPN network; The input end of the multi-scale kernel attention mechanism module is connected to the output end of the PAN network; the multi-scale kernel attention mechanism module is used to capture small-size target features in the feature maps output by the PAN network through multi-scale convolutional kernels in the attention branch.

4. The method for identifying the miner's in-well wearing device according to claim 3, wherein, The multi-scale kernel attention mechanism module specifically includes: 4 parallel branches with different sizes of convolutional kernels to capture features of different scales; among them, each branch includes a convolutional operation, batch normalization, and ReLU activation function connected in sequence, and the sizes of the convolutional kernels used in the convolutional operations in each branch are 1×1, 3×3, 5×5, and 7×7 respectively; A global average pooling layer connected to the output ends of the 4 branches, and the global average pooling layer is used to fuse the feature maps output by the 4 branches to obtain a fused feature vector; A first fully connected layer connected to the output end of the global average pooling layer, and the first fully connected layer is used to reduce the dimension of the fused feature vector to obtain a dimension-reduced feature vector; A second fully connected layer connected to the output end of the first fully connected layer, and the second fully connected layer is used to increase the dimension of the dimension-reduced feature vector to obtain a dimension-increased feature vector; 4 parallel softmax functions connected to the output end of the second fully connected layer, and the softmax functions are used to convert the dimension-increased feature vector into attention scores; 4 parallel weighting layers connected to the output ends of the 4 parallel softmax functions, and the weighting layers are used to weight the input features of the 4 parallel branches with different sizes of convolutional kernels using the attention scores to generate weighted feature maps; A fusion layer connected to the output ends of the 4 parallel weighting layers, and the fusion layer is used to fuse the outputs of the 4 parallel weighting layers.

5. An identification device for miners' in-well wearing equipment, characterized in that, It includes: A model construction module for forming an improved backbone network by adding a principal component convolution coupling module parallel to the backbone network in the backbone network of the original YOLOv5 network; specifically including a first principal component analysis layer connected to the output end of the input layer of the backbone network; a first feature fusion layer connected to the output end of the first principal component analysis layer and the output end of the first CBS module of the backbone network at the same time; A second principal component analysis layer connected to the output end of the first feature fusion layer; a second feature fusion layer connected to the output ends of the second principal component analysis layer and the output end of the second CBS module of the backbone network at the same time; a third principal component analysis layer connected to the output end of the second feature fusion layer; a third feature fusion layer connected to the output ends of the third principal component analysis layer and the output end of the third CBS module of the backbone network at the same time; a fourth principal component analysis layer connected to the output end of the third feature fusion layer; a fourth feature fusion layer connected to the output ends of the fourth principal component analysis layer and the output end of the fourth CBS module of the backbone network at the same time; the first principal component analysis layer is used for performing principal component extraction operation on the image to be recognized; the first to third feature fusion layers are used for fusing the input principal component feature map and the preliminary feature map; the second to fourth principal component analysis layers perform principal component extraction operation on the fused feature map output by the first to third feature fusion layers; the principal component convolution coupling module includes a plurality of sequentially stacked principal component analysis layers and feature fusion layers; The input end of the feature fusion layer is connected to the output ends of the CBS module in the backbone network and the output end of the principal component analysis layer at the same time; And an improved YOLOv5 network is constructed, which includes an improved backbone network, the neck network in the original YOLOv5 network, and the head network in the original YOLOv5 network; A model training module, which is used to collect the wearable device images before miners enter the well to construct a data set, and use the data set to train the improved YOLOv5 network to obtain a wearable device recognition model; A recognition module, which is used to input the image to be recognized collected before miners enter the well into the wearable device recognition model, perform feature extraction on the image to be recognized through the CBS module of the improved backbone network to obtain a preliminary feature map; perform principal component extraction operation on the image to be recognized through the principal component analysis layer of the improved backbone network to remove redundant features irrelevant to the wearable device in the image to be recognized and obtain a principal component feature map; Fuse the preliminary feature map and the principal component feature map through the feature fusion layer of the improved backbone network to obtain a fused feature map; Perform feature enhancement on the fused feature map through the neck network to obtain an enhanced feature map; Perform wearable device detection on the enhanced feature map through the head network, and output the category and position of the wearable device in the image to be recognized.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 4 above is implemented.

7. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in any one of claims 1 to 4 above is implemented.

Images