Mine safety helmet wearing detection method, device thereof, and electronic device

Abstract

The application provides a mining safety helmet wearing detection method, a device thereof and an electronic device. The method comprises the following steps: first, obtaining a target image; then, constructing a Faster RCNN neural network algorithm structure, training the Faster RCNN neural network algorithm structure, and obtaining a safety helmet wearing detection model, wherein the Faster RCNN neural network algorithm structure extracts image features by taking VGG16 as a convolutional neural network; then, performing image recognition on the target image by using the safety helmet wearing detection model, and determining whether there is a safety helmet above a face in the target image; and finally, determining that a person in the target image wears a safety helmet in the case that there is a safety helmet above the face. The method takes VGG16 as a backbone network of a feature extraction network, has high detection precision and high efficiency, and thus solves the problem of low efficiency in checking the safety helmet wearing condition in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep recognition and target recognition, in particular to a safety helmet wearing detection method for mine, a safety helmet wearing detection device for mine, a computer readable storage medium and an electronic device. BACKGROUND

[0002] Currently, the safety helmet detection in coal mine mainly includes two ways. One is sensor-based detection, which usually uses sensors to identify and detect the wearing situation of safety helmet. The other is traditional detection based on image feature extraction, which detects through traditional image recognition algorithm. According to different detection tasks, specific adaptive features need to be designed.

[0003] The sensor-based detection usually uses sensors to identify and detect the wearing situation of safety helmet. However, this kind of detection method based on sensors can only play a role in a limited range and cannot achieve good results in the environment of coal mine with serious interference and poor light conditions. The traditional image recognition algorithm is used for detection. According to different detection tasks, specific adaptive features need to be designed. The application range is small, the cost is high, and the comprehensive performance of the network is poor. SUMMARY

[0004] The main purpose of the present application is to provide a safety helmet wearing detection method for mine, a safety helmet wearing detection device for mine, a computer readable storage medium and an electronic device to at least solve the problem of low efficiency of checking the wearing situation of safety helmet in the prior art.

[0005] In order to achieve the above purpose, according to one aspect of the present application, a safety helmet wearing detection method for mine is provided, which comprises: acquiring a target image, the target image being an image including a face; constructing a Faster RCNN (Region-based Convolutional Neural Networks) neural network algorithm structure, training the Faster RCNN neural network algorithm structure to obtain a safety helmet wearing detection model, and the Faster RCNN neural network algorithm structure taking VGG16 as a convolutional neural network to extract image features; using the safety helmet wearing detection model to perform image recognition on the target image to determine whether there is a safety helmet above the face in the target image; and in the case that there is a safety helmet above the face, determining that the person in the target image wears a safety helmet.

[0006] Optionally, the acquiring the target image comprises: acquiring live video and extracting continuous image frames, the live video being a video of a construction site captured by a camera; and determining the image frame containing the face image as the target image.

[0007] Optionally, the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, an RPN network, an ROI pooling layer, and a full connection layer.

[0008] Optionally, the image recognition of the target image by the safety helmet wearing detection model comprises: inputting the target image into the convolutional neural network to calculate a feature map of the target image; inputting the feature map into the RPN network to obtain a first detection candidate frame map; inputting the first detection candidate frame map into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame map into a fixed size output to obtain a second detection candidate frame map; and performing target classification and bounding box regression on the second detection candidate frame map by the full connection layer to determine the positioning of the safety helmet in the target image.

[0009] Optionally, the inputting the feature map into the RPN network to obtain a first detection candidate frame map comprises: generating a plurality of initial detection frames according to the features of the feature map; performing foreground and background classification on the plurality of initial detection frames by using a softmax function; and correcting the plurality of initial detection frames by using a bounding box regression to obtain the first detection candidate frame map.

[0010] Optionally, after the plurality of initial detection frames are generated according to the features of the feature map, before the plurality of initial detection frames are corrected by using the bounding box regression, the method further comprises: removing redundant initial detection frames by using a Soft NMS algorithm.

[0011] Optionally, the activation function of the convolutional neural network is a ReLu function.

[0012] According to another aspect of the present application, a safety helmet wearing detection device for mine is provided, comprising an acquisition unit, a training unit, a first determination unit and a second determination unit, wherein the acquisition unit is configured to acquire a target image, the target image being an image including a face; the training unit is configured to construct a Faster RCNN neural network algorithm structure, and train the Faster RCNN neural network algorithm structure to obtain a safety helmet wearing detection model, the Faster RCNN neural network algorithm structure extracting image features by using VGG16 as a convolutional neural network; the first determination unit is configured to perform image recognition on the target image by using the safety helmet wearing detection model, and determine whether there is a safety helmet above the face in the target image; and the second determination unit is configured to determine that a person in the target image wears a safety helmet in the case that there is the safety helmet above the face.

[0013] According to still another aspect of the present application, a computer readable storage medium is provided, comprising a stored program, wherein the program, when executed, controls a device where the computer readable storage medium is located to perform any of the safety helmet wearing detection methods for mine.

[0014] According to still another aspect of the present application, an electronic device is provided, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise a program for performing any of the safety helmet wearing detection methods for mine.

[0015] According to the technical solution of the present application, in the safety helmet wearing detection method for mine, first, a target image is acquired, the target image being an image including a face; then, a Faster RCNN neural network algorithm structure is constructed, and the Faster RCNN neural network algorithm structure is trained to obtain a safety helmet wearing detection model, the Faster RCNN neural network algorithm structure extracting image features by using VGG16 as a convolutional neural network; then, image recognition is performed on the target image by using the safety helmet wearing detection model, and it is determined whether there is a safety helmet above the face in the target image; finally, in the case that there is the safety helmet above the face, it is determined that a person in the target image wears a safety helmet. This method uses a convolutional neural network to construct a safety helmet wearing detection model, uses a Faster RCNN algorithm, and uses VGG16 as a backbone network of a feature extraction network, so that the detection accuracy and efficiency are high, and automatic detection of safety helmet wearing conditions of personnel on a construction site is realized, thereby solving the problem of low efficiency in checking safety helmet wearing conditions in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which form a part of this specification, are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification. The embodiments of this application, and their

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for performing a method of detecting wearing of a mining safety helmet according to an embodiment of the present application is shown;

[0018] Figure 2 A flowchart of a method of detecting wearing of a mining safety helmet according to an embodiment of the present application is shown;

[0019] Figure 3 A feature map diagram of mapping a RoI to a VGG16 output according to an embodiment of the present application is shown;

[0020] Figure 4 A structure block diagram of a device for detecting wearing of a mining safety helmet according to an embodiment of the present application is shown.

[0021] Among the above-mentioned drawings, the following reference signs are included:

[0022] 102, processor; 104, memory; 106, transmission device; 108, input and output device. DETAILED DESCRIPTION

[0023] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0024] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the protection scope of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] As introduced in the background, the prior art safety hat wearing detection is inefficient, to solve the above problems, the embodiments of the present application provide a mine safety hat wearing detection method, a mine safety hat wearing detection device, a computer readable storage medium and an electronic device.

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking the case of running on a mobile terminal, Figure 1 is a hardware structure block diagram of a mobile terminal of a mine safety hat wearing detection method of the embodiments of the present application. As Figure 1 shown, the mobile terminal can include one or more (only one is shown in Figure 1 ) processor 102 (the processor 102 can include but not limited to processing devices such as microprocessor MCU or programmable logic device FPGA) and memory 104 for storing data, wherein the above mobile terminal can also include transmission device 106 for communication function and input output device 108. Those skilled in the art can understand, Figure 1 the structure shown is only schematic, which does not limit the structure of the above mobile terminal. For example, the mobile terminal can also include more or less components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0029] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the safety helmet wearing detection method for mine in the embodiments of the present application. The processor 102 can execute various functional applications and data processing, i.e., implement the above method, by running the computer program stored in the memory 104. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include memories remotely arranged with respect to the processor 102, which can be connected to the mobile terminal through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific examples of the above network can include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

[0030] In the present embodiment, a safety helmet wearing detection method for mine running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that herein.

[0031] Figure 2 is a flowchart of the safety helmet wearing detection method for mine according to the embodiments of the present application. As shown in Figure 2 , the method comprises the following steps:

[0032] In step S201, a target image is acquired, and the target image is an image including a face;

[0033] Specifically, the safety helmet wearing detection needs to be combined with human body detection, and only the safety helmet worn on the head is considered to meet the requirements, so an image including a face needs to be acquired.

[0034] In order to accurately acquire the image of the face, in step S201, the target image is acquired, including: acquiring a live video, and extracting a continuous image frame, the live video being a video of a construction site collected by a camera; and determining that the image frame containing the face image is the target image.

[0035] Step S202, a Faster RCNN neural network algorithm structure is constructed, and the Faster RCNN neural network algorithm structure is trained to obtain a safety helmet wearing detection model. The Faster RCNN neural network algorithm structure takes VGG16 as a convolutional neural network to extract image features.

[0036] Specifically, the Faster RCNN can be simply understood as a combination of the RPN and the Fast RCNN. The deep fully convolutional network RPN is used to generate candidate regions, and the Fast RCNN detector uses the candidate regions output by the RPN network for classification and bounding box regression.

[0037] In actual application, only after being well trained, the deep learning neural network algorithm can accurately identify the construction worker and the safety helmet in the picture, and provide accuracy for subsequent judgment. The Faster RCNN neural network algorithm structure is trained to obtain the safety helmet wearing detection model, including: obtaining a plurality of sample images; the plurality of sample images are input into the Faster RCNN neural network algorithm structure in sequence, and unsupervised training is performed to obtain the safety helmet wearing detection model. The traditional deep learning algorithm usually adopts a supervised learning manner, that is, a known data and its corresponding output are used for training, so as to obtain an optimal model, and then the model is used for judging new data samples to realize the classification purpose. For example, a large number of construction site pictures are collected, and then the pictures are identified by manual marking, and each picture sample is marked with a safety helmet or no safety helmet. The image data with the label is used for algorithm training. The application adopts an unsupervised training manner, that is, the picture without manual marking is put into the algorithm for unsupervised pre-training, and a small amount of image sample data with label information is combined for algorithm fine-tuning training to obtain a trained model. This manner can not only reduce the human cost of a large amount of manual labeling and greatly reduce the training cost, but also has a slightly better effect than the supervised learning manner, and realizes the improvement of the effect.

[0038] In order to realize the safety helmet recognition in a special environment, in an optional solution, the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, the RPN network, the ROI pooling layer and the full connection layer.

[0039] Step S203, the safety helmet wearing detection model is used for image recognition on the target image to determine whether there is a safety helmet above the face in the target image.

[0040] Specifically, the face position of the person in the target image is found, and then the safety helmet above the face is recognized according to the face position.

[0041] In order to improve the accuracy of safety helmet recognition, in an optional solution, the safety helmet wearing detection model is used for image recognition of the target image, including: inputting the target image into the convolutional neural network to calculate the feature map of the target image; inputting the feature map into the RPN network to obtain a first detection candidate frame; inputting the first detection candidate frame into the ROI pooling layer to convert the first candidate recognition region in the first detection candidate frame into a fixed size output to obtain a second detection candidate frame; and performing target classification and bounding box regression on the second detection candidate frame through the full connection layer to determine the positioning of the safety helmet in the target image.

[0042] In practical applications, the convolutional neural network is composed of three layers, i.e., a convolution layer (conv), an activation function layer (relu), and a pooling layer (pooling). As for VGG16, the network is composed of 13 convolution layers, 13 activation function layers, and 4 pooling layers. The key of the convolution layer is the convolution kernel, which is a matrix, and the commonly used size is 3x3. The convolution kernel can obtain the feature map of the original image by performing certain multiplication operation on the matrix of the input image according to the given step size.

[0043] In an optional solution, the activation function of the convolutional neural network adopts the ReLu function. A lot of calculation is saved, and the ReLu function outputs 0 in the case that the activation value of the neuron is negative, so that the network has sparsity and the overfitting problem can be alleviated.

[0044] In practical applications, the pooling layer is used to reduce the dimension of the input data, reduce the number of parameters and calculation amount, and make the input of the next layer more concise, so as to facilitate the network to extract features more widely, and also can avoid overfitting to some extent. In the convolutional neural network: 1) all the conv layers are kernel_size = 3, padding = 1, stride = 1; 2) all the pooling layers are kernel_size = 2, padding = 0, stride = 2. Among them, padding = 1 means adding a circle of 0 around the matrix, and padding = 0 means not adding 0. After each conv layer, the size of the feature map will not change, only the features are extracted by convolution; after each relu layer, the size of the feature map does not change, only the pixel points with negative values are reset to 0; after the pooling layer, the width and height of the feature map are reduced to half of the original. Therefore, after passing through the convolutional neural network, an image with a size of MxN will become a feature map with a size of (M / 16)x(N / 16).

[0045] The generation of the detection frame is the main bottleneck of the low real-time performance of some classic detection algorithms. For example, RCNN adopts selective search (Selective Search) to generate detection frames, which slows down the detection speed. The Faster RCNN algorithm introduces an RPN (Region Proposal Network) network for detection frame generation, which greatly improves the detection efficiency. In an optional solution, the above feature map is input into the above RPN network to obtain a first detection candidate frame diagram, including: generating a plurality of initial detection frames according to the features of the feature map, and classifying a plurality of the initial detection frames as foreground and background using a softmax function; and correcting a plurality of the initial detection frames using a bounding box regression to obtain the first detection candidate frame diagram.

[0046] Specifically, a very key concept in the RPN network is anchor. If it is simply summarized in one sentence, anchor is a reference frame of different sizes and aspect ratios preset on the image, that is, an initial detection frame. In the FasterRCNN algorithm, the number of anchors corresponding to each pixel point is preset to 9, and the 9 different anchors are obtained by combining three sizes [128x128, 256x256, 512x512] and three aspect ratios [1:1, 1:2, 2:1]. After traversing the feature map, each point corresponds to k anchors as an initial detection frame.

[0047] In practical applications, bounding box regression is used to fine-tune the candidate detection boxes, so that the fine-tuned window is closer to the Ground Truth, and more accurate positioning is achieved.

[0048] In order to enhance the detection effect, in an optional solution, after generating a plurality of initial detection boxes according to the feature map, before correcting the plurality of initial detection boxes by using the bounding box regression, the method further comprises: removing the redundant initial detection boxes by using a Soft NMS algorithm.

[0049] Specifically, for the anchor-based target detection algorithm, since the size and specific position of the target are not known in advance, it is necessary to use overlapping sampling or sliding window to exhaust all possible targets. However, this approach will cause many candidate boxes to point to the same target object, resulting in redundancy, and therefore it is necessary to delete the repeated candidate boxes. The non-maximum suppression algorithm (NMS) can be used to remove the redundant candidate boxes. The idea of the non-maximum suppression algorithm (NMS) is to suppress non-maximum values to find local maximum value targets, that is, to remove redundancy and find the optimal. The input of the algorithm is the detection boxes (the same target object may correspond to many boxes, and each box has its own classification score) detected in the previous step, and the output is the optimal box. The general process is as follows: 1) For a candidate box list B, find the box with the highest confidence (score), denoted as M; 2) delete M from the candidate box list B, and add the deleted M to the last reserved list D; 3) calculate the IoU (intersection over union) of all boxes in list B with M (the highest score), and delete all boxes in B whose IoU is greater than a given threshold (the degree of repetition is too high); 4) repeat the above steps until all candidate boxes are traversed; the boxes in list D are the required boxes. The traditional NMS algorithm has a problem - it is easy to mistakenly delete overlapping targets in the picture, that is, when there are overlapping targets in the detection box, the NMS algorithm will also directly delete them. The detection algorithm should output two boxes, but when the IoU of the two boxes is greater than the set threshold, the NMS algorithm will delete the green box with lower confidence, resulting in the detection of only one object, affecting the detection effect. In view of this problem, the Soft NMS algorithm is used instead of the NMS algorithm, which does not directly delete the detection boxes with IoU greater than the threshold during algorithm execution, but reduces the original confidence score. On the other hand, Soft NMS can be easily introduced into the target detection algorithm without the need for retraining the model, and the code is easy to implement with relatively small computational complexity. Therefore, the improved NMS algorithm Soft NMS is used to remove redundant detection boxes.

[0050] In practical applications, for the traditional convolutional neural network, the size of the input image to the trained network must be a fixed value, and the VGG16 network also needs to meet this requirement. In view of the fact that the original image often has different sizes, there are usually two solutions: one is to cut the image, and the other is to stretch or shorten the image. However, neither of these two methods is satisfactory, as they either result in incomplete images or change the original proportions of the images. Therefore, Faster RCNN proposes RoI Pooling. RoI (Region of Interest) is a candidate recognition region found from an image. The candidate recognition region (RoI) is obtained by mapping the candidate box output by the RPN network to the feature map, that is, the RoI is a candidate region (proposal region) for the original image, as shown in Figure 3 The role of RoI Pooling is to normalize each candidate region from the input list to a fixed-size target recognition region. The implementation of normalization is as follows: 1) According to the input image, map the candidate region to the corresponding position of the feature map 2) Divide the mapped region into equal-sized parts (the number of which is equal to the output dimension), find the maximum value (max pooling) of each part, 3) Keep and output the found maximum value. Using RoI Pooling, the training and testing speed can be significantly accelerated, the detection accuracy can be improved, and the model can be trained in an end-to-end manner.

[0051] The final classification part predicts the class of the detection box and the translation and scaling coefficients of the target box by Fast RCNN. The RoI after pooling is first converted from three dimensions to one dimension, and then passed to the fully connected layer. Using the obtained proposal feature map, the class to which each proposal belongs (whether the safety helmet is correctly worn) is determined by full connection and softmax; at the same time, the proposals are again subjected to bounding box regression to obtain a detection box with higher accuracy. Each node in the fully connected layer is connected to all nodes in the previous layer, which is used to integrate all the features extracted in the previous step. It maps the learned distributed feature representation to the sample label space and serves as the final "classifier" in the entire convolutional neural network.

[0052] In step S204, it is determined that the person in the target image wears a safety helmet when the safety helmet is above the face.

[0053] Specifically, it can be accurately determined that the person in the target image wears a safety helmet when the safety helmet is above the face.

[0054] Through the above embodiment, the safety helmet wearing detection model is constructed by using the convolutional neural network, the algorithm used is Faster RCNN, VGG16 is used as the backbone network of the feature extraction network, the detection precision is high and the efficiency is high, the automatic detection of the safety helmet wearing situation of the personnel on the construction site is realized, and thus the problem of low efficiency of checking the safety helmet wearing situation in the prior art is solved.

[0055] The embodiment of the present application also provides a safety helmet wearing detection device for mine. It should be noted that the safety helmet wearing detection device for mine provided in the embodiment of the present application can be used to execute the safety helmet wearing detection method for mine provided in the embodiment of the present application. The device is used to realize the above embodiment and the preferred embodiment, and the description has been made and will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the device described in the following embodiment is preferably realized in software, the realization of hardware or the combination of software and hardware is also possible and conceived.

[0056] The safety helmet wearing detection device for mine provided in the embodiment of the present application is introduced below.

[0057] Figure 4 is a schematic diagram of the safety helmet wearing detection device for mine according to the embodiment of the present application. As shown in Figure 4 , the device comprises an acquisition unit 10, a training unit 20, a first determination unit 30 and a second determination unit 40, wherein:

[0058] The acquisition unit 10 is configured to acquire a target image, wherein the target image is an image including a face.

[0059] Specifically, the safety helmet wearing detection needs to be combined with human body detection, and only the safety helmet worn on the head is considered to meet the requirements, so it is necessary to acquire an image including a face.

[0060] In order to accurately acquire the image of the face, the acquisition unit comprises an acquisition module and a determination module, wherein the acquisition module is configured to acquire a live video and extract continuous image frames, and the live video is a video of the construction site collected by a camera; and the determination module is configured to determine the image frame containing the face image as the target image.

[0061] The training unit 20 is configured to construct a Faster RCNN neural network algorithm structure, train the Faster RCNN neural network algorithm structure, and obtain a safety helmet wearing detection model, wherein the Faster RCNN neural network algorithm structure uses VGG16 as a convolutional neural network to extract image features.

[0062] Specifically, Faster RCNN can be simply understood as the combination of RPN and Fast RCNN-a deep fully convolutional network RPN is used to generate candidate regions, and a Fast RCNN detector uses the candidate regions output by the RPN network for classification and bounding box regression.

[0063] In actual applications, only after being well trained, the deep learning neural network algorithm can accurately identify the construction workers and safety hats in the pictures, and provide accuracy for subsequent judgment. The Faster RCNN neural network algorithm structure is trained to obtain a safety hat wearing detection model, including: obtaining a plurality of sample images; inputting the plurality of sample images into the Faster RCNN neural network algorithm structure in sequence, and performing unsupervised training to obtain the safety hat wearing detection model. Traditional deep learning algorithms usually use supervised learning, that is, training is performed through existing training samples (that is, known data and corresponding outputs) to obtain an optimal model, and then the model is used to judge new data samples to achieve the purpose of classification. For example, a large number of construction site pictures are collected, and then artificial recognition is performed, and each picture sample is marked with a safety hat or no safety hat, and then these labeled image data are used for algorithm training. The present application adopts an unsupervised training method, which means that pictures without artificial marking are put into the algorithm for unsupervised pre-training, and then a small amount of image sample data with label information is combined for algorithm fine-tuning training to obtain a trained model. This method not only reduces the labor cost of a large amount of manual labeling and greatly reduces the training cost, but also has a slightly better effect than the supervised learning method, and realizes the improvement of the effect.

[0064] In order to realize safety hat recognition in special environments, in an optional solution, the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, the RPN network, the ROI pooling layer, and the full connection layer.

[0065] The first determination unit 30 is configured to use the safety hat wearing detection model to perform image recognition on the target image, and determine whether there is a safety hat above the face in the target image.

[0066] Specifically, the face position of the personnel in the target image is found, and then the safety hat above the face is recognized according to the face position.

[0067] In order to improve the accuracy of the safety helmet recognition, in an optional solution, the first determining unit includes a first input module, a second input module, a third input module and a fourth input module. The first input module is configured to input the target image into the convolutional neural network to calculate a feature map of the target image. The second input module is configured to input the feature map into the RPN network to obtain a first detection candidate frame. The third input module is configured to input the first detection candidate frame into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame into a fixed size output to obtain a second detection candidate frame. The fourth input module is configured to perform target classification and bounding box regression on the second detection candidate frame through the full connection layer to determine the positioning of the safety helmet in the target image.

[0068] In practical applications, the convolutional neural network is composed of three layers, i.e., a convolution layer (conv), an activation function layer (relu) and a pooling layer (pooling). As for VGG16, the network is composed of 13 convolution layers, 13 activation function layers and 4 pooling layers. The key of the convolution layer is a convolution kernel, which is a matrix, and the commonly used size is 3x3. The convolution kernel can obtain a feature map of the original image by performing multiplication operation on the input image matrix according to a predetermined step size.

[0069] In an optional solution, the activation function of the convolutional neural network adopts ReLu function. A lot of calculation is saved, and the ReLu function outputs 0 in the case that the neuron activation value is negative, so that the network has sparsity and the overfitting problem can be alleviated.

[0070] In practical applications, the pooling layer is used to reduce the dimension of the input data, reduce the number of parameters and calculation amount, and make the input of the next layer more concise, so as to facilitate the network to extract features more widely and avoid overfitting to a certain extent. In the convolutional neural network: 1) all the conv layers are kernel_size=3, padding=1, stride=1; 2) all the pooling layers are kernel_size=2, padding=0, stride=2. Among them, padding=1 means adding a circle of 0 around the matrix, and padding=0 means not adding 0. After each conv layer, the size of the feature map will not change, only the feature will be extracted by convolution; after each relu layer, the size of the feature map will not change, only the pixel points with negative values in the feature map will be reset to 0; after the pooling layer, the width and height of the feature map are reduced to half of the original. Therefore, after passing through the convolutional neural network, an image with a size of MxN will become a feature map with a size of (M / 16)x(N / 16).

[0071] The generation of the detection frame is the main bottleneck of the low real-time of some classic detection algorithms, such as the selective search (Selective Search) method used by RCNN to generate the detection frame, which slows down the detection speed, and the Faster RCNN algorithm introduces the RPN (Region Proposal Network) network for the generation of the detection frame, which greatly improves the detection efficiency. In an optional solution, the second input module includes a classification submodule and a correction submodule. The classification submodule is configured to generate a plurality of initial detection frames according to the features of the feature map, and use a softmax function to classify the plurality of initial detection frames as foreground and background. The correction submodule is configured to use bounding box regression to correct the plurality of initial detection frames to obtain the first detection candidate frame map.

[0072] Specifically, a very key concept in the RPN network is anchor. If it is simply summarized in one sentence, anchor is a reference frame of different sizes and aspect ratios preset on the image, that is, an initial detection frame. In the Faster RCNN algorithm, the number of anchors corresponding to each pixel point is preset to 9, and the 9 different anchors are obtained by combining three sizes [128x128, 256x256, 512x512] and three aspect ratios [1:1, 1:2, 2:1]. After traversing the feature map, each point corresponds to k anchors as an initial detection frame.

[0073] In actual application, bounding box regression is used to fine-tune the candidate detection frame, so that the window after fine-tuning is closer to the ground truth, and more accurate positioning is achieved.

[0074] In order to enhance the detection effect, in an optional solution, after generating a plurality of initial detection frames according to the features of the feature map, before using bounding box regression to correct the plurality of initial detection frames, the device further includes a removal unit configured to use a Soft NMS algorithm to remove redundant initial detection frames.

[0075] Specifically, for anchor-based target detection algorithm, because the size and specific location of the target are not known in advance, it is necessary to use overlapping sampling or sliding window to exhaust all possible targets. But this approach will lead to many candidate boxes pointing to the same target object, resulting in redundancy, so it is necessary to delete the repeated candidate boxes, and the non-maximum suppression algorithm (NMS) can be used to remove redundant candidate boxes. The idea of non-maximum suppression algorithm (NMS) is to suppress non-maximum values to find local maximum value targets, that is, to remove redundancy and find the optimal. The input of the algorithm is the boxes detected in the last step (the same target object may correspond to many boxes, and each box has its own classification score), and the output is the optimal box. The general process is as follows: 1) For a candidate box list B, find the box with the highest confidence (score), denoted as M; 2) delete M from the candidate box list B, and add the deleted M to the last reserved list D; 3) calculate the IoU (intersection over union) of all boxes in list B and M (the highest score), delete all boxes in B whose IoU is greater than a given threshold (the degree of repetition is too high); 4) repeat the above steps until all candidate boxes are traversed; the boxes in list D are the required boxes. The traditional NMS algorithm has a problem-easy to delete the overlapping targets in the picture, that is, when there are overlapping targets in the detection box, the NMS algorithm will also directly delete them. The detection algorithm should output two boxes, but when the IoU of the two boxes is greater than the set threshold, the NMS algorithm will delete the green box with lower confidence, resulting in the detection of only one object, affecting the detection effect. To solve this problem, the Soft NMS algorithm is used instead of the NMS algorithm, which does not directly delete the detection box with IoU greater than the threshold during algorithm execution, but reduces its original confidence score. On the other hand, Soft NMS can be easily introduced into the target detection algorithm, without the need to retrain the model, the code is easy to implement, and the computational complexity is relatively small, so the improved NMS algorithm Soft NMS is used to remove redundant detection boxes.

[0076] In practical applications, for the traditional convolutional neural network, the size of the input image to the trained network must be a fixed value, and the VGG16 network also needs to meet this requirement. In view of the fact that the original image is often of different sizes, there are usually two solutions: one is to cut the image, and the other is to stretch or shorten the image. However, neither of these two methods is satisfactory, as they either result in incomplete images or change the original proportions of the images. Therefore, Faster RCNN proposes RoI Pooling. RoI (Region of Interest) is a candidate recognition region found from an image. The candidate recognition region (RoI) is obtained by mapping the candidate box output by the RPN network to the feature map, that is, the RoI is a candidate region (proposal region) for the original image, as shown in Figure 3 The role of RoI Pooling is to normalize each candidate region from the input list to a fixed-size target recognition region. The implementation of normalization is as follows: 1) According to the input image, map the candidate region to the corresponding position of the feature map 2) Divide the mapped region into equal-sized parts (the number of which is equal to the output dimension), find the maximum value (max pooling) of each part, 3) Keep and output the found maximum value. Using RoIPooling, the training and testing speed can be significantly accelerated, the detection accuracy can be improved, and the model can be trained in an end-to-end manner.

[0077] The final classification part predicts the class of the detection box and the translation and scaling coefficients of the target box by Fast RCNN. The RoI after pooling is first converted from three dimensions to one dimension, and then passed to the fully connected layer. Using the obtained proposal feature map, the class to which each proposal belongs (whether the safety helmet is correctly worn) is determined by full connection and softmax; at the same time, the proposals are subjected to bounding box regression again to obtain a detection box with higher accuracy. Each node in the fully connected layer is connected to all nodes in the previous layer, which is used to integrate all the features extracted before. It maps the learned distributed feature representation to the sample label space and serves as the final "classifier" in the entire convolutional neural network.

[0078] The second determination unit 40 is configured to determine that the person in the target image wears a safety helmet when the safety helmet is above the face.

[0079] Specifically, the fact that the safety helmet is above the face can accurately determine that the person in the target image wears a safety helmet.

[0080] Through the above embodiment, the safety helmet wearing detection model is constructed by using the convolutional neural network, the algorithm used is Faster RCNN, VGG16 is used as the backbone network of the feature extraction network, the detection precision is high and the efficiency is high, the automatic detection of the safety helmet wearing situation of the personnel on the construction site is realized, and thus the problem of low efficiency of checking the safety helmet wearing situation in the prior art is solved.

[0081] The wearing detection device of the mining safety helmet includes a processor and a memory, the acquisition unit, the training unit, the first determination unit and the second determination unit are all stored in the memory as program units, and the corresponding functions are realized by executing the program units stored in the memory by the processor. The modules are all located in the same processor, or the modules are respectively located in different processors in any combination.

[0082] The processor includes a core, and the core calls the corresponding program unit from the memory. One or more cores can be set, and the problem of low efficiency of checking the safety helmet wearing situation in the prior art is solved by adjusting the core parameters.

[0083] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0084] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium includes a stored program, wherein the program controls the device where the computer readable storage medium is located to execute the wearing detection method of the mining safety helmet when the program runs.

[0085] Specifically, the wearing detection method of the mining safety helmet includes:

[0086] In step S201, a target image is acquired, and the target image is an image including a face;

[0087] Specifically, the safety helmet wearing detection needs to be combined with human body detection, and only the safety helmet worn on the head is considered to meet the requirements, therefore, an image including a face needs to be acquired.

[0088] In step S202, a Faster RCNN neural network algorithm structure is constructed, the Faster RCNN neural network algorithm structure is trained, and a safety helmet wearing detection model is obtained, and the Faster RCNN neural network algorithm structure uses VGG16 as a convolutional neural network to extract image features.

[0089] Specifically, Faster RCNN can be simply understood as the combination of RPN and Fast RCNN - a deep fully convolutional network RPN is used to generate candidate regions, and a Fast RCNN detector uses the candidate regions output by the RPN network for classification and bounding box regression.

[0090] In step S203, the safety helmet wearing detection model is used to perform image recognition on the target image to determine whether the safety helmet is above the face in the target image.

[0091] Specifically, the face position of the person in the target image is found, and then the safety helmet above the face is recognized according to the face position.

[0092] In step S204, if the safety helmet is above the face, it is determined that the person in the target image wears a safety helmet.

[0093] Specifically, the safety helmet above the face can accurately determine that the person in the target image wears a safety helmet.

[0094] Optionally, the target image is obtained by: obtaining a live video, and extracting continuous image frames, wherein the live video is a video of a construction site captured by a camera; and determining the image frame containing the face image as the target image.

[0095] Optionally, the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, the RPN network, the ROI pooling layer, and the fully connected layer.

[0096] Optionally, the safety helmet wearing detection model is used to perform image recognition on the target image, including: inputting the target image into the convolutional neural network to calculate a feature map of the target image; inputting the feature map into the RPN network to obtain a first detection candidate frame map; inputting the first detection candidate frame map into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame map into a fixed size output to obtain a second detection candidate frame map; and performing target classification and bounding box regression on the second detection candidate frame map through the fully connected layer to determine the position of the safety helmet in the target image.

[0097] Optionally, the feature map is input into the RPN network to obtain a first detection candidate frame map, including: generating a plurality of initial detection frames according to the features of the feature map; using a softmax function to classify the plurality of initial detection frames as foreground and background; and using bounding box regression to correct the plurality of initial detection frames to obtain the first detection candidate frame map.

[0098] Optionally, after generating a plurality of initial detection boxes according to the features of the feature map, before correcting the plurality of initial detection boxes by using the bounding box regression, the method further comprises: removing redundant initial detection boxes by using a Soft NMS algorithm.

[0099] Optionally, the activation function of the convolutional neural network uses a ReLu function.

[0100] The embodiment of the present application provides a processor used for running a program, wherein the processor executes the safety helmet wearing detection method for mine when the program is running.

[0101] Specifically, the safety helmet wearing detection method for mine comprises:

[0102] In step S201, a target image is acquired, and the target image is an image comprising a face;

[0103] Specifically, the safety helmet wearing detection needs to be combined with human body detection, and only the safety helmet worn on the head is considered to meet the requirements, therefore, an image comprising a face needs to be acquired.

[0104] In step S202, a Faster RCNN neural network algorithm structure is constructed, and the Faster RCNN neural network algorithm structure is trained to obtain a safety helmet wearing detection model, and the Faster RCNN neural network algorithm structure uses VGG16 as a convolutional neural network to extract image features.

[0105] Specifically, the Faster RCNN can be simply understood as the combination of RPN and Fast RCNN, that is, a deep fully convolutional network RPN is used to generate a candidate region, and a Fast RCNN detector uses the candidate region output by the RPN network to perform classification and bounding box regression.

[0106] In step S203, the safety helmet wearing detection model is used for image recognition on the target image to determine whether there is a safety helmet above the face in the target image.

[0107] Specifically, the face position of a person in the target image is found, and then the safety helmet above the face is recognized according to the face position.

[0108] In step S204, in the case that there is the safety helmet above the face, it is determined that the person in the target image wears a safety helmet.

[0109] Specifically, it can be accurately determined that the person in the target image wears a safety helmet by the safety helmet above the face.

[0110] Optionally, the target image is acquired by acquiring live video and extracting continuous image frames, the live video being video of a construction site captured by a camera; and the image frame containing a face image is determined as the target image.

[0111] Optionally, the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, the RPN network, the ROI pooling layer and the full connection layer.

[0112] Optionally, the image recognition of the target image by the safety helmet wearing detection model comprises: inputting the target image into the convolutional neural network to calculate a feature map of the target image; inputting the feature map into the RPN network to obtain a first detection candidate frame map; inputting the first detection candidate frame map into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame map into a fixed size output to obtain a second detection candidate frame map; and performing target classification and bounding box regression on the second detection candidate frame map by the full connection layer to determine the positioning of the safety helmet in the target image.

[0113] Optionally, the inputting of the feature map into the RPN network to obtain a first detection candidate frame map comprises: generating a plurality of initial detection frames according to the features of the feature map; performing foreground and background classification on the plurality of initial detection frames by using a softmax function; and correcting the plurality of initial detection frames by using a bounding box regression to obtain the first detection candidate frame map.

[0114] Optionally, after the plurality of initial detection frames are generated according to the features of the feature map, before the plurality of initial detection frames are corrected by using the bounding box regression, the method further comprises: removing redundant initial detection frames by using a Soft NMS algorithm.

[0115] Optionally, the activation function of the convolutional neural network is a ReLu function.

[0116] An apparatus is provided, the apparatus comprising a processor, a memory, and a program stored on the memory and executable on the processor, the processor implementing at least the following steps when executing the program:

[0117] In step S201, a target image is acquired, the target image being an image containing a face;

[0118] Specifically, safety helmet wearing detection needs to be combined with human body detection, and only a safety helmet worn on the head is considered to meet the requirements, so an image containing a face needs to be acquired.

[0119] In step S202, a Faster RCNN neural network algorithm structure is constructed, and the Faster RCNN neural network algorithm structure is trained to obtain a safety helmet wearing detection model. The Faster RCNN neural network algorithm structure uses VGG16 as a convolutional neural network to extract image features.

[0120] Specifically, the Faster RCNN can be simply understood as a combination of the RPN and the Fast RCNN. The deep fully convolutional network RPN is used to generate a candidate region, and the Fast RCNN detector uses the candidate region output by the RPN network for classification and bounding box regression.

[0121] In step S203, the safety helmet wearing detection model is used for image recognition on the target image to determine whether the safety helmet is above the face in the target image.

[0122] Specifically, the face position of the personnel in the target image is found, and the safety helmet above the face is recognized according to the face position.

[0123] In step S204, it is determined that the personnel in the target image wears the safety helmet when the safety helmet is above the face.

[0124] Specifically, it can be accurately determined that the personnel in the target image wears the safety helmet when the safety helmet is above the face.

[0125] Optionally, the target image is obtained by obtaining a live video and extracting continuous image frames. The live video is a video of a construction site collected by a camera. The image frame containing the face image is determined as the target image.

[0126] Optionally, the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, the RPN network, the ROI pooling layer, and the full connection layer.

[0127] Optionally, the safety helmet wearing detection model is used for image recognition on the target image, including: inputting the target image into the convolutional neural network to calculate a feature map of the target image; inputting the feature map into the RPN network to obtain a first detection candidate frame map; inputting the first detection candidate frame map into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame map into a fixed size output to obtain a second detection candidate frame map; and performing target classification and bounding box regression on the second detection candidate frame map through the full connection layer to determine the positioning of the safety helmet in the target image.

[0128] Optionally, the feature map is input into the RPN network to obtain a first detection candidate frame map, including: generating a plurality of initial detection frames according to the features of the feature map, classifying the plurality of initial detection frames as foreground and background by using a softmax function; and correcting the plurality of initial detection frames by using a bounding box regression to obtain the first detection candidate frame map.

[0129] Optionally, after the plurality of initial detection frames are generated according to the features of the feature map, before the plurality of initial detection frames are corrected by using the bounding box regression, the method further includes: removing redundant initial detection frames by using a Soft NMS algorithm.

[0130] Optionally, the activation function of the convolutional neural network uses a ReLu function.

[0131] The device herein can be a server, a PC, a PAD, a mobile phone, etc.

[0132] The application also provides a computer program product adapted to execute a program including at least the following steps when executed on a data processing device:

[0133] Step S201, obtaining a target image, the target image being an image including a face;

[0134] Specifically, the safety hat wearing detection needs to be combined with human body detection, and only the safety hat worn on the head is considered to meet the requirements, so an image including a face needs to be obtained.

[0135] Step S202, constructing a Faster RCNN neural network algorithm structure, training the Faster RCNN neural network algorithm structure, and obtaining a safety hat wearing detection model, the Faster RCNN neural network algorithm structure using VGG16 as a convolutional neural network to extract image features;

[0136] Specifically, the Faster RCNN can be simply understood as the combination of RPN and Fast RCNN--a deep fully convolutional network RPN is used to generate candidate regions, and a Fast RCNN detector uses the candidate regions output by the RPN network for classification and bounding box regression.

[0137] Step S203, performing image recognition on the target image by using the safety hat wearing detection model to determine whether there is a safety hat above the face in the target image;

[0138] Specifically, the face position of a person in the target image is found, and then the safety hat above the face is recognized according to the face position.

[0139] Step S204, in the case that the safety helmet is above the face, it is determined that the person in the target image wears the safety helmet.

[0140] Specifically, it can be determined that the person in the target image wears the safety helmet by the safety helmet above the face.

[0141] Optionally, the target image is obtained by obtaining a live video and extracting a continuous image frame, wherein the live video is a video of a construction site captured by a camera, and the image frame containing the face image is the target image.

[0142] Optionally, the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, the RPN network, the ROI pooling layer and the full connection layer.

[0143] Optionally, the safety helmet wearing detection model is used for image recognition of the target image, including: inputting the target image into the convolutional neural network to calculate a feature map of the target image; inputting the feature map into the RPN network to obtain a first detection candidate frame map; inputting the first detection candidate frame map into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame map into a fixed size output to obtain a second detection candidate frame map; and performing target classification and bounding box regression on the second detection candidate frame map through the full connection layer to determine the positioning of the safety helmet in the target image.

[0144] Optionally, inputting the feature map into the RPN network to obtain the first detection candidate frame map includes: generating a plurality of initial detection frames according to the features of the feature map, classifying a plurality of the initial detection frames as foreground and background by using a softmax function; and correcting a plurality of the initial detection frames by using a bounding box regression to obtain the first detection candidate frame map.

[0145] Optionally, after the plurality of initial detection frames are generated according to the features of the feature map, before the plurality of initial detection frames are corrected by using the bounding box regression, the method further includes: removing redundant initial detection frames by using a Soft NMS algorithm.

[0146] Optionally, the activation function of the convolutional neural network is a ReLu function.

[0147] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with a general purpose computer, and can be centralized in a single computer or distributed among a network of computers, and can be implemented with program code executable by a computer, and thus can be stored in a storage device and executed by a computer, and in some cases, the steps shown or described can be executed in a different order than shown or described, or can be implemented as separate integrated circuit modules or as a single integrated circuit module, and thus the application is not limited to any particular combination of hardware and software.

[0148] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be in the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) embodying computer readable program code.

[0149] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 The means for implementing each one or more flow or steps and / or blocks in the flowchart illustrations and / or block diagrams can be implemented by a machine such as a computer, a processor, or other programmable data processing apparatus in the form of a series of one or more computer readable program code means or steps embodied in the computer readable program code means. Figure 1 The means for implementing each one or more flow or steps and / or blocks in the flowchart illustrations and / or block diagrams can be implemented by a machine such as a computer, a processor, or other programmable data processing apparatus in the form of a series of one or more computer readable program code means or steps embodied in the computer readable program code means.

[0150] The computer readable program code means or steps can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 The means for implementing each one or more flow or steps and / or blocks in the flowchart illustrations and / or block diagrams can be implemented by a machine such as a computer, a processor, or other programmable data processing apparatus in the form of a series of one or more computer readable program code means or steps embodied in the computer readable program code means. Figure 1 The means for implementing each one or more flow or steps and / or blocks in the flowchart illustrations and / or block diagrams can be implemented by a machine such as a computer, a processor, or other programmable data processing apparatus in the form of a series of one or more computer readable program code means or steps embodied in the computer readable program code means.

[0151] The computer readable program code means or steps can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams.Figure 1 one or more processes and / or functions specified in one or more blocks Figure 1 one or more processes and / or functions specified in one or more blocks

[0152] In one typical arrangement, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0153] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0154] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0155] It should also be noted that the terms "comprising", "containing", or any other variant thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0156] From the above description, it can be seen that the above-mentioned embodiments of the present application achieve the following technical effects:

[0157] 1) In the above-mentioned method for detecting the wearing of mining safety helmets in this application, firstly, a target image is acquired, which is an image including a face; then, a Faster R-CNN neural network algorithm structure is constructed and trained to obtain a safety helmet wearing detection model. The Faster R-CNN neural network algorithm structure uses VGG16 as a convolutional neural network to extract image features; then, the safety helmet wearing detection model is used to perform image recognition on the target image to determine whether there is a safety helmet above the face in the target image; finally, if there is a safety helmet above the face, it is determined that the person in the target image is wearing a safety helmet. This method uses a convolutional neural network to construct a safety helmet wearing detection model, uses the Faster R-CNN algorithm, and uses VGG16 as the backbone network of the feature extraction network. It has high detection accuracy and efficiency, realizes automatic detection of the safety helmet wearing status of personnel at construction sites, and thus solves the problem of low efficiency in checking the safety helmet wearing status in existing technologies.

[0158] 2) The helmet-wearing detection device for mining applications includes an acquisition unit, a training unit, a first determination unit, and a second determination unit. The acquisition unit acquires a target image, which is an image including a face. The training unit constructs a Faster R-CNN neural network algorithm structure and trains it to obtain a helmet-wearing detection model. The Faster R-CNN algorithm uses VGG16 as a convolutional neural network to extract image features. The first determination unit uses the helmet-wearing detection model to perform image recognition on the target image to determine whether a helmet is above the face in the target image. The second determination unit determines that the person in the target image is wearing a helmet if a helmet is above the face. This device utilizes a convolutional neural network to construct a helmet-wearing detection model, using the Faster R-CNN algorithm and VGG16 as the backbone of the feature extraction network. It achieves high detection accuracy and efficiency, enabling automatic detection of helmet-wearing status for personnel at construction sites, thus solving the problem of low efficiency in helmet-wearing checks in existing technologies.

[0159] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method of detecting the wearing of a mining safety helmet, characterized by, The method comprises the following steps: acquiring a target image, wherein the target image is an image comprising a human face; constructing a Faster RCNN neural network algorithm structure and training the Faster RCNN neural network algorithm structure to obtain a safety helmet wearing detection model, wherein the Faster RCNN neural network algorithm structure uses VGG16 as a convolutional neural network to extract image features; performing image recognition on the target image by using the safety helmet wearing detection model to determine whether there is a safety helmet above the human face in the target image; in the case that there is a safety helmet above the human face, determining that a person in the target image wears a safety helmet; the Faster RCNN neural network algorithm structure is composed of the convolutional neural network, an RPN network, an ROI pooling layer and a fully connected layer; the image recognition on the target image by using the safety helmet wearing detection model comprises the following steps: inputting the target image into the convolutional neural network to calculate a feature map of the target image; inputting the feature map into the RPN network to obtain a first detection candidate frame map; inputting the first detection candidate frame map into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame map into a fixed-size output to obtain a second detection candidate frame map; performing target classification and bounding box regression on the second detection candidate frame map by using the fully connected layer to determine the positioning of the safety helmet in the target image.

2. The method of claim 1, wherein, acquiring a target image comprises the following steps: acquiring a live video and extracting continuous image frames, wherein the live video is a video of a construction site collected by a camera; determining the image frames comprising human face images as the target images.

3. The method of claim 1, wherein, inputting the feature map into the RPN network to obtain a first detection candidate frame map comprises the following steps: generating a plurality of initial detection frames according to the features of the feature map, and classifying a plurality of the initial detection frames as foreground and background by using a softmax function; modifying a plurality of the initial detection frames by using bounding box regression to obtain the first detection candidate frame map.

4. The method of claim 3, wherein, after generating a plurality of initial detection frames according to the features of the feature map and before modifying a plurality of the initial detection frames by using bounding box regression, the method further comprises the following step: removing redundant initial detection frames by using a Soft NMS algorithm.

5. The method of claim 1, wherein, the activation function of the convolutional neural network uses a ReLu function.

6. A device for detecting the wearing of a safety cap for mining, characterized by The method comprises the following steps: an acquiring unit is configured to acquire a target image, wherein the target image is an image comprising a human face; a training unit is configured to construct a Faster RCNN neural network algorithm structure and train the Faster RCNN neural network algorithm structure to obtain a safety helmet wearing detection model, wherein the Faster RCNN neural network algorithm structure uses VGG16 as a convolutional neural network to extract image features; a first determining unit is configured to perform image recognition on the target image by using the safety helmet wearing detection model to determine whether there is a safety helmet above the human face in the target image; A second determination unit is configured to determine that the person in the target image wears a safety helmet when the safety helmet is above the face. The Faster RCNN neural network algorithm structure is composed of the convolutional neural network, the RPN network, the ROI pooling layer, and the full connection layer. The first determination unit comprises: A first input module is configured to input the target image into the convolutional neural network to calculate a feature map of the target image. A second input module is configured to input the feature map into the RPN network to obtain a first detection candidate frame map. A third input module is configured to input the first detection candidate frame map into the ROI pooling layer to convert a first candidate recognition region in the first detection candidate frame map into a fixed size output to obtain a second detection candidate frame map. A fourth input module is configured to perform target classification and frame regression on the second detection candidate frame map through the full connection layer to determine the positioning of the safety helmet in the target image.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program controls the device where the computer readable storage medium is located to perform the safety helmet wearing detection method of any one of claims 1 to 5 when the program is running.

8. An electronic device, comprising: comprise: one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise a program for executing the safety helmet wearing detection method of any one of claims 1 to 5.

Images