Safety helmet segmentation method based on electric power scene

Abstract

The invention relates to the technical field of electric power scene image segmentation, in particular to a safety helmet segmentation method based on an electric power scene, and the method comprises the following steps: collecting an input image containing a safety helmet in an electric power operation scene; constructing and training a safety helmet segmentation model; and inputting the input image into the safety helmet segmentation model to obtain a pixel-level prediction image of the to-be-input image, and analyzing the input electric power scene image into a pixel-level prediction image by the model so as to segment the safety helmet image in the electric power scene.

Description

Technical Field

[0001] The present invention relates to the technical field of power scene image segmentation, and in particular to a safety helmet segmentation method based on a power scene. Background Art

[0002] The method of helmet segmentation in power scenes involves multiple technologies. First of all, semantic segmentation technology plays an important role in this task. Semantic segmentation aims to assign each pixel in the image to a specific semantic category, so it can be used to distinguish and segment the helmet from other objects in the power scene. This technology can provide a basis for subsequent helmet detection and analysis.

[0003] Convolutional Neural Network (CNN) is a powerful tool for helmet segmentation. CNN has the ability to extract features from images and learn the mapping relationship between features and semantic categories. By using CNN, the features of helmets can be extracted from power scene images, helping the model to accurately segment the helmet area.

[0004] When training the helmet segmentation model, a large number of power scene image datasets with helmet annotations are required. These datasets need to be accurately annotated to classify each pixel as a helmet or non-helmet. The quality and diversity of the dataset are crucial to the training and generalization capabilities of the model.

[0005] Attention mechanism is another key technology that can help the model pay more attention to the features of the helmet area when processing power scene images. By introducing the attention mechanism, the model can automatically learn and emphasize the features related to the helmet, thereby improving the accuracy and robustness of segmentation.

[0006] A Chinese patent with publication number CN110070033B discloses "a method for detecting the wearing status of helmets in dangerous working areas in the power field". By detecting images on an improved deep learning model tiny-yolo, the images are divided into three categories, namely people, people wearing helmets and people not wearing helmets. The streamlined model can make the performance of mobile terminals, such as Android terminals, be best utilized within the allowed range. It can achieve high efficiency in detecting the wearing status of helmets in both videos and pictures, but it cannot detect whether the helmet is worn in compliance with regulations. Summary of the invention

[0007] In view of the above-mentioned prior art, the present invention provides a helmet segmentation method based on an electric power scenario, which mainly solves the technical problems existing in the above-mentioned background technology.

[0008] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0009] A safety helmet segmentation method based on an electric power scenario, the method comprising the following steps:

[0010] Collect input images of power work scenes including safety helmets;

[0011] Build and train a hard hat segmentation model;

[0012] The input image is input into the helmet segmentation model to obtain a pixel-level predicted image of the input image.

[0013] Optionally, the helmet segmentation model includes an encoding module, an attention module, a decoding module and a prediction module.

[0014] The encoding module is configured to perform convolution on the input image layer by layer, thereby extracting a low-resolution feature map of the image;

[0015] The attention module is configured to perform a pooling operation on the low-resolution feature map;

[0016] The decoding module is configured to generate a pixel-level predicted image matching the size of the input image based on the low-resolution feature map;

[0017] The prediction module is configured to generate a category label for each pixel, and classify each pixel in the pixel-level prediction image according to the category label to obtain a pixel-level image segmentation result.

[0018] Optionally, the encoding module is composed of a convolutional layer, a nonlinear layer, a pooling layer and a normalization layer.

[0019] The convolution layer is configured to extract local features of the input image;

[0020] The normalization layer is configured to perform a normalization operation on multiple local features;

[0021] The nonlinear layer is configured to introduce nonlinearity in the feature map through a ReLU activation function to enhance the expression capability;

[0022] The pooling layer is configured to reduce the spatial dimension of the extracted feature map and improve scale invariance and computational efficiency.

[0023] Optionally, the attention module is configured to perform a pooling operation on the low-resolution feature map, which specifically includes:

[0024] The low-resolution feature map output by the encoding module is used as input, and a global average pooling operation is performed on the output low-resolution feature map to convert the feature map of each channel into a scalar value;

[0025] Perform a one-dimensional convolution operation on the low-resolution feature map after global average pooling;

[0026] The low-resolution feature map after one-dimensional convolution is activated by the Sigmoid activation function;

[0027] The low-resolution feature map activated by the Sigmoid activation function is multiplied element by element with the low-resolution feature map output by the encoding module to achieve feature map scaling;

[0028] The scaled low-resolution feature map is added to the low-resolution feature map output by the encoding module to obtain a low-resolution feature map weighted by the attention mechanism.

[0029] Optionally, the decoding module generates a pixel-level prediction image matching the size of the input image, specifically including:

[0030] The low-resolution feature map output by the attention module is sent to the upsampling layer to restore the image to a higher spatial resolution;

[0031] The restored feature map with higher spatial resolution is sent to a convolutional layer for feature extraction and integration;

[0032] The feature extraction and integration results are input into a normalization layer for batch normalization, and the ReLU activation function of the nonlinear operation is performed during the batch normalization process.

[0033] Optionally, the specific steps of the prediction module obtaining the pixel-level image segmentation result include:

[0034] The decoder in the prediction module performs upsampling operations on the high-resolution feature map to generate a sparse upsampled feature map;

[0035] Generate dense feature maps using trainable filters through convolution operations;

[0036] The dense feature map is input into the softmax layer to predict the category of each pixel and finally obtain the pixel prediction result.

[0037] Optionally, a multi-classification loss function is used to train the helmet segmentation model, wherein the expression of the multi-classification loss function is:

[0038]

[0039] Where N is the total number of pixels in the image, n is the number of categories, and y is ac is the sign function, p ac is the predicted probability that sample a belongs to category c.

[0040] Optionally, judging whether the helmet is worn correctly based on the pixel-level predicted image includes the following specific steps:

[0041] The pixel classification result is converted into a binary image, where the pixel value of the helmet area is set to 1, the pixel value of the seat belt area is set to 2, and the pixel value of the background area is set to 0;

[0042] Use face detection algorithm to detect face area from pixel-level predicted image;

[0043] Find the connected domain in the binary image. If the connected domain is found, it means that there is a safety helmet and a safety belt in the safety helmet in the area. If the safety helmet area is located directly above the face area, and the safety belt area is located below the safety helmet area and the safety belt area overlaps with the face area, it means that the safety helmet is worn correctly.

[0044] The beneficial effects of the present invention are as follows: the method for segmenting a helmet based on an electric power scenario provided by the present application first effectively captures the semantic information in the image through an encoder and a decoder, and generates a prediction result that matches the size of the original image, and then improves the accuracy of model recognition by providing local attention; secondly, an innovative loss function is used to make the model training pay more attention to the segmentation boundary, and the training effect is better and more in line with the clothing image scenario; in addition, by inputting the electric power scene image into the model to generate an annotated picture, and then identifying whether the ear is between the two safety helmet straps, it is possible to detect whether the staff is wearing the safety helmet correctly. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flowchart of a method for segmenting a helmet based on an electric power scenario in an embodiment of the present application;

[0046] Figure 2 Schematic diagram of the helmet segmentation model in the embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is further elaborated in detail below in conjunction with the drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. In the following description, the expression "some embodiments" is related to a subset of all possible embodiments, but it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0048] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0049] It should be understood that the present invention can be implemented in different forms and should not be interpreted as being limited to the embodiments proposed herein. On the contrary, providing these embodiments will make the disclosure thorough and complete, and the scope of the present invention will be fully conveyed to those skilled in the art. And the purpose of the terms used herein is only to describe specific embodiments and is not intended to be a limitation of the present invention. When used herein, the singular forms of "one", "one" and "said / the" are also intended to include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "compose" and / or "include" when used in this specification determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0050] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "inside", "outside", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only implementation method.

[0051] In order to fully understand the present invention, a detailed structure will be proposed in the following description to illustrate the technical solution proposed by the present invention. The optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementations.

[0052] Please refer to the attached Figure 1 The present application provides a method for segmenting a helmet based on an electric power scenario, the method comprising the following steps:

[0053] S1, collecting input images of power operation scenes including safety helmets;

[0054] Among them, power operation scenes usually include actual application scenes such as power stations, power towers, indoor and outdoor operations. The input images can come from photos taken, or from videos taken by mobile terminals, and the images are output at a speed of 20 frames per second.

[0055] After obtaining the input image, the input image is also preprocessed by Gaussian smoothing and noise removal;

[0056] S2. Build and train a helmet segmentation model;

[0057] S3: Input the input image into the helmet segmentation model to obtain a pixel-level predicted image of the input image.

[0058] See also Figure 2 In an optional embodiment, in step S2, the helmet segmentation model includes an encoding module, an attention module, a decoding module and a prediction module.

[0059] The encoding module is configured to perform convolution on the input image layer by layer, thereby extracting a low-resolution feature map of the image;

[0060] The attention module is configured to perform a pooling operation on the low-resolution feature map;

[0061] The decoding module is configured to generate a pixel-level predicted image matching the size of the input image based on the low-resolution feature map;

[0062] The prediction module is configured to generate a category label for each pixel, and classify each pixel in the pixel-level prediction image according to the category label to obtain a pixel-level image segmentation result.

[0063] Specifically, the input image passes through the encoding module, and convolution operations are performed layer by layer to extract the feature map of the image. The feature map of the image is input into the attention module to make it pay more attention to small targets, and pooling operations are performed. The feature information after the pooling operation is sent to the decoding module, and the low-resolution feature map is mapped to the full input resolution. The generated feature map passes through the prediction module to classify each pixel and obtain the pixel-level semantic segmentation result.

[0064] In an optional embodiment, the encoding module is composed of a convolutional layer, a nonlinear layer, a pooling layer and a normalization layer.

[0065] The convolution layer is configured to extract local features of the input image;

[0066] The normalization layer is configured to perform a normalization operation on multiple local features;

[0067] The nonlinear layer is configured to introduce nonlinearity in the feature map through a ReLU activation function to enhance the expression capability;

[0068] The pooling layer is configured to reduce the spatial dimension of the extracted feature map, improve scale invariance and computational efficiency, and the pooling layer uses a sliding window of 2*2 size and a step size of 2 for downsampling operations. In each pooling layer of the encoder network, the index value of the maximum pooling operation is recorded for use when performing upsampling operations in the decoder.

[0069] In an optional embodiment, the attention module is configured to perform a pooling operation on the low-resolution feature map through an attention mechanism, and the attention mechanism can adaptively learn the importance of channels in the feature map and weight the feature map according to the importance to enhance the performance of the network on the helmet segmentation task. The attention module is configured to perform a pooling operation on the low-resolution feature map through an attention mechanism, which specifically includes:

[0070] The low-resolution feature map output by the encoding module is taken as input, and a global average pooling operation is performed on the output low-resolution feature map to convert the feature map of each channel into a scalar value, which can be achieved by calculating the average value on each channel;

[0071] Perform a one-dimensional convolution operation on the low-resolution feature map after global average pooling. The purpose of this convolution operation is to introduce dependencies between channels to capture long-distance context information in the feature map;

[0072] The low-resolution feature map after one-dimensional convolution is activated by the Sigmoid activation function, which will limit each element in the feature map to between 0 and 1, indicating its importance or weight;

[0073] The low-resolution feature map activated by the Sigmoid activation function is multiplied element by element with the low-resolution feature map output by the encoding module to achieve feature map scaling, so that the weight of the input feature map can be adjusted according to the importance of each channel;

[0074] The scaled low-resolution feature map is added to the low-resolution feature map output by the encoding module to obtain a low-resolution feature map weighted by the attention mechanism, which can highlight important channel features and suppress unimportant channel features.

[0075] In an optional implementation, the decoding module generates a pixel-level prediction image matching the size of the input image, specifically comprising:

[0076] The low-resolution feature map output by the attention module is sent to the upsampling layer to restore the image to a higher spatial resolution;

[0077] The restored feature map with higher spatial resolution is sent to a convolutional layer for feature extraction and integration;

[0078] The feature extraction and integration results are input into a normalization layer for batch normalization, and the ReLU activation function of the nonlinear operation is performed during the batch normalization process.

[0079] In an optional implementation, the specific steps of the prediction module obtaining the pixel-level image segmentation result include:

[0080] The decoder in the prediction module performs upsampling operations on the high-resolution feature map to generate a sparse upsampled feature map;

[0081] Generate dense feature maps using trainable filters through convolution operations;

[0082] The dense feature map is input into the softmax layer to predict the category of each pixel and finally obtain the pixel prediction result.

[0083] Furthermore, a multi-classification loss function is used to train the helmet segmentation model, wherein the expression of the multi-classification loss function is:

[0084]

[0085] Where N is the total number of pixels in the image, n is the number of categories, and y is ac is the sign function, p ac is the predicted probability that sample a belongs to category c.

[0086] In an optional embodiment, after step S3, when the predicted image based on the pixel level is obtained, the method further includes: judging whether the helmet is worn correctly based on the predicted image at the pixel level, and the specific steps include:

[0087] The pixel classification result is converted into a binary image, where the pixel value of the helmet area is set to 1, the pixel value of the seat belt area is set to 2, and the pixel value of the background area is set to 0;

[0088] Detecting the face region from the pixel-level predicted image using a face detection algorithm, such as a Haar cascade classifier or a deep learning face detection model;

[0089] Find connected domains in the binary image. Connected regions refer to regions with the same adjacent pixel values, such as regions composed entirely of pixels with a pixel value of 1, regions composed entirely of pixels with a pixel value of 2, and regions composed entirely of pixels with a pixel value of 0. If a connected domain is found, it means that a safety helmet and a safety belt in the safety helmet exist in the region. If the safety helmet region is located directly above the face region, and the safety belt region is located below the safety helmet region and the safety belt region overlaps with the face region, it means that the safety helmet is worn correctly, otherwise the safety helmet is worn incorrectly.

[0090] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A helmet segmentation method based on power scenarios, characterized in that: The method comprises the following steps: Collect input images of power work scenes including safety helmets; Build and train a hard hat segmentation model; The input image is input into the helmet segmentation model to obtain a pixel-level predicted image of the input image.

2. According to the method for segmenting safety helmets based on power scenarios in claim 1, it is characterized in that: The helmet segmentation model includes an encoding module, an attention module, a decoding module and a prediction module. The encoding module is configured to perform convolution on the input image layer by layer, thereby extracting a low-resolution feature map of the image; The attention module is configured to perform a pooling operation on the low-resolution feature map; The decoding module is configured to generate a pixel-level predicted image matching the size of the input image based on the low-resolution feature map; The prediction module is configured to generate a category label for each pixel, and classify each pixel in the pixel-level prediction image according to the category label to obtain a pixel-level image segmentation result.

3. According to the method for segmenting safety helmets based on power scenarios in claim 2, it is characterized in that: The encoding module consists of a convolutional layer, a nonlinear layer, a pooling layer and a normalization layer. The convolution layer is configured to extract local features of the input image; The normalization layer is configured to perform a normalization operation on multiple local features; The nonlinear layer is configured to introduce nonlinearity in the feature map through a ReLU activation function to enhance the expression capability; The pooling layer is configured to reduce the spatial dimension of the extracted feature map and improve scale invariance and computational efficiency.

4. The method for segmenting a helmet based on an electric power scenario according to claim 3 is characterized in that: The attention module is configured to perform a pooling operation on the low-resolution feature map, which specifically includes: The low-resolution feature map output by the encoding module is used as input, and a global average pooling operation is performed on the output low-resolution feature map to convert the feature map of each channel into a scalar value; Perform a one-dimensional convolution operation on the low-resolution feature map after global average pooling; The low-resolution feature map after one-dimensional convolution is activated by the Sigmoid activation function; The low-resolution feature map activated by the Sigmoid activation function is multiplied element by element with the low-resolution feature map output by the encoding module to achieve feature map scaling; The scaled low-resolution feature map is added to the low-resolution feature map output by the encoding module to obtain a low-resolution feature map weighted by the attention mechanism.

5. The method for segmenting a helmet based on an electric power scenario according to claim 4 is characterized in that: The decoding module generates a pixel-level predicted image that matches the size of the input image, specifically comprising: The low-resolution feature map output by the attention module is sent to the upsampling layer to restore the image to a higher spatial resolution; The restored feature map with higher spatial resolution is sent to a convolutional layer for feature extraction and integration; The feature extraction and integration results are input into a normalization layer for batch normalization, and the ReLU activation function of the nonlinear operation is performed during the batch normalization process.

6. The method for segmenting safety helmets based on power scenarios according to claim 5 is characterized in that: The specific steps of obtaining the pixel-level image segmentation result by the prediction module include: The decoder in the prediction module performs upsampling operations on the high-resolution feature map to generate a sparse upsampled feature map; Generate dense feature maps using trainable filters through convolution operations; The dense feature map is input into the softmax layer to predict the category of each pixel and finally obtain the pixel prediction result.

7. The method for segmenting a helmet based on an electric power scenario according to claim 6 is characterized in that: The helmet segmentation model is trained using a multi-classification loss function, where the expression of the multi-classification loss function is: Where N is the total number of pixels in the image, n is the number of categories, and y is ac is the sign function, p ac is the predicted probability that sample a belongs to category c.

8. The method for segmenting a helmet based on an electric power scenario according to claim 7, characterized in that: Based on the pixel-level prediction image, the specific steps of judging whether the helmet is worn correctly include: The pixel classification result is converted into a binary image, where the pixel value of the helmet area is set to 1, the pixel value of the seat belt area is set to 2, and the pixel value of the background area is set to 0; Use face detection algorithm to detect face area from pixel-level predicted image; Find the connected domain in the binary image. If the connected domain is found, it means that there is a safety helmet and a safety belt in the safety helmet in the area. If the safety helmet area is located directly above the face area, and the safety belt area is located below the safety helmet area and the safety belt area overlaps with the face area, it means that the safety helmet is worn correctly.

Images