Safety helmet monitoring method and system in power operation based on Faster-RCNN algorithm

Abstract

The present invention discloses an intelligent helmet detection method and system based on Faster-RCNN algorithm, which integrates transfer learning and Faster-RCNN algorithm to detect helmets. First, the Faster-RCNN network is improved, and the anchor frame generation scheme is improved by using k-means II, and the residual network and attention mechanism modules are introduced to enhance the detection accuracy. At the same time, the network fusion transfer learning solves the problem of insufficient samples, and then the entire built network is trained, and the training set is input into the network after the μth iteration for training, and its weight parameters are retained. A benchmark network of the test set is constructed, and the network parameters with the best effect in the μth iteration results are selected as the benchmark network of the test set, and the test set is input into the benchmark network for testing to obtain the test results. The detection accuracy is greatly improved, and the intelligent recognition technology of graphics and images is used to identify whether the personnel within the scope of the on-site operation are wearing helmets in a standardized manner, and timely warnings are given to prevent accidents in the on-site operation.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and system for monitoring safety helmets in electric power operations based on a Faster-RCNN algorithm. Background Art

[0002] During the inspection of power transmission lines, workers wearing helmets can not only protect their safety, but also collect and locate surrounding information through sensors in the helmets, which can be used as the location positioning basis for environmental status monitoring and inspection management of transmission lines. The existing helmet monitoring platform provides inspection personnel with functions such as equipment fault entry, task query, and equipment information query. However, due to factors such as light intensity and occlusion during transmission line inspection, helmet monitoring is not accurate enough. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for monitoring safety helmets in power operations based on the Faster-RCNN algorithm, so as to solve the problem of inaccurate safety helmet monitoring of personnel within the on-site operation range due to factors such as light intensity and shading.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] The method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm includes:

[0006] Collect inspection videos, obtain target images through inspection videos, use annotation tools to annotate the helmets in the single-frame target images, and save their information to obtain sample data;

[0007] Faster-RCNN is used to detect the target area in the image: first, the image samples in the pre-training database ImageNet are resampled according to the similarity of the target, and then the network parameters are pre-trained using the resampled data set;

[0008] Combine transfer learning and Faster-RCNN network to train the network, perform μ iterations of training, and retain the best network weight parameters;

[0009] Construct a benchmark network for the test set, select the network parameters with the best effect in μ iteration results as the benchmark network for the test set, and input the test set into the benchmark network for testing to obtain the test results.

[0010] Further, obtaining the target image includes:

[0011] Determine the hard hat to be marked, and obtain the corresponding marking box according to the area where the hard hat is located; divide the marked hard hat into a pre-set state label; generate the corresponding xml document from the marked hard hat image, and the marked hard hat information in each document contains four position attributes and 1 category attribute.

[0012] Furthermore, the original Faster-RCNN network specifically consists of the following four parts:

[0013] Backbone network: used to extract input image feature representation and provide effective input features for the RPN network;

[0014] RPN network: receives the feature representation extracted by the backbone network and constructs candidate regions for detection targets;

[0015] Region of Interest (ROI) Pooling Layer: Maps candidate regions to feature maps and pools them to a uniform scale for subsequent detection and classification;

[0016] Classification and regression: Classify the target categories in the feature area and use bounding box regression to correct the target box to obtain the precise position offset.

[0017] Furthermore, the Faster-RCNN network is adjusted and optimized, and the detection of the target area in the image using Faster-RCNN specifically includes:

[0018] Improvements to the anchor box generation scheme, where the anchor is a set of rectangular boxes generated by RPN, using three scales and three aspect ratios, generating k = 9 anchors at the center of each sliding window; making the anchor scheme correspond to the size of the defect being detected;

[0019] Use the k-means II algorithm to classify the input data set containing N points into k clusters, so that all points in the data set are classified into the class where the cluster center is closest to them, and use the Euclidean distance as the standard for measuring distance during classification;

[0020] The Res2Net module is introduced. This module uses a smaller convolution kernel group to replace the original one, and uses the smaller convolution kernel to capture more fine-grained features. At the same time, the input feature maps are grouped and the groups are connected in the form of residuals.

[0021] First, the input features are obtained through a 1*1 convolutional layer; then, they are divided into S groups along the channel dimension, and each feature subset is denoted as x i ∈R H×W×C / S,i=2,.....,S, the convolution layer corresponding to the i-th group is recorded as Fi, and its output is recorded as fi; except for the first group, all other xi pass through a 3*3 convolution layer respectively, and when i is greater than 2, xi and fi-1 are fused and passed through a 3*3 convolution layer to obtain the output of the i-th group; secondly, the outputs of each group are fused and passed through a 1*1 convolution layer; the above process is defined as follows:

[0022]

[0023] Finally, the input features are fused with the output of formula (3) to obtain the output features of the Res2Net module;

[0024] The channel attention module and the spatial attention module are introduced into the above Res2Net module, and the SCA-Res2Net module is proposed. Two SE blocks and one SA block are added after the second 1*1 convolution layer, and the input features are cascaded after the first SE block. The effective multi-scale discriminative information is enhanced from the spatial and channel dimensions, the background interference information is suppressed, and the loss of spatial information of the input features is reduced through the residual structure.

[0025] Furthermore, the specific steps of the k-means II algorithm are as follows:

[0026] (1) Input a data set containing k points;

[0027] (2) Randomly select k points in the data set as cluster centers, and then repeat this sampling process 5 times to obtain 5k data points. Then cluster these data points to obtain k initial cluster centers.

[0028] (3) Divide each point in the data set into the class with the closest cluster center according to the Euclidean distance; the Euclidean distance formula is:

[0029]

[0030] (4) Recalculate the cluster center of each class using the following calculation method:

[0031]

[0032] where |c i | is the number of samples of this type;

[0033] (5) Set a threshold. If the distance between the new center obtained by equation (2) and the original center is less than the threshold, clustering ends. Otherwise, go to step (3).

[0034] (6) Output k clusters and k cluster centers;

[0035] Furthermore, the image samples in the pre-training database ImageNet are resampled, and the network parameters are pre-trained using the resampled data set:

[0036] Based on similarity resampling, a data set of migration samples is constructed and an evaluation function of target similarity is established; the measurement method used is:

[0037]

[0038] Where: I1 and I2 represent any two images in the training sample set and the target sample set respectively;

[0039] f(I1) represents the feature vector extracted from the image, which is calculated using the gray-level co-occurrence matrix. The similarity between two images is the negative exponential power of the gray-level co-occurrence matrix distance.

[0040] The value obtained by formula (4) is between 0 and 1, which is interpreted as the similarity probability between the training sample and the target sample. The training data set is sampled according to this probability distribution to obtain the sampled training data set. In this data set, samples with higher similarity have a greater probability of being drawn multiple times; samples with lower similarity have a lower frequency of occurrence. The resampled data set is then used to pre-train the network parameters.

[0041] Furthermore, the network is trained by combining transfer learning and Faster-RCNN network to retain the best network weight parameters:

[0042] After the pre-training is completed, the parameters are optimized using the prepared helmet dataset based on the pre-trained model, and finally used in helmet detection;

[0043] The loss in Faster-RCNN is divided into two categories: regression loss and classification loss. The classification loss is Softmax, and the regression loss is the loss caused by Smooth L1. The total loss function expression is as follows:

[0044]

[0045] The classification loss expressions of RPN and ROI are the same, both are cross entropy losses; however, RPN loss is a binary cross entropy loss, while ROI loss is a multi-classification cross entropy loss; their expressions are shown in formula (6);

[0046]

[0047] Among them, N cls is the total number of anchors; in the RPN classification loss function, p i is the predicted classification probability of the i-th anchor; p* iis the label. When the anchor is positive, p* i =1, when the anchor is negative, p* i =0;L cls (p i , p* i ) is the binary cross entropy loss function, and its expression is shown in formula (7);

[0048]

[0049] The regression loss functions of RPN and ROI are both derived from Smooth L1 Loss calculation, Smooth L1 The expression of Loss is shown in formula (8); the expression of regression loss function is shown in formula (9);

[0050]

[0051]

[0052] Among them, N reg Determined by the number of anchor positions; p* i Lreg(t i ,t* i ) indicates that only positive samples have bounding box regression loss; Lreg(t i ,t* i )=R(t i -t* i ),R(t i -t* i ) is Smooth L1 function; λ is the weight balancing parameter; t i It is the parameterized coordinate of the bounding box regression predicted by the i-th anchor; perform μ iterations of training and save the trained network parameters.

[0053] Furthermore, a benchmark network for the test set is constructed, and the network parameters with the best effect in μ iteration results are selected as the benchmark network for the test set. The test set is input into the benchmark network for testing to obtain the test results:

[0054] The network accuracy evaluation criteria are mainly based on precision, recall, and mean average precision (mAP). The calculation formula is as follows:

[0055]

[0056]

[0057]

[0058] in:

[0059] TP represents the detected positive example; FP represents the detected negative example; TN represents the undetected negative example; FN represents the undetected positive example; t_bbox represents the marked real box of the helmet; p_bbox represents the box of the helmet calculated by the algorithm; mAP is the area enclosed by the Precision-recall curve and the coordinate axis; n is the number of groups calculated, and the approximate formula of the difference is:

[0060]

[0061] Furthermore, the safety helmet monitoring system for power transmission line inspection based on the Faster-RCNN algorithm includes:

[0062] The acquisition module is used to acquire inspection videos, obtain target images through inspection videos, use annotation tools to annotate the helmets in the single-frame target images, and save their information to obtain sample data;

[0063] The target region detection module is used to detect the target region in the image using Faster-RCNN: first, the image samples in the pre-training database ImageNet are resampled according to the similarity of the target, and then the network parameters are pre-trained using the resampled data set;

[0064] The training module is used to train the network by combining transfer learning and Faster-RCNN network, perform μ iterations of training, and retain the best network weight parameters;

[0065] The test module is used to construct a benchmark network for the test set, select the best network parameters in μ iteration results as the benchmark network for the test set, and input the test set into the benchmark network for testing to obtain the test results.

[0066] Compared with the prior art, the present invention has the following technical effects:

[0067] The safety helmets are detected by integrating transfer learning and Faster-RCNN algorithm. First, the Faster-RCNN network is improved, and the anchor box generation scheme is improved by using k-means II. The residual network and attention mechanism modules are introduced to enhance the detection accuracy. At the same time, the network is integrated with transfer learning to solve the problem of insufficient samples. Then the entire network is trained, and the training set is input into the network after the μth iteration for training, and its weight parameters are retained. The benchmark network of the test set is constructed, and the network parameters with the best effect in the μth iteration result are selected as the benchmark network of the test set. The test set is input into the benchmark network for testing to obtain the test results. In view of the problem of insufficient sample data sets, transfer learning is integrated to improve the Faster-RCNN network. By improving its anchor mechanism, the detection accuracy and efficiency are improved; the SCA-Res2Net module is integrated on the original basis to greatly improve the detection accuracy. The intelligent recognition technology of graphics and images is used to identify whether the personnel within the on-site operation range are wearing safety helmets in a standardized manner, and timely warnings are given to prevent accidents in on-site operations. Safety helmets are detected in different lighting, visibility, and occlusion environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a diagram of the “smart helmet” system;

[0069] Figure 2 This is the Faster-RCNN network structure diagram;

[0070] Figure 3 This is the SCA-Res2Net module structure diagram;

[0071] Figure 4 Safety helmet detection effect diagram; DETAILED DESCRIPTION

[0072] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] The present invention is an intelligent helmet detection method based on the Faster-RCNN algorithm, which is implemented in the following steps:

[0074] Step 1: Collect inspection videos, obtain target images through inspection videos, use annotation tools to annotate the helmets in single-frame images, save their information, and obtain sample data.

[0075] Step 2: Adjust and optimize the Faster-RCNN network and use Faster-RCNN to detect the target area in the image. It is a flexible and general target segmentation and recognition framework. The network can detect the target area in the image, such as Figure 2 shown.

[0076] Step 3: Since a large number of data sets are required during network training, transfer learning is introduced to solve the problem of insufficient samples. First, the image samples in the pre-training database ImageNet are resampled according to the similarity of the target; then the resampled data set is used to pre-train the network parameters.

[0077] Step 4: Combine transfer learning and Faster-RCNN network to train the network and retain the best network weight parameters.

[0078] Step 5: Test the test set to obtain the target detection results.

[0079] The present invention is also characterized in that:

[0080] Step 1 collects inspection videos, obtains target images through inspection videos, uses annotation tools to annotate the images of safety helmets, and saves their information to obtain sample data.

[0081] Step 1.1 Determine the safety helmet to be marked, and obtain the corresponding marking box according to the area of ​​the safety helmet.

[0082] Step 1.2 classifies the marked helmet into a certain pre-set state label.

[0083] Step 1.3 generates a corresponding XML document from the marked helmet image, and the marked helmet information in each document contains four position attributes and one category attribute.

[0084] Obtain sample data through steps 1.1, 1.2, and 1.3.

[0085] Step 2: Adjust and optimize the Faster-RCNN network and use Faster-RCNN to detect the target area in the image. It is a flexible and universal target segmentation and recognition framework. The network can detect the target area in the image.

[0086] The original Faster-RCNN network specifically consists of the following four parts:

[0087] (1) Backbone network: used to extract input image feature representation and provide effective input features for the RPN network.

[0088] (2)RPN network: receives the feature representation extracted by the backbone network and constructs candidate regions for detection.

[0089] (3) Region of Interest (ROI) Pooling Layer: Map the candidate region to the feature map and

[0090] Pooling is unified to a unified scale for subsequent detection and classification.

[0091] (4) Classification and regression: Classify the target categories in the feature area and use bounding box regression to correct the target box to obtain the accurate position offset.

[0092] Step 2.1 Improvement of the anchor box generation scheme. Anchors are a set of rectangular boxes generated by RPN, using 3 scales and 3 aspect ratios (1:1, 1:2, 2:1), generating k = 9 anchors at the center of each sliding window. The original anchor generation scheme of the FasterR-CNN algorithm is a size set based on experience of the data set. For the helmet to be detected, the original anchor cannot correspond to the size of the detected defect, which requires a large number of regression operations when generating region proposals in RPN, affecting the detection efficiency of the algorithm. Therefore, in RPN, the anchor scheme should correspond to the size of the detected defect, so that when RPN performs bounding box regression, it can reduce regression work, improve computational efficiency and reduce network training time, so as to obtain better region proposals and improve detection accuracy.

[0093] The k-means II algorithm overcomes the randomness of the initial cluster center and the complexity of traversing the entire data set. In the process of selecting the initial cluster center, it selects multiple samples each time when traversing the data set. The algorithm classifies the input data set containing N points into k clusters, so that all points in the data set are divided into the class where the closest cluster center is located, and uses the Euclidean distance as the standard for measuring distance during classification. The specific steps of the k-means II algorithm are as follows:

[0094] (1) Input a data set containing k points;

[0095] (2) Randomly select k points in the data set as cluster centers, and then repeat this sampling process 5 times to obtain 5k data points. Then cluster these data points to obtain k initial cluster centers.

[0096] (3) Divide each point in the data set into the class with the closest cluster center according to the Euclidean distance. The Euclidean distance formula is:

[0097]

[0098] (4) Recalculate the cluster center of each class using the following calculation method:

[0099]

[0100] where |c i | is the number of samples of this type.

[0101] (5) Set a threshold. If the distance between the new center obtained by equation (2) and the original center is less than the threshold, clustering ends. Otherwise, go to step (3).

[0102] (6) Output k clusters and k cluster centers.

[0103] Step 2.2 In order to divide the input features into different groups and realize hierarchical connection in the residual block, the Res2Net module is introduced to improve the selectivity of the model structure and the effectiveness of multi-scale feature extraction. Figure 3 The figure shows the internal structure of the Res2Net module. The module uses a smaller convolution kernel group for replacement to capture more fine-grained features. At the same time, the input feature maps are grouped and the groups are connected in the form of residuals.

[0104] First, to reduce computational complexity, the input features are obtained through a 1*1 convolutional layer. Then, they are divided into S groups along the channel dimension, and each feature subset is denoted as x i ∈R H×W×C / S ,i=2,.....,S, the i-th group corresponding to the convolutional layer is denoted as F i , and its output is recorded as f i Except for the first group, all other x i Pass through a 3*3 convolution layer respectively, and when i is greater than 2, fuse x i With f i-1 And through a 3*3 convolution layer, the output of the i-th group is obtained. Secondly, the outputs of each group are fused and passed through a 1*1 convolution layer. The above process is defined as follows:

[0105]

[0106] Finally, the input features are fused with the output of formula (3) to obtain the output features of the Res2Net module.

[0107] Therefore, through the above-mentioned intra-group residual structure, the entire structure can effectively extract global and local features, represent multi-scale features in a finer granularity and improve the receptive field of each network layer under similar computational load.

[0108] Step 2.3 introduces the channel attention module and the spatial attention module into the above Res2Net module, and proposes the SCA-Res2Net module. Add two SE blocks and one SA block after the second 1*1 convolution layer, and cascade the input features after the first SE block to enhance the effective multi-scale discriminant information from the spatial and channel dimensions, suppress background interference information, and reduce the loss of input feature spatial information through the residual structure. The specific structure is as follows Figure 3 shown.

[0109] Step 3: Since a large number of data sets are required during network training, transfer learning is introduced to solve the problem of insufficient samples. First, the image samples in the pre-training database ImageNet are resampled according to the similarity of the target; then the resampled data sets are used to pre-train the network parameters.

[0110] Step 3.1: Construct a dataset of migration samples based on similarity resampling. In order to evaluate the similarity between the training samples from the ImageNet source domain and the samples from the target domain, we first need to establish an evaluation function for target similarity. Considering both accuracy and complexity, the metric used is:

[0111]

[0112] Where: I1 and I2 represent any two images in the training sample set and the target sample set respectively;

[0113] f(I1) represents the feature vector extracted from the image, which is calculated here using the gray-level co-occurrence matrix; the similarity between two images is taken as the negative exponential power of the gray-level co-occurrence matrix distance.

[0114] It is easy to see that the value obtained by formula (3) is between 0 and 1, which can be interpreted as the similarity probability between the training sample and the target sample. According to this probability distribution, the training data set is sampled to obtain the sampled training data set. In this data set, samples with higher similarity have a higher probability of being selected multiple times; the frequency of samples with lower similarity is significantly reduced.

[0115] Step 3.2: Use the resampled data set to pre-train the network parameters.

[0116] Step 4 combines transfer learning and Faster-RCNN network to train the network and retain the best network weight parameters.

[0117] Step 4.1 Faster-RCNN is very suitable for the task of detecting helmets. The network is pre-trained by the samples generated in step 3.1. After the pre-training is completed, the parameters are optimized based on the pre-trained model using the prepared helmet dataset, and finally used in helmet detection. This can improve the robustness of the algorithm model and reduce overfitting.

[0118] Step 4.2 The loss in Faster-RCNN is divided into two categories: regression loss and classification loss. The classification loss is Softmax and the regression loss is Smooth. L1 The total loss function expression is as follows:

[0119]

[0120] The classification loss expressions of RPN and ROI are the same, both are cross entropy losses. However, RPN loss is a binary cross entropy loss, while ROI loss is a multi-classification cross entropy loss. The expressions are shown in formula (6).

[0121]

[0122] Among them, N cls is the total number of anchors; in the RPN classification loss function, p i is the predicted classification probability of the i-th anchor; p* i is the label. When the anchor is positive, p* i =1, when the anchor is negative, p* i =0;L cls (p i , p* i ) is the binary cross entropy loss function, and its expression is shown in formula (7).

[0123]

[0124] The regression loss functions of RPN and ROI are both derived from Smooth L1 Loss calculation, Smooth L1 Loss solves the problem of gradient explosion when the predicted value is close to the true value and the problem of the function being undifferentiable at point 0 affecting convergence. L1 The expression of Loss is shown in formula (8). The expression of regression loss function is shown in formula (9).

[0125]

[0126]

[0127] Among them, N reg Determined by the number of anchor positions; p* i Lreg(t i ,t* i ) means that only positive samples have bounding box regression loss; L reg (t i ,t* i )=R(t i -t* i ),R(t i -t* i ) is Smooth L1 function; λ is the weight balancing parameter; t i is the parameterized coordinate of the bounding box regression predicted by the i-th anchor.

[0128] Step 4.3 performs μ iterations of training and saves the trained network parameters.

[0129] Step 5 builds a benchmark network for the test set, selects the network parameters with the best effect in μ iteration results as the benchmark network for the test set, and inputs the test set into the benchmark network for testing to obtain the test results.

[0130] The network accuracy evaluation criteria are mainly based on precision, recall and mean average precision (mAP). The calculation formula is as follows:

[0131]

[0132]

[0133]

[0134] in:

[0135] TP represents the detected positive example; FP represents the detected negative example; TN represents the undetected negative example; FN represents the undetected positive example; t_bbox represents the marked real box of the helmet; p_bbox represents the box of the helmet calculated by the algorithm.

[0136] If only precision or recall is used to measure the accuracy of a model, it is obviously inappropriate. Therefore, mAP, one of the most important indicators in target detection, is also needed. It is the average AP value of multiple validation sets. mAP is the area enclosed by the Precision-recall curve and the coordinate axis. n is the number of groups calculated, and the approximate formula of the difference is:

[0137]

[0138] The beneficial results of the present invention are that, in order to solve the problem of insufficient sample data sets, transfer learning is integrated; the Faster R-CNN network is improved, and the detection accuracy and efficiency are improved by improving its anchor mechanism; the SCA-Res2Net module is introduced into the Faster R-CNN network, which greatly improves the detection accuracy.

Claims

1. A method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm, characterized in that: include: Collect inspection videos, obtain target images through inspection videos, use annotation tools to annotate the helmets in the single-frame target images, and save their information to obtain sample data; Faster-RCNN is used to detect the target area in the image: first, the image samples in the pre-training database ImageNet are resampled according to the similarity of the target, and then the network parameters are pre-trained using the resampled data set; Combine transfer learning and Faster-RCNN network to train the network, perform μ iterations of training, and retain the best network weight parameters; Construct a benchmark network for the test set, select the best network parameters in μ iteration results as the benchmark network for the test set, input the test set into the benchmark network for testing and obtain the test results; Combine transfer learning and Faster-RCNN network to train the network and retain the best network weight parameters: After the pre-training is completed, the parameters are optimized using the prepared helmet dataset based on the pre-trained model, and finally used in helmet detection; The loss in Faster-RCNN is divided into two categories: regression loss and classification loss. The classification loss is Softmax, and the regression loss is the loss caused by Smooth L1. The total loss function expression is as follows: The classification loss expressions of RPN and ROI are the same, both are cross entropy losses; however, RPN loss is a binary cross entropy loss, while ROI loss is a multi-classification cross entropy loss; their expressions are shown in formula (6); Among them, N cls is the total number of anchors; in the RPN classification loss function, p i is the predicted classification probability of the i-th anchor; p * i is the label. When the anchor is positive, p * i =1, when the anchor is negative, p * i =0;L cls (p i , p * i ) is the binary cross entropy loss function, and its expression is shown in formula (7); The regression loss functions of RPN and ROI are both derived from Smooth L1 Loss calculation, Smooth L1 The expression of Loss is shown in formula (8); the expression of regression loss function is shown in formula (9); Among them, N reg Determined by the number of anchor positions; p * i Lreg(t i ,t * i ) indicates that only positive samples have bounding box regression loss; Lreg(t i ,t * i )=R(t i -t * i ),R(t i -t * i ) is Smooth L1 function; λ is the weight balancing parameter; t i It is the parameterized coordinate of the bounding box regression predicted by the i-th anchor; perform μ iterations of training and save the trained network parameters.

2. The method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm according to claim 1 is characterized in that: Obtaining the target image includes: Determine the hard hat to be marked, and obtain the corresponding marking box according to the area where the hard hat is located; divide the marked hard hat into a pre-set state label; generate the corresponding xml document from the marked hard hat image, and the marked hard hat information in each document contains four position attributes and 1 category attribute.

3. The method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm according to claim 1 is characterized in that: The original Faster-RCNN network specifically consists of the following four parts: Backbone network: used to extract input image feature representation and provide effective input features for the RPN network; RPN network: receives the feature representation extracted by the backbone network and constructs candidate regions for detection targets; Region of Interest (ROI) pooling layer: maps candidate regions to feature maps and pools them to a uniform scale for subsequent detection and classification; Classification and regression: Classify the target categories in the feature area and use bounding box regression to correct the target box to obtain the precise position offset.

4. The method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm according to claim 1 is characterized in that: Adjust and optimize the Faster-RCNN network, and use Faster-RCNN to detect the target area in the image, including: Improvements to the anchor box generation scheme, where the anchor is a set of rectangular boxes generated by RPN, using three scales and three aspect ratios, generating k = 9 anchors at the center of each sliding window; making the anchor scheme correspond to the size of the defect being detected; Use the k-meansII algorithm to classify the input data set containing N points into k clusters, so that all points in the data set are classified into the class where the cluster center is closest to them, and use the Euclidean distance as the standard for measuring distance during classification; The Res2Net module is introduced. This module uses a smaller convolution kernel group to replace the original one, and uses the smaller convolution kernel to capture more fine-grained features. At the same time, the input feature maps are grouped and the groups are connected in the form of residuals. First, the input features are obtained through a 1*1 convolutional layer; then, they are divided into S groups along the channel dimension, and each feature subset is denoted as x i ∈R H×W×C / S ,i=2,.....,S, the convolution layer corresponding to the i-th group is recorded as Fi, and its output is recorded as fi; except for the first group, all other xi pass through a 3*3 convolution layer respectively, and when i is greater than 2, xi and fi-1 are fused and passed through a 3*3 convolution layer to obtain the output of the i-th group; secondly, the outputs of each group are fused and passed through a 1*1 convolution layer; the above process is defined as follows: Finally, the input features are fused with the output of formula (3) to obtain the output features of the Res2Net module; The channel attention module and the spatial attention module are introduced into the above Res2Net module, and the SCA-Res2Net module is proposed. Two SE blocks and one SA block are added after the second 1*1 convolution layer, and the input features are cascaded after the first SEblock to enhance the effective multi-scale discriminative information from the spatial and channel dimensions, suppress background interference information, and reduce the loss of spatial information of the input features through the residual structure.

5. The method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm according to claim 4 is characterized in that: The specific steps of the k-meansII algorithm are as follows: (1) Input a data set containing k points; (2) Randomly select k points in the data set as cluster centers, and then repeat this sampling process 5 times to obtain 5k data points. Then cluster these data points to obtain k initial cluster centers. (3) Divide each point in the data set into the class with the closest cluster center according to the Euclidean distance; the Euclidean distance formula is: (4) Recalculate the cluster center of each class using the following calculation method: where |c i | is the number of samples of this type; (5) Set a threshold. If the distance between the new center obtained by equation (2) and the original center is less than the threshold, clustering ends. Otherwise, go to step (3). (6) Output k clusters and k cluster centers.

6. The method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm according to claim 1 is characterized in that: Resample the image samples in the pre-training database ImageNet, and then use the resampled data set to pre-train the network parameters: Based on similarity resampling, a data set of migration samples is constructed and an evaluation function of target similarity is established; the measurement method used is: Where: I1 and I2 represent any two images in the training sample set and the target sample set respectively; f(I1) represents the feature vector extracted from the image, which is calculated using the gray-level co-occurrence matrix. The similarity between two images is the negative exponential power of the gray-level co-occurrence matrix distance. The value obtained by formula (4) is between 0 and 1, which is interpreted as the similarity probability between the training sample and the target sample; The training data set is sampled according to the probability distribution to obtain the sampled training data set; In this data set, samples with higher similarity have a greater probability of being drawn multiple times; samples with lower similarity appear less frequently; the resampled data set is then used to pre-train the network parameters.

7. The method for monitoring safety helmets in power operations based on the Faster-RCNN algorithm according to claim 1, characterized in that: Construct a benchmark network for the test set, select the best network parameters in the μ iteration results as the benchmark network for the test set, input the test set into the benchmark network for testing, and obtain the test results: The network accuracy evaluation criteria are mainly based on precision, recall, and mean average precision (mAP). The calculation formula is as follows: in: TP represents the detected positive example; FP represents the detected negative example; TN represents the undetected negative example; FN represents the undetected positive example; t_bbox represents the marked real box of the helmet; p_bbox represents the box of the helmet calculated by the algorithm; mAP is the area enclosed by the Precision-recall curve and the coordinate axis; n is the number of groups calculated, and the approximate formula of the difference is:

8. The safety helmet monitoring system for power transmission line inspection based on Faster-RCNN algorithm is characterized by: include: The acquisition module is used to collect inspection videos, obtain target images through the inspection videos, use annotation tools to annotate the helmets in the single-frame target images, and save their information to obtain sample data; The target region detection module is used to detect the target region in the image using Faster-RCNN: first, the image samples in the pre-training database ImageNet are resampled according to the similarity of the target, and then the network parameters are pre-trained using the resampled data set; The training module is used to train the network by combining transfer learning and Faster-RCNN network, perform μ iterations of training, and retain the best network weight parameters; The test module is used to construct a benchmark network for the test set, select the network parameters with the best effect in μ iteration results as the benchmark network for the test set, and input the test set into the benchmark network for testing to obtain the test results; Combine transfer learning and Faster-RCNN network to train the network and retain the best network weight parameters: After the pre-training is completed, the parameters are optimized using the prepared helmet dataset based on the pre-trained model, and finally used in helmet detection; The loss in Faster-RCNN is divided into two categories: regression loss and classification loss. The classification loss is Softmax, and the regression loss is the loss caused by Smooth L1. The total loss function expression is as follows: The classification loss expressions of RPN and ROI are the same, both are cross entropy losses; however, RPN loss is a binary cross entropy loss, while ROI loss is a multi-classification cross entropy loss; their expressions are shown in formula (6); Among them, N cls is the total number of anchors; in the RPN classification loss function, p i is the predicted classification probability of the i-th anchor; p * i is the label. When the anchor is positive, p * i =1, when the anchor is negative, p * i =0;L cls (p i , p * i ) is the binary cross entropy loss function, and its expression is shown in formula (7); The regression loss functions of RPN and ROI are both derived from Smooth L1 Loss calculation, Smooth L1 The expression of Loss is shown in formula (8); the expression of regression loss function is shown in formula (9); Among them, N reg Determined by the number of anchor positions; p * i Lreg(t i ,t * i ) indicates that only positive samples have bounding box regression loss; Lreg(t i ,t * i )=R(t i -t * i ),R(t i -t * i ) is Smooth L1 function; λ is the weight balancing parameter; t i It is the parameterized coordinate of the bounding box regression predicted by the i-th anchor; perform μ iterations of training and save the trained network parameters.

Images