Power operation risk assessment method and system based on deep learning

Abstract

The invention provides an electric power operation risk assessment method and system based on deep learning, and the method comprises the steps: collecting a picture of an electric power operation site, and carrying out the expansion of the picture through data enhancement; dividing the expanded pictures into a training set and a verification set; a backbone network and a loss function are replaced for the convolutional neural network, and an electric power operation detection model suitable for a self-attention mechanism of an electric power operation scene is constructed; using the training set to train a power operation detection model; based on an entropy weight method, verifying the power operation detection model by using the verification set; when it is judged that the comprehensive score of the accuracy of the illegal labels labeled in the verification set is larger than a preset threshold value, it is determined that the power operation detection model passes verification; and detecting a picture of a to-be-evaluated electric power operation site by using the electric power operation detection model to obtain a risk level of the electric power operation site. And the detection accuracy and the detection speed of the power operation violation behaviors are improved.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a method and system for risk assessment of power operations based on deep learning. Background Art

[0002] As the most popular single-stage target detection network, convolutional neural network has attracted the attention of many researchers, and many improvements have been made on its basis to improve the accuracy of power operation recognition.

[0003] The power system is a highly complex system involving multiple devices and multiple levels of control and monitoring. Traditional convolutional neural networks often have difficulty coping with the challenges brought by the complexity and variability of the environment. Deep learning technology can learn the complex laws and patterns of the system from raw data through end-to-end learning, providing more accurate and comprehensive analysis for power operation risk assessment.

[0004] Traditional convolutional neural networks can only detect ordinary targets, but there are problems with occlusion and small targets in power scene operations. In the process of detecting such targets, the accuracy will be reduced, especially when detecting extremely small targets such as safety helmet ropes and supervisor armbands in dark scenes. In addition, when the target is occluded, traditional algorithms are difficult to accurately identify, resulting in insufficient accuracy.

[0005] The Chinese patent with publication number CN115359360A discloses "a method, system, device and storage medium for detecting power field operation scenes". By adding an attention module, the multi-scale feature information of the feature map can be better extracted, and the channel attention and spatial attention modules are used to adjust the feature map to better obtain spatial information. However, this method has low detection accuracy for small targets and occluded targets, and it is impossible to evaluate the effectiveness of the model and solve the problem of sample imbalance. Sample imbalance will cause the training results of each behavior to be very different. Summary of the invention

[0006] In view of the above technical problems, the present invention provides a method for risk assessment of power operations based on deep learning, comprising:

[0007] Collecting pictures of power operation sites and augmenting the pictures through data enhancement;

[0008] Divide the expanded images into training set and validation set;

[0009] By replacing the backbone network and loss function of the convolutional neural network, a power operation detection model with a self-attention mechanism suitable for power operation scenarios is constructed; and the power operation detection model is trained using the training set;

[0010] Based on the entropy weight method, the verification set is used to verify the power operation detection model; when the comprehensive score of the accuracy of the violation labels marked in the verification set is judged to be greater than a preset threshold, it is determined that the power operation detection model has passed the verification; using the power operation detection model, the pictures of the power operation site to be evaluated are detected to obtain the risk level of the power operation site.

[0011] Furthermore, the image is expanded by data enhancement, including:

[0012] Read multiple pictures of the pictures of the power operation site at any time, perform zooming operations on the multiple pictures, and adjust the saturation, brightness, contrast and hue;

[0013] Combine multiple images.

[0014] Furthermore, it also includes:

[0015] The training set and the validation set are divided into a ratio of 8:2.

[0016] Furthermore, by replacing the backbone network and loss function of the convolutional neural network, a power operation detection model with a self-attention mechanism suitable for power operation scenarios is constructed, including:

[0017] The convolutional neural network resizes the input image, and the size of the resized image is limited to 640×640;

[0018] The backbone network used three 3×3 convolutional layers to extract features from the image, and obtained three feature maps of different scales, namely 80×80, 40×40 and 20×20.

[0019] The feature maps of three different scales are sent to the detection layer respectively. After the global average pooling operation, the feature maps are reduced to one-dimensional vectors, and the feature maps are classified and regressed.

[0020] Use the Softmax activation function to convert a one-dimensional vector into a probability distribution for classification tasks;

[0021] The loss function EC-Loss is used to calculate the difference between the predicted box and the true box and the back-propagation algorithm is used to update the model parameters.

[0022] Furthermore, the backbone network is an improved Transformer network. The improved Transformer network establishes a cross-window self-attention mechanism based on the traditional Transformer architecture. The cross-window self-attention mechanism includes two modules, namely horizontal self-attention and vertical self-attention.

[0023] Furthermore, the horizontal self-attention is used to input the feature map Split into The feature map of the horizontal bar, where sω represents the width of the horizontal bar feature map, C, H, W represent the dimension, height and width of the feature map respectively;

[0024] For each feature X of the horizontal feature map i , i = 1, 2, ..., M, use Transformer to process and get feature Y i ;

[0025] Merge the M features to obtain the lateral features sent to the detection layer;

[0026] Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention L-ATT is as follows:

[0027]

[0028] X=[X 1 , X 2 , ..., X M ]

[0029]

[0030]

[0031] Among them, H, W, C represent the height, width and dimension of the feature map respectively, and Q, K, V represent the query vector, key vector and value vector in the horizontal attention.

[0032] Furthermore, the vertical self-attention is used to split the input feature map A(C×H×W) into The feature map of the vertical bars, where sω represents the width of the vertical bar feature map;

[0033] For each feature X of the vertical bar feature map i , i = 1, 2, ..., M, and processed by Transformer, we can get the feature Y i ;

[0034] Merge the M features to obtain the longitudinal features sent to the detection layer;

[0035] Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention R-ATT is as follows:

[0036]

[0037] X=[X1 , X 2 , ..., X M ]

[0038]

[0039]

[0040] In the improved Transformer network, there are M detection layers, which are evenly divided into two groups. The first detection layer uses horizontal features, and the second detection layer uses vertical features. Finally, the features of the two groups are merged. The formula used in the merging process is shown in the following formula:

[0041]

[0042] The expression of the cross-window self-attention mechanism CR-Window-ATT is shown in the following formula:

[0043] CR-Window-ATT(X)=Concat(head1,head2,…,head M )W O

[0044] Where W O Represents the commonly used projection matrix, which projects the self-attention result to the dimension of the target output.

[0045] Furthermore, the formula of the loss function EC-Loss is as follows:

[0046]

[0047] Where y∈[-1,1] represents the real object box, p∈[0,1] represents the predicted probability of the foreground category, the adjustment factor γ is used to focus on difficult-to-distinguish samples, and the weight factor α∈(0,1) is used when the model predicts a positive sample. If it is a negative sample, the weight factor is 1-α.

[0048] Where q represents the actual category label. When q>0, it means that the actual label is a positive example, and the loss function is the binary cross entropy loss function, which is used to measure the gap between the probability p predicted by the model and the actual positive example; when q=0, it means that the actual label is a negative example, and the loss function is the negative example loss function.

[0049] Further, based on the entropy weight method, the verification set is used to verify the power operation detection model; when it is determined that the comprehensive score of the accuracy of the violation labels marked in the verification set is greater than a preset threshold, it is determined that the power operation detection model has passed the verification, including:

[0050] Use the trained power operation detection model to detect the image or video, store the label name of the violation and the number of violations detected in a list, and use the accuracy (P) as an evaluation indicator;

[0051] Assume there are n violation labels, for each violation label i, m violations are detected, and its evaluation index is x ij , j represents the label index of the violation, and the violation is normalized to obtain the normalized value p ij , calculated as follows:

[0052]

[0053] Where min(x i ) and max(x i ) represent the maximum and minimum values ​​of label i in the evaluation index accuracy respectively;

[0054] For the i-th violation label, the information entropy H i The calculation formula is as follows:

[0055]

[0056] According to the entropy weight method, the weight W of the i-th violation label on the accuracy i The calculation formula is as follows:

[0057]

[0058] Where n is the number of violation labels, is the sum of the information entropy of all violation labels;

[0059] The weight W of all violation labels on accuracy i Perform weighted summation to obtain W P ;

[0060] Calculate the comprehensive evaluation score as shown in the following formula:

[0061] Score=W P ×P

[0062] The calculated comprehensive evaluation score Score is compared with a preset threshold value. If it is higher than the preset threshold value, it is determined that the power operation detection model has passed the verification.

[0063] The present invention also provides a power operation risk assessment system based on deep learning, comprising:

[0064] A picture acquisition module, used to collect pictures of the power operation site and expand the pictures through data enhancement;

[0065] An image division module is used to divide the expanded images into a training set and a validation set;

[0066] A model building module, used to build a power operation detection model with a self-attention mechanism suitable for power operation scenarios by replacing the backbone network and loss function of the convolutional neural network; and train the power operation detection model using the training set;

[0067] The model verification module is used to verify the power operation detection model using the verification set based on the entropy weight method; when the comprehensive score of the accuracy of the violation labels marked in the verification set is judged to be greater than a preset threshold, it is determined that the power operation detection model has passed the verification; the power operation detection model is used to detect pictures of the power operation site to be evaluated to obtain the risk level of the power operation site.

[0068] Furthermore, the model building module includes:

[0069] The image resizing submodule is used by the convolutional neural network to resize the input image. The size of the resized image is limited to 640×640.

[0070] The feature extraction submodule is used to extract features from images by using three 3×3 convolutional layers in a backbone network to obtain feature maps of three different scales, namely 80×80, 40×40 and 20×20.

[0071] The classification and prediction submodule is used to send the feature maps of three different scales to the detection layer respectively, reduce the dimension of the feature map to a one-dimensional vector after the global average pooling operation, and perform classification and regression prediction on the feature map;

[0072] The conversion submodule is used to convert the one-dimensional vector into a probability distribution using the Softmax activation function for classification tasks;

[0073] The parameter update submodule is used to calculate the difference between the predicted box and the true box using the loss function EC-Loss and update the model parameters using the back propagation algorithm.

[0074] Furthermore, the model validation module includes:

[0075] A detection submodule, used to detect images or videos using the trained power operation detection model, store the label names of the violations and the number of violations detected in a list, and use the accuracy (P) as an evaluation indicator;

[0076] The normalization submodule is used to assume that there are n violation labels. For each violation label i, m violations are detected, and its evaluation index is x ij , j represents the label index of the violation, and the violation is normalized to obtain the normalized value p ij , calculated as follows:

[0077]

[0078] Where min(x i ) and max(x i ) represent the maximum and minimum values ​​of label i in the evaluation index accuracy respectively;

[0079] The information entropy calculation submodule is used to calculate the information entropy H for the i-th violation label. i The calculation formula is as follows:

[0080]

[0081] The weight calculation submodule is used to calculate the weight W of the i-th violation label in terms of accuracy according to the entropy weight method. i The calculation formula is as follows:

[0082]

[0083] Where n is the number of violation labels, is the sum of the information entropy of all violation labels;

[0084] The weighted summation submodule is used to weight all illegal labels on the accuracy W i Perform weighted summation to obtain W P ;

[0085] The verification submodule is used to calculate the comprehensive evaluation score, which is calculated as shown in the following formula:

[0086] Score=W P ×P

[0087] The calculated comprehensive evaluation score Score is compared with a preset threshold value. If it is higher than the preset threshold value, it is determined that the power operation detection model has passed the verification.

[0088] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the above claims when executing the computer program.

[0089] The present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any one of the above claims are implemented.

[0090] The present invention provides a method and system for assessing the risk of power operations based on deep learning. The deep learning algorithm is used to effectively detect power scene operations, improve the detection accuracy and speed of violations, and the model is evaluated by the entropy weight method to maximize the effectiveness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 It is a flowchart of a method for risk assessment of power operations based on deep learning provided by an embodiment of the present invention;

[0092] Figure 2 It is a technical solution flow chart of a method for risk assessment of power operations based on deep learning according to an embodiment of the present invention;

[0093] Figure 3 is a cross-window attention mechanism involved in an embodiment of the present invention;

[0094] Figure 4 It is a structural schematic diagram of a power operation risk assessment system based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0095] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0096] Example 1

[0097] In view of the above defects or improvement needs of the prior art, the present invention provides a power operation risk assessment method based on deep learning, the process of which is as follows: Figure 1 As shown, the purpose is to effectively detect power scene operations through deep learning algorithms, improve the detection accuracy and speed of violations, and evaluate the model through entropy weight method to maximize the effectiveness of the model. The method includes the following steps:

[0098] Step S101, collecting pictures of the power operation site, and expanding the pictures through data enhancement.

[0099] Read multiple pictures of the power operation site at any time, perform zooming operations on the multiple pictures, and adjust the saturation, brightness, contrast and hue; and splice the multiple pictures.

[0100] Step S102, dividing the expanded images into a training set and a validation set.

[0101] The training set and the validation set are divided into a ratio of 8:2.

[0102] Step S103, by replacing the backbone network and loss function of the convolutional neural network, a power operation detection model with a self-attention mechanism suitable for power operation scenarios is constructed; and the power operation detection model is trained using the training set.

[0103] The convolutional neural network resizes the input image, and the size of the image after resizing is limited to 640×640; the backbone network uses three 3×3 convolutional layers in succession to extract features from the image, and obtains three feature maps of different scales, which are 80×80, 40×40 and 20×20 respectively; the three feature maps of different scales are sent to the detection layer respectively, and the feature maps are reduced to one-dimensional vectors after global average pooling operation, and the feature maps are classified and regressed.

[0104] The Softmax activation function is used to convert the one-dimensional vector into a probability distribution for classification tasks; the loss function EC-Loss is used to calculate the difference between the predicted box and the true box and the back-propagation algorithm is used to update the model parameters.

[0105] The backbone network is an improved Transformer network. The improved Transformer network establishes a cross-window self-attention mechanism based on the traditional Transformer architecture. The principle is as follows: Figure 3 As shown in the figure, the cross-window self-attention mechanism consists of two modules, namely horizontal self-attention and vertical self-attention.

[0106] The horizontal self-attention is used to transform the input feature map Split into The feature map of the horizontal bar, where sω represents the width of the horizontal bar feature map, C, H, W represent the dimension, height and width of the feature map respectively;

[0107] For each feature X of the horizontal feature map i , i = 1, 2, ..., M, use Transformer to process and get feature Y i ;

[0108] Merge the M features to obtain the lateral features sent to the detection layer;

[0109] Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention L-ATT is as follows:

[0110]

[0111] X=[X 1 , X 2 , ..., X M ]

[0112]

[0113]

[0114] Among them, H, W, C represent the height, width and dimension of the feature map respectively, and Q, K, V represent the query vector, key vector and value vector in the horizontal attention.

[0115] The vertical self-attention is used to split the input feature map A(C×H×W) into The feature map of the vertical bars, where sω represents the width of the vertical bar feature map;

[0116] For each feature X of the vertical bar feature map i , i = 1, 2, ..., M, and processed by Transformer, we can get the feature Y i ;

[0117] Merge the M features to obtain the longitudinal features sent to the detection layer;

[0118] Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention R-ATT is as follows:

[0119]

[0120] X=[X 1 , X 2 , ..., X M ]

[0121]

[0122]

[0123] In the improved Transformer network, there are M detection layers, which are evenly divided into two groups. The first detection layer uses horizontal features, and the second detection layer uses vertical features. Finally, the features of the two groups are merged. The formula used in the merging process is shown in the following formula:

[0124]

[0125] The expression of the cross-window self-attention mechanism CR-Window-ATT is shown in the following formula:

[0126] CR-Window-ATT(X)=Concat(head1, head2,..., head M )W O

[0127] Where W O Represents the commonly used projection matrix, which projects the self-attention result to the dimension of the target output.

[0128] The formula of the loss function EC-Loss is as follows:

[0129]

[0130] Where y∈[-1, 1] represents the real object box, p∈[0, 1] represents the predicted probability of the foreground category, the adjustment factor γ is used to focus on difficult-to-distinguish samples, and the weight factor α∈(0, 1) is used when the model predicts a positive sample. If it is a negative sample, the weight factor is 1-α.

[0131] Where q represents the actual category label. When q>0, it means that the actual label is a positive example, and the loss function is the binary cross entropy loss function, which is used to measure the gap between the probability p predicted by the model and the actual positive example; when q=0, it means that the actual label is a negative example, and the loss function is the negative example loss function.

[0132] Step S104, based on the entropy weight method, use the verification set to verify the power operation detection model; when it is determined that the comprehensive score of the accuracy of the violation labels marked in the verification set is greater than a preset threshold, it is determined that the power operation detection model has passed the verification; use the power operation detection model to detect pictures of the power operation site to be evaluated to obtain the risk level of the power operation site.

[0133] Use the trained power operation detection model to detect the image or video, store the label name of the violation and the number of violations detected in a list, and use the accuracy (P) as an evaluation indicator;

[0134] Assume there are n violation labels, for each violation label i, m violations are detected, and its evaluation index is x ij , j represents the label index of the violation, and the violation is normalized to obtain the normalized value p ij , calculated as follows:

[0135]

[0136] Where min(x i ) and max(x i ) represent the maximum and minimum values ​​of label i in the evaluation index accuracy respectively;

[0137] For the i-th violation label, the information entropy H i The calculation formula is as follows:

[0138]

[0139] According to the entropy weight method, the weight W of the i-th violation label on the accuracy i The calculation formula is as follows:

[0140]

[0141] Where n is the number of violation labels, is the sum of the information entropy of all violation labels;

[0142] The weight W of all violation labels on accuracy i Perform weighted summation to obtain W P ;

[0143] Calculate the comprehensive evaluation score as shown in the following formula:

[0144] Score=W P ×P

[0145] The calculated comprehensive evaluation score Score is compared with a preset threshold value. If it is higher than the preset threshold value, it is determined that the power operation detection model has passed the verification.

[0146] Example 2

[0147] The present invention provides an implementation scheme of a power operation risk assessment method based on deep learning, please refer to

[0148] Attached Figure 2 , Figure 2 This is a technical solution flow chart of a power operation risk assessment method based on deep learning provided by the implementation case, which specifically includes the following steps:

[0149] (1) Perform data enhancement operations on the data set to expand the data set

[0150] (2) Use annotation software to label the images and remove blurred and blank images.

[0151] (3) Divide all the labeled images according to their proportions to obtain training sets and validation sets.

[0152] (4) Design a risk assessment model for power operations based on deep learning.

[0153] (5) Use the entropy weight method to evaluate the effectiveness of the model.

[0154] The power operation risk assessment model based on deep learning includes a feature extraction module, a backbone

[0155] Network module and prediction module; specifically include the following steps:

[0156] (4.1) The image is input into the network, and the network resizes the input image. The size of the image after resizing is limited to 640×640.

[0157] (4.2) In the backbone network, three 3×3 convolutional layers are used continuously to extract features from the image, and three feature maps of different scales are obtained. The scales of the feature maps are 80×80, 40×40, and 20×20 respectively.

[0158] (4.3) The feature maps of three different scales are sent to the detection layer respectively. After the global average pooling operation, the feature maps are reduced to one-dimensional vectors, and the feature maps are classified and regressed.

[0159] (4.4) Use the Softmax activation function to convert the one-dimensional vector into a probability distribution for classification tasks. At the same time, use EC-Loss to calculate the difference between the predicted box and the true box and use the back propagation algorithm to update the model parameters to improve the generalization ability of the model.

[0160] The backbone network uses an improved Transformer network. The improved Transformer network establishes a cross-window self-attention mechanism based on the traditional Transformer architecture, which improves computational efficiency by forming a cross-window. The cross-window self-attention mechanism contains two modules, horizontal self-attention and vertical self-attention. It includes the following sub-steps:

[0161] (4.2.1) Input feature map Split into The feature map of the horizontal stripe is sω, where sω represents the width of the horizontal stripe feature map, and C, H, and W represent the dimension, height, and width of the feature map, respectively.

[0162] (4.2.2) For each feature X of the horizontal feature map i , i = 1, 2, ..., M, and processed using Transformer, we can get feature Y i .

[0163] (4.2.3) These M features are merged to obtain the lateral features sent to the detection layer.

[0164] (4.2.4) Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention L-ATT is as follows:

[0165]

[0166] X=[X 1 ,X 2 ,…,X M ]

[0167]

[0168]

[0169] Among them, H, W, C represent the height, width and dimension of the feature map respectively, and Q, K, V represent the query vector, key vector and value vector in the horizontal attention.

[0170] For vertical attention, the following sub-steps are included:

[0171] (4.2.5) Split the input feature map A(C×H×W) into The feature map of the vertical strips is sω, where sω represents the width of the vertical strip feature map.

[0172] (4.2.6) For each feature X of the vertical bar feature map i , i = 1, 2, ..., M, and processed using Transformer, we can get feature Y i .

[0173] (4.2.7) These M features are merged to obtain the longitudinal features sent to the detection layer.

[0174] (4.2.8) Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention R-ATT is as follows:

[0175]

[0176] X=[X 1 ,X 2 ,…,X M ]

[0177]

[0178]

[0179] (4.2.9) In the improved Transformer, there are M detection layers, which are evenly divided into two groups, each with The first detection layer uses horizontal features, and the second detection layer uses vertical features. Finally, the features of the two groups are merged. The formula used in the merging process is shown below:

[0180]

[0181] Finally, the expression of the cross-window self-attention mechanism CR-Window-ATT is shown below:

[0182] CR-Window-ATT(X)=Concat(head1,head2,…,head M )W O

[0183] Where W O Represents the commonly used projection matrix, which projects the self-attention result to the dimension of the target output.

[0184] Among them, in the process of analyzing the difference between the predicted box and the real box, the EC-Loss loss function is used. It includes the following sub-steps:

[0185] EC-Loss uses weighted thinking to solve the problem of class imbalance caused by equal treatment of positive and negative samples.

[0186] (4.4.1)The formula of EC-Loss is as follows:

[0187]

[0188] Where y∈[-1,1] represents the real object box, p∈[0,1] represents the predicted probability of the foreground category, the adjustment factor γ is used to focus on difficult-to-distinguish samples, and the weight factor α∈(0,1) is used when the model predicts a positive sample. If it is a negative sample, the weight factor is 1-α.

[0189] Where q represents the actual category label. When q>0, it means that the actual label is a positive example. At this time, the loss function is the binary cross entropy loss function, which is used to measure the difference between the probability p predicted by the model and the actual positive example. When q=0, it means that the actual label is a negative example. At this time, the loss function is the negative example loss function.

[0190] The entropy weight method is used to evaluate the effectiveness of the model. It includes the following sub-steps:

[0191] (4.5.1) Use the trained model to detect images or videos, store the label names of violations and the number of violations detected in a list, and use accuracy (P) as the evaluation indicator.

[0192] (4.5.2) Assume there are n violation labels. For each violation label i, m violations are detected and its evaluation index is x ij , j represents the label index of the violation, and the violation is normalized to obtain the normalized value p ij , calculated as follows:

[0193]

[0194] Where min(x i ) and max(x i ) represent the maximum and minimum values ​​of label i in the evaluation index accuracy.

[0195] (4.5.3) For the i-th violation label, the information entropy H i The calculation formula is as follows:

[0196]

[0197] (4.5.4) According to the entropy weight method, the weight W of the i-th violation label on the accuracy i The calculation formula is as follows:

[0198]

[0199] Where n is the number of violation labels, is the sum of the information entropy of all violation labels.

[0200] (4.5.5) Weight W of all illegal labels on accuracy i Perform weighted summation to obtain W P

[0201] (4.5.6) Calculate the comprehensive evaluation score as shown in the following formula:

[0202] Score=W P ×P

[0203] (4.5.7) The calculated comprehensive evaluation score Score is compared with the set threshold. If it is higher than the threshold, the model performance is considered good. If it is lower than the threshold, the model structure, hyperparameters or optimization algorithm need to be further adjusted.

[0204] Specifically, by loading the pre-trained power scene operation detection model, pre-processing and model inference of the operation site pictures or videos to be identified, the image results containing the target box, confidence, and label name are generated. After evaluating the model through the entropy weight method, the detection effectiveness of the model in the actual scene can be effectively understood, and the practical application ability of the model can be improved.

[0205] Based on the same inventive concept, the present invention also provides a power operation risk assessment system 400 based on deep learning, such as Figure 4 As shown, including:

[0206] The picture acquisition module 410 is used to acquire pictures of the power operation site and expand the pictures through data enhancement;

[0207] The picture division module 420 is used to divide the expanded pictures into a training set and a validation set;

[0208] A model building module 430 is used to build a power operation detection model with a self-attention mechanism suitable for power operation scenarios by replacing the backbone network and loss function of the convolutional neural network; and train the power operation detection model using the training set;

[0209] The model verification module 440 is used to verify the power operation detection model based on the entropy weight method using the verification set; when the comprehensive score of the accuracy of the violation labels marked in the verification set is greater than a preset threshold, it is determined that the power operation detection model has passed the verification; the power operation detection model is used to detect pictures of the power operation site to be evaluated to obtain the risk level of the power operation site.

[0210] Furthermore, the model building module includes:

[0211] The image resizing submodule is used by the convolutional neural network to resize the input image. The size of the resized image is limited to 640×640.

[0212] The feature extraction submodule is used to extract features from images by using three 3×3 convolutional layers in a backbone network to obtain feature maps of three different scales, namely 80×80, 40×40 and 20×20.

[0213] The classification and prediction submodule is used to send the feature maps of three different scales to the detection layer respectively, reduce the dimension of the feature map to a one-dimensional vector after the global average pooling operation, and perform classification and regression prediction on the feature map;

[0214] The conversion submodule is used to convert the one-dimensional vector into a probability distribution using the Softmax activation function for classification tasks;

[0215] The parameter update submodule is used to calculate the difference between the predicted box and the true box using the loss function EC-Loss and update the model parameters using the back propagation algorithm.

[0216] Furthermore, the model validation module includes:

[0217] A detection submodule, used to detect images or videos using the trained power operation detection model, store the label names of the violations and the number of violations detected in a list, and use the accuracy (P) as an evaluation indicator;

[0218] The normalization submodule is used to assume that there are n violation labels. For each violation label i, m violations are detected, and its evaluation index is x ij , j represents the label index of the violation, and the violation is normalized to obtain the normalized value p ij , calculated as follows:

[0219]

[0220] Where min(x i ) and max(x i ) represent the maximum and minimum values ​​of label i in the evaluation index accuracy respectively;

[0221] The information entropy calculation submodule is used to calculate the information entropy H for the i-th violation label. i The calculation formula is as follows:

[0222]

[0223] The weight calculation submodule is used to calculate the weight W of the i-th violation label in terms of accuracy according to the entropy weight method. i The calculation formula is as follows:

[0224]

[0225] Where n is the number of violation labels, is the sum of the information entropy of all violation labels;

[0226] The weighted summation submodule is used to weight all illegal labels on the accuracy W i Perform weighted summation to obtain W P ;

[0227] The verification submodule is used to calculate the comprehensive evaluation score, which is calculated as shown in the following formula:

[0228] Score=W P×P

[0229] The calculated comprehensive evaluation score Score is compared with a preset threshold value. If it is higher than the preset threshold value, it is determined that the power operation detection model has passed the verification.

[0230] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0231] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0232] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0233] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalents that do not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A power operation risk assessment method based on deep learning, characterized in that: include: Collecting pictures of power operation sites and augmenting the pictures through data enhancement; Divide the expanded images into training set and validation set; By replacing the backbone network and loss function of the convolutional neural network, a power operation detection model with a self-attention mechanism suitable for power operation scenarios is constructed; and the power operation detection model is trained using the training set; Based on the entropy weight method, the power operation detection model is verified using the verification set; When the comprehensive score of the accuracy of the violation labels marked in the verification set is judged to be greater than a preset threshold, it is determined that the power operation detection model has passed the verification; using the power operation detection model, the risk level of the power operation site is obtained by detecting the pictures of the power operation site to be evaluated.

2. The method according to claim 1, characterized in that The image is expanded by data augmentation, including: Read multiple pictures of the pictures of the power operation site at any time, perform zooming operations on the multiple pictures, and adjust the saturation, brightness, contrast and hue; Combine multiple images.

3. The method according to claim 1, characterized in that Also includes: The training set and the validation set are divided into a ratio of 8:

2.

4. The method according to claim 1, characterized in that: By replacing the backbone network and loss function of the convolutional neural network, a power operation detection model with a self-attention mechanism suitable for power operation scenarios is constructed, including: The convolutional neural network resizes the input image, and the size of the resized image is limited to 640×640; The backbone network used three 3×3 convolutional layers to extract features from the image, and obtained three feature maps of different scales, namely 80×80, 40×40 and 20×20. The feature maps of three different scales are sent to the detection layer respectively. After the global average pooling operation, the feature maps are reduced to one-dimensional vectors, and the feature maps are classified and regressed. Use the Softmax activation function to convert a one-dimensional vector into a probability distribution for classification tasks; The loss function EC-Loss is used to calculate the difference between the predicted box and the true box and the back-propagation algorithm is used to update the model parameters.

5. The method according to claim 4, characterized in that The backbone network is an improved Transformer network. The improved Transformer network establishes a cross-window self-attention mechanism based on the traditional Transformer architecture. The cross-window self-attention mechanism includes two modules, namely horizontal self-attention and vertical self-attention.

6. The method according to claim 5, characterized in that The horizontal self-attention is used to transform the input feature map Split into The feature map of the horizontal bar, where sω represents the width of the horizontal bar feature map, C, H, W represent the dimension, height and width of the feature map respectively; For each feature X of the horizontal feature map i , i = 1, 2, ..., M, use Transformer to process and get feature Y i ; Merge the M features to obtain the lateral features sent to the detection layer; Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention L-ATT is as follows: X=[X 1 ,X 2 ,...,X M ] Among them, H, W, C represent the height, width and dimension of the feature map respectively, and Q, K, V represent the query vector, key vector and value vector in the horizontal attention.

7. The method according to claim 5, characterized in that The vertical self-attention is used to split the input feature map A(C×H×W) into The feature map of the vertical bars, where sω represents the width of the vertical bar feature map; For each feature X of the vertical bar feature map i , i = 1, 2, ..., M, and processed using Transformer, we can get feature Y i ; Merge the M features to obtain the longitudinal features sent to the detection layer; Assuming that the feature belongs to the kth detection layer, the calculation formula of the horizontal attention R-ATT is as follows: X=[X 1 ,X 2 ,...,X M ] In the improved Transformer network, there are M detection layers, which are evenly divided into two groups. The first detection layer uses horizontal features, and the second detection layer uses vertical features. Finally, the features of the two groups are merged. The formula used in the merging process is shown in the following formula: The expression of the cross-window self-attention mechanism CR-Window-ATT is shown in the following formula: CR-Window-ATT(X)=Concat(head1,head2,...,head M )W O Where W O Represents the commonly used projection matrix, which projects the self-attention result to the dimension of the target output.

8. The method according to claim 1 or 4, characterized in that: The formula of the loss function EC-Loss is as follows: Where y∈[-1,1] represents the real object box, p∈[0,1] represents the predicted probability of the foreground category, the adjustment factor γ is used to focus on difficult-to-distinguish samples, and the weight factor α∈(0,1) is used when the model predicts a positive sample. If it is a negative sample, the weight factor is 1-α. Where q represents the actual category label. When q>0, it means that the actual label is a positive example, and the loss function is the binary cross entropy loss function, which is used to measure the gap between the probability p predicted by the model and the actual positive example; when q=0, it means that the actual label is a negative example, and the loss function is the negative example loss function.

9. The method according to claim 1, characterized in that: Based on the entropy weight method, the power operation detection model is verified using the verification set; when it is determined that the comprehensive score of the accuracy of the violation labels marked in the verification set is greater than a preset threshold, it is determined that the power operation detection model has passed the verification, including: Use the trained power operation detection model to detect the image or video, store the label name of the violation and the number of violations detected in a list, and use the accuracy (P) as an evaluation indicator; Assume there are n violation labels, for each violation label i, m violations are detected, and its evaluation index is x ij , j represents the label index of the violation, and the violation is normalized to obtain the normalized value p ij , calculated as follows: Where min(x i ) and max(x i ) represent the maximum and minimum values ​​of label i in the evaluation index accuracy respectively; For the i-th violation label, the information entropy H i The calculation formula is as follows: According to the entropy weight method, the weight W of the i-th violation label on the accuracy i The calculation formula is as follows: Where n is the number of violation labels, is the sum of the information entropy of all violation labels; The weight W of all violation labels on accuracy i Perform weighted summation to obtain W P ; Calculate the comprehensive evaluation score as shown in the following formula: Score=W P ×P The calculated comprehensive evaluation score Score is compared with a preset threshold value. If it is higher than the preset threshold value, it is determined that the power operation detection model has passed the verification.

10. A power operation risk assessment system based on deep learning, characterized in that: include: A picture acquisition module, used to collect pictures of the power operation site and expand the pictures through data enhancement; An image division module is used to divide the expanded images into a training set and a validation set; A model building module, used to build a power operation detection model with a self-attention mechanism suitable for power operation scenarios by replacing the backbone network and loss function of the convolutional neural network; and train the power operation detection model using the training set; A model verification module, used to verify the power operation detection model using the verification set based on the entropy weight method; When the comprehensive score of the accuracy of the violation labels marked in the verification set is judged to be greater than a preset threshold, it is determined that the power operation detection model has passed the verification; using the power operation detection model, the risk level of the power operation site is obtained by detecting the pictures of the power operation site to be evaluated.

11. The system according to claim 10, characterized in that Model building modules, including: The image resizing submodule is used by the convolutional neural network to resize the input image. The size of the resized image is limited to 640×640. The feature extraction submodule is used to extract features from images by using three 3×3 convolutional layers in a backbone network to obtain feature maps of three different scales, namely 80×80, 40×40 and 20×20. The classification and prediction submodule is used to send the feature maps of three different scales to the detection layer respectively, reduce the dimension of the feature map to a one-dimensional vector after the global average pooling operation, and perform classification and regression prediction on the feature map; The conversion submodule is used to convert the one-dimensional vector into a probability distribution using the Softmax activation function for classification tasks; The parameter update submodule is used to calculate the difference between the predicted box and the true box using the loss function EC-Loss and update the model parameters using the back propagation algorithm.

12. The system according to claim 10, characterized in that Model validation module, including: A detection submodule, used to detect images or videos using the trained power operation detection model, store the label names of the violations and the number of violations detected in a list, and use the accuracy (P) as an evaluation indicator; The normalization submodule is used to assume that there are n violation labels. For each violation label i, m violations are detected, and its evaluation index is x ij , j represents the label index of the violation, and the violation is normalized to obtain the normalized value p ij , calculated as follows: Where min(x i ) and max(x i ) represent the maximum and minimum values ​​of label i in the evaluation index accuracy respectively; The information entropy calculation submodule is used to calculate the information entropy H for the i-th violation label. i The calculation formula is as follows: The weight calculation submodule is used to calculate the weight W of the i-th violation label in terms of accuracy according to the entropy weight method. i The calculation formula is as follows: Where n is the number of violation labels, is the sum of the information entropy of all violation labels; The weighted summation submodule is used to weight all illegal labels on the accuracy W i Perform weighted summation to obtain W P ; The verification submodule is used to calculate the comprehensive evaluation score, which is calculated as shown in the following formula: Score=W P ×P The calculated comprehensive evaluation score Score is compared with a preset threshold value. If it is higher than the preset threshold value, it is determined that the power operation detection model has passed the verification.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

14. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images