A method and device for detecting abnormal temperature of power equipment

Abstract

The application discloses a kind of electric power equipment abnormal temperature detection method and device, including obtaining the infrared image to be identified of electric power equipment, the infrared image to be identified is input to improved target detection model, improved target detection model includes target feature extraction network, target feature fusion network and target detection network;Adopt target feature extraction network to carry out feature strengthening extraction operation to the infrared image to be identified, determine multiple infrared feature maps, target feature extraction network includes cascaded attention activation convolution module, multi-branch feature fusion module and spatial pyramid pooling module;Each infrared feature map is respectively input to target feature fusion network and carries out feature deep fusion operation, and outputs corresponding fusion feature map;Through target detection network, each fusion feature map is carried out temperature detection, and the temperature detection result of electric power equipment is output. Through the model, the temperature of electric power equipment is detected, and the detection effect is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a power equipment abnormal temperature detection method and device. BACKGROUND

[0002] With the increasing of life electricity and industrial electricity, the load of power equipment is increasing, and the power equipment bears high load power transmission. The insulation of the equipment is not only directly affected by electricity and heat in the power transmission process, but also gradually weakened by various factors such as increasing use time or poor environment. High load inevitably causes the power equipment to heat up, and the abnormal temperature of the power equipment leads to increasing failures.

[0003] Timely discovering and processing the abnormal temperature of the power equipment and taking corresponding control measures play a great role in the safe and stable operation of the power equipment. At present, in the traditional power equipment abnormal temperature detection technology, the power equipment is generally detected for abnormal temperature based on a deep learning neural network combined with image recognition. Due to the simple structure of the neural network and the weak generalization ability, the detection accuracy of the network model is easily disturbed in the case of low image resolution and small target defects. At the same time, in the usual case, the infrared image of the power equipment in the actual operation of the substation is less, and the image has a lot of noise and unobvious edge details, which leads to unsatisfactory detection effect. SUMMARY

[0004] The present application provides a power equipment abnormal temperature detection method and device, which is used to solve the technical problem that the detection accuracy of the network model is easily disturbed in the case of low image resolution and small target defects due to the simple structure of the neural network and the weak generalization ability, and the infrared image of the power equipment in the substation has a lot of noise and unobvious edge details in the usual case, which leads to unsatisfactory detection effect.

[0005] The power equipment abnormal temperature detection method provided by the present application comprises:

[0006] An infrared image to be recognized of the power equipment is acquired, and the infrared image to be recognized is input into an improved target detection model. The improved target detection model comprises a target feature extraction network, a target feature fusion network and a target detection network.

[0007] The target feature extraction network is used for performing a feature strengthening extraction operation on the infrared image to be recognized, and a plurality of infrared feature maps are determined. The target feature extraction network comprises a cascaded attention activation convolution module, a multi-branch feature fusion module and a spatial pyramid pooling module.

[0008] The infrared feature maps are respectively input into the target feature fusion network for deep feature fusion operation, and corresponding fusion feature maps are output.

[0009] The target detection network is used to detect the temperature of the power equipment according to the fusion feature maps, and a temperature detection result of the power equipment is output.

[0010] Optionally, the step of using the target feature extraction network to perform feature enhancement extraction operation on the to-be-recognized infrared image to determine a plurality of infrared feature maps comprises:

[0011] The attention activation convolution module is used to perform feature extraction on the input to-be-recognized infrared image, and a vector feature fusion map is output.

[0012] The multi-branch feature fusion module is used to perform multi-branch feature fusion on the vector feature fusion map, and a first infrared feature map is generated.

[0013] The multi-branch feature fusion module is used to perform multi-branch feature fusion on the first infrared feature map, and a second infrared feature map is generated.

[0014] The multi-branch feature fusion module and the spatial pyramid pooling module are used to perform multi-branch feature fusion and pooling operation on the second infrared feature map, and a third infrared feature map is output.

[0015] Optionally, the multi-branch feature fusion module comprises a first branch fusion module and a second branch fusion module in cascade; the attention activation convolution module comprises a spatial-to-depth layer, a non-convolution step layer, an attention mechanism layer, a convolution group, and the second branch fusion module; the step of using the attention activation convolution module to perform feature extraction on the input to-be-recognized infrared image to output a vector feature fusion map comprises:

[0016] The spatial-to-depth layer is used to perform down-sampling and splicing operation on the to-be-recognized infrared image, and a sampling feature map is constructed.

[0017] The sampling feature map is input into the non-convolution step layer for channel number replacement operation, and an intermediate feature map is output.

[0018] The attention mechanism layer is used to remove redundant features in the intermediate feature map, and a target extraction feature map is generated.

[0019] The convolution group is used to perform non-linear mapping on the target extraction feature map, and a non-linear vector feature map is output.

[0020] The second branch fusion module is used to perform branch fusion operation on the non-linear vector feature map, and a vector feature fusion map is determined.

[0021] Optionally, the spatial pyramid pooling module includes a convolutional module with FReLU activation function, a convolutional group, a first pooling layer, a second pooling layer, a third pooling layer, a fourth pooling layer, and an attention mechanism layer; the step of performing feature fusion and pooling operations on the second infrared feature map through a cascaded multi-branch feature fusion module and a spatial pyramid pooling module to output a third infrared feature map includes:

[0022] The second infrared feature map is input into the multi-branch feature fusion module to perform multi-branch feature fusion operation and determine the infrared fused feature map;

[0023] The infrared fused feature map is nonlinearly mapped by convolutional modules and convolutional groups with FReLU activation functions to generate corresponding first and second feature maps. The convolutional group includes three cascaded convolutional modules with FReLU activation functions.

[0024] An attention mechanism layer is used to remove redundant features from the first feature map and output the third feature map.

[0025] The second feature map is input into the first pooling layer, the second pooling layer, the third pooling layer, and the fourth pooling layer respectively to perform pooling operations. Then, it passes through the attention mechanism layer to perform redundant feature removal operations, generating multiple second sub-feature maps.

[0026] After concatenating multiple second sub-feature maps, a fourth feature map is generated by performing non-linear mapping through a convolution module with FReLU activation function.

[0027] After stacking the third feature map and the fourth feature map, a nonlinear mapping is performed through a convolution module with the FReLU activation function to output the third infrared feature map.

[0028] Optionally, the convolutional module with FReLU activation function includes a convolutional layer, a batch regularization layer, and a FReLU activation function layer; the specific processing procedure of the convolutional module with FReLU activation function is as follows:

[0029] The convolutional input map, which is input to the convolutional module with the FReLU activation function, is convolved by a convolutional layer to generate a spatial feature vector map.

[0030] After performing a normalization operation on the spatial feature vector map using a batch regularization layer, a nonlinear mapping is performed through a FReLU activation function layer to output a convolutional output map.

[0031] The FReLU activation function is specifically:

[0032]

[0033]

[0034] where x c,i,j is the feature vector at the two-dimensional position (i, j) on the cth feature channel of the convolution input image; FReLU() is a funnel activation function; is the region centered at the two-dimensional position (i, j) on the cth feature channel of the convolution input image; is the shared parameter of the region centered at the two-dimensional position (i, j) in the cth feature channel; out1 is the first tensor graph; out2 is the second tensor graph; out is the convolution output image; cat() is a concatenation function; dim is a concatenation dimension; and T() is a two-dimensional space condition.

[0035] Optionally, the target feature fusion network comprises an attention convolution module, a sampling convolution module, a feature fusion module, and a first branch fusion module; and the step of performing feature fusion on each of the infrared feature maps by using the target feature fusion network to generate a corresponding fusion feature map comprises:

[0036] inputting the first infrared feature map and the second infrared feature map into the attention convolution module respectively to perform feature extraction and generate a first output feature map and a second output feature map;

[0037] performing feature extraction and up-sampling operation on the third infrared feature map by using the sampling convolution module to output a third output feature map;

[0038] stacking and branch fusing the second output feature map and the third output feature map by using the feature fusion module to determine a fourth output feature map;

[0039] inputting the fourth output feature map into the sampling convolution module to perform feature extraction and up-sampling operation and generate a first initial feature map;

[0040] stacking and branch fusing the first initial feature map and the first output feature map by using the feature fusion module to determine a first fusion feature map;

[0041] branch fusing the first fusion feature map by using the first branch fusion module to obtain a first subgraph, and stacking and branch fusing the first subgraph and the fourth output feature map by using the feature fusion module to output a second fusion feature map;

[0042] branch fusing the second fusion feature map by using the first branch fusion module to obtain a second subgraph, and stacking and branch fusing the second subgraph and the third infrared feature map by using the feature fusion module to generate a third fusion feature map.

[0043] Optionally, the first branch fusion module comprises a convolution module with a FReLU activation function, a maximum value pooling layer and a convolution fusion module; the specific processing process of the first branch fusion module is:

[0044] The fusion input image input into the first branch fusion module is subjected to non-linear mapping and maximum value pooling operation by the convolution module with the FReLU activation function and the maximum value pooling layer respectively to generate corresponding first mapping image and maximum value pooling image;

[0045] The first mapping image and the first pooling image are subjected to non-linear mapping by the convolution module with the FReLU activation function to output second mapping image and third mapping image;

[0046] After the second mapping image and the third mapping image are spliced, convolution operation and non-linear mapping operation are performed by the convolution fusion module to determine the fusion output image.

[0047] Optionally, the target detection network comprises a re-parameterized convolution module and a prediction convolution layer; the step of performing temperature detection on each fusion feature map by the target detection network to output the temperature detection result of the power equipment corresponding to each fusion feature map comprises:

[0048] The re-parameterized convolution module is used to perform adjustment feature vector operation on each fusion feature map to generate corresponding adjustment feature map;

[0049] The prediction convolution layer is used to perform prediction on each adjustment feature map to output the temperature detection result of the power equipment.

[0050] Optionally, the training and testing process of the improved target detection model comprises:

[0051] An abnormal temperature infrared image set of the power equipment is obtained, the improved Mahalanobis distance function and the non-local mean filtering algorithm are used to perform denoising operation on the abnormal temperature infrared image set to generate denoised image data, and then edge detail enhancement operation is performed on the denoised image data to output binary image data;

[0052] The binary image data and the denoised image data are used to construct a target image set, and the target image set is normalized to determine a sample set;

[0053] A preset clustering algorithm is used to perform clustering calculation on the sample set to determine a prior box size, and the prior box size is used to assist the model in target positioning during model training;

[0054] A preset image data set is used to pre-train the improved target detection model to obtain a pre-trained model;

[0055] Based on the weight of the pre-training model and the prior box size, the target feature fusion module and the target detection module of the improved target detection model are locally trained first using the sample set, and then the improved target detection model is trained as a whole.

[0056] The improved Mahalanobis distance function is specifically:

[0057]

[0058] In the formula, C is a target covariance matrix, alpha is a shrinkage coefficient, diag(D) is a new diagonal matrix composed of diagonal elements of the covariance matrix D, mu min is the minimum eigenvalue of the covariance matrix D, mu max is the maximum eigenvalue of the covariance matrix D, D is a covariance matrix, F m,n is a pixel with coordinates (m, n) in a local region centered at pixel coordinates (i, j), F i.j is a pixel with coordinates (i, j), and dm(F m,n , F i.j ) represents the Mahalanobis distance between F(m, n) and F(i, j), and r represents the field radius with (i, j) as the pixel center coordinate.

[0059] The second aspect of the present application provides an abnormal temperature detection device for power equipment, comprising:

[0060] An input module is configured to acquire an infrared image to be identified of the power equipment, and input the infrared image to be identified into an improved target detection model, wherein the improved target detection model comprises a target feature extraction network, a target feature fusion network and a target detection network.

[0061] A feature extraction module is configured to perform a feature enhancement extraction operation on the infrared image to be identified by using the target feature extraction network, and determine a plurality of infrared feature maps, wherein the target feature extraction network comprises a cascaded attention activation convolution module, a multi-branch feature fusion module and a spatial pyramid pooling module.

[0062] A feature fusion module is configured to input each of the infrared feature maps into the target feature fusion network to perform a feature deep fusion operation, and output a corresponding fusion feature map.

[0063] A detection module is configured to perform temperature detection on each of the fusion feature maps by using the target detection network, and output a temperature detection result of the power equipment.

[0064] From the above technical solutions, the present application has the following advantages:

[0065] The improved Mahalanobis distance and non-local mean filtering algorithm is used to denoise the temperature abnormal infrared image of the power equipment of the substation first, and then the edge detail enhancement processing is carried out on the denoised infrared image, so that the edge details of the infrared image are enhanced and the noise in the infrared image is reduced. Secondly, the new attention convolution module is used to replace the ordinary convolution module with a step of 2 in the original feature extraction module, which avoids the non-discriminative loss of feature information. At the same time, the spatial pyramid pooling module is used to improve the original feature extraction module, which eliminates irrelevant redundant feature information and enhances the detection ability of low-resolution images and temperature abnormal small targets. The convolution module with FReLU activation function is used to improve the original feature fusion module, which improves the nonlinear fitting ability of the model. Finally, the corresponding temperature detection result is output through the target detection module, which realizes the rapid and accurate detection of the abnormal temperature area of the power equipment infrared image, improves the detection precision and obtains better detection effect. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0067] Figure 1 The step flow chart of the power equipment abnormal temperature detection method provided by the embodiment of the present application is shown in the figure.

[0068] Figure 2 The structure diagram of the improved target detection model provided by the embodiment of the present application is shown in the figure.

[0069] Figure 3 The step flow chart of the power equipment abnormal temperature detection method provided by the embodiment of the present application is shown in the figure.

[0070] Figure 4 The structure diagram of the first branch fusion module in the target detection model provided by the embodiment of the present application is shown in the figure.

[0071] Figure 5 The structure diagram of the second branch fusion module in the target detection model provided by the embodiment of the present application is shown in the figure.

[0072] Figure 6 The step flow chart of the training process of the improved target detection model provided by the embodiment of the present application is shown in the figure.

[0073] Figure 7 The effect diagram of the image detection test using the improved target detection model provided by the embodiment of the present application is shown in the figure.

[0074] Figure 8 A structural block diagram of an abnormal temperature detection device for power equipment is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The embodiment of the present application provides an abnormal temperature detection method and device for power equipment, and aims to solve the technical problem that in the prior art, due to simple neural network construction and weak generalization ability, the detection precision of a network model is easily disturbed in the case of low image resolution and small target defects, and meanwhile, in a normal case, an infrared image of a power equipment of a transformer substation has a large amount of noise and unobvious edge details, so that the detection effect is not ideal.

[0076] In order to make the technical scheme of the present application clearer and easier to understand, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the embodiments described below are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0077] Please refer to Figure 1 , Figure 1 A step flow chart of an abnormal temperature detection method for power equipment is provided in an embodiment of the present application.

[0078] In step 101, an infrared image to be recognized of power equipment is acquired, and the infrared image to be recognized is input to an improved target detection model, the improved target detection model including a target feature extraction network, a target feature fusion network and a target detection network.

[0079] The infrared image to be recognized refers to constructing an infrared image set of power equipment by collecting infrared images of daily inspection of a transformer substation, and screening out infrared images of abnormal temperature of power equipment from the infrared image set.

[0080] Please refer to Figure 2The improved target detection model provided by the embodiment of the present application is a YOLOv7-SPD model, which is improved on the basis of the original YOLOv7 target detection model. Specifically, the improvement is as follows: the normal convolution module with a step of 2 in the original YOLOv7 detection model is replaced by an SPD-NAM convolution module (an attention convolution module composed of a space-to-depth layer, a convolution step-free layer and an attention mechanism layer), the NAM attention mechanism is embedded into the SPPCSPC (SPP (Spatial Pyramid Pooling) structure used in the original YOLOv7 target detection model) to construct an SPPCSPC-NAM module (a spatial pyramid pooling module), then the original CBS module is improved into a CBF module (a convolution module with a FReLU activation function) based on the FReLU activation function, and an MB_CBF module (a second branch fusion module) and a TB_CBF module (a first branch fusion module) are constructed, thereby obtaining the improved YOLOv7-SPD model in the embodiment.

[0081] The model mainly includes three parts, namely a target feature extraction module (i.e. a Backbone backbone extraction network in the neural network model), a target feature fusion module (i.e. a Neck feature fusion network in the neural network model) and a target detection module (i.e. a Head detection network in the neural network model).

[0082] In the embodiment, when receiving an infrared image to be identified of a power equipment, the infrared image to be identified is input into the improved target detection model.

[0083] In step 102, a target feature extraction network is used to perform a feature strengthening extraction operation on the infrared image to be identified, and a plurality of infrared feature maps are determined. The target feature extraction network includes a cascaded attention activation convolution module, a multi-branch feature fusion module and a spatial pyramid pooling module.

[0084] In this step, the target feature extraction network includes a cascaded attention activation convolution module, a multi-branch feature fusion module and a spatial pyramid pooling module (SPPCSPC-NAM module); the infrared feature maps include a first infrared feature map, a second infrared feature map and a third infrared feature map.

[0085] In the embodiment, a target feature extraction network is used to perform a feature strengthening extraction operation on the infrared image to be identified, and a plurality of infrared feature maps are determined.

[0086] In step 103, each infrared feature map is input into a target feature fusion network for a feature deep layer fusion operation, and a corresponding fusion feature map is output.

[0087] In the embodiment, each infrared feature map is input into a target feature fusion network for feature deep fusion operation, and a corresponding fusion feature map is output.

[0088] In step 104, the temperature detection network is used to detect the temperature of each fusion feature map, and the temperature detection result of the power equipment is output.

[0089] In the embodiment, the temperature detection network is used to detect the temperature of each fusion feature map, and the temperature detection result of the power equipment is output.

[0090] In the embodiment, the original YOLOv7 target detection model is improved to build a YOLOv7-SPD model for power equipment in a substation, and the YOLOv7-SPD model is trained and tested based on an infrared image dataset of power equipment in a substation. The YOLOv7-SPD model is used in combination with the power equipment abnormal temperature detection method to detect the abnormal temperature area in the power equipment image. In the embodiment, the original YOLOv7 detection model with a step of 2 is replaced by an SPD-NAM convolution module to avoid non-discriminative loss of feature information. The NAM attention mechanism is embedded into the SPPCSPC to build an SPPCSPC-NAM module, which can eliminate irrelevant redundant feature information and enhance the detection capability of low-resolution images and small temperature abnormal targets. Then, the original CBS module is improved to a CBF module based on the FReLU activation function, and the MB_CBF module and the TB_CBF module are built to improve the nonlinear fitting capability of the model. Finally, the corresponding temperature detection result is output by the target detection module to quickly and accurately detect the abnormal temperature area of the power equipment infrared image, improve the detection precision, and obtain better detection effect.

[0091] Please refer to Figure 3 , Figure 3 The step flow chart of the power equipment abnormal temperature detection method provided in the embodiment of the application is shown in the figure.

[0092] The power equipment abnormal temperature detection method provided by the application comprises the following steps:

[0093] In step 301, the infrared image to be recognized of the power equipment is obtained, and the infrared image to be recognized is input into the improved target detection model. The improved target detection model comprises a target feature extraction network, a target feature fusion network and a target detection network.

[0094] Specifically, when the power equipment needs to be detected for abnormal temperature, the infrared image of the corresponding power equipment abnormal temperature can be input into the pre-trained improved target detection model for processing.

[0095] In the embodiment, an infrared image to be recognized of the power equipment is acquired, and the infrared image to be recognized is input into the improved target detection model.

[0096] In step 302, the input infrared image to be recognized is subjected to feature extraction by using the attention activation convolution module, and a vector feature fusion graph is output.

[0097] The vector feature fusion graph refers to a feature graph output after image processing by the attention activation convolution module, and it can be understood that it can correspond to any output feature graph after image processing by the attention activation convolution module in the image detection process.

[0098] In this step, the multi-branch feature fusion module includes a cascaded first branch fusion module (TB_CBF module) and a second branch fusion module (MB_CBF module), and the attention activation convolution module includes a space-to-depth layer (SPD layer), a non-strided convolution layer, an attention mechanism layer (NAM attention mechanism), a convolution group, and the second branch fusion module.

[0099] Further, step 302 can include the following sub-steps:

[0100] S21, the space-to-depth layer is used to perform downsampling and splicing operations on the infrared image to be recognized, and a sampling feature graph is constructed.

[0101] For convenience of description, it is assumed that, in the feature extraction process, an infrared image to be recognized F of any size (S, S, C1) is input into the attention activation convolution module. in After the SPD layer, the image F′ in has a size of (S / N sacle , S / N sacle , N2sacleC1). sacle A N2sacle sub-feature graph (with a size of S / N sacle , S / N sacle , C1) is randomly extracted, and then a sampling feature graph F′ in is obtained after being serially stacked.

[0102] S22, the sampling feature graph is input into the non-strided convolution layer to perform channel number replacement, and an intermediate feature graph is output.

[0103] Specifically, the channel number is changed by the non-strided convolution layer to obtain an intermediate feature graph F′ out , where the convolution layer filter size is C2×C2, C2<N2sacleC1, and the intermediate feature graph F′ out has a size of (S / N sacle , S / Nsacle ,C2)

[0104] S23, the redundant features in the intermediate feature map are removed through the attention mechanism layer to generate a target extraction feature map.

[0105] The redundant features refer to the features corresponding to the smaller weights in the feature map.

[0106] It is worth mentioning that the NAM attention (normalization-based attention module, NAM) is an efficient and lightweight attention mechanism, and its core is to use the batch normalization layer (BN) to process the scaling factor of the input feature map, which represents the importance of each feature information, and the calculation formula is as follows:

[0107]

[0108] In the formula, μ B and σ B are the mean and standard deviation of the small batch B; γ and β are trainable affine transformation parameters (i.e. scale and displacement); B in and B out represent the input batch and the output batch respectively.

[0109] Exemplarily, the redundant features in the intermediate feature map are removed through the attention mechanism layer, that is, the BN processing (normalization) is performed on the input intermediate feature map according to the channel attention submodule to obtain the scaling factor γ and the channel weight w γ for each channel, and finally the pixel-level accumulation is performed with the intermediate feature map to obtain the output feature Mc; then the feature Mc is input into the spatial attention submodule to obtain the spatial scaling factor λ and the spatial weight w λ , and the target extraction feature map Ms after NAM attention processing is obtained after pixel-level accumulation with the intermediate feature map. Through the above processing procedure, the features corresponding to the smaller weights in the feature map are removed, so that the features with larger weights and stronger importance are obtained.

[0110] S24, the target extraction feature map is nonlinearly mapped by a convolution group to output a nonlinear vector feature map.

[0111] The nonlinear vector feature map refers to the feature map after image processing by the convolution group, and it can be understood that it can correspond to any output feature map after image processing by the convolution group in the image detection process.

[0112] S25, the second branch fusion module is used to perform branch fusion operation on the nonlinear vector feature map to determine a vector feature fusion map.

[0113] The vector feature fusion map refers to the feature map after image processing by the second branch fusion module. It can be understood that it can correspond to any output feature map after image processing by the second branch fusion module during image detection.

[0114] In this embodiment, by replacing the ordinary convolutional module with a stride of 2 in the original YOLOv7 detection model with the SPD-NAM convolutional module, irrelevant and redundant feature information is eliminated, enhancing the detection capability for low-resolution images and small targets with temperature anomalies. Furthermore, the original CBS module is improved by using the FReLU activation function to construct the MB_CBF module (second branch fusion module) based on the CBF module, which improves the model's nonlinear fitting capability and avoids non-discriminatory loss of feature information.

[0115] Step 303: Perform multi-branch feature fusion on the vector feature fusion map through the multi-branch feature fusion module to generate the first infrared feature map.

[0116] Optionally, refer to Figure 4 and Figure 5 The first branch fusion module (TB_CBF module) includes a convolutional module with FReLU activation, a max pooling layer, and a convolutional fusion module. The specific processing steps of the first branch fusion module are as follows: The fusion input map input to the first branch fusion module is subjected to nonlinear mapping and max pooling operations using the convolutional module with FReLU activation and the max pooling layer, respectively, to generate corresponding first mapping map and max pooling map. The first mapping map and first pooling map are then subjected to nonlinear mapping using the convolutional module with FReLU activation, outputting a second mapping map and a third mapping map. The second and third mapping maps are then concatenated, and convolutional operations and nonlinear mapping operations are performed by the convolutional fusion module to determine the fusion output map.

[0117] For example, the specific processing procedure of the second branch fusion module (MB_CBF module) is as follows: the feature maps are respectively input into the CBF module (a convolutional module with FReLU activation function) for nonlinear mapping to obtain two first feature sub-maps; then one of the first feature sub-maps is input into the cascaded CBF module for nonlinear mapping to obtain a second feature sub-map, and then the second feature sub-map is input into the cascaded CBF module for nonlinear mapping to obtain a third feature sub-map; finally, the two first feature sub-maps, the second feature sub-map and the third feature sub-map are fused, and then input into the convolutional layer (Conv) for convolution operation, and then through the CBF module for nonlinear mapping to obtain the final first branch fusion map.

[0118] In the embodiment, the multi-branch feature fusion module is used for performing multi-branch feature fusion on the vector feature fusion graph to generate a first infrared feature graph.

[0119] In step 304, the multi-branch feature fusion module is used for performing multi-branch feature fusion on the first infrared feature graph to generate a second infrared feature graph.

[0120] In the embodiment, the multi-branch feature fusion module is used for performing multi-branch feature fusion on the first infrared feature graph to generate a second infrared feature graph.

[0121] In step 305, the multi-branch feature fusion module and the spatial pyramid pooling module are cascaded to perform multi-branch feature fusion and pooling operation on the second infrared feature graph to output a third infrared feature graph.

[0122] The spatial pyramid pooling module (SPPCSPC-NAM) includes a convolution module with a FReLU activation function, a convolution group, a first pooling layer, a second pooling layer, a third pooling layer, a fourth pooling layer, and an attention mechanism layer.

[0123] The convolution group includes three convolution modules with FReLU activation functions cascaded.

[0124] Further, step 305 can include the following sub-steps:

[0125] S51, inputting the second infrared feature graph into the multi-branch feature fusion module to perform a multi-branch feature fusion operation and determining an infrared fusion feature graph.

[0126] S52, performing nonlinear mapping on the infrared fusion feature graph by the convolution module with the FReLU activation function and the convolution group respectively to generate corresponding first and second feature graphs, and the convolution group includes three convolution modules with FReLU activation functions cascaded.

[0127] The first feature graph refers to the feature graph processed by the convolution module with the FReLU activation function, and it can be understood that it can correspond to any output feature graph processed by the convolution module with the FReLU activation function in the image detection process.

[0128] Optionally, the convolution module with the FReLU activation function includes a convolution layer, a batch normalization layer, and a FReLU activation function layer; and the specific processing process of the convolution module with the FReLU activation function is as follows:

[0129] The convolution input graph input into the convolution module with the FReLU activation function is subjected to convolution operation by the convolution layer to generate a spatial feature vector graph;

[0130] The batch normalization layer is used to perform normalization on the spatial feature vector map, and the FReLU activation function layer is used for non-linear mapping, and the convolution output map is output.

[0131] The FReLU activation function is specifically:

[0132]

[0133]

[0134] In the formula, x c,i,j is a feature vector at a two-dimensional position (i, j) on the cth feature channel in the convolution input map; FReLU() is a funnel activation function; is a region centered at the two-dimensional position (i, j) on the cth feature channel in the convolution input map; is a shared parameter in the cth feature channel for the region centered at the two-dimensional position (i, j); out1 is the first tensor map; out2 is the second tensor map; out is the convolution output map; cat() is a concatenation function; dim is the concatenation dimension; T() is a two-dimensional space condition.

[0135] It is worth mentioning that the input convolution input map x c,i,j is applied to FReLU to obtain out1 and out2, and then the two vectors are concatenated along dim=1 using cat to obtain the final output convolution output map out.

[0136] S53, using the attention mechanism layer to remove redundant features in the first feature map, outputting a third feature map.

[0137] S54, inputting the second feature map into the first pooling layer, the second pooling layer, the third pooling layer, and the fourth pooling layer respectively to perform pooling operation, and then sequentially passing through the attention mechanism layer to remove redundant features, generating multiple second sub-feature maps.

[0138] Specifically, the first pooling layer, the second pooling layer, the third pooling layer, and the fourth pooling layer are pooling layers with pooling kernel sizes of 1, 5, 9, and 13 respectively; the second sub-feature map refers to an intermediate map generated in the spatial pyramid pooling module.

[0139] S55, after concatenating multiple second sub-feature maps, and through the convolution module with FReLU activation function for non-linear mapping, a fourth feature map is generated.

[0140] The fourth feature map refers to an intermediate map generated in the spatial pyramid pooling module.

[0141] S56, after stacking the third feature map and the fourth feature map, performing nonlinear mapping on the third infrared feature map through a convolution module with a FReLU activation function.

[0142] It is worth mentioning that after the feature fusion of the third feature map and the fourth feature map, the third infrared feature map is output through the processing of the CBF module, realizing the fusion of different scale feature information.

[0143] For example, after the input feature map is processed by the 3 CBF modules in the first branch and the 1, 5, 9, and 13 pooling kernel size pooling layers, the NAM attention module is added, and then the CBF module is processed to obtain the feature F1; in addition, the NAM attention module is added after the input feature map is processed by the CBF module in the second branch to obtain the feature F2; finally, the F1 and F2 features are stacked (Concat) and processed by the CBF module, realizing the fusion of different scale feature information and improving the precision of the YOLOv7-SPD target detection model.

[0144] In this embodiment, the multi-branch feature fusion module and the spatial pyramid pooling module are cascaded to perform multi-branch feature fusion and pooling operations on the second infrared feature map, and output the third infrared feature map.

[0145] Step 306, input each infrared feature map into a target feature fusion network for feature deep fusion operation, and output a corresponding fusion feature map.

[0146] The target feature fusion network includes an attention convolution module, a sampling convolution module, a feature fusion module, and a first branch fusion module.

[0147] The fusion feature map includes a first fusion feature map, a second fusion feature map, and a third fusion feature map.

[0148] Further, step 306 can include the following sub-steps:

[0149] S61, input the first infrared feature map and the second infrared feature map into the attention convolution module for feature extraction, to generate a first output feature map and a second output feature map.

[0150] S62, perform feature extraction and up-sampling operation on the third infrared feature map through the sampling convolution module, to output a third output feature map.

[0151] S63, stack and branch fuse the second output feature map and the third output feature map using the feature fusion module to determine a fourth output feature map.

[0152] S64, input the fourth output feature map into the sampling convolution module to perform feature extraction and up-sampling operation, to generate a first initial feature map.

[0153] S65, stack and branch fuse the first initial feature map and the first output feature map through the feature fusion module to determine a first fused feature map.

[0154] S66, branch fuse the first fused feature map through the first branch fusion module to obtain a first sub-map, and input the first sub-map and the fourth output feature map into the feature fusion module to perform stack and branch fusion, and output a second fused feature map.

[0155] S67, branch fuse the second fused feature map through the first branch fusion module to obtain a second sub-map, and input the second sub-map and the third infrared feature map into the feature fusion module to perform stack and branch fusion, and generate a third fused feature map.

[0156] Specifically, the sampling convolution module includes an up-sampling layer and an SPD-NAM convolution module, and the feature fusion module includes a Concat layer and a second branch fusion module; the first initial feature map refers to a feature map output after the sampling convolution module performs image processing, and it can be understood that it can correspond to any output feature map after the sampling convolution module performs image processing in the image detection process; the fourth output feature map refers to a feature map output after the feature fusion module performs image processing, and it can be understood that it can correspond to any output feature map after the feature fusion module performs image processing in the image detection process.

[0157] In this embodiment, each infrared feature map is input into the target feature fusion network to perform a feature deep fusion operation, and a corresponding fused feature map is output.

[0158] Step 307, perform temperature detection on each fused feature map through the target detection network, and output a temperature detection result of the power equipment.

[0159] The target detection network includes a reparameterization convolution module and a prediction convolution layer.

[0160] The adjusted feature map refers to a feature map output after the reparameterization convolution module performs image processing.

[0161] Further, step 307 includes the following sub-steps:

[0162] S71, perform an adjusted feature vector operation on each fused feature map through the reparameterization convolution module to generate a corresponding adjusted feature map.

[0163] S72, perform prediction on each adjusted feature map through the prediction convolution layer to output a temperature detection result of the power equipment.

[0164] It is worth mentioning that the three scale features (corresponding to three YOLO_Head) extracted by the target feature extraction network and the target feature fusion network, that is, the fusion feature map, are input into the RepConv module, that is, the reparameterization convolution module, and the adjusted feature vector size is output. Since there is only one type of target, the output size is (1+5) x 3 = 18, where 1 represents the number of label categories, 5 represents the coordinate position (x, y, w, h) of the detection box and the confidence score C, and 3 represents the number of prior boxes. Finally, the 1x1 convolution in YOLO_Head, that is, the prediction convolution layer, is used for prediction, and the temperature detection result of the power equipment is output, so that the staff can clearly understand the temperature abnormal area.

[0165] In the embodiment, the target detection network is used to perform temperature detection on each fusion feature map, and output the temperature detection result of the power equipment.

[0166] In the embodiment of the present application, by improving the original YOLOv7 target detection model, a YOLOv7-SPD model for power equipment in a substation is constructed, and the YOLOv7-SPD model is trained and tested based on the infrared image dataset of the power equipment in the substation. The YOLOv7-SPD model is combined with the abnormal temperature detection method for power equipment to realize the detection of the abnormal temperature area in the power equipment image. In the embodiment of the present application, the ordinary convolution module with a step of 2 in the original YOLOv7 detection model is replaced by the SPD-NAM convolution module, which avoids non-discriminative loss of feature information. The NAM attention mechanism is embedded into the SPPCSPC to construct the SPPCSPC-NAM module, which can eliminate irrelevant redundant feature information, thereby enhancing the detection capability for low-resolution images and small temperature abnormal targets. Then, the original CBS module is improved into a CBF module based on the FReLU activation function, and the MB_CBF module and the TB_CBF module are constructed to improve the nonlinear fitting capability of the model. Finally, the corresponding temperature detection result is output through the target detection module, which realizes the rapid and accurate detection of the abnormal temperature area of the power equipment infrared image, improves the detection precision and obtains better detection effect.

[0167] Please refer to Figure 6 , Figure 6 The step flow chart of the improved target detection model training process provided in the embodiment of the present application can include the following steps:

[0168] Step 601, acquire an abnormal temperature infrared image set of power equipment, perform a denoising operation on the abnormal temperature infrared image set using an improved Mahalanobis distance function and a non-local mean filtering algorithm, generate denoised image data, and then perform an edge detail enhancement operation on the denoised image data, output binary image data.

[0169] In this step, the power equipment infrared image set is constructed by collecting the infrared images of daily inspection of the substation, and the infrared images of abnormal temperature of the power equipment are screened out from the infrared image set to construct the infrared image set of abnormal temperature of the power equipment of the substation, which contains infrared images of various shooting angles, backgrounds, environments and weather conditions.

[0170] The images in the abnormal temperature infrared image set are labeled, specifically, the LabelImg labeling tool is used to frame the temperature abnormal area with a rectangular frame (Rectangle), and the label is defined as "abnormal".

[0171] Further, all pixel values corresponding to each image in the abnormal temperature infrared image set are obtained, all covariances of pixel values i and pixel values j corresponding to each image are calculated, all covariances are used as matrix elements to form a covariance matrix, all covariance matrices are substituted into the improved Mahalanobis distance function for operation, and the non-local mean filtering algorithm is combined to perform a denoising operation on the abnormal temperature infrared image set to generate denoised image data.

[0172] In a specific implementation, for the convenience of implementation of the method, the process of denoising the image by combining the covariance matrix, the improved Mahalanobis distance function and the non-local mean filtering algorithm can be converted into a formula encapsulation form, and the abnormal temperature noise-containing infrared image in the temperature infrared image set of the power equipment of the substation actually collected can be expressed as:

[0173] F i,j =X i,j +Z i,j ;

[0174] In the formula, F represents the power equipment temperature abnormal infrared image containing noise, X is the real power equipment temperature abnormal infrared image, Z is an additive white noise signal, and (i,j) is the pixel coordinate corresponding to the abnormal temperature noise-containing infrared image.

[0175] The improved Mahalanobis distance function is used to measure the similarity, specifically:

[0176]

[0177]

[0178] In the formula, C is the target covariance matrix; a is the shrinkage coefficient; diag(D) is a new diagonal matrix composed of diagonal elements of the covariance matrix D; μ min is the minimum eigenvalue of the covariance matrix D; μ max is the maximum eigenvalue of the covariance matrix D; D is the covariance matrix; F m,nF(m,n) is a pixel with coordinate (m,n) in the local region centered at pixel coordinate (i,j); F i.j F(i,j) is a pixel with coordinate (i,j); dm(F m,n F(i,j) is a pixel with coordinate (i,j); dm(F i.j ) represents the Mahalanobis distance between F(m,n) and F(i,j); r represents the radius of the region centered at pixel coordinate (i,j).

[0179] In combination with the Non-Local Means (NLM) algorithm, the abnormal temperature infrared image containing noise is processed, and the obtained infrared image, i.e., the denoised image data, can be expressed as:

[0180]

[0181] In the formula, is the denoised image data, Ω i,j represents a local region centered at pixel (i,j), represents the distance between F(m,n) and F(i,j), F(m,n) is a pixel with coordinate (m,n) in the local region centered at pixel coordinate (i,j), and F(i,j) is a pixel with coordinate (i,j).

[0182] Finally, the denoised image is input into the edge detail enhancement module. The module first performs median filtering on the denoised image using a convolution kernel size of 7x7, then performs erosion and dilation operations on the denoised image through three iterations of convolution with a kernel size of 5x5, i.e., 3 times of erosion and 3 times of dilation, and then performs Otsu binarization. Finally, the image is dilated through three iterations of convolution with a kernel size of 5x5, and a binary image that highlights the edge details of the power equipment in the infrared image, i.e., binary image data, is finally obtained.

[0183] In this embodiment, an abnormal temperature infrared image set of power equipment is obtained, a denoising operation is performed on the abnormal temperature infrared image set using the improved Mahalanobis distance function and the non-local mean filtering algorithm, denoised image data is generated, and then an edge detail enhancement operation is performed on the denoised image data, and binary image data is output.

[0184] Step 602, construct a target image set using the binary image data and the denoised image data, and normalize the target image set to determine a sample set.

[0185] Specifically, the binary image data is added to the initially input denoised image data to obtain the combined enhanced infrared image of the power equipment, i.e., the target image set. Then, the image set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The images in the target image set are normalized to a size of 640×640. The sample set includes the training set, the validation set, and the test set.

[0186] In this embodiment, a target image set is constructed using binary image data and denoised image data, and the target image set is normalized to determine the sample set.

[0187] Step 603: Use a preset clustering algorithm to perform clustering calculations on the sample set to determine the prior bounding box size. The prior bounding box size is used to assist the model in target localization during model training.

[0188] In this step, the K-means clustering algorithm is preset. Before training and testing, the K-means clustering algorithm is first used to cluster the images and their labels in the training set to obtain the optimal prior box size [12 20; 20 30; 30, 47; 36 76; 53 56; 67 98; 132 110; 92 169; 211 236] to assist the model in target localization.

[0189] In this embodiment, a preset clustering algorithm is used to perform clustering calculations on the sample set to determine the prior bounding box size. The prior bounding box size is used to assist the model in target localization during model training.

[0190] Step 604: Pre-train the improved target detection model using a pre-set image dataset to obtain a pre-trained model.

[0191] In this step, the preset image dataset is the COCO dataset. The YOLOv7-SPD model is trained using the COCO dataset to obtain a pre-trained model, which is then transferred to the sample set of this invention for training and testing.

[0192] In this embodiment, a pre-trained model is obtained by pre-training the improved target detection model using a preset image dataset.

[0193] Step 605: Based on the weights and prior box size of the pre-trained model, use the sample set to first perform local model training on the target feature fusion module and the target detection module of the improved target detection model, and then perform overall model training on the improved target detection model.

[0194] Specifically, the training process can be divided into two parts. In the first part, based on the weights of the pre-trained model and the prior box size, the weight parameters of the feature extraction network are first frozen, and the target feature fusion network and the target detection network of the improved target detection model are locally trained for 100 rounds with a batch size of 8 and a learning rate of 1x10 -2 ; in the second part, after unfreezing, the entire model is retrained for 100 rounds with a batch size of 2 and a learning rate of 1x10 -3 , combined with the SGD optimizer and the Cos learning rate reduction strategy, a weight model is saved every 10 rounds; a total of 200 rounds are trained, and finally, the model with the smallest loss value and the highest validation set accuracy is saved as the optimal YOLOv7-SPD target detection model, i.e., the improved target detection model. Then, the test set is input into the optimal YOLOv7-SPD target detection model for detection, and the detection results, including the class, the prediction box, and the confidence, are obtained. The prediction box is filtered through the non-maximum suppression (NMS) to retain the detection box that best fits the target region. Finally, the average precision (AP) and the FPS value of the test set results are calculated according to the calculation principle of the detection precision (AP) and the detection speed (FPS) evaluation indicators, which are used to verify the actual detection effect of the present application, as shown in Table 1. The target detection model includes the target feature extraction network (Backbone), the target feature fusion network (Neck), and the target detection network (Head).

[0195] Table 1 Test set detection results

[0196]

[0197] Exemplarily, referring to Figure 7 , an effect schematic diagram of image detection test using the improved target detection model trained is shown, and it can be seen that, by using the improved target detection model provided by the present embodiment, the position of the abnormal temperature in the power equipment can be clearly and accurately located, and the effective data such as the corresponding abnormal temperature, the environmental temperature, and the distance can be detected, so that the power operation and maintenance personnel can quickly confirm the specific position of the power equipment with abnormal temperature problem, thereby timely processing and avoiding greater loss.

[0198] In the present embodiment, based on the weights of the pre-trained model and the prior box size, the sample set is used to locally train the target feature fusion module and the target detection module of the improved target detection model, and then the improved target detection model is trained as a whole.

[0199] In the embodiment of the present application, by improving the original YOLOv7 target detection model, a YOLOv7-SPD model for power equipment in a substation is constructed, and the YOLOv7-SPD model is trained and tested based on an infrared image dataset of power equipment in a substation. The YOLOv7-SPD model is used in combination with an abnormal temperature detection method for power equipment to detect the abnormal temperature area in the image of the power equipment. In the embodiment of the present application, the original YOLOv7 detection model with a step of 2 is replaced by an SPD-NAM convolution module to avoid non-discriminative loss of feature information. The NAM attention mechanism is embedded into the SPPCSPC to construct an SPPCSPC-NAM module, which can eliminate irrelevant redundant feature information and thus enhance the detection capability for low-resolution images and small temperature abnormal targets. Then, the original CBS module is improved into a CBF module based on the FReLU activation function, and the MB_CBF module and the TB_CBF module are constructed to improve the nonlinear fitting capability of the model. Finally, the corresponding temperature detection result is output by the target detection module to quickly and accurately detect the abnormal temperature area of the infrared image of the power equipment, improve the detection precision, and obtain better detection effect.

[0200] Please refer to Figure 8 , Figure 8 The structure block diagram of the power equipment abnormal temperature detection device provided in the embodiment of the present application is shown in FIG. 1.

[0201] The input module 801 is configured to acquire an infrared image to be identified of power equipment, and input the infrared image to be identified into an improved target detection model. The improved target detection model includes a target feature extraction network, a target feature fusion network, and a target detection network.

[0202] The feature extraction module 802 is configured to perform a feature strengthening extraction operation on the infrared image to be identified by using the target feature extraction network, and determine a plurality of infrared feature maps. The target feature extraction network includes a cascaded attention activation convolution module, a multi-branch feature fusion module, and a spatial pyramid pooling module.

[0203] The feature fusion module 803 is configured to input each infrared feature map into the target feature fusion network to perform a feature deep fusion operation, and output a corresponding fusion feature map.

[0204] The detection module 804 is configured to perform a temperature detection operation on each fusion feature map by using the target detection network, and output a temperature detection result of the power equipment.

[0205] Further, the feature extraction module 802 includes:

[0206] The vector fusion submodule is configured to perform feature extraction on the input infrared image to be recognized by using an attention activation convolution module, and output a vector feature fusion image.

[0207] The first extraction submodule is configured to perform multi-branch feature fusion on the vector feature fusion image by using a multi-branch feature fusion module, and generate a first infrared feature image.

[0208] The second extraction submodule is configured to perform multi-branch feature fusion on the first infrared feature image by using a multi-branch feature fusion module, and generate a second infrared feature image.

[0209] The third extraction submodule is configured to perform multi-branch feature fusion and pooling operations on the second infrared feature image by using a cascaded multi-branch feature fusion module and a spatial pyramid pooling module, and output a third infrared feature image.

[0210] Further, the vector fusion submodule comprises:

[0211] The sampling unit is configured to perform downsampling and splicing operations on the infrared image to be recognized by using a spatial-to-depth layer, and construct a sampling feature image.

[0212] The replacement unit is configured to input the sampling feature image into a non-convolution step layer to perform a channel number replacement operation, and output an intermediate feature image.

[0213] The elimination unit is configured to eliminate redundant features in the intermediate feature image by using an attention mechanism layer, and generate a target extraction feature image.

[0214] The mapping unit is configured to perform nonlinear mapping on the target extraction feature image by using a convolution group, and output a nonlinear vector feature image.

[0215] The fusion unit is configured to perform branch fusion operations on the nonlinear vector feature image by using a second branch fusion module, and determine a vector feature fusion image.

[0216] Further, the third extraction submodule comprises:

[0217] The multi-branch fusion unit is configured to input the second infrared feature image into a multi-branch feature fusion module to perform multi-branch feature fusion operations, and determine an infrared fusion feature image.

[0218] The second mapping unit is configured to perform nonlinear mapping on the infrared fusion feature image by using a convolution module with a FReLU activation function and a convolution group, respectively, to generate a corresponding first feature image and a second feature image, and the convolution group comprises three convolution modules with FReLU activation functions in cascade.

[0219] The second elimination unit is configured to eliminate redundant features in the first feature image by using an attention mechanism layer, and output a third feature image.

[0220] The pooling unit is configured to sequentially perform the redundant feature elimination operation on the second feature maps after the second feature maps are input into the first pooling layer, the second pooling layer, the third pooling layer and the fourth pooling layer respectively and the pooling operation is performed, to generate a plurality of second sub-feature maps.

[0221] The splicing unit is configured to splice the plurality of second sub-feature maps, perform the non-linear mapping on the plurality of second sub-feature maps through the convolution module with the FReLU activation function, and generate a fourth feature map.

[0222] The third mapping unit is configured to stack the third feature map and the fourth feature map, perform the non-linear mapping on the third feature map and the fourth feature map through the convolution module with the FReLU activation function, and output a third infrared feature map.

[0223] Optionally, the convolution module with the FReLU activation function comprises a convolution layer, a batch normalization layer and a FReLU activation function layer. The image processing step with the FReLU activation function comprises:

[0224] The convolution input map input into the convolution module with the FReLU activation function is subjected to the convolution operation by the convolution layer to generate a spatial feature vector map.

[0225] The spatial feature vector map is subjected to the normalization operation by the batch normalization layer, and is subjected to the non-linear mapping by the FReLU activation function layer to output a convolution output map.

[0226] The FReLU activation function is specifically as follows:

[0227]

[0228]

[0229] In the formula, x c,i,j is a feature vector at a two-dimensional position (i, j) on a cth feature channel in the convolution input map; FReLU() is a funnel activation function; is a region centered at the two-dimensional position (i, j) on the cth feature channel in the convolution input map; is a shared parameter of the region centered at the two-dimensional position (i, j) in the cth feature channel; out1 is a first tensor map; out2 is a second tensor map; out is a convolution output map; cat() is a splicing function; dim is a splicing dimension; and T() is a two-dimensional space condition.

[0230] Further, the feature fusion module 803 comprises:

[0231] The attention extraction sub-module is configured to input the first infrared feature map and the second infrared feature map into the attention convolution module respectively to perform feature extraction, and generate a first output feature map and a second output feature map.

[0232] The second sampling sub-module is configured to perform feature extraction and up-sampling operation on the third infrared feature map by the sampling convolutional module, and output a third output feature map.

[0233] The fourth fusion sub-module is configured to stack and branch fuse the second output feature map and the third output feature map by the feature fusion module, and determine a fourth output feature map.

[0234] The third sampling sub-module is configured to input the fourth output feature map into the sampling convolutional module to perform feature extraction and up-sampling operation, and generate a first initial feature map.

[0235] The fifth fusion sub-module is configured to stack and branch fuse the first initial feature map and the first output feature map by the feature fusion module, and determine a first fusion feature map.

[0236] The second branch fusion sub-module is configured to branch fuse the first fusion feature map by the first branch fusion module to obtain a first sub-map, and input the first sub-map and the fourth output feature map into the feature fusion module to perform stacking and branch fusion, and output a second fusion feature map.

[0237] The third branch fusion sub-module is configured to branch fuse the second fusion feature map by the first branch fusion module to obtain a second sub-map, and input the second sub-map and the third infrared feature map into the feature fusion module to perform stacking and branch fusion, and generate a third fusion feature map.

[0238] Optionally, the first branch fusion module includes a convolutional module with a FReLU activation function, a maximum value pooling layer, and a convolutional fusion module. The image processing steps of the first branch fusion module include:

[0239] The fusion input map input into the first branch fusion module is subjected to non-linear mapping and maximum value pooling operation by the convolutional module with the FReLU activation function and the maximum value pooling layer respectively, to generate a corresponding first mapping map and a maximum value pooling map;

[0240] The first mapping map and the first pooling map are subjected to non-linear mapping by the convolutional module with the FReLU activation function, to output a second mapping map and a third mapping map;

[0241] After the second mapping map and the third mapping map are spliced, convolution operation and non-linear mapping operation are performed by the convolutional fusion module to determine a fusion output map.

[0242] Further, the detection module 804 includes:

[0243] The adjustment sub-module is configured to perform adjustment feature vector operation on each fusion feature map by the re-parameterized convolutional module, to generate a corresponding adjustment feature map.

[0244] a prediction submodule configured to predict each adjusted feature map by using a prediction convolutional layer, and output a temperature detection result of the power equipment.

[0245] In an optional embodiment, the device further comprises:

[0246] an output binary image module configured to obtain a set of abnormal temperature infrared images of the power equipment, perform a denoising operation on the set of abnormal temperature infrared images by using an improved Mahalanobis distance function and a non-local mean filtering algorithm, generate denoised image data, and then perform an edge detail enhancement operation on the denoised image data, and output binary image data.

[0247] a determination sample module configured to construct a target image set by using the binary image data and the denoised image data, and normalize the target image set to determine a sample set.

[0248] a size module configured to perform clustering calculation on the sample set by using a preset clustering algorithm, and determine a prior box size, the prior box size being used to assist the model in target positioning during model training.

[0249] a pre-training module configured to pre-train the improved target detection model by using a preset image data set, and obtain a pre-trained model.

[0250] a frozen training module configured to perform local model training on a target feature fusion module and a target detection module of the improved target detection model by using the sample set based on the weights of the pre-trained model and the prior box size, and then perform overall model training on the improved target detection model.

[0251] The improved Mahalanobis distance function is specifically as follows:

[0252]

[0253] In the formula, C is a target covariance matrix; a is a shrinkage coefficient; diag(D) is a new diagonal matrix composed of diagonal elements of the covariance matrix D; μ min is a minimum eigenvalue of the covariance matrix D; μ max is a maximum eigenvalue of the covariance matrix D; D is a covariance matrix; F m,n is a pixel with coordinates (m, n) in a local region centered at pixel coordinates (i, j); F i.j is a pixel with coordinates (i, j); dm(F m,n ,F i.j ) represents the Mahalanobis distance between F(m, n) and F(i, j); and r represents a field radius with (i, j) as the pixel center coordinate.

[0254] Those skilled in the art can clearly understand the specific working process of the above-described apparatuses, modules and sub-modules for the convenience and brevity of description, and the corresponding processes in the foregoing method embodiments can be referred to, which will not be described here.

[0255] In several embodiments provided in the present application, it should be understood that the disclosed apparatuses and methods can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. Taking the division of the units as an example, the division can be changed in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, apparatuses or units, and can be in electrical, mechanical or other forms.

[0256] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0257] The above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of detecting an abnormal temperature of an electric power device, characterized by, The method comprises the following steps: acquiring an infrared image to be identified of a power equipment, and inputting the infrared image to be identified into an improved target detection model, wherein the improved target detection model comprises a target feature extraction network, a target feature fusion network and a target detection network; performing feature strengthening extraction operation on the infrared image to be identified by using the target feature extraction network, and determining a plurality of infrared feature maps, wherein the target feature extraction network comprises a cascaded attention activation convolution module, a multi-branch feature fusion module and a spatial pyramid pooling module; inputting each of the infrared feature maps into the target feature fusion network to perform feature deep fusion operation, and outputting a corresponding fusion feature map; performing temperature detection on each of the fusion feature maps by using the target detection network, and outputting a temperature detection result of the power equipment corresponding to each of the fusion feature maps; the plurality of infrared feature maps comprise a first infrared feature map, a second infrared feature map and a third infrared feature map; the multi-branch feature fusion module comprises a cascaded first branch fusion module and a second branch fusion module; and the attention activation convolution module comprises a spatial-to-depth layer, a non-convolution step layer, an attention mechanism layer, a convolution group and a second branch fusion module; the step of performing feature extraction on the input infrared image to be identified by using the attention activation convolution module and outputting a vector feature fusion map comprises the following steps: performing down-sampling and splicing operation on the infrared image to be identified by using the spatial-to-depth layer to construct a sampling feature map; inputting the sampling feature map into the non-convolution step layer to perform channel number replacement operation and output an intermediate feature map; performing redundancy feature elimination on the intermediate feature map by using the attention mechanism layer to generate a target extraction feature map; performing non-linear mapping on the target extraction feature map by using the convolution group to output a non-linear vector feature map; performing branch fusion operation on the non-linear vector feature map by using the second branch fusion module to determine a vector feature fusion map; the convolution module with FReLU activation function comprises a convolution layer, a batch normalization layer and a FReLU activation function layer; and the specific processing process of the convolution module with FReLU activation function is as follows: performing convolution operation on the convolution input map input into the convolution module with FReLU activation function by using the convolution layer to generate a spatial feature vector map; performing normalization operation on the spatial feature vector map by using the batch normalization layer, and then performing non-linear mapping by using the FReLU activation function layer to output a convolution output map; the FReLU activation function is specifically as follows: ； ； wherein, is a feature vector at a two-dimensional position (i, j) on the c-th feature channel of the convolution input map; is a funnel activation function; is a region centered at a two-dimensional position (i, j) on the c-th feature channel of the convolution input map; is a shared parameter in the c-th feature channel for the region centered at a two-dimensional position (i, j); is a first tensor graph; is a second tensor graph; is a convolution output map; is a concatenation function; is a concatenation dimension; is a two-dimensional spatial condition; the first branch fusion module comprises a convolution module with FReLU activation function, a maximum value pooling layer and a convolution fusion module; and the specific processing process of the first branch fusion module is as follows: performing non-linear mapping and maximum value pooling operation on the fusion input map input into the first branch fusion module by using the convolution module with FReLU activation function and the maximum value pooling layer respectively to generate a corresponding first mapping map and a maximum value pooling map; performing non-linear mapping on the first mapping map and the maximum value pooling map by using the convolution module with FReLU activation function to output a second mapping map and a third mapping map; and The second map and the third map are spliced, and convolution operation and nonlinear mapping operation are performed by a convolution fusion module to determine a fusion output map; The second branch fusion module includes a convolution module with a FReLU activation function and a convolution layer.

2. The power equipment abnormal temperature detection method according to claim 1, characterized by, The step of performing feature enhancement extraction on the to-be-identified infrared image by using the target feature extraction network to determine a plurality of infrared feature maps includes: An attention activation convolution module is used to extract features of the input to-be-identified infrared image to output a vector feature fusion map; A multi-branch feature fusion module is used to perform multi-branch feature fusion on the vector feature fusion map to generate a first infrared feature map; A multi-branch feature fusion module is used to perform multi-branch feature fusion on the first infrared feature map to generate a second infrared feature map; A multi-branch feature fusion module and a spatial pyramid pooling module are cascaded to perform multi-branch feature fusion and pooling operation on the second infrared feature map to output a third infrared feature map.

3. The power equipment abnormal temperature detection method according to claim 2, characterized by, The spatial pyramid pooling module includes a convolution module with a FReLU activation function, a convolution group, a first pooling layer, a second pooling layer, a third pooling layer, a fourth pooling layer, and an attention mechanism layer; the step of performing feature fusion and pooling operation on the second infrared feature map by using the cascaded multi-branch feature fusion module and the spatial pyramid pooling module to output a third infrared feature map includes: The second infrared feature map is input into a multi-branch feature fusion module to perform multi-branch feature fusion operation to determine an infrared fusion feature map; A convolution module with a FReLU activation function and a convolution group are used to respectively perform nonlinear mapping on the infrared fusion feature map to generate corresponding first and second feature maps, and the convolution group includes three convolution modules with FReLU activation functions in cascade; An attention mechanism layer is used to remove redundant features in the first feature map to output a third feature map; The second feature map is input into a first pooling layer, a second pooling layer, a third pooling layer, and a fourth pooling layer to perform pooling operation, and then sequentially passes through an attention mechanism layer to perform redundant feature removal operation to generate a plurality of second sub-feature maps; The plurality of second sub-feature maps are spliced, and then a convolution module with a FReLU activation function is used to perform nonlinear mapping to generate a fourth feature map; The third feature map and the fourth feature map are stacked, and then a convolution module with a FReLU activation function is used to perform nonlinear mapping to output a third infrared feature map.

4. The power equipment abnormal temperature detection method according to claim 2, characterized by, The target feature fusion network includes an attention convolution module, a sampling convolution module, a feature fusion module, and a first branch fusion module; The step of performing feature fusion on each infrared feature map by using the target feature fusion network to generate a corresponding fusion feature map includes: The first infrared feature map and the second infrared feature map are input into an attention convolution module to perform feature extraction to generate a first output feature map and a second output feature map; A sampling convolution module is used to perform feature extraction and up-sampling operation on the third infrared feature map to output a third output feature map; and The second output feature map and the third output feature map are stacked and branch fused by a feature fusion module to determine a fourth output feature map; The fourth output feature map is input into a sampling convolution module to perform feature extraction and up-sampling operations to generate a first initial feature map; The first initial feature map and the first output feature map are stacked and branch fused by a feature fusion module to determine a first fusion feature map; The first fusion feature map is branch fused by a first branch fusion module to obtain a first sub-map, and the first sub-map and the fourth output feature map are input into a feature fusion module to be stacked and branch fused to output a second fusion feature map; The second fusion feature map is branch fused by a first branch fusion module to obtain a second sub-map, and the second sub-map and the third infrared feature map are input into a feature fusion module to be stacked and branch fused to generate a third fusion feature map.

5. The power equipment abnormal temperature detection method according to claim 1, characterized by, The target detection network includes a reparameterization convolution module and a prediction convolution layer; the step of performing temperature detection on each fusion feature map by the target detection network to output the temperature detection result of the power equipment corresponding to each fusion feature map includes: Each fusion feature map is adjusted by a reparameterization convolution module to generate a corresponding adjusted feature map; The adjusted feature map is predicted by a prediction convolution layer to output the temperature detection result of the power equipment.

6. The power equipment abnormal temperature detection method according to claim 1, wherein The training process of the improved target detection model includes: An abnormal temperature infrared image set of the power equipment is obtained, an improved Mahalanobis distance function and a non-local mean filtering algorithm are used to perform denoising operations on the abnormal temperature infrared image set to generate denoised image data, and then edge detail enhancement operations are performed on the denoised image data to output binary image data; The binary image data and the denoised image data are used to construct a target image set, and the target image set is normalized to determine a sample set; A preset clustering algorithm is used to perform clustering calculation on the sample set to determine a prior box size, which is used to assist the model in target positioning during model training; A preset image data set is used to pre-train the improved target detection model to obtain a pre-trained model; Based on the weights of the pre-trained model and the prior box size, the sample set is used to first perform local model training on the target feature fusion module and the target detection module of the improved target detection model, and then perform overall model training on the improved target detection model; The improved Mahalanobis distance function is specifically: ； ； wherein is a target covariance matrix; is a shrinkage coefficient; is a covariance matrix is a new diagonal matrix composed of diagonal elements; is a covariance matrix is the minimum eigenvalue in is a covariance matrix is the maximum eigenvalue in is a covariance matrix; is a pixel with coordinates (m, n) in a local region centered at (i, j); is a pixel with coordinates (i, j); dm denotes the Mahalanobis distance between F(m, n) and F(i, j); r denotes the radius of the field centered at (i, j).

7. An abnormal temperature detection device for electric power equipment, applied to the abnormal temperature detection method for electric power equipment according to claim 1, characterized in that, It includes: An input module is configured to obtain an infrared image to be identified of a power equipment, and input the infrared image to be identified into an improved target detection model, wherein the improved target detection model includes a target feature extraction network, a target feature fusion network, and a target detection network; A feature extraction module is configured to perform feature enhancement extraction operations on the infrared image to be identified by using the target feature extraction network to determine a plurality of infrared feature maps, wherein the target feature extraction network includes a cascaded attention activation convolution module, a multi-branch feature fusion module, and a spatial pyramid pooling module; The feature fusion module is configured to input each of the infrared feature maps into the target feature fusion network for feature deep fusion operation, and output a corresponding fusion feature map. The detection module is configured to perform temperature detection on each of the fusion feature maps by using the target detection network, and output a temperature detection result of the power equipment.

Images