A method for detecting equipment defects

Abstract

The present invention provides a device defect detection method, comprising: obtaining an original infrared image and an original ultraviolet image of a device; preprocessing the original infrared image and the original ultraviolet image to obtain a preprocessed original infrared image and a preprocessed original ultraviolet image; extracting the features of the preprocessed original infrared image and the preprocessed original ultraviolet image respectively to obtain an infrared feature map and an ultraviolet feature map; performing information enhancement and compression on the infrared feature map and the ultraviolet feature map respectively through a double-branch enhancement module to obtain a compressed infrared enhanced feature map and a compressed ultraviolet enhanced feature map; processing the compressed feature map through a unique attribute selection module and a common attribute selection module to obtain a final fused feature map; inputting the final fused feature map into a Mobile‑Aim network to obtain a device defect detection result. The present invention can improve the accuracy of defect recognition results.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method for detecting equipment defects. Background Art

[0002] Since the scope and environment of high-voltage transmission lines are very wide and complex, the transmission lines are often affected by harsh environments (such as swamps, lakes, mountains, etc.) and extreme climates (such as lightning, strong winds, heavy rain, heavy snow, etc.). These factors will cause aging, wear and corrosion of transmission lines and power supply equipment to varying degrees, thus affecting the normal operation of the power grid and causing power outages. Therefore, it is very necessary to conduct regular equipment defect detection and maintenance on power lines and equipment, timely understand the operating status of the lines, prevent power outages, and ensure the reliability of power transmission.

[0003] As a common fault in high-voltage transmission lines, power equipment defects are a harbinger of aging of electrical and mechanical components, and may be accompanied by electrical pulses, ultrasound, electromagnetic radiation, light, chemical reactions, and cause local heating, etc. These characteristics make equipment defects a latent fault that needs to be discovered and eliminated through specific detection methods.

[0004] The existing equipment defect detection methods have low definition of the collected equipment images and lack of effective methods to fuse and enhance image features, resulting in insufficient feature information extracted and low accuracy of defect identification results. Summary of the invention

[0005] The purpose of the present invention is to provide a device defect detection method to improve the accuracy of defect identification results.

[0006] A method for detecting equipment defects comprises the following steps;

[0007] S1, using an infrared thermal imager and an ultraviolet camera to collect the original image of the device, and obtaining the original infrared image and the original ultraviolet image of the device;

[0008] S2, preprocessing the original infrared image and the original ultraviolet image respectively to obtain a preprocessed original infrared image and a preprocessed original ultraviolet image;

[0009] S3, extracting features of the preprocessed original infrared image and features of the preprocessed original ultraviolet image respectively to obtain an infrared feature map and an ultraviolet feature map;

[0010] S4, performing information enhancement and compression on the infrared feature map and the ultraviolet feature map respectively through a dual-branch enhancement module to obtain a compressed infrared enhanced feature map and a compressed ultraviolet enhanced feature map;

[0011] S5, fusing the compressed infrared enhancement feature map and the compressed ultraviolet enhancement feature map through a unique attribute selection module to obtain a fused feature map containing unique attributes, fusing the compressed infrared enhancement feature map and the compressed ultraviolet enhancement feature map through a common attribute selection module to obtain a fused feature map containing common attributes, and then fusing the fused feature map containing unique attributes and the fused feature map containing common attributes to obtain a final fused feature map;

[0012] S6, inputting the final fused feature map into the Mobile-Aim network to obtain the equipment defect detection result, wherein the Mobile-Aim network is based on the MobileSAM network, and an a-Aim module is added between the convolution layer and the fully connected layer, wherein the convolution layer and the a-Aim module are connected with an L2 regularization layer and a residual network, the a-Aim module and the fully connected layer are connected with a Dropout layer and a residual network, and a skip connection is introduced between the convolution layer and the fully connected layer;

[0013] Among them, the dual-branch enhancement module includes a position information enhancement unit and an image information enhancement unit. In the position information enhancement unit, an average pooling operation is performed on the channels of the infrared feature map and the channels of the ultraviolet feature map along the horizontal coordinate direction to obtain an infrared horizontal coordinate direction feature map and an ultraviolet horizontal coordinate direction feature map. An average pooling operation is performed on the channels of the infrared feature map and the channels of the ultraviolet feature map along the vertical coordinate direction to obtain an infrared vertical coordinate direction feature map and an ultraviolet vertical coordinate direction feature map. The image information enhancement unit splices the infrared horizontal coordinate direction feature map and the infrared vertical coordinate direction feature map to obtain an infrared enhanced feature map, and then compresses the infrared enhanced feature map to obtain a compressed infrared enhanced feature map. The image information enhancement unit splices the ultraviolet horizontal coordinate direction feature map and the ultraviolet vertical coordinate direction feature map to obtain an ultraviolet enhanced feature map, and then compresses the ultraviolet enhanced feature map to obtain a compressed ultraviolet enhanced feature map.

[0014] According to the device defect detection method provided by the present invention, an infrared thermal imager and an ultraviolet camera are first used to collect the original image of the device, and the original infrared image and the original ultraviolet image of the device under multiple viewing angles are obtained. After preprocessing and feature extraction, an infrared feature map and an ultraviolet feature map are obtained. Then, a dual-branch enhancement module is introduced to enhance the position information and image information of the infrared feature map and the ultraviolet feature map, so that the position information and image information of the two feature maps are more obvious. Then, compression is performed to obtain a compressed infrared enhanced feature map and a compressed ultraviolet enhanced feature map, which are then fused through a unique attribute selection module and a common attribute selection module to obtain a The fused feature map of unique attributes and the fused feature map of common attributes are further fused to obtain the final fused feature map, which can effectively realize the fusion enhancement of image features and make the extracted features richer, thereby improving the defect recognition accuracy. In addition, the Mobile-Aim network in this application is based on the MobileSAM network, and adds an a-Aim module between the convolutional layer and the fully connected layer. The a-Aim module does not require redundant parameters. On the basis of two dimensions, it pays more attention to three-dimensional information. By calculating the importance of each feature map and dynamically adjusting the weights, the network's representation ability can be improved, thereby further improving the defect recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the process of the equipment defect detection method of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] See also Figure 1 The device defect detection method provided by the embodiment of the present invention includes steps S1 to S6:

[0018] S1, using an infrared thermal imager and an ultraviolet camera to collect the original image of the device, to obtain the original infrared image and the original ultraviolet image of the device.

[0019] Wherein, the infrared thermal imager and the ultraviolet camera are both provided with positioning modules, so that the original infrared image and the original ultraviolet image are both marked with the acquisition area and the acquisition time.

[0020] S2, preprocessing the original infrared image and the original ultraviolet image respectively to obtain a preprocessed original infrared image and a preprocessed original ultraviolet image.

[0021] Wherein, step S2 specifically includes:

[0022] The original infrared image and the original ultraviolet image are cropped respectively, the sizes of the original infrared image and the original ultraviolet image are adjusted to 256×64×64, and the invalid edge areas are removed to reduce the interference of irrelevant factors, so as to obtain the preprocessed original infrared image and the preprocessed original ultraviolet image.

[0023] S3, extracting the features of the preprocessed original infrared image and the preprocessed original ultraviolet image respectively to obtain an infrared feature map and an ultraviolet feature map.

[0024] Wherein, step S3 specifically includes:

[0025] The preprocessed original infrared image and the preprocessed original ultraviolet image are subjected to denoising and contrast enhancement respectively, and finally texture features and shape features are extracted to generate an infrared feature map and an ultraviolet feature map.

[0026] S4, performing information enhancement and compression on the infrared feature map and the ultraviolet feature map respectively through a dual-branch enhancement module to obtain a compressed infrared enhanced feature map and a compressed ultraviolet enhanced feature map.

[0027] Among them, the dual-branch enhancement module includes a position information enhancement unit and an image information enhancement unit. In the position information enhancement unit, an average pooling operation is performed on the channels of the infrared feature map and the channels of the ultraviolet feature map along the horizontal coordinate direction to obtain an infrared horizontal coordinate direction feature map and an ultraviolet horizontal coordinate direction feature map. An average pooling operation is performed on the channels of the infrared feature map and the channels of the ultraviolet feature map along the vertical coordinate direction to obtain an infrared vertical coordinate direction feature map and an ultraviolet vertical coordinate direction feature map. The image information enhancement unit splices the infrared horizontal coordinate direction feature map and the infrared vertical coordinate direction feature map to obtain an infrared enhanced feature map, and then compresses the infrared enhanced feature map to obtain a compressed infrared enhanced feature map. The image information enhancement unit splices the ultraviolet horizontal coordinate direction feature map and the ultraviolet vertical coordinate direction feature map to obtain an ultraviolet enhanced feature map, and then compresses the ultraviolet enhanced feature map to obtain a compressed ultraviolet enhanced feature map.

[0028] Specifically, step S4 includes:

[0029] In the position information enhancement unit, the channels of the infrared feature map are averaged and pooled along the horizontal axis to obtain a size of t 1 ×k 1×1 infrared horizontal coordinate direction feature map, and perform average pooling operation on the channels of the infrared feature map along the vertical coordinate direction to obtain a size of t 1 ×g 1 ×1 infrared ordinate direction feature map, the image information enhancement unit splices the infrared abscissa direction feature map and the infrared ordinate direction feature map to obtain a size of t 1 ×1×(k 1 +g 1 ) infrared enhanced feature map, and then shrunk by a factor of s 1 The infrared enhanced feature map is transformed from t 1 Dimension compression to t 1 / s 1 dimensional, and obtain the compressed infrared enhanced feature map, where t 1 is the input dimension of the infrared feature map, k 1 is the input width of the infrared feature map, g 1 is the input height of the infrared feature map, s 1 is the shrinkage factor of the infrared enhanced feature map;

[0030] At the same time, in the position information enhancement unit, the channels of the ultraviolet feature map are averaged and pooled along the horizontal axis to obtain a size of t 2 ×k 2 ×1 ultraviolet horizontal coordinate direction feature map, and perform average pooling operation on the channels of the ultraviolet feature map along the vertical coordinate direction to obtain a size of t 2 ×g 2 The image information enhancement unit splices the ultraviolet horizontal coordinate feature map and the ultraviolet vertical coordinate feature map to obtain a size of t 2 ×1×(k 2 +g 2 ) UV-enhanced feature map, and then shrunk by a factor of s 2 The UV enhanced feature map is transformed from t 2 Dimension compression to t 2 / s 2 dimension, and obtain the compressed UV enhanced feature map, where t 2 is the input dimension of the UV feature map, k 2 is the input width of the UV feature map, g 2 is the input height of the UV feature map, s 2 is the shrinkage factor of the UV-enhanced feature map.

[0031] S5, fusing the compressed infrared enhancement feature map and the compressed ultraviolet enhancement feature map through a unique attribute selection module to obtain a fused feature map containing unique attributes, fusing the compressed infrared enhancement feature map and the compressed ultraviolet enhancement feature map through a common attribute selection module to obtain a fused feature map containing common attributes, and then fusing the fused feature map containing unique attributes and the fused feature map containing common attributes to obtain a final fused feature map.

[0032] Among them, the compressed infrared enhanced feature map and the compressed ultraviolet enhanced feature map are fused by the unique attribute selection module to achieve feature refinement. Specifically, in the process of obtaining the fused feature map containing unique attributes, the following conditional formula is satisfied:

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] in, represents a fused feature map containing unique attributes, represents the compressed infrared enhanced feature map, represents the compressed UV-enhanced feature map, represents element-wise multiplication, represents the unique attribute feature selection coefficient, represents the Sigmoid activation function, Represents the result of global average pooling calculation, Represents the result of global maximum pooling calculation, represents a 1×1 convolution function, represents the 2×2 convolution function, R represents the ReLU6 activation function, represents the global average pooling function, represents the global maximum pooling function, Indicates unique attribute characteristics;

[0039] By fusing the compressed infrared enhanced feature map and the compressed ultraviolet enhanced feature map through the common attribute selection module, feature refinement can be achieved. Specifically, in the process of obtaining the fused feature map containing the common attributes, the following conditional formula is satisfied:

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] in, Represents a fused feature map containing common attributes, represents the infrared feature selection coefficient, represents the UV characteristic selection coefficient, represents the Softmax function, and Represents two two-layer fully connected networks that share the first layer weights;

[0045] The final fusion feature map satisfies the following conditional formula:

[0046] ;

[0047] in, Represents the final fused feature map.

[0048] S6, input the final fused feature map into the Mobile-Aim network to obtain the equipment defect detection result.

[0049] Specifically, the equipment defect detection result may include the type, location and recognition rate of the equipment defect.

[0050] Among them, the Mobile-Aim network is based on the MobileSAM network, and an a-Aim module is added between the convolution layer and the fully connected layer, wherein the convolution layer and the a-Aim module are connected by an L2 regularization layer and a residual network, and the a-Aim module and the fully connected layer are connected by a Dropout layer and a residual network, and jump connections are introduced in the convolution layer and the fully connected layer, so that the gradient flows directly to the deeper layers of the network. The a-Aim module does not require redundant parameters, and pays more attention to three-dimensional information on the basis of two dimensions. The weights are dynamically adjusted by calculating the importance of each feature map to improve the representation ability of the network.

[0051] Wherein, the a-Aim module satisfies the following conditional formula:

[0052] ;

[0053] ;

[0054] in, represents the image features after being processed by the a-Aim module, Represents the correlation between the target pixel and the surrounding pixels. n Represents the pixel value of the target pixel, is a constant, and Respectively represent the mean and variance of the input image after removing the target pixels.

[0055] According to the device defect detection method provided by the present invention, an infrared thermal imager and an ultraviolet camera are first used to collect the original image of the device, and the original infrared image and the original ultraviolet image of the device under multiple viewing angles are obtained. After preprocessing and feature extraction, an infrared feature map and an ultraviolet feature map are obtained. Then, a dual-branch enhancement module is introduced to enhance the position information and image information of the infrared feature map and the ultraviolet feature map, so that the position information and image information of the two feature maps are more obvious. Then, compression is performed to obtain a compressed infrared enhanced feature map and a compressed ultraviolet enhanced feature map, which are then fused through a unique attribute selection module and a common attribute selection module to obtain a The fused feature map of unique attributes and the fused feature map of common attributes are further fused to obtain the final fused feature map, which can effectively realize the fusion enhancement of image features and make the extracted features richer, thereby improving the defect recognition accuracy. In addition, the Mobile-Aim network in this application is based on the MobileSAM network, and adds an a-Aim module between the convolutional layer and the fully connected layer. The a-Aim module does not require redundant parameters. On the basis of two dimensions, it pays more attention to three-dimensional information. By calculating the importance of each feature map and dynamically adjusting the weights, the network's representation ability can be improved, thereby further improving the defect recognition accuracy.

[0056] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0057] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for detecting equipment defects, characterized in that: The steps include: S1, using an infrared thermal imager and an ultraviolet camera to collect the original image of the device, and obtaining the original infrared image and the original ultraviolet image of the device; S2, preprocessing the original infrared image and the original ultraviolet image respectively to obtain a preprocessed original infrared image and a preprocessed original ultraviolet image; S3, extracting features of the preprocessed original infrared image and features of the preprocessed original ultraviolet image respectively to obtain an infrared feature map and an ultraviolet feature map; S4, performing information enhancement and compression on the infrared feature map and the ultraviolet feature map respectively through a dual-branch enhancement module to obtain a compressed infrared enhanced feature map and a compressed ultraviolet enhanced feature map; S5, fusing the compressed infrared enhancement feature map and the compressed ultraviolet enhancement feature map through a unique attribute selection module to obtain a fused feature map containing unique attributes, fusing the compressed infrared enhancement feature map and the compressed ultraviolet enhancement feature map through a common attribute selection module to obtain a fused feature map containing common attributes, and then fusing the fused feature map containing unique attributes and the fused feature map containing common attributes to obtain a final fused feature map; S6, inputting the final fused feature map into the Mobile-Aim network to obtain the equipment defect detection result, wherein the Mobile-Aim network is based on the MobileSAM network, and an a-Aim module is added between the convolution layer and the fully connected layer, wherein the convolution layer and the a-Aim module are connected with an L2 regularization layer and a residual network, the a-Aim module and the fully connected layer are connected with a Dropout layer and a residual network, and a skip connection is introduced between the convolution layer and the fully connected layer; Wherein, the dual-branch enhancement module includes a position information enhancement unit and an image information enhancement unit, in which the position information enhancement unit performs an average pooling operation on the channels of the infrared feature map and the channels of the ultraviolet feature map along the horizontal coordinate direction to obtain an infrared horizontal coordinate direction feature map and an ultraviolet horizontal coordinate direction feature map, performs an average pooling operation on the channels of the infrared feature map and the channels of the ultraviolet feature map along the vertical coordinate direction to obtain an infrared vertical coordinate direction feature map and an ultraviolet vertical coordinate direction feature map, the image information enhancement unit splices the infrared horizontal coordinate direction feature map and the infrared vertical coordinate direction feature map to obtain an infrared enhanced feature map, and then compresses the infrared enhanced feature map to obtain a compressed infrared enhanced feature map, and the image information enhancement unit splices the ultraviolet horizontal coordinate direction feature map and the ultraviolet vertical coordinate direction feature map to obtain an ultraviolet enhanced feature map, and then compresses the ultraviolet enhanced feature map to obtain a compressed ultraviolet enhanced feature map; Step S4 specifically includes: In the position information enhancement unit, an average pooling operation is performed on the channels of the infrared feature map along the horizontal axis direction to obtain an infrared horizontal axis direction feature map with a size of t1×k1×1, and an average pooling operation is performed on the channels of the infrared feature map along the vertical axis direction to obtain an infrared vertical axis direction feature map with a size of t1×g1×1. The image information enhancement unit splices the infrared horizontal axis direction feature map and the infrared vertical axis direction feature map to obtain an infrared enhanced feature map with a size of t1×1×(k1+g1), and then compresses the infrared enhanced feature map from t1 dimension to t1 / s1 dimension with a contraction multiple s1 to obtain a compressed infrared enhanced feature map, wherein t1 is the input dimension of the infrared feature map, k1 is the input width of the infrared feature map, g1 is the input height of the infrared feature map, and s1 is the contraction multiple of the infrared enhanced feature map; At the same time, in the position information enhancement unit, an average pooling operation is performed on the channels of the ultraviolet feature map along the horizontal axis direction to obtain an ultraviolet horizontal axis feature map with a size of t2×k2×1, and an average pooling operation is performed on the channels of the ultraviolet feature map along the vertical axis direction to obtain an ultraviolet vertical axis feature map with a size of t2×g2×1. The image information enhancement unit splices the ultraviolet horizontal axis feature map and the ultraviolet vertical axis feature map to obtain an ultraviolet enhanced feature map with a size of t2×1×(k2+g2), and then compresses the ultraviolet enhanced feature map from t2 dimension to t2 / s2 dimension with a contraction multiple s2 to obtain a compressed ultraviolet enhanced feature map, wherein t2 is the input dimension of the ultraviolet feature map, k2 is the input width of the ultraviolet feature map, g2 is the input height of the ultraviolet feature map, and s2 is the contraction multiple of the ultraviolet enhanced feature map; The compressed infrared enhanced feature map and the compressed ultraviolet enhanced feature map are fused by the unique attribute selection module to obtain a fused feature map containing unique attributes, and the following conditional formula is satisfied: ； ； ； ； ； in, represents a fused feature map containing unique attributes, represents the compressed infrared enhanced feature map, represents the compressed UV-enhanced feature map, represents element-wise multiplication, represents the unique attribute feature selection coefficient, represents the Sigmoid activation function, Represents the result of global average pooling calculation, Represents the result of global maximum pooling calculation, represents a 1×1 convolution function, represents the 2×2 convolution function, R represents the ReLU6 activation function, represents the global average pooling function, represents the global maximum pooling function, Indicates unique attribute characteristics; The compressed infrared enhanced feature map and the compressed ultraviolet enhanced feature map are fused by the common attribute selection module to obtain a fused feature map containing common attributes, and the following conditional formula is satisfied: ； ； ； ； in, Represents a fused feature map containing common attributes, represents the infrared feature selection coefficient, represents the UV characteristic selection coefficient, represents the Softmax function, and Represents two two-layer fully connected networks that share the first layer weights; The final fusion feature map satisfies the following conditional formula: ； in, Represents the final fused feature map; The a-Aim module satisfies the following conditional formula: ； ； in, represents the image features after being processed by the a-Aim module, Represents the correlation between the target pixel and the surrounding pixels. n Represents the pixel value of the target pixel, is a constant, and Respectively represent the mean and variance of the input image after removing the target pixels.

2. The device defect detection method according to claim 1, characterized in that: The infrared thermal imager and the ultraviolet camera are both provided with positioning modules, so that the original infrared image and the original ultraviolet image are both marked with the acquisition area and the acquisition time.

3. The device defect detection method according to claim 1, characterized in that: Step S2 specifically includes: The original infrared image and the original ultraviolet image are cropped respectively, the sizes of the original infrared image and the original ultraviolet image are adjusted to 256×64×64, and the invalid edge areas are removed to obtain the preprocessed original infrared image and the preprocessed original ultraviolet image.

4. The device defect detection method according to claim 1, characterized in that: Step S3 specifically includes: The preprocessed original infrared image and the preprocessed original ultraviolet image are subjected to denoising and contrast enhancement respectively, and finally texture features and shape features are extracted to generate an infrared feature map and an ultraviolet feature map.

Images