Glass insulator dirt identification method and device, computer equipment and storage medium

Abstract

The invention relates to a glass insulator dirt identification method and device, computer equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: obtaining a to-be-identified picture of each glass insulator shot by an unmanned aerial vehicle; determining a multi-scale feature map of the to-be-recognized picture based on a pre-constructed backbone network; the backbone network is determined based on a pixel focusing attention mechanism and a network model of the multi-head self-attention network; the pixel focusing attention mechanism is used for determining the local features of the pixels according to the correlation degree between the pixels and determining the context information of the pixels; inputting the multi-scale feature map into a feature fusion network, and performing feature fusion on the multi-scale feature map through a feature pyramid network FPN and a path aggregation network PAN in the feature fusion network to obtain a fused feature map; and inputting the fused feature map into a detection head to obtain a pollution identification result corresponding to the to-be-identified picture. By adopting the method, the glass insulator pollution identification accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment, storage medium and computer program product for identifying contamination of glass insulators. Background Art

[0002] Glass insulators in transmission lines are one of the important components of the power grid in the process of transporting electricity. Their main function is to support the transmission wires, isolate the current, and protect the power components on the transmission lines. Since the glass insulators on the transmission lines are in an open-air environment all day long, they will always accumulate dirt after a period of time. When dirt and dust accumulate on the surface of the insulator, a conductive layer will be formed, which will reduce the insulation resistance, thereby increasing the risk of line leakage and breakdown, posing a threat to electrical equipment and personnel safety. When the surface of the insulator is seriously polluted, its flashover voltage will drop significantly, resulting in the insulator being prone to flashover and breakdown at the same operating voltage, which will lead to line failures and equipment damage.

[0003] In the related technology, the current defect detection of glass insulators generally uses the yolo series target detection algorithm based on convolutional neural networks for direct detection. However, the convolution structure of the convolutional neural network has advantages only in local short-distance modeling, and is weak in global long-distance modeling capabilities, and its receptive field is small. Glass insulator contamination often shows different appearances due to different light and environment. The convolutional neural network has insufficient receptive field, resulting in insufficient accuracy in detecting glass insulator contamination. In addition, the difference between glass insulator contamination and normal glass insulators is mainly the difference in spectral color, and spectral color belongs to the shallow features in the neural network. The yolo series convolutional neural network only uses the features of the last three stages of resolution in the feature fusion stage, resulting in insufficient utilization of shallow features and insufficient accuracy in identifying glass insulator contamination. Summary of the invention

[0004] Based on this, it is necessary to provide a glass insulator contamination identification method, device, computer equipment, computer readable storage medium and computer program product that can improve the accuracy of glass insulator contamination identification in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for identifying contamination of a glass insulator. The method comprises:

[0006] Obtain the images of each glass insulator to be identified taken by the drone;

[0007] Based on a pre-built backbone network, a multi-scale feature map of the image to be identified is determined; the backbone network is determined based on a network model of a pixel-focused attention mechanism and a multi-head self-attention network; the pixel-focused attention mechanism is used to determine the local features of each pixel according to the degree of association between the pixels, and to determine the context information of each pixel; the multi-head self-attention network is used to determine a triple of a linear transformation of the image, and based on the attention heads corresponding to the query, key and value contained in the triple, independently calculate the attention weight corresponding to each pixel; the multi-scale feature map is a feature map obtained by calculating the attention weight corresponding to each pixel, and contains the local features and context information of each pixel; the multi-scale feature map includes multiple feature maps of different resolutions;

[0008] Inputting the multi-scale feature map into a feature fusion network, performing feature fusion on the multi-scale feature map through a feature pyramid network FPN and a path aggregation network PAN in the feature fusion network to obtain a fused feature map;

[0009] The fused feature map is input into a detection head to obtain a contamination recognition result corresponding to the image to be recognized; the detection head is a classification network composed of multiple convolutional layers; the contamination recognition result includes an output image with glass insulators marked, and the contamination category of each glass insulator in the output image; the contamination category includes contamination and non-contamination.

[0010] In one embodiment, the multi-scale feature map includes a plurality of target feature maps; and determining the multi-scale feature map of the image to be identified based on a pre-built backbone network includes:

[0011] Based on the patch embedding layer of the backbone network, the image to be identified is segmented to obtain feature vectors corresponding to a plurality of image blocks, and the feature vectors are determined as the initial feature map corresponding to the image to be identified;

[0012] The initial feature map is subjected to multiple downsampling processes to obtain multiple target feature maps with different resolutions; the downsampling process is determined by weighted summing of feature vectors of each pixel after convolution based on the pixel focus attention mechanism;

[0013] For the target feature map with the lowest resolution, weighted summation is performed on the triplets of each pixel after convolution through the multi-head self-attention network to obtain an updated target feature map with the lowest resolution;

[0014] It is determined that the updated target feature map with the lowest resolution and other target feature maps with different resolutions are multi-scale feature maps.

[0015] In one embodiment, the feature fusion of the multi-scale feature map is performed through the feature pyramid network FPN and the path aggregation network PAN in the feature fusion network to obtain a fused feature map, including:

[0016] The multi-scale feature map is input into the FPN in the feature fusion network, and the feature maps of different resolutions in the multi-scale feature map are up-sampled by the FPN to obtain the initial fused feature map corresponding to the image to be identified; the FPN is used to transfer the semantic information of the high-resolution feature map in the multi-scale feature map to the low-resolution feature map;

[0017] The initial fused feature map is input into the PAN in the feature fusion network, and the feature maps of different resolutions in the initial fused feature map are downsampled by the PAN to obtain a fused feature map composed of multiple feature maps of different resolutions; the PAN is used to transfer the position information of the low-resolution feature map in the initial fused feature map to the corresponding high-resolution feature map.

[0018] In one embodiment, the step of inputting the fused feature map into a detection head to obtain a contamination recognition result corresponding to the image to be recognized includes:

[0019] Inputting the fused feature map into multiple convolutional layers of a trained detection head to obtain an output result of the detection head; the output result includes bounding box parameters, category probability and confidence of each glass insulator in the image to be identified;

[0020] For each of the glass insulators, if the confidence corresponding to the current glass insulator is greater than the first threshold and the category probability corresponding to the current glass insulator is greater than the second threshold, then the contamination category of the current glass insulator is determined to be contaminated;

[0021] Marking the bounding box parameters and the pollution category corresponding to the current glass insulator in the to-be-recognized image to obtain a marked image, and storing the pollution category of the current glass insulator to obtain a stored pollution category;

[0022] After the traversal of each glass insulator is completed, the marked image is determined to be the output image, and the stored contamination category is determined to be the contamination category of each glass insulator in the output image.

[0023] In one embodiment, the method further comprises:

[0024] Determine the backbone network, the feature fusion network and the detection head as the recognition model to be trained, and obtain a training sample set of the recognition model to be trained; the training sample set includes pictures containing a contamination category marked with glass insulators;

[0025] Inputting the training sample set into the recognition model to be trained to obtain a loss value of the recognition model to be trained; the loss value is determined based on the actual result and the predicted result of the training sample set;

[0026] The model parameters in the recognition model to be trained are adjusted to obtain an updated recognition model to be trained, and the step of inputting the training sample set into the recognition model to be trained to obtain the loss value of the recognition model to be trained is returned to execute until the loss value meets the training end condition, thereby obtaining a trained backbone network, a feature fusion network and a detection head.

[0027] In one embodiment, the training sample set includes a plurality of insulator pictures with position information and pollution categories of glass insulators marked; the step of obtaining the training sample set of the recognition model to be trained includes:

[0028] The initial insulator images corresponding to the glass insulators of each tower are taken by drone;

[0029] Performing image preprocessing on each of the initial insulator images to obtain a preprocessed insulator image; the image preprocessing includes rotation, cropping and enhancement processing;

[0030] The position of each glass insulator in the preprocessed insulator image and the contamination category are marked to obtain a marked insulator image, and a plurality of marked insulator images are determined as a training sample set.

[0031] In a second aspect, the present application also provides a glass insulator contamination identification device. The device comprises:

[0032] An image acquisition module is used to acquire the images to be identified of each glass insulator taken by the drone;

[0033] A feature determination module is used to determine the multi-scale feature map of the image to be identified based on a pre-built backbone network; the backbone network is determined based on a network model of a pixel-focused attention mechanism and a multi-head self-attention network; the pixel-focused attention mechanism is used to determine the local features of each pixel according to the degree of association between the pixels, and to determine the context information of each pixel; the multi-head self-attention network is used to determine the triple of the linear transformation of the image, and independently calculate the attention weight corresponding to each pixel based on the attention heads corresponding to the query, key and value contained in the triple; the multi-scale feature map is a feature map obtained by calculating the attention weight corresponding to each pixel, and contains the local features and context information of each pixel; the multi-scale feature map includes multiple feature maps of different resolutions;

[0034] A feature fusion module is used to input the multi-scale feature map into a feature fusion network, and perform feature fusion on the multi-scale feature map through a feature pyramid network FPN and a path aggregation network PAN in the feature fusion network to obtain a fused feature map;

[0035] The fused feature map is input into a detection head to obtain a contamination recognition result corresponding to the image to be recognized; the detection head is a classification network composed of multiple convolutional layers; the contamination recognition result includes an output image with glass insulators marked, and the contamination category of each glass insulator in the output image; the contamination category includes contamination and non-contamination.

[0036] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0038] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0039] The above-mentioned glass insulator contamination identification method, device, computer equipment, storage medium and computer program product, by inputting the image to be identified taken by the drone into the pre-built backbone network, the backbone network can determine the local features of each pixel in the image to be identified through the pixel focus attention mechanism and the multi-head self-attention network, and obtain the feature map determined by the local features, thereby obtaining multiple feature maps of different resolutions corresponding to the image to be identified, and determining them as multi-scale feature maps. The multi-scale feature map can represent the shallow features and deep features of the image to be identified in different feature maps. Based on this, the server can input the multi-scale feature map into the feature fusion network, and perform feature fusion on the multi-scale feature map through the feature pyramid network FPN and the path aggregation network PAN in the feature fusion network to obtain a fused feature map. The fused feature map can enhance the shallow features in the multi-scale feature map. Finally, the fused feature map is input into the detection head, and the monitoring head can identify and classify the glass insulator contained in the image to be identified according to the fused feature map, determine the position and category of the glass insulator, and output an output image with the contamination category of the glass insulator marked. Since the multi-head self-attention network has the ability to establish long distances, it can use the super-large receptive field enhancement algorithm to perceive the surrounding environment, and can more accurately identify glass insulator contamination. In addition, the process of fusion feature maps can enhance the shallow features of glass insulator contamination and normal glass insulators, better identify glass insulator contamination with different spectral colors, and thus improve the accuracy of glass insulator contamination identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A diagram showing an application environment of a method for identifying contamination of a glass insulator in an embodiment;

[0042] Figure 2 A schematic diagram of a flow chart of a method for identifying contamination of a glass insulator in one embodiment;

[0043] Figure 3 A schematic diagram of a flow chart of a method for identifying contamination of a glass insulator in another embodiment;

[0044] Figure 4 A schematic diagram of a backbone network model in one embodiment;

[0045] Figure 5A schematic diagram of a model of a backbone network and a feature fusion network in one embodiment;

[0046] Figure 6 A schematic diagram of a model of a backbone network and a feature fusion network in another embodiment;

[0047] Figure 7 is a structural block diagram of a glass insulator contamination identification device in one embodiment;

[0048] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The glass insulator contamination identification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The terminal 102 can receive the picture to be identified taken by the drone or the shooting device, and the terminal 102 can also take the picture to be identified by itself. The terminal 102 sends the picture to be identified to the server 104. The server 104 can determine the multi-scale feature map of the picture to be identified through the pre-built backbone network. And the multi-scale feature map is input into the feature fusion network, and the multi-scale feature map is feature fused through the feature pyramid network FPN (Feature Pyramid Networks) and the path aggregation network PAN (Path Aggregation Network) in the feature fusion network to obtain a fused feature map. The server 104 can input the fused feature map into the detection head to obtain the dirt recognition result corresponding to the picture to be identified. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0051] In an exemplary embodiment, Figure 2 As shown, a method for identifying contamination of glass insulators is provided, and the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S202 to S208. Among them:

[0052] Step S202, obtaining the images of each glass insulator to be identified taken by the drone.

[0053] Among them, glass insulators are devices used to support and insulate conductors. In power systems, glass insulators are widely used in high-voltage and ultra-high-voltage AC and DC transmission lines to insulate and suspend conductors.

[0054] Specifically, the server may receive photos of glass insulators on each tower taken by a terminal or a drone. The terminal or drone may take photos of at least one glass insulator included on multiple towers according to predetermined shooting parameters such as shooting time, shooting height, shooting angle, etc. All the photos taken may be determined as images to be identified.

[0055] Step S204: determining a multi-scale feature map of the image to be identified based on the pre-built backbone network.

[0056] Among them, the backbone network is determined by the network model based on the pixel-focused attention mechanism and the multi-headed self-attention network; the pixel-focused attention mechanism is used to determine the local features of each pixel according to the degree of correlation between the pixels, and determine the context information of each pixel; the multi-headed self-attention network is used to determine the triple of the linear transformation of the image, and based on the attention heads corresponding to the query, key and value contained in the triple, independently calculate the attention weight corresponding to each pixel; the multi-scale feature map is a feature map obtained by calculating the attention weights corresponding to each pixel, which contains the local features and context information of each pixel; the multi-scale feature map includes multiple feature maps of different resolutions.

[0057] Specifically, the server can pre-build a network model including a pixel-focused attention mechanism and a multi-head self-attention network. The pixel-focused attention mechanism can integrate local and global information. For example, the pixel-focused attention mechanism can map the input feature map into multiple representations by pre-processing it through convolutional layers, linear layers, etc., decompose the feature map into local features and global features, and generate interactions between local features and global features, thereby enhancing the contextual information of local pixels. In the interaction of multiple local pixels, the pixel-focused attention mechanism can use a soft attention mechanism to calculate the importance of each pixel to other pixels, and use the importance as a weight, which is used to weight local features to emphasize important areas in the image.

[0058] The multi-head self-attention network can divide the image into several blocks, each of which may be flattened into a one-dimensional vector. At this time, the input of the entire image consists of these blocks. The multi-head self-attention network can generate three different attention heads (triplets) through linear transformation of the blocks: query, key, and value. The multi-head self-attention network can determine the attention weights of each pixel in different attention heads. Finally, the outputs corresponding to each attention head are spliced ​​to obtain the feature map corresponding to the image to be recognized.

[0059] In one example, the backbone network can downsample the image to be identified, and obtain a feature map with a lower resolution each time it is sampled, thereby obtaining multiple feature maps with different resolutions according to the number of sampling times, and determining them as multi-scale feature maps.

[0060] Step S206, input the multi-scale feature map into the feature fusion network, and perform feature fusion on the multi-scale feature map through the feature pyramid network FPN and the path aggregation network PAN in the feature fusion network to obtain a fused feature map.

[0061] Among them, the feature fusion network includes the feature pyramid network FPN and the path aggregation network PAN. FPN is mainly used to improve the performance of glass insulator detection and segmentation. FPN adopts a top-down feature fusion method. By constructing multi-scale feature maps, the network can simultaneously utilize high-level semantic features and low-level detail features. PAN is a network structure used to enhance semantic segmentation and target detection. PAN aggregates multiple layers of feature maps through bottom-up and top-down data flows, thereby fusing multiple layers of feature maps. The combination of PAN and FPN can further improve the representation ability of features.

[0062] Specifically, the server inputs the multi-scale feature map of the image to be identified into FPN, which can upsample the multi-scale feature map to obtain a processed multi-scale feature map. After that, the processed multi-scale feature map is downsampled by PAN to obtain a fused feature map. The fused feature map contains high-level semantic features, low-level detail features, and location information of the glass insulator.

[0063] Step S208, input the fused feature map to the detection head to obtain the contamination recognition result corresponding to the image to be recognized.

[0064] Among them, the detection head is a classification network composed of multiple convolutional layers; the contamination recognition result includes an output image of the glass insulators marked, and the contamination category of each glass insulator in the output image; the contamination category includes contamination and non-contamination.

[0065] Specifically, the server can input the fused feature map to the detection head, and the detection head can process the fused feature map through multiple convolutional layers to obtain the position information of the normal glass insulators or the contaminated glass insulators identified in the image to be processed, as well as the contamination category of each glass insulator. The server can annotate the original image to be identified according to the contamination category and position information to obtain the annotated output image, thereby determining that the output image and the contamination category of each glass insulator are the contamination identification results corresponding to the image to be identified.

[0066] In the above-mentioned glass insulator contamination identification method, by inputting the image to be identified taken by the drone into the pre-built backbone network, the backbone network can determine the local features of each pixel in the image to be identified through the pixel focus attention mechanism and the multi-head self-attention network, and obtain the feature map determined by the local features, thereby obtaining multiple feature maps of different resolutions corresponding to the image to be identified, and determining them as multi-scale feature maps. The multi-scale feature map can represent the shallow features and deep features of the image to be identified in different feature maps. Based on this, the server can input the multi-scale feature map into the feature fusion network, and perform feature fusion on the multi-scale feature map through the feature pyramid network FPN and the path aggregation network PAN in the feature fusion network to obtain a fused feature map. The fused feature map can enhance the shallow features in the multi-scale feature map. Finally, the fused feature map is input into the detection head, and the monitoring head can identify and classify the glass insulator contained in the image to be identified according to the fused feature map, determine the position and category of the glass insulator, and output an output image with the contamination category of the glass insulator marked. Since the multi-head self-attention network has the ability to establish long distances, it can use the super-large receptive field enhancement algorithm to perceive the surrounding environment, and can more accurately identify glass insulator contamination. In addition, the process of fusion feature maps can enhance the shallow features of glass insulator contamination and normal glass insulators, better identify glass insulator contamination with different spectral colors, and thus improve the accuracy of glass insulator contamination identification.

[0067] In an exemplary embodiment, the multi-scale feature map includes multiple target feature maps, and the specific implementation process of the step of "determining the multi-scale feature map of the image to be identified based on the pre-built backbone network" includes:

[0068] Based on the patch embedding layer of the backbone network, the image to be identified is segmented to obtain feature vectors corresponding to multiple blocks, and determined as the initial feature map corresponding to the image to be identified; the initial feature map is downsampled multiple times to obtain multiple target feature maps with different resolutions; the downsampling process is determined by weighted summing up the feature vectors of each pixel after convolution based on the pixel focus attention mechanism; for the target feature map with the lowest resolution, the weighted summation of the triplets of each pixel after convolution is performed through a multi-head self-attention network to obtain the updated target feature map with the lowest resolution; the updated target feature map with the lowest resolution and other target feature maps with different resolutions are determined to be multi-scale feature maps.

[0069] The patch embedding layer is used to segment the image to be identified, obtain multiple patches, and map each patch to a one-dimensional vector through the fully connected layer to obtain the feature vector of the image to be identified. For example, the processing of the patch embedding layer can be represented by Patch Embedding. Downsampling is used to reduce the spatial size of the initial feature map.

[0070] Specifically, the server can input the image to be identified into the patch embedding layer, segment the image to be identified, obtain multiple segments (i.e., patches), and determine the one-dimensional vector of each segment, thereby obtaining an initial feature map composed of the one-dimensional vectors of each segment. The server can downsample the initial feature map through the focused attention layer and the convolution GLU (Gated Linear Unit) layer in the backbone network to obtain the target feature map of the first downsampling. Based on this, the server can iterate the above process, for example, further segment the target feature map of the first downsampling through the patch embedding layer to obtain the segmented initial feature map, and downsample the segmented initial feature map through the focused attention layer and the convolution GLU layer in the backbone network to obtain the target feature map of the second downsampling. The resolution of the target feature map of the second downsampling is less than the resolution of the target feature map of the first downsampling. Accordingly, the server can iterate the downsampling behavior multiple times to obtain multiple target feature maps of different resolutions. During the downsampling process, the server can use the pixel-focused attention mechanism to perform weighted summation on the feature vectors of each pixel after convolution in the feature map, and add the features of the previous sampling to the target feature map after the next sampling.

[0071] In addition, during the last downsampling process, the server can first segment the target feature map obtained by the last sampling through the patch embedding layer to obtain the initial feature map after segmentation. The server performs the last downsampling process on the initial feature map after segmentation through the multi-head self-attention layer and the convolution GLU layer in the backbone network to obtain the target feature map for the last downsampling. This target feature map is the target feature map with the lowest resolution. During the generation process of the target feature map with the lowest resolution, it is determined by weighted summing of the triplets of each pixel after convolution through the multi-head self-attention network. At this time, there are target feature maps corresponding to the number of downsampling times for the picture to be identified, and the features corresponding to the multiple pictures to be identified are removed to determine a multi-scale feature map. In other words, the target feature map with the lowest resolution and multiple target feature maps with different resolutions are determined to be multi-scale feature maps.

[0072] In this embodiment, the resolution of the downsampled feature map can be reduced through block segmentation through multiple iterative downsampling processes. The pixel focus attention mechanism and the multi-head self-attention network in the downsampling process can add features to each downsampled feature map, thereby enhancing the feature information of each target feature map, and then increasing the feature information of different depths in the multi-scale feature map, which can improve the feature information and feature quantity of the multi-scale feature map.

[0073] In an exemplary embodiment, the specific implementation process of the step of "performing feature fusion on the multi-scale feature map through the feature pyramid network FPN and the path aggregation network PAN in the feature fusion network to obtain a fused feature map" includes:

[0074] First, the multi-scale feature map is input into the FPN in the feature fusion network. The feature maps of different resolutions in the multi-scale feature map are upsampled by FPN to obtain the initial fused feature map corresponding to the image to be identified.

[0075] Among them, FPN is used to transfer the semantic information of the high-resolution feature map in the multi-scale feature map to the low-resolution feature map.

[0076] Specifically, the server inputs the multi-scale feature map into the FPN in the feature fusion network. Since there is a feature pyramid in the FPN, the feature pyramid can receive target feature maps of different resolutions in the multi-scale feature map. The server starts with the target feature map with a small resolution according to the resolution size of the target feature map in the feature pyramid, and adjusts the resolution of the target feature map to be the same as the resolution of the target feature map of the previous layer through upsampling processing (such as nearest neighbor interpolation or bilinear interpolation method). The server performs feature fusion on the upsampled target feature map with the target feature map of the previous layer to obtain the fused feature map of the previous layer. Based on this, the server can iterate, for example, upsample the fused feature map of the previous layer so that the resolution of the fused feature map of the previous layer is the same as the resolution of the target feature map of the previous layer, and perform feature fusion on the fused feature map of the previous layer with the target feature map of the previous layer. Through multiple upsampling iterations, feature fusion can be completed for the multi-scale feature map to obtain the initial fused feature map corresponding to the image to be identified. The initial fused feature map may include multiple feature maps that have been fused with FPN features.

[0077] In an example, if there are four target feature maps at different levels of C2, C3, C4, and C5 in the multi-scale feature map. C2 represents a lower-level feature, that is, high resolution and contains rich detail information; C5 represents a higher-level feature, that is, low resolution and contains more semantic information. At this time, the server can start from the highest-level feature C5, and C5 adjusts the spatial dimension through an upsampling so that C5 matches the feature map of the lower layer (such as C4, C3, etc.). The upsampled feature map will be fused with the feature map of the corresponding layer below, usually using the element-wise sum method. For C4: add the upsampled C5 feature map to the C4 feature map to obtain the fused feature F4. For C3: add the upsampled F4 feature to the C3 feature to obtain the fused feature F3. For C2: combine C2 and the upsampled F3 to generate the final F2.

[0078] Finally, the initial fused feature map is input into the PAN in the feature fusion network, and the feature maps of different resolutions in the initial fused feature map are downsampled by PAN to obtain a fused feature map composed of multiple feature maps with different resolutions.

[0079] Among them, PAN is used to transfer the position information of the low-resolution feature map in the initial fusion feature map to the corresponding high-resolution feature map. PAN adds a feature pyramid in the opposite direction after FPN to supplement FPN and transfer high-resolution feature information.

[0080] Specifically, the server inputs the initial fused feature map into the PAN in the feature fusion network. The PAN receives feature maps of different resolutions in the initial fused feature map through the internal feature pyramid. At this time, the server starts from the feature map with a large resolution according to the resolution size of the feature map in the feature pyramid, and adjusts the resolution of the feature map to be the same as the resolution of the feature map of the next layer through downsampling. The server performs feature fusion on the downsampled feature map with the target feature map of the next layer to obtain a fused feature map of the next layer. Based on this, the server can iterate, for example, downsample the fused feature map of the next layer so that the resolution of the fused feature map of the next layer is the same as the resolution of the feature map of the next layer, and perform feature fusion on the fused feature map of the next layer with the feature map of the next layer. Through multiple downsampling iterations, the feature fusion of the initial fused feature map can be completed to obtain a fused feature map corresponding to the image to be identified.

[0081] In this embodiment, the multi-scale feature map can be subjected to two rounds of feature fusion through the feature pyramid of FPN+PAN. The first round of fusion is to upsample the multi-scale feature map through FPN, and the second round of fusion is to downsample the upsampled initial fused feature map through PAN to obtain a fused feature map, so that deep semantic information can be transferred to shallow features, and fine semantic information can be transferred to deep features, thereby improving the semantic information amount and semantic richness of the fused feature map.

[0082] In an exemplary embodiment, the specific implementation process of the step of "inputting the fused feature map into the detection head to obtain the contamination recognition result corresponding to the image to be recognized" includes:

[0083] The fused feature map is input into multiple convolutional layers of the trained detection head to obtain the output result of the detection head. For each glass insulator, if the confidence corresponding to the current glass insulator is greater than the first threshold and the category probability corresponding to the current glass insulator is greater than the second threshold, the contamination category of the current glass insulator is determined to be contaminated; the bounding box parameters and contamination category corresponding to the current glass insulator are marked in the image to be identified to obtain a marked image, and the contamination category of the current glass insulator is stored to obtain a stored contamination category; after completing the traversal of each glass insulator, the marked image is determined to be the output image, and the stored contamination category is determined to be the contamination category of each glass insulator in the output image.

[0084] The output results include the bounding box parameters, category probability and confidence of each glass insulator in the image to be identified. The bounding box parameters can be the center point coordinates and width, height or upper left corner and lower right corner coordinates. The category probability is the probability value of the contamination category of the glass insulator being contaminated or not contaminated. The confidence is the possibility of the existence of a glass insulator in each bounding box.

[0085] Specifically, the server can input the fused feature map into multiple convolutional layers of the trained detection head. The detection head can process the fused feature map to obtain the bounding box parameters, class probability and confidence of each glass insulator detected in the fused feature map. For example, the server can determine two glass insulators, namely insulator A and insulator B, through the detection head, and obtain the bounding box parameters, class probability and confidence of insulator A. And the bounding box parameters, class probability and confidence of insulator B. For insulator A, if the confidence corresponding to insulator A is greater than the first threshold and the class probability corresponding to insulator A is greater than the second threshold, then the contamination category of the current glass insulator is determined to be contaminated. If the confidence corresponding to insulator A is greater than the first threshold and the class probability corresponding to insulator A is less than or equal to the second threshold, then the contamination category of the current glass insulator is determined to be non-contaminated. If the confidence corresponding to insulator A is less than the first threshold, then insulator A is determined to be not an insulator, and insulator A is not treated as an insulator for subsequent processing.

[0086] Afterwards, if it is determined that both insulator A and insulator B are insulators, the server can mark the bounding box parameters and contamination category corresponding to insulator A in the image to be identified, and mark the bounding box parameters and contamination category corresponding to insulator B in the image to be identified, to obtain a marked image. The contamination categories of insulators A and B are stored in a pre-established storage structure to obtain a stored contamination category. At this point, the marked image is determined to be the output image, and the stored contamination category is determined to be the contamination category of each glass insulator in the output image.

[0087] In this embodiment, the bounding box parameters, category probability and confidence of the glass insulator can be obtained through the detection head, and the contamination category of the glass insulator can be determined through the category probability and confidence, so that the glass insulator can be marked in the image to be identified and the contamination identification result can be obtained, which can improve the accuracy of the contamination identification result.

[0088] In an exemplary embodiment, the method further comprises:

[0089] Determine the backbone network, feature fusion network and detection head as the recognition model to be trained, and obtain the training sample set of the recognition model to be trained; input the training sample set into the recognition model to be trained to obtain the loss value of the recognition model to be trained; adjust the model parameters in the recognition model to be trained to obtain the updated recognition model to be trained, return to execute the step of inputting the training sample set into the recognition model to be trained to obtain the loss value of the recognition model to be trained, until the loss value meets the training end condition, and obtain the trained backbone network, feature fusion network and detection head.

[0090] The training sample set includes pictures of a contamination category marked with glass insulators; and the loss value is determined based on the actual results and the predicted results of the training sample set.

[0091] Specifically, the server can determine the model composed of the backbone network, the feature fusion network and the detection head as the recognition model to be trained. The server can obtain annotated pictures containing glass insulators in multiple scenes through drone photography and use them as training sample sets. The annotated pictures contain the location information and pollution category of the glass insulator. The scene can be a scene where glass insulators of different models work under different weather conditions at different times and locations. The server can input the training sample set into the recognition model to be trained to obtain the predicted value corresponding to each annotated picture in the training sample set. The server determines the loss value between the predicted value and the location information and the pollution category through the loss function. The server updates the parameters of the backbone network, the feature fusion network and the detection head in the recognition model to be trained with the goal of optimizing the loss value to the minimum value to obtain the trained recognition model to be trained. Then, the training sample set is input into the trained recognition model to be trained through iteration to determine the loss value until the loss value meets the training end condition. At this time, the recognition model to be trained is a trained recognition model. If the loss value does not meet the training end condition, the parameters in the recognition model to be trained are continued to be adjusted, and the training sample set is input for iterative training.

[0092] In this embodiment, the backbone network, feature fusion network and detection head can be iteratively trained through the training sample set of glass insulators. When the loss value meets the training end condition, the trained backbone network, feature fusion network and detection head can be obtained, which can improve the training efficiency and the recognition accuracy of the trained backbone network, feature fusion network and detection head.

[0093] In an exemplary embodiment, the training sample set includes multiple insulator pictures with the position information and contamination categories of the glass insulators marked. The specific implementation process of the step "obtaining the training sample set of the recognition model to be trained" includes:

[0094] Initial insulator images corresponding to the glass insulators of each tower are taken based on drones; each initial insulator image is preprocessed to obtain a preprocessed insulator image; the image preprocessing includes rotation, cropping and enhancement processing; the position of each glass insulator in the preprocessed insulator image and the contamination category are marked to obtain a marked insulator image, and multiple marked insulator images are determined as a training sample set.

[0095] Specifically, the drone can photograph the glass insulators on each pole tower in various scenarios to obtain an initial insulator picture. The server can receive the initial insulator picture and rotate, crop and perform image enhancement processing on the initial insulator picture. Insulator pictures with consistent specifications can be obtained by rotation and cropping. For example, the specifications can be that the insulators in the insulator picture are oriented in the same direction, the insulator pictures are the same size, etc. Image enhancement can perform operations such as decolorizing the image and increasing the color saturation, which are not limited in this application. The server can mark the location of each glass insulator in the insulator picture and the contamination category according to the instructions sent by the user side to obtain a marked insulator picture. The server can determine multiple marked insulator pictures as a training sample set.

[0096] In this embodiment, the initial insulator image is obtained by using a drone, and the initial insulator image is preprocessed and labeled, so as to obtain a high-quality training sample set and improve the completeness and richness of the training sample set.

[0097] like Figure 3 As shown, the specific execution process of the above-mentioned glass insulator contamination identification method is described in detail below in combination with a specific embodiment. The innovation of the detection scheme of the embodiment of the present application is mainly reflected in the construction of the network model training stage, building a network model for glass insulator contamination detection, and using the multi-head self-attention network and the shallow feature full fusion algorithm to identify the glass insulator contamination. The network consists of a multi-head self-attention network backbone and a feature fusion part.

[0098] Specifically, the convolutional network backbone of the embodiment of the present application adopts TransNeXt, an excellent achievement in the forefront of the visual field, and simulates the biological visual system, especially the visual perception of the fovea, by combining aggregated pixel focused attention and convolution GLU. This method enables each pixel to achieve global perception and strengthens the information mixing and natural visual perception capabilities of the model. TransNeXt has shown excellent performance in various visual tasks, including image classification, object detection, and semantic segmentation. The network structure is as follows Figure 4 As shown in the figure, the network structure has four layers. The first three layers contain convolutional GLU layers and focus attention layers. The last layer uses a multi-head self-attention layer. Each layer can be downsampled through image segmentation (Patch Embedding) to adjust the resolution of the feature map.

[0099] For the pixel-focused attention mechanism, the mechanism has fine-grained perception near each query while maintaining coarse-grained awareness of global information. In order to achieve the pixel-level translation equivalence inherent in eye movements, the embodiment of the present application adopts a dual-path design, including query-centered sliding window attention and pooling attention. In addition, in order to induce coupling between the two attention paths, the server can use the same softmax to calculate the query-key similarity results of the two paths. This leads to competition between fine-grained and coarse-grained features, transforming the pixel-focused attention into a multi-scale attention mechanism. Now focus on the operation of a single pixel on the input feature map. Define the set of pixels in the sliding window centered on (i, j) on the input feature map as p(i, j). For a fixed window size k*k, ||p(i, ​​j)||=k 2 At the same time, define the pixel set obtained from the feature map pooling as σ(X). For the pooling size H p *H p ||σ(X)||=H p W p . Therefore, pixel-focused attention (PFA) can be described as follows.

[0100]

[0101]

[0102]

[0103]

[0104]

[0105] For the feature fusion part: the FPN+PAN structure is used in the feature fusion part. FPN can pass deep semantic information to shallow features with higher resolution, which can make large targets clearer. PAN passes the fine semantic information of shallow features to deep features, which makes small targets clearer. Unlike the Yolo series, the embodiment of this application fuses the features of 4 different levels extracted by the multi-head self-attention network backbone. The structure is as follows Figure 5 As shown in the figure, the backbone network obtains the features of S1, S2, S3, and S4,4 layers by downsampling. S1, S2, S3, and S4 are upsampled by FPN, and the sampling results of FPN are downsampled by PAN to complete feature fusion.

[0106] Finally, the network structure of the embodiment of the present application is summarized as follows: Figure 6As shown in the figure, the image to be identified is input, multi-level features are obtained through TransNext, and feature fusion is performed through FPN+PAN to obtain the final feature map, which is then input into the detection head to obtain the contamination identification result of the glass insulator.

[0107] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0108] Based on the same inventive concept, the embodiment of the present application also provides a glass insulator contamination identification device for implementing the above-mentioned glass insulator contamination identification method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the glass insulator contamination identification device provided below can refer to the limitations of the glass insulator contamination identification method above, and will not be repeated here.

[0109] In an exemplary embodiment, Figure 7 As shown, a glass insulator contamination identification device 700 is provided, comprising: an image acquisition module 701, a feature determination module 702, a feature fusion module 703 and a contamination identification module 704, wherein:

[0110] The image acquisition module 701 is used to acquire the image to be identified of each glass insulator taken by the drone;

[0111] The feature determination module 702 is used to determine the multi-scale feature map of the image to be identified based on the pre-built backbone network; the backbone network is determined based on the network model of the pixel-focused attention mechanism and the multi-head self-attention network; the pixel-focused attention mechanism is used to determine the local features of each pixel according to the degree of association between the pixels, and determine the context information of each pixel; the multi-head self-attention network is used to determine the triple of the linear transformation of the image, and independently calculate the attention weight corresponding to each pixel based on the attention heads corresponding to the query, key and value contained in the triple; the multi-scale feature map is a feature map obtained by calculating the attention weight corresponding to each pixel, and contains the local features and context information of each pixel; the multi-scale feature map includes multiple feature maps of different resolutions;

[0112] The feature fusion module 703 is used to input the multi-scale feature map into the feature fusion network, and perform feature fusion on the multi-scale feature map through the feature pyramid network FPN and the path aggregation network PAN in the feature fusion network to obtain a fused feature map;

[0113] The contamination recognition module 704 is used to input the fused feature map into the detection head to obtain the contamination recognition result corresponding to the image to be recognized; the detection head is a classification network composed of multiple convolutional layers; the contamination recognition result includes an output image with glass insulators marked, and the contamination category of each glass insulator in the output image; the contamination category includes contamination and non-contamination.

[0114] Furthermore, the multi-scale feature map includes multiple target feature maps, and the feature determination module 702 is specifically used to: segment the image to be identified based on the patch embedding layer of the backbone network, obtain feature vectors corresponding to multiple blocks, and determine them as the initial feature map corresponding to the image to be identified; perform multiple downsampling processes on the initial feature map to obtain multiple target feature maps with different resolutions; the downsampling process is based on the pixel focus attention mechanism to perform weighted summation on the feature vectors of each pixel after convolution; for the target feature map with the lowest resolution, perform weighted summation on the triplets of each pixel after convolution through a multi-head self-attention network to obtain an updated target feature map with the lowest resolution; determine that the updated target feature map with the lowest resolution, as well as other target feature maps with different resolutions, are multi-scale feature maps.

[0115] Furthermore, the feature fusion module 703 is specifically used to: input the multi-scale feature map into the FPN in the feature fusion network, and upsample the feature maps of different resolutions in the multi-scale feature map through the FPN to obtain an initial fused feature map corresponding to the image to be identified; the FPN is used to transfer the semantic information of the high-resolution feature map in the multi-scale feature map to the low-resolution feature map; input the initial fused feature map into the PAN in the feature fusion network, and downsample the feature maps of different resolutions in the initial fused feature map through the PAN to obtain a fused feature map composed of multiple feature maps of different resolutions; the PAN is used to transfer the position information of the low-resolution feature map in the initial fused feature map to the corresponding high-resolution feature map.

[0116] Furthermore, the contamination identification module 704 is specifically used to: input the fused feature map into multiple convolutional layers of the trained detection head to obtain the output result of the detection head; the output result includes the bounding box parameters, category probability and confidence of each glass insulator in the image to be identified; for each glass insulator, if the confidence corresponding to the current glass insulator is greater than the first threshold, and the category probability corresponding to the current glass insulator is greater than the second threshold, then the contamination category of the current glass insulator is determined to be contaminated; the bounding box parameters and contamination category corresponding to the current glass insulator are marked in the image to be identified to obtain a marked image, and the contamination category of the current glass insulator is stored to obtain a stored contamination category; after completing the traversal of each glass insulator, the marked image is determined to be the output image, and the stored contamination category is determined to be the contamination category of each glass insulator in the output image.

[0117] Furthermore, the device also includes a model training module, which is specifically used to: determine that the backbone network, feature fusion network and detection head are the recognition models to be trained, and obtain the training sample set of the recognition model to be trained; the training sample set includes pictures containing pollution categories marked with glass insulators; input the training sample set into the recognition model to be trained to obtain the loss value of the recognition model to be trained; the loss value is determined based on the actual results and predicted results of the training sample set; adjust the model parameters in the recognition model to be trained to obtain an updated recognition model to be trained, return to execute the step of inputting the training sample set into the recognition model to be trained to obtain the loss value of the recognition model to be trained, until the loss value meets the training end condition, and obtain the trained backbone network, feature fusion network and detection head.

[0118] Furthermore, the training sample set includes multiple insulator pictures with the position information and pollution categories of the glass insulators marked. The model training module is also specifically used for: taking initial insulator pictures corresponding to the glass insulators of each tower based on drones; performing image preprocessing on each initial insulator picture to obtain a preprocessed insulator picture; image preprocessing includes rotation, cropping and enhancement processing; marking the position of each glass insulator in the preprocessed insulator picture and the pollution category to obtain a marked insulator picture, and determining multiple marked insulator pictures as the training sample set.

[0119] Each module in the above-mentioned glass insulator contamination identification device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0120] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store training sample sets, pictures to be identified taken by drones, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying contamination of a glass insulator is implemented.

[0121] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0122] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0123] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0124] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0125] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0127] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying contamination of glass insulators, characterized in that: The method comprises: Obtain the images of each glass insulator to be identified taken by the drone; Based on a pre-built backbone network, a multi-scale feature map of the image to be identified is determined; the backbone network is determined based on a network model of a pixel-focused attention mechanism and a multi-head self-attention network; the pixel-focused attention mechanism is used to determine the local features of each pixel according to the degree of association between the pixels, and to determine the context information of each pixel; the multi-head self-attention network is used to determine a triple of a linear transformation of the image, and based on the attention heads corresponding to the query, key and value contained in the triple, independently calculate the attention weight corresponding to each pixel; the multi-scale feature map is a feature map obtained by calculating the attention weight corresponding to each pixel, and contains the local features and context information of each pixel; the multi-scale feature map includes multiple feature maps of different resolutions; Inputting the multi-scale feature map into a feature fusion network, performing feature fusion on the multi-scale feature map through a feature pyramid network FPN and a path aggregation network PAN in the feature fusion network to obtain a fused feature map; The fused feature map is input into a detection head to obtain a contamination recognition result corresponding to the image to be recognized; the detection head is a classification network composed of multiple convolutional layers; the contamination recognition result includes an output image with glass insulators marked, and the contamination category of each glass insulator in the output image; the contamination category includes contamination and non-contamination.

2. The method according to claim 1, characterized in that The multi-scale feature map includes a plurality of target feature maps; the multi-scale feature map of the image to be identified is determined based on the pre-built backbone network, including: Based on the patch embedding layer of the backbone network, the image to be identified is segmented to obtain feature vectors corresponding to a plurality of image blocks, and the feature vectors are determined as the initial feature map corresponding to the image to be identified; The initial feature map is subjected to multiple downsampling processes to obtain multiple target feature maps with different resolutions; the downsampling process is determined by weighted summing of feature vectors of each pixel after convolution based on the pixel focus attention mechanism; For the target feature map with the lowest resolution, weighted summation is performed on the triplets of each pixel after convolution through the multi-head self-attention network to obtain an updated target feature map with the lowest resolution; It is determined that the updated target feature map with the lowest resolution and other target feature maps with different resolutions are multi-scale feature maps.

3. The method according to claim 1, characterized in that The method of performing feature fusion on the multi-scale feature map through the feature pyramid network FPN and the path aggregation network PAN in the feature fusion network to obtain a fused feature map includes: The multi-scale feature map is input into the FPN in the feature fusion network, and the feature maps of different resolutions in the multi-scale feature map are up-sampled by the FPN to obtain the initial fused feature map corresponding to the image to be identified; the FPN is used to transfer the semantic information of the high-resolution feature map in the multi-scale feature map to the low-resolution feature map; The initial fused feature map is input into the PAN in the feature fusion network, and the feature maps of different resolutions in the initial fused feature map are downsampled by the PAN to obtain a fused feature map composed of multiple feature maps of different resolutions; the PAN is used to transfer the position information of the low-resolution feature map in the initial fused feature map to the corresponding high-resolution feature map.

4. The method according to claim 1, characterized in that: The step of inputting the fused feature map into the detection head to obtain the contamination recognition result corresponding to the image to be recognized includes: Inputting the fused feature map into multiple convolutional layers of a trained detection head to obtain an output result of the detection head; the output result includes bounding box parameters, category probability and confidence of each glass insulator in the image to be identified; For each of the glass insulators, if the confidence corresponding to the current glass insulator is greater than the first threshold and the category probability corresponding to the current glass insulator is greater than the second threshold, then the contamination category of the current glass insulator is determined to be contaminated; Marking the bounding box parameters and the pollution category corresponding to the current glass insulator in the to-be-recognized image to obtain a marked image, and storing the pollution category of the current glass insulator to obtain a stored pollution category; After the traversal of each glass insulator is completed, the marked image is determined to be the output image, and the stored contamination category is determined to be the contamination category of each glass insulator in the output image.

5. The method according to claim 1, characterized in that The method further comprises: Determine the backbone network, the feature fusion network and the detection head as the recognition model to be trained, and obtain a training sample set of the recognition model to be trained; the training sample set includes pictures containing a contamination category marked with glass insulators; Inputting the training sample set into the recognition model to be trained to obtain a loss value of the recognition model to be trained; the loss value is determined based on the actual result and the predicted result of the training sample set; The model parameters in the recognition model to be trained are adjusted to obtain an updated recognition model to be trained, and the step of inputting the training sample set into the recognition model to be trained to obtain the loss value of the recognition model to be trained is returned to execute until the loss value meets the training end condition, thereby obtaining a trained backbone network, a feature fusion network and a detection head.

6. The method according to claim 5, characterized in that The training sample set includes a plurality of insulator pictures with marked position information and pollution categories of glass insulators; the step of obtaining the training sample set of the recognition model to be trained includes: The initial insulator images corresponding to the glass insulators of each tower are taken by drone; Performing image preprocessing on each of the initial insulator images to obtain a preprocessed insulator image; the image preprocessing includes rotation, cropping and enhancement processing; The position of each glass insulator in the preprocessed insulator image and the contamination category are marked to obtain a marked insulator image, and a plurality of marked insulator images are determined as a training sample set.

7. A glass insulator contamination identification device, characterized in that: The device comprises: An image acquisition module is used to acquire the images to be identified of each glass insulator taken by the drone; A feature determination module is used to determine the multi-scale feature map of the image to be identified based on a pre-built backbone network; the backbone network is determined based on a network model of a pixel-focused attention mechanism and a multi-head self-attention network; the pixel-focused attention mechanism is used to determine the local features of each pixel according to the degree of association between the pixels, and to determine the context information of each pixel; the multi-head self-attention network is used to determine the triple of the linear transformation of the image, and independently calculate the attention weight corresponding to each pixel based on the attention heads corresponding to the query, key and value contained in the triple; the multi-scale feature map is a feature map obtained by calculating the attention weight corresponding to each pixel, and contains the local features and context information of each pixel; the multi-scale feature map includes multiple feature maps of different resolutions; A feature fusion module is used to input the multi-scale feature map into a feature fusion network, and perform feature fusion on the multi-scale feature map through a feature pyramid network FPN and a path aggregation network PAN in the feature fusion network to obtain a fused feature map; A contamination recognition module is used to input the fused feature map into a detection head to obtain a contamination recognition result corresponding to the image to be recognized; the detection head is a classification network composed of multiple convolutional layers; the contamination recognition result includes an output image with glass insulators marked, and the contamination category of each glass insulator in the output image; the contamination category includes contamination and non-contamination.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images