Outdoor insulator surface defect identification method, device and electronic equipment

Abstract

The present invention provides a method, device, and electronic device for identifying surface defects on outdoor insulators. The method comprises: acquiring a target image; locating the insulator in the target image based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates; inputting the insulator image to be identified, after cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be identified and a normal insulator image, and a second difference feature map between the features of the insulator image to be identified and the features of the normal insulator image; fusing the first difference feature map and the second difference feature map, performing threshold segmentation on the fused feature map, and determining the surface defect identification result of the corresponding insulator based on the segmentation result. The present invention can automatically and accurately identify surface defects on insulators in high-voltage power systems and outdoor high-speed railway traction substation systems.

Description

Technical Field

[0001] The present invention relates to the technical field of traction substation operation status inspection and maintenance, and in particular to a method, device and electronic equipment for identifying surface defects of outdoor insulators. Background Art

[0002] Traditional equipment status inspections in the high-voltage power transmission and transformation sector suffer from limited inspection ranges, low detection efficiency, and potential safety hazards. Currently, drone-based inspections of high-voltage power transmission and transformation equipment are widely used by major power companies in my country. These inspections offer flexibility and maneuverability, enabling comprehensive inspections and diagnostics by hovering around equipment. However, current drone-based inspections for surface defect detection on outdoor insulators typically rely on manual inspection after drone-generated images are captured, preventing automated testing. Manual inspections can reduce defect identification efficiency, and detection accuracy can vary from person to person. Summary of the Invention

[0003] The embodiments of the present invention provide a method, device and electronic equipment for identifying surface defects of outdoor insulators, so as to solve the problem that the current manual-based surface defect identification of outdoor insulators has low efficiency and accuracy.

[0004] In a first aspect, an embodiment of the present invention provides a method for identifying surface defects of an outdoor insulator, comprising:

[0005] Acquire a target image; the target image is an image including an insulator and on which surface defects are to be identified;

[0006] Positioning the insulator in the target image based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates;

[0007] Inputting the insulator image to be identified, which is obtained by cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be identified and a normal insulator image and a second difference feature map between features of the insulator image to be identified and features of the normal insulator image;

[0008] The first difference feature map and the second difference feature map are fused, and the fused feature map is threshold segmented, and the surface defect recognition result of the corresponding insulator is determined according to the segmentation result.

[0009] In a possible implementation, the preset insulator detection model includes: a first feature extraction network, a feature fusion network, a feature attention network, a candidate region generation network, and a classification and bounding box regression network;

[0010] The first feature extraction network is used to extract features of different scales on the target image to obtain extracted feature maps of different scales of the target image;

[0011] The feature fusion network is used to perform feature fusion on the extracted feature maps of different scales of the target image to obtain a multi-scale fused feature map of the target image;

[0012] The feature attention network is used to weight the features of different channels in the multi-scale fusion feature map to obtain a weighted multi-scale fusion feature map;

[0013] The candidate region generation network is used to generate candidate regions according to the weighted multi-scale fusion feature map to obtain candidate regions with a high probability of containing insulators and corresponding candidate region features;

[0014] The classification and bounding box regression network is used to classify the features of each candidate region and regress each candidate region, and determine the coordinates of the candidate region with the smallest loss as the insulator positioning coordinates of the target image.

[0015] In one possible implementation, the training process of the preset insulator detection model includes:

[0016] Inputting each training image in the training set into the first feature extraction network, the feature fusion network, the feature attention network, the candidate region generation network, and the classification and bounding box regression network in sequence;

[0017] Based on the insulator positioning predicted bounding box of each training image output by the classification and bounding box regression network, the insulator positioning true bounding box of each training image and the IoU loss function, the weights of different channels in the feature attention network and the candidate regions generated in the candidate region generation network are iteratively trained. When the IoU loss function converges, a trained preset insulator detection model is obtained.

[0018] In a possible implementation, the angle of the insulator in each training image in the training set is different;

[0019] Based on the insulator positioning predicted bounding boxes of each training image output by the classification and bounding box regression network, the insulator positioning true bounding boxes of each training image and the IoU loss function, the candidate regions generated in the candidate region generation network are iteratively trained, including:

[0020] Iteratively training the candidate regions generated in the candidate region generation network based on the frame rotation angle of the predicted frame of the insulator positioning of each training image output by the classification and bounding box regression network, the frame rotation angle of the true frame of the insulator positioning of each training image, and the IoU loss function;

[0021] Positioning the insulator in the target image based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates includes:

[0022] Positioning the insulator in the target image based on a preset insulator detection model to obtain a target image including insulator positioning coordinates including a frame rotation angle;

[0023] The step of inputting the insulator image to be identified, which is obtained by cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model includes:

[0024] The insulator image to be identified, which is rotated according to the frame rotation angle in the insulator positioning coordinates and cropped according to the insulator positioning coordinates, is input into a preset image reconstruction and feature reconstruction fusion model.

[0025] In a possible implementation, the preset image reconstruction and feature reconstruction fusion model includes: an image reconstruction network, a second feature extraction network and a feature reconstruction network;

[0026] The image reconstruction network is used to reconstruct the image of the insulator to be identified to obtain a first difference feature map between the image of the insulator to be identified and a normal insulator image;

[0027] The second feature extraction network is used to extract features from the insulator image to be identified;

[0028] The feature reconstruction network is used to reconstruct the extracted features of the insulator image to be identified, and obtain a second difference feature map between the features of the insulator image to be identified and the features of a normal insulator image.

[0029] In a possible implementation, fusing the first difference feature map and the second difference feature map includes:

[0030] according to , fusing the first difference feature map and the second difference feature map;

[0031] in, is the fused feature map, is the weight of the first difference feature map, is the first difference feature map, is the weight of the second difference feature map, This is the second difference feature map.

[0032] In a possible implementation, performing threshold segmentation on the fused feature map and determining the surface defect recognition result of the corresponding insulator according to the segmentation result includes:

[0033] The foreground and background of the fused feature map are segmented based on the large-law threshold segmentation method. The presence of surface defects and the location of surface defects of the corresponding insulator are determined according to the foreground area in the segmentation result.

[0034] In a second aspect, an embodiment of the present invention provides a device for identifying surface defects of an outdoor insulator, comprising:

[0035] An acquisition module, configured to acquire a target image; the target image is an image including an insulator and on which surface defects are to be identified;

[0036] a positioning module, configured to locate the insulator in the target image based on a preset insulator detection model, and obtain a target image including the insulator positioning coordinates;

[0037] a first recognition module, configured to input the insulator image to be recognized, which is obtained by cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model, to obtain a first difference feature map between the insulator image to be recognized and a normal insulator image, and a second difference feature map between features of the insulator image to be recognized and features of the normal insulator image;

[0038] The second recognition module is used to fuse the first difference feature map and the second difference feature map, perform threshold segmentation on the fused feature map, and determine the surface defect recognition result of the corresponding insulator according to the segmentation result.

[0039] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0041] Embodiments of the present invention provide a method, device, and electronic device for identifying surface defects on outdoor insulators. These methods acquire a target image, locate the insulator in the target image based on a preset insulator detection model, and obtain a target image including the insulator's location coordinates. The target image, cropped according to the insulator's location coordinates, is then input into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be identified and a normal insulator image, and a second difference feature map between the features of the insulator image to be identified and the features of the normal insulator image. The first difference feature map and the second difference feature map are fused, and the fused feature map is threshold-segmented. The surface defect identification result of the corresponding insulator is determined based on the segmentation result. This method not only automatically identifies surface defects on insulators in the target image, but also improves detection accuracy based on the feature map fused from the first and second difference feature maps. This method can meet the operational status inspection and maintenance needs of most high-voltage power systems and outdoor high-speed railway traction substations, providing the necessary technical support for the maintenance of their key components. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 This is a flow chart of an implementation method for identifying surface defects of outdoor insulators provided by an embodiment of the present invention;

[0044] Figure 2 is a flow chart for implementing a method for identifying surface defects of outdoor insulators provided by another embodiment of the present invention;

[0045] Figure 3 1 is a schematic structural diagram of an outdoor insulator surface defect identification device provided by an embodiment of the present invention;

[0046] Figure 4 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0048] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below with reference to the accompanying drawings.

[0049] See also Figure 1 , which shows a flow chart for implementing the method for identifying surface defects of outdoor insulators provided by an embodiment of the present invention, and is described in detail as follows:

[0050] In step 101, a target image is acquired.

[0051] The target image is an image of an insulator for which surface defects are to be identified. This image can be obtained by photographing insulators in high-voltage power systems or outdoor high-speed railway traction substations using a vision system / image acquisition system consisting of a camera and lens. After acquiring the target image, median filtering can be performed on the image to suppress noise.

[0052] In step 102, the insulator in the target image is located based on a preset insulator detection model to obtain a target image including the insulator location coordinates.

[0053] The preset insulator detection model is a target detection model, and the insulators in the target image are the targets to be detected by the preset insulator detection model. To address the small size of insulator targets, complex backgrounds, and arbitrary angles, the preset insulator detection model was developed by improving the structure of commonly used target detection models such as Faster RCNN and Yolov3, and then training them based on training images from the training set.

[0054] Optional, combined Figure 2 ,The preset insulator detection model includes: the first feature extraction network, the feature fusion network, the feature attention network, the candidate region generation network, the classification and bounding box regression network.

[0055] The first feature extraction network is used to extract features of the target image at different scales to obtain extracted feature maps of the target image at different scales.

[0056] The feature fusion network is used to fuse the extracted feature maps of the target image at different scales to obtain a multi-scale fused feature map of the target image.

[0057] The feature attention network is used to weight the features of different channels in the multi-scale fusion feature map to obtain the weighted multi-scale fusion feature map.

[0058] The candidate region generation network is used to generate candidate regions according to the weighted multi-scale fusion feature map, and obtain candidate regions with a high probability of containing insulators and corresponding candidate region features.

[0059] The classification and bounding box regression network is used to classify the features of each candidate region and regress each candidate region, and determine the coordinates of the candidate region with the smallest loss as the insulator positioning coordinates of the target image.

[0060] Optionally, based on the above structure of the preset insulator detection model, the training process of the preset insulator detection model includes:

[0061] Each training image in the training set is input into the first feature extraction network, feature fusion network, feature attention network, candidate region generation network, classification and bounding box regression network in sequence.

[0062] Based on the predicted bounding boxes of the insulator positioning of each training image output by the classification and bounding box regression network, the true bounding boxes of the insulator positioning of each training image and the Intersection over Union (IoU) loss function, the weights of different channels in the feature attention network and the candidate regions generated in the candidate region generation network are iteratively trained. When the IoU loss function converges, the trained preset insulator detection model is obtained.

[0063] Among them, the angle of the insulator in each training image in the training set is different. Therefore, based on the insulator positioning predicted bounding box of each training image output by the classification and bounding box regression network, the insulator positioning true bounding box of each training image and the IoU loss function, the candidate regions generated in the candidate region generation network are iteratively trained, including:

[0064] Based on the bounding box rotation angle of the insulator localization predicted bounding box of each training image output by the classification and bounding box regression network, the bounding box rotation angle of the insulator localization true bounding box of each training image and the IoU loss function, the candidate regions generated in the candidate region generation network are iteratively trained.

[0065] Accordingly, the insulator in the target image is located based on the preset insulator detection model to obtain the target image including the insulator location coordinates, including:

[0066] The insulator in the target image is positioned based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates of the frame rotation angle.

[0067] For example, the insulator positioning coordinates including the frame rotation angle can be ,in, It can be the upper left corner coordinate or center coordinate of the bounding box where the insulator is located in the target image. It can be the length and width of the bounding box where the insulator is located in the target image, is the angle between the bounding box of the insulator in the target image and the vertical / horizontal direction.

[0068] In this embodiment, a feature fusion network is added after the feature extraction network of a commonly used object detection model to superimpose or add the multi-scale features extracted by the feature extraction network to fuse features of different scales. This improves the detection performance of commonly used object detection models for small objects such as insulators. Furthermore, a large number of samples can be taken from the anchor frame regions of training images in a training set (i.e., the regions where insulators are located in the training images). The insulator detection model with the above structure is trained based on the sampled images and the original training images in the training set, thereby obtaining a preset insulator detection model with good detection performance for small objects such as insulators. Furthermore, a feature attention network is added after the feature fusion network to assign different weights to features of different dimensions / channels. This can reduce background noise and highlight the target (i.e., insulator) in the presence of cluttered background. Furthermore, training images in which the angles of insulators in the training images are not exactly the same are used to form a training set, and candidate regions including the rotation angles of the bounding boxes are generated in the candidate region generation network. Then, the IoU loss is used as the training loss of the network, which can solve the bounding box regression problem of detecting arbitrarily rotated targets and obtain a preset insulator detection model that can locate insulators at any angle.

[0069] In step 103, the insulator image to be identified, which is obtained by cropping the target image according to the insulator positioning coordinates, is input into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be identified and the normal insulator image and a second difference feature map between the features of the insulator image to be identified and the features of the normal insulator image.

[0070] In this embodiment, the target image including the insulator location coordinates can be cropped based on the input image requirements of the preset image reconstruction and feature reconstruction fusion model and uniformly scaled to a preset size. The cropped insulator image to be identified includes at least the target image corresponding to its corresponding insulator location coordinates. Because the insulator location coordinates include the frame rotation angle, the insulator image to be identified, after being cropped according to the insulator location coordinates, is input into the preset image reconstruction and feature reconstruction fusion model, which may include:

[0071] The insulator image to be identified, which is rotated according to the frame rotation angle in the insulator positioning coordinates and cropped according to the insulator positioning coordinates, is input into the preset image reconstruction and feature reconstruction fusion model.

[0072] In this embodiment, since the insulator positioning coordinates after the preset insulator detection model detects and positions the target image include a frame rotation angle, the target image is rotated to a direction consistent with the input image of the preset image reconstruction and feature reconstruction fusion model according to the frame rotation angle, and then cropped and input into the preset image reconstruction and feature reconstruction fusion model, the preset image reconstruction and feature reconstruction fusion model can be enabled to identify insulator surface defects at any angle.

[0073] Optional, combined Figure 2 , the preset image reconstruction and feature reconstruction fusion model includes: image reconstruction network, second feature extraction network and feature reconstruction network.

[0074] The image reconstruction network is used to reconstruct the image of the insulator to be identified and obtain a first difference feature map between the image of the insulator to be identified and a normal insulator image.

[0075] The second feature extraction network is used to extract features from the insulator image to be identified.

[0076] The feature reconstruction network is used to reconstruct the features of the extracted insulator image to be identified, and obtain a second difference feature map between the features of the insulator image to be identified and the features of a normal insulator image.

[0077] In this embodiment, the structure of the preset image reconstruction and feature reconstruction fusion model can be divided into two parts. One part is the image reconstruction network, which can be implemented using an autoencoder structure. When training the preset image reconstruction and feature reconstruction fusion model, defect-free insulator images are used to enable the image reconstruction network to reconstruct the insulator image to be identified into a normal insulator image. Based on this, the difference image between the reconstructed image (i.e., the normal insulator image) and the insulator image to be identified is calculated, i.e., the defect image of the insulator image to be identified. The other part includes a feature extraction network and a feature reconstruction network. The feature extraction network is used to extract features of the input image (i.e., the image to be identified). The feature reconstruction network can also be implemented using an autoencoder structure. The trained feature reconstruction network is capable of reconstructing the features of the insulator image to be identified into the features of the normal insulator image. Based on this, the difference features between the reconstructed features (i.e., the features of the normal insulator image) and the features of the insulator image to be identified are calculated, i.e., the defect features of the insulator image to be identified.

[0078] In step 104, the first difference feature map and the second difference feature map are fused, and the fused feature map is segmented using a threshold value, and a surface defect recognition result of the corresponding insulator is determined based on the segmentation result.

[0079] In this embodiment, since the first difference feature map and the second difference feature map show the difference between the image of the insulator to be identified and the normal insulator image from different angles, threshold segmentation of the foreground and background is performed based on the fused feature map, and based on the segmentation result, it is determined whether the insulator corresponding to the image of the insulator to be identified has surface defects and the location of the surface defects, which can improve the accuracy of surface defect identification of outdoor insulators.

[0080] Optionally, fusing the first difference feature map and the second difference feature map includes:

[0081] according to , the first difference feature map and the second difference feature map are fused.

[0082] in, is the fused feature map, is the weight of the first difference feature map, is the first difference feature map, is the weight of the second difference feature map, It is the second difference feature map.

[0083] In this embodiment, weighted fusion is directly adopted for the first difference feature map and the second difference feature map. and the weight of the second difference feature map It can be determined based on experience or by training a neural network model.

[0084] Optionally, threshold segmentation is performed on the fused feature map, and a surface defect recognition result of the corresponding insulator is determined based on the segmentation result, including:

[0085] The foreground and background of the fused feature map are segmented based on the large-law threshold segmentation method. The presence of surface defects and the location of surface defects of the corresponding insulator are determined according to the foreground area in the segmentation result.

[0086] Among them, the large-law threshold segmentation method, also known as the inter-class variance threshold segmentation method, separates the image into two parts: background and target based on the image's grayscale characteristics. The larger the inter-class variance between the background and target, the greater the difference between the two parts of the image. When part of the target is misclassified as background, or part of the background is misclassified as target, the difference between the two parts decreases. Therefore, the segmentation that maximizes the inter-class variance minimizes the probability of misclassification. Therefore, based on the large-law threshold segmentation method, the foreground and background of the fused feature map are segmented. Based on the foreground area in the segmentation result, the presence and location of surface defects in the corresponding insulator can be more accurately determined.

[0087] The embodiment of the present invention obtains a target image, and then locates the insulator in the target image based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates. The target image is then cropped according to the insulator positioning coordinates and the insulator image to be identified is input into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be identified and a normal insulator image, and a second difference feature map between the features of the insulator image to be identified and the features of the normal insulator image. The first difference feature map and the second difference feature map are fused, and the fused feature map is threshold segmented, and the surface defect identification result of the corresponding insulator is determined based on the segmentation result. Not only can the surface defects of the insulator in the target image be automatically identified, but the detection accuracy can also be improved based on the feature map fused from the first difference feature map and the second difference feature map, thereby meeting the operation status maintenance requirements of most high-voltage power systems and outdoor high-speed railway traction substation systems, and providing the necessary technical support for the maintenance of their key components.

[0088] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0089] The following are device embodiments of the present invention. For details not fully described therein, reference may be made to the corresponding method embodiments described above.

[0090] Figure 3 The following is a schematic diagram showing the structure of an outdoor insulator surface defect identification device provided by an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown, which are detailed as follows:

[0091] like Figure 3 As shown, the device for identifying surface defects of outdoor insulators includes: an acquisition module 31 , a positioning module 32 , a first identification module 33 and a second identification module 34 .

[0092] An acquisition module 31 is configured to acquire a target image; the target image is an image including an insulator and on which surface defects are to be identified;

[0093] a positioning module 32, configured to locate the insulator in the target image based on a preset insulator detection model, and obtain a target image including the insulator positioning coordinates;

[0094] The first recognition module 33 is configured to input the insulator image to be recognized, which is obtained by cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be recognized and a normal insulator image, and a second difference feature map between the features of the insulator image to be recognized and the features of the normal insulator image;

[0095] The second recognition module 34 is configured to fuse the first difference feature map and the second difference feature map, perform threshold segmentation on the fused feature map, and determine a surface defect recognition result of the corresponding insulator based on the segmentation result.

[0096] The embodiment of the present invention obtains a target image, and then locates the insulator in the target image based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates. The target image is then cropped according to the insulator positioning coordinates and the insulator image to be identified is input into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be identified and a normal insulator image, and a second difference feature map between the features of the insulator image to be identified and the features of the normal insulator image. The first difference feature map and the second difference feature map are fused, and the fused feature map is threshold segmented, and the surface defect identification result of the corresponding insulator is determined based on the segmentation result. Not only can the surface defects of the insulator in the target image be automatically identified, but the detection accuracy can also be improved based on the feature map fused from the first difference feature map and the second difference feature map, thereby meeting the operation status maintenance requirements of most high-voltage power systems and outdoor high-speed railway traction substation systems, and providing the necessary technical support for the maintenance of their key components.

[0097] In a possible implementation, the preset insulator detection model includes: a first feature extraction network, a feature fusion network, a feature attention network, a candidate region generation network, and a classification and bounding box regression network;

[0098] The first feature extraction network is used to extract features of different scales on the target image to obtain extracted feature maps of different scales of the target image;

[0099] The feature fusion network is used to perform feature fusion on the extracted feature maps of different scales of the target image to obtain a multi-scale fused feature map of the target image;

[0100] The feature attention network is used to weight the features of different channels in the multi-scale fusion feature map to obtain a weighted multi-scale fusion feature map;

[0101] The candidate region generation network is used to generate candidate regions according to the weighted multi-scale fusion feature map to obtain candidate regions with a high probability of containing insulators and corresponding candidate region features;

[0102] The classification and bounding box regression network is used to classify the features of each candidate region and regress each candidate region, and determine the coordinates of the candidate region with the smallest loss as the insulator positioning coordinates of the target image.

[0103] In one possible implementation, the training process of the preset insulator detection model includes:

[0104] Inputting each training image in the training set into the first feature extraction network, the feature fusion network, the feature attention network, the candidate region generation network, and the classification and bounding box regression network in sequence;

[0105] Based on the insulator positioning predicted bounding box of each training image output by the classification and bounding box regression network, the insulator positioning true bounding box of each training image and the IoU loss function, the weights of different channels in the feature attention network and the candidate regions generated in the candidate region generation network are iteratively trained. When the IoU loss function converges, a trained preset insulator detection model is obtained.

[0106] In a possible implementation, the angle of the insulator in each training image in the training set is different;

[0107] Based on the insulator positioning predicted bounding boxes of each training image output by the classification and bounding box regression network, the insulator positioning true bounding boxes of each training image and the IoU loss function, the candidate regions generated in the candidate region generation network are iteratively trained, including:

[0108] Iteratively training the candidate regions generated in the candidate region generation network based on the frame rotation angle of the predicted frame of the insulator positioning of each training image output by the classification and bounding box regression network, the frame rotation angle of the true frame of the insulator positioning of each training image, and the IoU loss function;

[0109] A positioning module 32 may be configured to locate the insulator in the target image based on a preset insulator detection model, and obtain a target image including insulator positioning coordinates including a frame rotation angle;

[0110] The first recognition module 33 may be configured to input the insulator image to be recognized, which is rotated according to the frame rotation angle in the insulator positioning coordinates and cropped according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model.

[0111] In a possible implementation, the preset image reconstruction and feature reconstruction fusion model includes: an image reconstruction network, a second feature extraction network and a feature reconstruction network;

[0112] The image reconstruction network is used to reconstruct the image of the insulator to be identified to obtain a first difference feature map between the image of the insulator to be identified and a normal insulator image;

[0113] The second feature extraction network is used to extract features from the insulator image to be identified;

[0114] The feature reconstruction network is used to reconstruct the extracted features of the insulator image to be identified, and obtain a second difference feature map between the features of the insulator image to be identified and the features of a normal insulator image.

[0115] In a possible implementation, the second identification module 34 may be configured to: , fusing the first difference feature map and the second difference feature map;

[0116] in, is the fused feature map, is the weight of the first difference feature map, is the first difference feature map, is the weight of the second difference feature map, This is the second difference feature map.

[0117] In one possible implementation, the second recognition module 34 can be used to segment the foreground and background of the fused feature map based on a large-law threshold segmentation method, and determine whether the corresponding insulator has surface defects and the location of the surface defects based on the foreground area in the segmentation result.

[0118] Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, the steps in the above-mentioned embodiments of the method for identifying surface defects of outdoor insulators are implemented, such as Figure 1 Alternatively, when the processor 40 executes the computer program 42, the functions of the modules / units in the above-mentioned device embodiments are realized, for example Figure 3 Functions of the modules / units 31 to 34 are shown.

[0119] For example, the computer program 42 may be divided into one or more modules / units, one or more modules / units being stored in the memory 41 and executed by the processor 40 to implement the present invention. One or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 42 in the electronic device 4. For example, the computer program 42 may be divided into Figure 3 Modules / units 31 to 34 are shown.

[0120] The electronic device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 4 may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0121] The processor 40 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0122] Memory 41 can be an internal storage unit of electronic device 4, such as a hard drive or memory within electronic device 4. Memory 41 can also be an external storage device within electronic device 4, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, memory 41 can include both an internal storage unit and an external storage device within electronic device 4. Memory 41 is used to store computer programs and other programs and data required by the electronic device. Memory 41 can also be used to temporarily store data that has been output or is about to be output.

[0123] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0124] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0125] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0126] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0127] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0128] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0129] If the integrated module / unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned outdoor insulator surface defect identification method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media. It should be noted that the content of computer-readable media can be appropriately expanded or reduced based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, legislation and patent practice do not require computer-readable media to include electrical carrier signals and telecommunications signals.

[0130] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for identifying surface defects of outdoor insulators, characterized in that: include: Acquire the target image; The target image is an image including an insulator and on which surface defects are to be identified; Positioning the insulator in the target image based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates; Inputting the insulator image to be identified, which is obtained by cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model to obtain a first difference feature map between the insulator image to be identified and a normal insulator image and a second difference feature map between features of the insulator image to be identified and features of the normal insulator image; fusing the first difference feature map and the second difference feature map, performing threshold segmentation on the fused feature map, and determining a surface defect recognition result of the corresponding insulator according to the segmentation result; The preset insulator detection model includes: a first feature extraction network, a feature fusion network, a feature attention network, a candidate region generation network, and a classification and bounding box regression network; The first feature extraction network is used to extract features of different scales on the target image to obtain extracted feature maps of different scales of the target image; The feature fusion network is used to perform feature fusion on the extracted feature maps of different scales of the target image to obtain a multi-scale fused feature map of the target image; The feature attention network is used to weight the features of different channels in the multi-scale fusion feature map to obtain a weighted multi-scale fusion feature map; The candidate region generation network is used to generate candidate regions according to the weighted multi-scale fusion feature map to obtain candidate regions with a high probability of containing insulators and corresponding candidate region features; The classification and bounding box regression network is used to classify the features of each candidate region and regress each candidate region, and determine the coordinates of the candidate region with the smallest loss as the insulator positioning coordinates of the target image; The preset image reconstruction and feature reconstruction fusion model includes: an image reconstruction network, a second feature extraction network and a feature reconstruction network; The image reconstruction network is used to reconstruct the image of the insulator to be identified to obtain a first difference feature map between the image of the insulator to be identified and a normal insulator image; The second feature extraction network is used to extract features from the insulator image to be identified; The feature reconstruction network is used to reconstruct the extracted features of the insulator image to be identified, and obtain a second difference feature map between the features of the insulator image to be identified and the features of a normal insulator image.

2. The method for identifying surface defects of outdoor insulators according to claim 1, characterized in that: The training process of the preset insulator detection model includes: Inputting each training image in the training set into the first feature extraction network, the feature fusion network, the feature attention network, the candidate region generation network, and the classification and bounding box regression network in sequence; Based on the insulator positioning predicted bounding box of each training image output by the classification and bounding box regression network, the insulator positioning true bounding box of each training image and the IoU loss function, the weights of different channels in the feature attention network and the candidate regions generated in the candidate region generation network are iteratively trained. When the IoU loss function converges, a trained preset insulator detection model is obtained.

3. The method for identifying surface defects of outdoor insulators according to claim 2, characterized in that: The angle of the insulator in each training image in the training set is different; Based on the insulator positioning predicted bounding boxes of each training image output by the classification and bounding box regression network, the insulator positioning true bounding boxes of each training image and the IoU loss function, the candidate regions generated in the candidate region generation network are iteratively trained, including: Iteratively training the candidate regions generated in the candidate region generation network based on the frame rotation angle of the predicted frame of the insulator positioning of each training image output by the classification and bounding box regression network, the frame rotation angle of the true frame of the insulator positioning of each training image, and the IoU loss function; Positioning the insulator in the target image based on a preset insulator detection model to obtain a target image including the insulator positioning coordinates includes: Positioning the insulator in the target image based on a preset insulator detection model to obtain a target image including insulator positioning coordinates including a frame rotation angle; The step of inputting the insulator image to be identified, which is obtained by cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model includes: The insulator image to be identified, which is rotated according to the frame rotation angle in the insulator positioning coordinates and cropped according to the insulator positioning coordinates, is input into a preset image reconstruction and feature reconstruction fusion model.

4. The method for identifying surface defects of outdoor insulators according to claim 1, characterized in that: The fusing the first difference feature map and the second difference feature map includes: according to , fusing the first difference feature map and the second difference feature map; in, is the fused feature map, is the weight of the first difference feature map, is the first difference feature map, is the weight of the second difference feature map, This is the second difference feature map.

5. The method for identifying surface defects of outdoor insulators according to claim 1, characterized in that: The step of performing threshold segmentation on the fused feature map and determining the surface defect recognition result of the corresponding insulator according to the segmentation result includes: The foreground and background of the fused feature map are segmented based on the large-law threshold segmentation method. The presence of surface defects and the location of surface defects of the corresponding insulator are determined according to the foreground area in the segmentation result.

6. An outdoor insulator surface defect identification device, characterized in that: include: An acquisition module, used to acquire a target image; The target image is an image including an insulator and on which surface defects are to be identified; a positioning module, configured to locate the insulator in the target image based on a preset insulator detection model, and obtain a target image including the insulator positioning coordinates; a first recognition module, configured to input the insulator image to be recognized, which is obtained by cropping the target image according to the insulator positioning coordinates, into a preset image reconstruction and feature reconstruction fusion model, to obtain a first difference feature map between the insulator image to be recognized and a normal insulator image, and a second difference feature map between features of the insulator image to be recognized and features of the normal insulator image; a second recognition module, configured to fuse the first difference feature map and the second difference feature map, perform threshold segmentation on the fused feature map, and determine a surface defect recognition result of the corresponding insulator based on the segmentation result; The preset insulator detection model includes: a first feature extraction network, a feature fusion network, a feature attention network, a candidate region generation network, and a classification and bounding box regression network; The first feature extraction network is used to extract features of different scales on the target image to obtain extracted feature maps of different scales of the target image; The feature fusion network is used to perform feature fusion on the extracted feature maps of different scales of the target image to obtain a multi-scale fused feature map of the target image; The feature attention network is used to weight the features of different channels in the multi-scale fusion feature map to obtain a weighted multi-scale fusion feature map; The candidate region generation network is used to generate candidate regions according to the weighted multi-scale fusion feature map to obtain candidate regions with a high probability of containing insulators and corresponding candidate region features; The classification and bounding box regression network is used to classify the features of each candidate region and regress each candidate region, and determine the coordinates of the candidate region with the smallest loss as the insulator positioning coordinates of the target image; The preset image reconstruction and feature reconstruction fusion model includes: an image reconstruction network, a second feature extraction network and a feature reconstruction network; The image reconstruction network is used to reconstruct the image of the insulator to be identified to obtain a first difference feature map between the image of the insulator to be identified and a normal insulator image; The second feature extraction network is used to extract features from the insulator image to be identified; The feature reconstruction network is used to reconstruct the extracted features of the insulator image to be identified, and obtain a second difference feature map between the features of the insulator image to be identified and the features of a normal insulator image.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images