Insulator breakage detection method, device, equipment, storage medium and program product
By employing a multi-task collaborative learning method for the damage detection model, the problem of low accuracy in insulator damage detection in existing technologies is solved, enabling high-precision detection of insulator structures and abnormal areas, thereby improving detection accuracy and positioning capabilities.
Patent Information
- Application Number
- CN202611116634.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-27
- Publication Date
- 2026-08-25
AI Technical Summary
Existing insulator damage detection methods suffer from low detection accuracy, especially for damage that is structurally complex, highly localized, and scarce, which may lead to decreased insulation performance and safety hazards.
A multi-task collaborative learning method is adopted, which integrates image reconstruction, anomaly localization, and target detection branches in the damage detection model. This method improves the accuracy of insulator damage detection through feature extraction, reconstruction, localization, and detection.
It realizes multi-task collaborative learning of insulator structure distribution and abnormal areas, improves the accuracy and positioning capability of damage detection, and can more accurately identify the damage location and level of insulators.
Smart Images
Figure CN122636614A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of electrical equipment technology, and in particular to a method, apparatus, equipment, storage medium, and program product for detecting insulator damage. Background Technology
[0002] During the inspection of transmission lines and substations, insulators are exposed to complex environments for extended periods, making them prone to defects such as cracks, porcelain chipping, and contamination. These types of damage have the following characteristics: strong structural integrity (arranged in a string-like pattern), strong localization (minor damage), and scarce sample size (abnormalities are far fewer than normal). Insulator damage can lead to a decline in insulation performance, and in severe cases, may cause flashover or power outages.
[0003] Currently, insulator damage detection mainly relies on manual inspection or automatic identification methods based on target detection algorithms. That is, the degree of damage to insulators is determined by human observation.
[0004] However, the above-mentioned damage detection methods suffer from low damage detection accuracy. Summary of the Invention
[0005] Therefore, it is necessary to provide an insulator damage detection method, device, equipment, storage medium, and program product that can improve the prediction accuracy of insulator contamination levels, addressing the aforementioned technical problems.
[0006] Firstly, this application provides a method for detecting insulator damage, including:
[0007] The insulator image to be detected is acquired, and the insulator image is subjected to feature extraction and fusion to obtain fused features;
[0008] The image reconstruction branch of the damage detection model is used to reconstruct the fused features to obtain the reconstructed image;
[0009] The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map;
[0010] The target detection branch of the damage detection model is used to perform detection based on anomaly probability maps and fused features to obtain the insulator damage detection results.
[0011] In one embodiment, the image reconstruction branch includes a feature mapping module, a multi-scale feature decomposition module, a structure-aware attention module, and a multi-branch residual reconstruction module. The image reconstruction branch of the damage detection model is used to perform feature reconstruction on the fused features to obtain a reconstructed image, including:
[0012] The fused features are input into the feature mapping module and projected to obtain the latent spatial features;
[0013] The latent spatial features are input into the multi-scale feature decomposition module for decomposition processing to obtain features at multiple scales.
[0014] Each scale feature is input into the structure-aware attention module for weighted processing to obtain the attention residual fusion feature corresponding to each scale feature;
[0015] The attention residual fusion features corresponding to multiple scale features are input into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image.
[0016] In one embodiment, attention residual fusion features corresponding to multiple scale features are input into a multi-branch residual reconstruction module for fusion processing to obtain a reconstructed image, including:
[0017] For each scale feature, the attention residual fusion feature is deconvolutionally processed by the generator to obtain the sub-reconstructed image corresponding to the scale feature.
[0018] The sub-reconstructed images corresponding to multiple scale features are fused to obtain the reconstructed image.
[0019] In one embodiment, the anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map, including:
[0020] The insulator image and the reconstructed image are compared to obtain the reconstruction error map.
[0021] The reconstruction error map is convolved to obtain the anomaly probability map.
[0022] In one embodiment, the target detection branch of the damage detection model is used to perform detection based on anomaly probability maps and fused features to obtain insulator damage detection results, including:
[0023] The anomaly probability map and the fused features are concatenated to obtain the target detection features;
[0024] The target detection features are calculated using a target detection algorithm to obtain the insulator damage detection results.
[0025] In one embodiment, the damage detection model is trained in the following manner:
[0026] Acquire training samples. In each training round, input the training samples into the three branches of the initial damage detection model for forward propagation to obtain the predicted values of each branch and determine the loss value of each branch in the current round.
[0027] The total loss value is calculated based on the loss value of each branch in the current round, and the total loss value is backpropagated to obtain the gradient information of the loss function of each branch.
[0028] Update the weights of the loss function of each branch based on the gradient information of the loss function of each branch;
[0029] The new total loss value is calculated based on the loss function of each branch and the updated weights. The network parameters of each branch are then jointly updated based on the new total loss value. The training samples are then re-inputted into the three branches for forward propagation until the total loss value converges or reaches the preset training rounds, thus obtaining the trained damage detection model.
[0030] Secondly, this application also provides an insulator damage detection device, comprising:
[0031] The acquisition module is used to acquire images of insulators to be detected, and to extract and fuse features from the insulator images to obtain fused features;
[0032] The reconstruction module is used to perform feature reconstruction on the fused features using the image reconstruction branch of the damage detection model to obtain the reconstructed image.
[0033] The localization module is used to perform anomaly localization processing on the reconstructed image and insulator image using the anomaly localization branch of the damage detection model to obtain an anomaly probability map.
[0034] The detection module is used to perform detection based on anomaly probability maps and fused features using the target detection branch of the damage detection model to obtain the insulator damage detection results.
[0035] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0036] The insulator image to be detected is acquired, and the insulator image is subjected to feature extraction and fusion to obtain fused features;
[0037] The image reconstruction branch of the damage detection model is used to reconstruct the fused features to obtain the reconstructed image;
[0038] The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map;
[0039] The target detection branch of the damage detection model is used to perform detection based on anomaly probability maps and fused features to obtain the insulator damage detection results.
[0040] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0041] The insulator image to be detected is acquired, and the insulator image is subjected to feature extraction and fusion to obtain fused features;
[0042] The image reconstruction branch of the damage detection model is used to reconstruct the fused features to obtain the reconstructed image;
[0043] The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map;
[0044] The target detection branch of the damage detection model is used to perform detection based on anomaly probability maps and fused features to obtain the insulator damage detection results.
[0045] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0046] The insulator image to be detected is acquired, and the insulator image is subjected to feature extraction and fusion to obtain fused features;
[0047] The image reconstruction branch of the damage detection model is used to reconstruct the fused features to obtain the reconstructed image;
[0048] The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map;
[0049] The target detection branch of the damage detection model is used to perform detection based on anomaly probability maps and fused features to obtain the insulator damage detection results.
[0050] The aforementioned insulator damage detection method, apparatus, equipment, storage medium, and program products acquire images of the insulator to be inspected, and extract and fuse features from the insulator images to obtain fused features. The image reconstruction branch of the damage detection model is used to reconstruct the fused features, resulting in a reconstructed image. The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image, obtaining an anomaly probability map. The target detection branch of the damage detection model is used to perform detection based on the anomaly probability map and fused features, yielding the insulator damage detection result. By learning the insulator structure distribution through generative reconstruction and achieving anomaly localization through reconstruction errors, while simultaneously combining this with a target detection network for damage detection, multi-task collaborative learning of insulator structure distribution, anomaly regions, and detection targets is realized, thereby improving the accuracy and localization capability of insulator damage detection. Attached Figure Description
[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0052] Figure 1 This is a diagram illustrating the application environment of an insulator damage detection method in one embodiment;
[0053] Figure 2 This is a flowchart illustrating an insulator damage detection method in one embodiment;
[0054] Figure 3 This is a schematic diagram of the process of using the image reconstruction branch of the damage detection model to reconstruct features from fused features in one embodiment;
[0055] Figure 4 This is a schematic diagram of the process in one embodiment of inputting attention residual fusion features corresponding to multiple scale features into a multi-branch residual reconstruction module for fusion processing;
[0056] Figure 5 This is a schematic diagram of the process of performing anomaly localization processing on reconstructed images and insulator images using the anomaly localization branch of the damage detection model in one embodiment.
[0057] Figure 6 This is a schematic diagram of the process of using the target detection branch of the damage detection model to perform detection based on anomaly probability maps and fused features in one embodiment;
[0058] Figure 7 This is a flowchart illustrating the training process of a damage detection model in one embodiment;
[0059] Figure 8 This is a flowchart illustrating the insulator damage detection method in another embodiment;
[0060] Figure 9 This is a structural block diagram of an insulator damage detection device in one embodiment;
[0061] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0063] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0064] During the inspection of transmission lines and substations, insulators are exposed to complex environments for extended periods, making them prone to defects such as cracks, porcelain chipping, and contamination. These types of damage are characterized by: strong structural integrity (arranged in a string-like pattern), strong localization (minor damage), and scarce sample size (abnormalities are far fewer than normal). Insulator damage leads to a decline in insulation performance and, in severe cases, may cause flashovers or power outages. Currently, insulator damage detection mainly relies on manual inspection or automatic identification methods based on target detection algorithms. That is, the degree of insulator damage is determined by human observation. However, the above-mentioned damage detection methods suffer from low accuracy.
[0065] In view of the above-mentioned technical problems, this application provides an insulator damage detection method that can improve the accuracy of damage detection. The following embodiments will specifically illustrate the insulator damage detection method.
[0066] The insulator damage detection method provided in this application embodiment can be applied to, for example... Figure 1 The application environment is shown. Drone 102 is used to inspect power transmission line 104. Drone 102 is equipped with a detection device, an infrared camera, a visible light camera, etc.; power transmission line 104 is a power transmission line in a power system. When power transmission line 104 is in operation, drone 102 starts, and its detection device can control the infrared camera, visible light camera, zoom camera, and fixed-focus camera to collect image data of the insulators in power transmission line 104, and perform fault detection based on the collected image data to obtain the detection results. The detection device can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.
[0067] In one exemplary embodiment, such as Figure 2As shown, an insulator damage detection method is provided. This embodiment illustrates the application of this method to a detection device. In this embodiment, the method includes:
[0068] S201, acquire the image of the insulator to be detected, and perform feature extraction and fusion on the insulator image to obtain fused features.
[0069] In the embodiments of this application, the detection device can acquire insulator images using any type of camera configured on the drone; it can also acquire initial insulator images using various types of cameras configured on the drone, and filter the initial insulator images according to their clarity, selecting the insulator image acquired by the camera with the highest clarity as the insulator image to be detected.
[0070] In some embodiments, for the insulator image to be detected, a preset convolutional model is used to extract features from the insulator image to obtain multiple network layer features. An alignment module is then used to upsample and align these multiple network layer features to obtain aligned network layer features. Finally, a fusion module is used to scale and align the aligned network layer features and perform weighted summation to obtain fused features.
[0071] In some embodiments, it is assumed that the image of the insulator to be detected is Where X is the insulator image, H is the image height, W is the image width, and C is the number of image channels. The convolution model can be represented by the following relationship (1):
[0072] (1);
[0073] In the formula, Indicates the first Layered convolutional module, Indicates the first Layer features, number of network layers is It can be Treated as a feature of layer 0, i.e.: .
[0074] To construct multi-scale feature fusion, the alignment module can be represented by the following relation (2):
[0075]
[0076] (2);
[0077] in, For the first Layer fusion characteristics, For channel-aligned convolution, This is an upsampling operation.
[0078] The fusion module can be represented by the following relation (3):
[0079] (3);
[0080] in, For scale alignment function, Indicates the first Layer fusion weights, And satisfy: .
[0081] S202, the image reconstruction branch of the damage detection model is used to reconstruct the fused features to obtain the reconstructed image.
[0082] The image reconstruction branch includes a feature mapping module, a multi-scale feature decomposition module, a structure-aware attention module, and a multi-branch residual reconstruction module.
[0083] In the embodiments of this application, after obtaining the fused features, the detection device inputs the fused features into the image reconstruction branch for feature reconstruction. That is, it uses the feature mapping module of the image reconstruction branch to perform mapping and projection processing on the fused features to obtain latent spatial features. Then, it uses the multi-scale feature decomposition module of the image reconstruction branch to perform feature decomposition to obtain multiple scale features. It uses the structure-aware attention module of the image reconstruction branch to perform weighted processing on the scale features to obtain attention residual fusion features corresponding to multiple scale features. Finally, it uses the multi-branch residual reconstruction module of the image reconstruction branch to reconstruct the attention residual fusion features to obtain the reconstructed image.
[0084] In some embodiments, the detection device can also perform feature reconstruction on the fused features using any one of the following methods: matrix method, iterative method, back projection method, two-dimensional Fourier reconstruction method, filtered back projection method, fan-beam and spiral CT reconstruction algorithms, to obtain a reconstructed image.
[0085] S203, using the anomaly localization branch of the damage detection model to perform anomaly localization processing on the reconstructed image and the insulator image, to obtain an anomaly probability map.
[0086] In the embodiments of this application, the detection device can input the insulator image and the reconstructed image into the anomaly localization branch of the damage detection model for difference processing, and take the absolute value of the difference to obtain the reconstruction error map; finally, the reconstruction error map is convolved, and the convolution result is activated to obtain the anomaly probability map.
[0087] S204. The target detection branch of the damage detection model is used to perform detection based on the anomaly probability map and fusion features to obtain the insulator damage detection results.
[0088] The insulator damage detection results can include whether the insulator is damaged or not, as well as the location and severity of damage in cases of damage. Damage severity can be classified as minor, moderate, or severe.
[0089] In the embodiments of this application, the detection device can input the anomaly probability map and fusion features into the target detection branch of the damage detection model to perform splicing processing on the anomaly probability map and fusion features to obtain target detection features; finally, the target detection algorithm is used to calculate the target detection features to obtain the insulator damage detection result.
[0090] In some embodiments, the detection device can also simultaneously input the anomaly probability map and fused features into different types of target detection algorithms for processing, obtaining initial damage detection results for different types of target detection algorithms. Finally, the initial damage detection results of different types of target detection algorithms are subjected to union processing, or the initial damage detection result with the largest number of targets is determined as the insulator damage detection result. The target detection algorithm can be any one of the following: Cascade+HOG, DPM+Haar, SVM, R-CNN, Fast R-CNN, Faster R-CNN, SPP-net, R-FCN, YOLO series, SSD, and DenseBox.
[0091] The aforementioned insulator damage detection method acquires an image of the insulator to be inspected, and performs feature extraction and fusion to obtain fused features. The image reconstruction branch of the damage detection model is then used to reconstruct the fused features, resulting in a reconstructed image. The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image, obtaining an anomaly probability map. Finally, the target detection branch of the damage detection model performs detection based on the anomaly probability map and fused features, yielding the insulator damage detection result. By learning the insulator structure distribution through generative reconstruction and achieving anomaly localization through reconstruction errors, while simultaneously combining this with a target detection network for damage detection, multi-task collaborative learning of insulator structure distribution, anomaly regions, and detection targets is achieved, thereby improving the accuracy and localization capability of insulator damage detection.
[0092] In one exemplary embodiment, such as Figure 3 As shown, the image reconstruction branch of the damage detection model is used to perform feature reconstruction on the fused features to obtain the reconstructed image, including:
[0093] S301, the fused feature input feature mapping module is projected to obtain latent spatial features.
[0094] In the embodiments of this application, the detection device inputs the fused features into the feature mapping module, first performs convolution processing on the fused features to obtain convolution features, then performs normalization processing on the convolution features to obtain normalized features, and finally uses an activation function to perform nonlinear transformation on the normalized features to obtain latent space features.
[0095] In some embodiments, the mapping processing operation of the feature mapping module can be represented by the following relation (4):
[0096] (4);
[0097] In the formula, Represents potential spatial characteristics; and For feature size; The number of channels for the feature can be 256; Indicates size is convolution kernel, For normalization function, For example, ReLU is an activation function.
[0098] S302 inputs the latent spatial features into the multi-scale feature decomposition module for decomposition processing to obtain features at multiple scales.
[0099] In the embodiments of this application, the detection device inputs latent spatial features into a multi-scale feature decomposition module. For each latent spatial feature, a preset downsampling function is used to decompose the latent spatial feature to obtain multiple scale features. For example, the downsampling function can be a convolution or pooling function.
[0100] In some embodiments, the decomposition operation of the multi-scale feature decomposition module can be represented by the following relation (5):
[0101] (5);
[0102] in, Indicates the first Characteristics of scale; To determine the number of scales, considering the trade-off between computational performance and effectiveness, the number of scales can be 3 or 4. This represents the downsampling function.
[0103] S303, input the features at each scale into the structure-aware attention module for weighted processing to obtain the attention residual fusion features corresponding to each scale feature.
[0104] In the embodiments of this application, the detection device inputs the features at each scale into the structure-aware attention module. For each scale feature, the scale feature is first multiplied with the preset linear matrix parameters to obtain the query, key, and value matrix. The attention matrix is obtained through the query matrix, key matrix, and feature dimension. The attention matrix is multiplied with the value matrix to obtain the weighted feature of the corresponding scale. Finally, the scale feature is summed with the weighted feature of the corresponding scale to obtain the attention residual fusion feature corresponding to each scale feature.
[0105] In some embodiments, the weighted features of the structure-aware attention module can be represented by the following relation (6):
[0106]
[0107]
[0108] (6);
[0109]
[0110]
[0111] In the formula, , and These are query, key, and value matrices, respectively. , and These are the parameters of the corresponding learnable linear mapping matrix; Represents the attention matrix. Indicates the feature dimension (number of channels).
[0112] In some embodiments, the attention residual fusion feature can be represented by the following relation (7):
[0113] (7);
[0114] In the formula, Fusion features after introducing attention residuals These are weighted features.
[0115] S304: Input the attention residual fusion features corresponding to multiple scale features into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image.
[0116] In the embodiments of this application, the detection device inputs the attention residual fusion features corresponding to multiple scale features into the multi-branch residual reconstruction module. For the attention residual fusion features corresponding to each scale feature, the generator performs deconvolution processing on the attention residual fusion features to obtain the sub-reconstructed image corresponding to the scale feature. The generator can be constructed by the deconvolution function. Finally, the sub-reconstructed images corresponding to multiple scale features are fused to obtain the reconstructed image.
[0117] By learning the insulator structure distribution through generative reconstruction, the accuracy and location capability of insulator damage detection can be improved.
[0118] In one exemplary embodiment, such as Figure 4 As shown, the attention residual fusion features corresponding to multiple scale features are input into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image, including:
[0119] S401, for the attention residual fusion feature corresponding to each scale feature, the generator is used to perform deconvolution processing on the attention residual fusion feature to obtain the sub-reconstructed image corresponding to the scale feature.
[0120] In the image reconstruction branch, each scale corresponds to a generator, and the generator for each scale can be constructed by deconvolution.
[0121] In the embodiments of this application, after the detection device inputs the attention residual fusion features corresponding to multiple scale features into the multi-branch residual reconstruction module, it substitutes the attention residual fusion features into the generator for calculation for each scale to obtain the sub-reconstructed image corresponding to the scale feature.
[0122] In some embodiments, the generator can be represented by the following relation (8):
[0123] (8);
[0124] In the formula, For the first A scale generator (which can be constructed using a deconvolution function). This represents the reconstructed image.
[0125] S402, the sub-reconstructed images corresponding to multiple scale features are fused to obtain the reconstructed image.
[0126] In the embodiments of this application, the detection device performs upsampling processing on the sub-reconstructed images corresponding to each scale feature to obtain upsampling features of different layers, and combines the fusion weights of each layer to perform fusion processing on the upsampling features of different layers to obtain the reconstructed image.
[0127] In some embodiments, the reconstructed image can be represented by the following relation (9):
[0128] (9);
[0129] In the formula, For upsampling operation, Indicates the first Layer fusion weights, And satisfy:
[0130]
[0131] By learning the insulator structure distribution through generative reconstruction, the accuracy and location capability of insulator damage detection can be improved.
[0132] In one exemplary embodiment, such as Figure 5 As shown, the anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image, resulting in an anomaly probability map, including:
[0133] S501, perform difference processing on the insulator image and the reconstructed image to obtain the reconstruction error map.
[0134] In the embodiments of this application, the detection device can input the insulator image and the reconstructed image into the abnormal location branch of the damage detection model for difference processing, and take the absolute value of the difference to obtain the reconstruction error map;
[0135] In some embodiments, the reconstruction error map can be represented by the following relationship (10):
[0136] (10);
[0137] In the formula, Image of an insulator. To reconstruct the image, To reconstruct the error map.
[0138] S502, perform convolution processing on the reconstruction error map to obtain the anomaly probability map.
[0139] In the embodiments of this application, the detection device performs convolution processing on the reconstructed error map, and uses an activation function to perform a nonlinear transformation operation on the convolution result to obtain an anomaly probability map.
[0140] In some embodiments, the anomaly probability graph can be represented by the following relationship (11):
[0141] (11);
[0142] In the formula, This is an anomaly probability map, where σ is the activation function. It is a convolution function.
[0143] By learning the insulator structure distribution through generative reconstruction and realizing anomaly location through reconstruction error, the accuracy and location capability of insulator damage detection are improved.
[0144] In one exemplary embodiment, such as Figure 6 As shown, the target detection branch of the damage detection model performs detection based on anomaly probability maps and fused features to obtain the insulator damage detection results, including:
[0145] S601, the anomaly probability map and fused features are spliced together to obtain the target detection features.
[0146] In the embodiments of this application, the detection device substitutes the anomaly probability map and the fused features into a preset feature splicing algorithm for splicing processing to obtain target detection features.
[0147] In some embodiments, the target detection features can be represented by the following relation (12):
[0148] (12);
[0149] In the formula, This is an anomaly probability plot. As a feature of fusion, For target detection features, Concat is the feature concatenation operation.
[0150] S602, the target detection features are calculated using a target detection algorithm to obtain the insulator damage detection results.
[0151] In the embodiments of this application, the target detection algorithm can be represented by the following relation (13):
[0152] (13);
[0153] In the formula, D represents the output of the detection model, and H represents the output head branch of the detection.
[0154] In one exemplary embodiment, such as Figure 7 As shown, the damage detection model is trained in the following way:
[0155] S701. Obtain training samples. In each training round, input the training samples into the three branches of the initial damage detection model for forward propagation to obtain the predicted values of each branch and determine the loss value of each branch in the current round.
[0156] The training samples can be batches of insulator images labeled with information such as whether they are damaged, the location of the damage, and the level of damage. The initial damage detection model includes three branches: image reconstruction, anomaly localization, and target detection. Each branch network processes the same or different modalities of input data and generates a first prediction result, a second prediction result, and a third prediction result, respectively. The image reconstruction branch is pre-configured with a first loss function, the anomaly localization branch is pre-configured with a second loss function, and the target detection branch is pre-configured with a third loss function.
[0157] In the embodiments of this application, after acquiring the training samples, the detection device inputs the training samples into the three branches of the damage detection model in the training round for forward propagation, obtains the predicted value of each branch, and determines the loss value of each branch in the current round based on the preset loss function.
[0158] In some embodiments, the first loss function of the image reconstruction branch can be represented by the following relation (14):
[0159] (14);
[0160] In the formula, Original image Corresponding position The value, To reconstruct the image correspond The value of .
[0161] The second loss function of the anomaly localization branch can be expressed by the following relation (15):
[0162] (15);
[0163] In the formula, Indicates whether a pixel is a damaged area. This represents the probability of an anomaly.
[0164] The third loss function of the object detection branch can be expressed by the following relation (16):
[0165] (16);
[0166] In the formula, Cross-entropy is used for classification loss; The regression loss is for the bounding box; is the weighting coefficient, which can be set to 1. The regression loss of the detection box can be expressed by the following equation (17):
[0167] (17);
[0168] in, For the prediction box, This represents the actual annotation box. The calculation is as follows: .
[0169] S702 calculates the total loss value based on the loss value of each branch in the current round, and backpropagates the total loss value to obtain the gradient information of the loss function of each branch.
[0170] The gradient information can include the descent rate of the loss function and the degree of gradient conflict.
[0171] In the embodiments of this application, the detection device performs a weighted summation of the loss values of each branch in the current round to obtain the total loss value. The weights of the loss function of each branch can be set according to the actual training effect. Finally, the total loss value is backpropagated to obtain the gradient information of the loss function of each branch.
[0172] In some embodiments, the total loss function can be expressed by the following relation (18):
[0173] (18);
[0174] in, , and For weighting coefficients, preferably, .
[0175] S703 updates the weights of the loss function of each branch based on the gradient information of the loss function of each branch.
[0176] In the embodiments of this application, the detection device adaptively updates the weights of each loss function based on the descent rate and gradient conflict degree of the loss function of each branch, so that loss functions with slower descent or greater gradient conflict receive higher weights. Specifically, it first records the change in each loss value in the current round relative to the previous round and calculates the descent rate; then it calculates the cosine similarity matrix between the gradients of each branch network in the current round, and obtains the gradient conflict index by maximizing the value of the reciprocal of the cosine similarity matrix by multiplying it by 0; finally, it calculates the unnormalized weight correction factor based on the descent rate and the gradient conflict index; it normalizes the correction factor to obtain the updated weights, and replaces the previous weights with the updated weights.
[0177] S704: Calculate the new total loss value based on the loss function of each branch and the updated weights, and jointly update the network parameters of each branch based on the new total loss value. Then, re-input the training samples into the three branches for forward propagation until the total loss value converges or reaches the preset training rounds, and obtain the trained damage detection model.
[0178] In the embodiments of this application, the detection device calculates a new total loss value based on the loss function of each branch and the updated weights, and jointly updates the network parameters of each branch based on the new total loss value. That is, for the bottom feature extraction layer shared by the three branches, the cumulative gradient generated by the recalculated total loss is used for a one-time update; for the top task layer independent of each branch, the independent gradient of its corresponding loss function is used for update, and the update step size is scaled by the current weight coefficient of that branch. Then, the training samples are input into the three branches for forward propagation until the total loss value converges or reaches a preset number of training rounds, thus obtaining the trained damage detection model.
[0179] In addition to the methods of all the above embodiments, a method for detecting insulator damage is also provided, such as... Figure 8 As shown, the method includes:
[0180] S801: Acquire the image of the insulator to be detected, and perform feature extraction and fusion on the insulator image to obtain fused features;
[0181] S802, the fused feature input feature mapping module is projected to obtain latent spatial features;
[0182] S803 inputs the latent spatial features into the multi-scale feature decomposition module for decomposition processing to obtain features at multiple scales.
[0183] S804, input the features at each scale into the structure-aware attention module for weighted processing to obtain the attention residual fusion features corresponding to each scale feature;
[0184] S805: For the attention residual fusion feature corresponding to each scale feature, the generator is used to perform deconvolution processing on the attention residual fusion feature to obtain the sub-reconstructed image corresponding to the scale feature.
[0185] S806, fuses the sub-reconstructed images corresponding to multiple scale features to obtain the reconstructed image;
[0186] S807 performs difference processing on the insulator image and the reconstructed image to obtain the reconstruction error map;
[0187] S808 performs convolution processing on the reconstruction error map to obtain the anomaly probability map;
[0188] S809, the anomaly probability map and fused features are spliced together to obtain the target detection features;
[0189] S810 uses a target detection algorithm to calculate the target detection features and obtain the insulator damage detection results.
[0190] Each of the above steps has been described in the foregoing embodiments. For details, please refer to the foregoing content. They will not be repeated here.
[0191] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0192] Based on the same inventive concept, this application also provides an insulator damage detection device for implementing the insulator damage detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the insulator damage detection device provided below can be found in the limitations of the insulator damage detection method described above, and will not be repeated here.
[0193] In one exemplary embodiment, such as Figure 9 As shown, an insulator damage detection device is provided, comprising: an acquisition module 91, a reconstruction module 92, a positioning module 93, and a detection module 94, wherein:
[0194] The acquisition module 91 is used to acquire the image of the insulator to be detected, and to extract and fuse the features of the insulator image to obtain fused features;
[0195] Reconstruction module 92 is used to perform feature reconstruction on the fused features using the image reconstruction branch of the damage detection model to obtain a reconstructed image;
[0196] The positioning module 93 is used to perform anomaly positioning processing on the reconstructed image and the insulator image using the anomaly positioning branch of the damage detection model to obtain an anomaly probability map;
[0197] The detection module 94 is used to perform detection based on the anomaly probability map and the fusion features using the target detection branch of the damage detection model to obtain the insulator damage detection result.
[0198] In an exemplary embodiment, the reconstruction module 92 described above is specifically used for:
[0199] The fused features are input into the feature mapping module for projection processing to obtain latent spatial features;
[0200] The latent spatial features are input into the multi-scale feature decomposition module for decomposition processing to obtain multiple scale features.
[0201] Each scale feature is input into the structure-aware attention module for weighted processing to obtain the attention residual fusion feature corresponding to each scale feature;
[0202] The attention residual fusion features corresponding to multiple scale features are input into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image.
[0203] In an exemplary embodiment, the reconstruction module 92 described above is specifically used for:
[0204] For each scale feature, the attention residual fusion feature is deconvolutionally processed by the generator to obtain the sub-reconstructed image corresponding to the scale feature.
[0205] The reconstructed image is obtained by fusing the sub-reconstructed images corresponding to the multiple scale features.
[0206] In an exemplary embodiment, the positioning module 93 described above is specifically used for:
[0207] The insulator image and the reconstructed image are subjected to difference processing to obtain a reconstruction error map;
[0208] The reconstruction error map is convolved to obtain the anomaly probability map.
[0209] In an exemplary embodiment, the detection module 94 described above is specifically used for:
[0210] The anomaly probability map and the fused features are concatenated to obtain the target detection features;
[0211] The target detection features are calculated using a target detection algorithm to obtain the insulator damage detection result.
[0212] In one exemplary embodiment, the above-described apparatus further includes a training module, specifically used for:
[0213] Acquire training samples. In each training round, input the training samples into the three branches of the initial damage detection model for forward propagation to obtain the predicted value of each branch and determine the loss value of each branch in the current round.
[0214] The total loss value is calculated based on the loss value of each branch in the current round, and the total loss value is backpropagated to obtain the gradient information of the loss function of each branch;
[0215] The weights of the loss function of each branch are updated based on the gradient information of the loss function of each branch;
[0216] Calculate a new total loss value based on the loss function and updated weights of each branch, and jointly update the network parameters of each branch based on the new total loss value. Then, re-input the training samples into the three branches for forward propagation until the total loss value converges or reaches the preset training rounds, thereby obtaining the trained damage detection model.
[0217] Each module in the aforementioned insulator damage detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0218] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores insulator images and training samples. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements an insulator damage detection method.
[0219] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0220] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, performs the following steps:
[0221] The insulator image to be detected is acquired, and the insulator image is subjected to feature extraction and fusion to obtain fused features;
[0222] The image reconstruction branch of the damage detection model is used to reconstruct the fused features to obtain the reconstructed image;
[0223] The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map;
[0224] The target detection branch of the damage detection model is used to perform detection based on anomaly probability maps and fused features to obtain the insulator damage detection results.
[0225] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0226] The fused features are input into the feature mapping module and projected to obtain the latent spatial features;
[0227] The latent spatial features are input into the multi-scale feature decomposition module for decomposition processing to obtain features at multiple scales.
[0228] Each scale feature is input into the structure-aware attention module for weighted processing to obtain the attention residual fusion feature corresponding to each scale feature;
[0229] The attention residual fusion features corresponding to multiple scale features are input into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image.
[0230] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0231] For each scale feature, the attention residual fusion feature is deconvolutionally processed by the generator to obtain the sub-reconstructed image corresponding to the scale feature.
[0232] The sub-reconstructed images corresponding to multiple scale features are fused to obtain the reconstructed image.
[0233] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0234] The insulator image and the reconstructed image are compared to obtain the reconstruction error map.
[0235] The reconstruction error map is convolved to obtain the anomaly probability map.
[0236] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0237] The anomaly probability map and the fused features are concatenated to obtain the target detection features;
[0238] The target detection features are calculated using a target detection algorithm to obtain the insulator damage detection results.
[0239] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0240] Acquire training samples. In each training round, input the training samples into the three branches of the initial damage detection model for forward propagation to obtain the predicted values of each branch and determine the loss value of each branch in the current round.
[0241] The total loss value is calculated based on the loss value of each branch in the current round, and the total loss value is backpropagated to obtain the gradient information of the loss function of each branch.
[0242] Update the weights of the loss function of each branch based on the gradient information of the loss function of each branch;
[0243] The new total loss value is calculated based on the loss function of each branch and the updated weights. The network parameters of each branch are then jointly updated based on the new total loss value. The training samples are then re-inputted into the three branches for forward propagation until the total loss value converges or reaches the preset training rounds, thus obtaining the trained damage detection model.
[0244] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0245] The insulator image to be detected is acquired, and the insulator image is subjected to feature extraction and fusion to obtain fused features;
[0246] The image reconstruction branch of the damage detection model is used to reconstruct the fused features to obtain the reconstructed image;
[0247] The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map;
[0248] The target detection branch of the damage detection model is used to perform detection based on anomaly probability maps and fused features to obtain the insulator damage detection results.
[0249] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0250] The fused features are input into the feature mapping module and projected to obtain the latent spatial features;
[0251] The latent spatial features are input into the multi-scale feature decomposition module for decomposition processing to obtain features at multiple scales.
[0252] Each scale feature is input into the structure-aware attention module for weighted processing to obtain the attention residual fusion feature corresponding to each scale feature;
[0253] The attention residual fusion features corresponding to multiple scale features are input into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image.
[0254] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0255] For each scale feature, the attention residual fusion feature is deconvolutionally processed by the generator to obtain the sub-reconstructed image corresponding to the scale feature.
[0256] The sub-reconstructed images corresponding to multiple scale features are fused to obtain the reconstructed image.
[0257] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0258] The insulator image and the reconstructed image are compared to obtain the reconstruction error map.
[0259] The reconstruction error map is convolved to obtain the anomaly probability map.
[0260] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0261] The anomaly probability map and the fused features are concatenated to obtain the target detection features;
[0262] The target detection features are calculated using a target detection algorithm to obtain the insulator damage detection results.
[0263] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:
[0264] Acquire training samples. In each training round, input the training samples into the three branches of the initial damage detection model for forward propagation to obtain the predicted values of each branch and determine the loss value of each branch in the current round.
[0265] The total loss value is calculated based on the loss value of each branch in the current round, and the total loss value is backpropagated to obtain the gradient information of the loss function of each branch.
[0266] Update the weights of the loss function of each branch based on the gradient information of the loss function of each branch;
[0267] The new total loss value is calculated based on the loss function of each branch and the updated weights. The network parameters of each branch are then jointly updated based on the new total loss value. The training samples are then re-inputted into the three branches for forward propagation until the total loss value converges or reaches the preset training rounds, thus obtaining the trained damage detection model.
[0268] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0269] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0270] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0271] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for detecting insulator damage, characterized in that, The method includes: The image of the insulator to be detected is acquired, and the insulator image is subjected to feature extraction and fusion to obtain fused features; The fused features are reconstructed using the image reconstruction branch of the damage detection model to obtain a reconstructed image; The anomaly localization branch of the damage detection model is used to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map. The target detection branch of the damage detection model is used to perform detection based on the anomaly probability map and the fused features to obtain the insulator damage detection results.
2. The method according to claim 1, characterized in that, The image reconstruction branch includes a feature mapping module, a multi-scale feature decomposition module, a structure-aware attention module, and a multi-branch residual reconstruction module. The image reconstruction branch using the damage detection model performs feature reconstruction on the fused features to obtain a reconstructed image, including: The fused features are input into the feature mapping module for projection processing to obtain latent spatial features; The latent spatial features are input into the multi-scale feature decomposition module for decomposition processing to obtain multiple scale features. Each scale feature is input into the structure-aware attention module for weighted processing to obtain the attention residual fusion feature corresponding to each scale feature; The attention residual fusion features corresponding to multiple scale features are input into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image.
3. The method according to claim 2, characterized in that, The step of inputting the attention residual fusion features corresponding to multiple scale features into the multi-branch residual reconstruction module for fusion processing to obtain the reconstructed image includes: For each scale feature, the attention residual fusion feature is deconvolutionally processed by the generator to obtain the sub-reconstructed image corresponding to the scale feature. The reconstructed image is obtained by fusing the sub-reconstructed images corresponding to the multiple scale features.
4. The method according to claim 2, characterized in that, The step of using the anomaly localization branch of the damage detection model to perform anomaly localization processing on the reconstructed image and the insulator image to obtain an anomaly probability map includes: The insulator image and the reconstructed image are subjected to difference processing to obtain a reconstruction error map; The reconstruction error map is convolved to obtain the anomaly probability map.
5. The method according to claim 1, characterized in that, The target detection branch of the damage detection model performs detection based on the anomaly probability map and the fused features to obtain the insulator damage detection result, including: The anomaly probability map and the fused features are concatenated to obtain the target detection features; The target detection features are calculated using a target detection algorithm to obtain the insulator damage detection result.
6. The method according to any one of claims 1-4, characterized in that, The damage detection model is trained in the following way: Acquire training samples. In each training round, input the training samples into the three branches of the initial damage detection model for forward propagation to obtain the predicted value of each branch and determine the loss value of each branch in the current round. The total loss value is calculated based on the loss value of each branch in the current round, and the total loss value is backpropagated to obtain the gradient information of the loss function of each branch; The weights of the loss function of each branch are updated based on the gradient information of the loss function of each branch; Calculate a new total loss value based on the loss function and updated weights of each branch, and jointly update the network parameters of each branch based on the new total loss value. Then, re-input the training samples into the three branches for forward propagation until the total loss value converges or reaches the preset training rounds, thereby obtaining the trained damage detection model.
7. An insulator damage detection device, characterized in that, The device includes: The acquisition module is used to acquire an image of the insulator to be detected, and to extract and fuse features from the insulator image to obtain fused features; The reconstruction module is used to perform feature reconstruction on the fused features using the image reconstruction branch of the damage detection model to obtain a reconstructed image; The positioning module is used to perform anomaly positioning processing on the reconstructed image and the insulator image using the anomaly positioning branch of the damage detection model to obtain an anomaly probability map; The detection module is used to perform detection based on the anomaly probability map and the fusion features using the target detection branch of the damage detection model to obtain the insulator damage detection result.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.