Umbrella skirt damage detection method, storage medium and device

Abstract

The present invention discloses a shed damage detection method, storage medium, and device. The method comprises acquiring an image; obtaining shed area coordinates; extracting the shed image, performing damage prediction, and obtaining a shed damage heat map; generating a peak map within the heat map, and screening out points with peak values greater than a threshold to obtain the corresponding shed damage points. The present invention predicts shed area damage and damage to damaged portions of the shed area, effectively reducing environmental impacts. Training using a two-layer loss function enhances the supervision of shed damage prediction. The present invention offers high recognition accuracy, efficiency, and robustness, and is applicable to various environmental conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, storage medium and device for detecting shed damage. Background Art

[0002] The complex and ever-changing environment surrounding insulator string sheds, coupled with the effects of prolonged strong electric fields and mechanical loads, the inherent quality and structural constraints of the insulators, and bird pecking, can easily lead to shed damage. If these damages are not detected promptly, they can develop into more serious damage, resulting in serious power grid incidents. With the increasing advancement of drone technology, intelligent image-based identification of insulator damage is becoming increasingly important. Currently, the primary method for detecting insulator damage relies on object detection algorithms, which can directly locate defects on the insulator. Other methods include, but are not limited to, fully connected convolutional networks (FCNs), which directly identify damaged areas at the pixel level. However, this approach can lead to sample imbalance due to the small size of the damaged area, making model training difficult. It also makes it difficult to accurately locate the damaged shed area. Summary of the Invention

[0003] To solve the above problems, a method, storage medium, and device for detecting and identifying shed heat maps are provided, which have high recognition accuracy, high efficiency, and strong robustness and can be applied to various environmental conditions. The present invention adopts the following technical solutions:

[0004] In a first aspect, the present invention provides a method for detecting shed damage, comprising the following steps:

[0005] S1. Get the picture;

[0006] S2, obtaining the regional coordinates of the shed;

[0007] S3. Extract the shed image, perform damage prediction, and obtain a heat map of shed damage;

[0008] S4. Generate a peak map in the heat map, filter out points with peak values greater than a threshold, and obtain corresponding damage points of the shed.

[0009] Furthermore, obtaining the regional coordinates of the shed specifically includes: using a target detection algorithm to detect the area where the shed is located, wherein the target detection algorithm uses a convolutional network to predict the target and obtain the position of the target.

[0010] Furthermore, the damage prediction in step S3 to obtain the heat map of shed damage is specifically performed by extracting features from the shed image and obtaining the heat map of shed damage points in the shed image using the shed prediction model.

[0011] Furthermore, the shed prediction model is trained in advance, specifically including: making labels: marking sample images, generating a peak map of shed damage as a label, wherein the peak map includes a peak probability map of the damage point and a peak probability map of the background, and marking pixel values of the peak map in a range of 0 to 1, where the closer to the damage point, the larger the pixel value; and training the model: using a two-layer loss function for training.

[0012] Furthermore, the peak probability of any point p in the peak graph conforms to the following formula

[0013]

[0014] Among them, j represents the intersection point, p represents a point in the peak graph, l k Represents the intersection point in the image that has the greatest impact on point p, Dist(p,l k ) represents the distance between point p and point l k The distance between points; σ represents the decay rate of the control peak probability.

[0015] Furthermore, the training model specifically includes inputting a sample feature map, using σ as a transitivity parameter to measure the peak value, predicting the pixel points in the sample feature map, outputting an activation function tanh(x) with a linear property between 0 and 1, and using the Euclidean distance as the loss function; wherein,

[0016]

[0017] Loss Function Where x and y represent the horizontal and vertical coordinates on the heat map respectively. The value of the point with x and y coordinates on the heat map representing the damage point. Indicates the label point corresponding to the label heat map, l, w, h are the length, width, and height of the data respectively.

[0018] Furthermore, the umbrella skirt prediction model includes a normal convolutional network, a deconvolutional network, a vertical convolutional network and a horizontal convolutional network, wherein the normal convolutional network downsamples the image, the deconvolutional network expands the plane dimension of the feature attributes, the vertical convolutional network extracts features from the vertical direction, and the horizontal convolutional network extracts features from the horizontal direction.

[0019] In a second aspect of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the method described in any one of the above technical solutions when running.

[0020] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method described in any one of the above technical solutions.

[0021] The present invention first predicts the shed area and the damaged parts of the shed area, effectively reducing the impact of the environment. It then uses a heat map to annotate the shed image, which is then used to train the model to predict the probability map of the damaged shed area for the entire image. Training using a two-layer loss function enhances the supervision of shed damage prediction. This method has high recognition accuracy, high efficiency, and strong robustness, and can be applied to various environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 This is a flow chart of a method for detecting shed damage according to an embodiment of the present invention;

[0024] Figure 2 for Figure 1 Schematic diagram of different types of convolution modules used in the embodiments;

[0025] Figure 3 for Figure 1 Schematic diagram of the main structure of the umbrella skirt damage model in the embodiment. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0027] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products or apparatus.

[0028] This embodiment provides a method for detecting shed damage. Figure 1 As shown, the following steps are included:

[0029] S1. Get the picture;

[0030] S2, obtaining the regional coordinates of the shed;

[0031] Specifically, the area where the shed skirt is located is detected using a target detection algorithm. The target detection algorithm uses a convolutional network to predict the target and obtain the target's position. Common target detection algorithms include Faster RCNN, SSD, and YOLO, but the target detection algorithms included in this embodiment are not limited to the aforementioned ones.

[0032] S3. Extract the shed image, perform damage prediction, and obtain a heat map of shed damage;

[0033] Specifically, the method comprises the following steps: performing feature extraction on the shed image, and using a shed prediction model to obtain a heat map of shed damage points in the shed image.

[0034] This embodiment uses a convolutional network to predict shed damage, similar to the pixel-level recognition algorithm of FCN. Unlike FCN, this embodiment proposes a shed damage heat map prediction. The closer the distance to the shed damage is, the greater the probability value is, and the farther the distance is, the smaller the probability value is. In this regard, the difference between the algorithm proposed in this embodiment and FCN is that FCN classifies the pixels of the image, and the classification result is either 0 (background) or 1 (damaged area), which is a discrete solution. In contrast, this embodiment proposes a heat map-like method to predict the image, and the predicted value is between 0 and 1, which is a regression variable. For convolutional networks, their prediction ability for linear quantities is better than that for discrete quantities.

[0035] Before executing this step, the shed prediction model needs to be trained in advance, including:

[0036] Create labels: Label the sample images and generate a peak map of shed damage as a label. The peak map includes a peak probability map of the damage point and a peak probability map of the background. The pixel values of the peak map are marked in the range of 0 to 1. The closer to the damage point, the larger the pixel value.

[0037] Training model: Two-layer loss function is used for training.

[0038] Similar to a fully connected convolutional network, in order to train a model capable of detecting straight lines in a form, this embodiment designs and generates a probability peak map of shed damage as a label. Each point on the probability map is inversely correlated with the distance to its nearest shed damage point.

[0039] In the process of generating the label map of the shed, the peak probability map of the shed damage is defined.

[0040]

[0041] Among them, j represents the peak map of the damaged point or the background, p represents a point on the probability peak map, l k Represents the intersection point in the image that has the greatest impact on point p, Dist(p,l k ) represents the distance between point p and point l k The distance between points; σ represents the decay rate of the control peak probability.

[0042] Usually, since the graph contains more than one shed damage probability graph, the probability values of certain points in the probability peak graph will be affected by different damage points. In this case, the point with the greatest impact is set as the final value. The points on the final probability graph can be expressed as:

[0043]

[0044] The final probability peak map is calculated using the shed damage point that has the greatest impact on the current point as the center line. During the prediction phase, the predicted result will be a probability map similar to the original label. The original image is labeled to generate a probability peak map for the damage point and a probability peak map for the background. The pixel values in the labeled map range from 0 to 1, with larger values indicating whiter colors. Zooming in on the damage hotspot map clearly shows that points closer to the damage point have larger values, exhibiting a stair-step pattern similar to the peak map.

[0045] During the actual training process, σ is used as the transitivity of the metric peak to represent the peak value from the current point to the nearest damage point. The original image and the extracted feature map are used as the scale measurement units respectively. Through testing, it is found that although the peak range of the final extracted feature map is wider, the effect of using the original image as the scale is better. This may be related to the peak transitivity represented by the line segment during the training process. The peak value of the line segment can be represented within a certain range. Forcibly expanding its peak performance value will produce counterproductive effects.

[0046] Loss function: The algorithm of this embodiment is similar to the prediction results of FCN, and also predicts the pixels on the entire image. In FCN, the model uses the ReLU activation function as the last layer. The prediction result of the model in this embodiment is a heat map of the damage points, so the model output should preferably have an activation function with linear properties between 0 and 1. During the training process, the tanh function is used as the final activation function of the model. The analytical function of tanh is:

[0047]

[0048] The prediction result is a heat map of the damage point, and the annotation map is also a heat map of the horizontal and vertical lines. In this case, this embodiment uses Euclidean distance as the loss function for training, as shown in the following formula, where j represents the damage point.

[0049]

[0050] Heat map or background heat map, the value range is (0,1), x, y represent the horizontal and vertical coordinates on the heat map, respectively. The value of the point with x and y coordinates on the heat map representing the damage point. The label points corresponding to the label heatmap are represented by l, w, and h, respectively, representing the length, width, and height of the data. Training is performed using the SGD gradient descent algorithm to update the gradients. Data augmentation is performed on the dataset using random brightness, contrast, random cropping, and random rotation.

[0051] S4. Generate a peak map in the heat map, filter out points with peak values greater than a threshold, and obtain corresponding damage points of the shed.

[0052] It should be noted that the umbrella skirt prediction model in this embodiment includes a normal convolutional network, a deconvolutional network, a vertical convolutional network and a horizontal convolutional network, wherein the normal convolutional network downsamples the image, the deconvolutional network expands the plane dimension of the feature attributes, the vertical convolutional network extracts features from the vertical direction, and the horizontal convolutional network extracts features from the horizontal direction.

[0053] First, the data for each layer in the model is three-dimensional data of size h, w, c, where h and w are the height and width of the data, respectively, and c represents the number of channels. For example, the model input has three channels. The network architecture used in the model does not alter the planar relative relationships of the extracted features. The model utilizes only convolutional, pooling, activation, normalization, and deconvolutional layers. For example, convolutional layers are generated by mapping onto regions of the previous layer, while activation layers correspond one-to-one with the corresponding coordinates of the previous layer. Pooling layers serve a similar purpose and are typically used for downsampling to reduce the model's computational workload. However, experiments have shown that while more downsampling improves model performance, the accuracy of damage point identification decreases dramatically. One reason for this is that pixels representing damage points often cover a limited area. Excessive downsampling results in a larger coverage area for the damage point, resulting in the loss of details and other features near the damage point. Experiments have shown that a cumulative downsampling factor of 4 is optimal for the current network architecture.

[0054] In this embodiment, the structure used is mainly composed of the following main modules, namely, the common convolution structure PC, the upsampling convolution structure UC, the vertical convolution structure HC and the horizontal convolution structure WC. Figure 2 As shown in the figure, the ordinary convolution structure is mainly composed of convolution and pooling operations, which is a module for downsampling images; the upsampling module contains an upsampling convolution layer, which can expand the plane dimension of feature attributes; the vertical convolution structure HC module is composed of an average convolution layer and a convolution layer for feature extraction, and finally uses a concat layer to merge the features. The use of an average pooling layer can reduce the amount of calculation; the horizontal convolution WC extracts features in the horizontal direction, which can extract and analyze the horizontal features of feature attributes.

[0055] The above modules are the main components of the network model. In the process of designing the network, the network model used is as follows Figure 3 As shown: the images enter the PC structure for downsampling in turn. Except for the first one that does not use the BN normalization layer after the PC structure, the other PC structures all use the BN structure for normalization. During the experiment, we found that using the BN layer makes the model converge better; after the above steps, the horizontal dimensions of the feature attributes extracted by the model are: / 2, / 4, / 8, / 16, / 32 relative to the original image; then the last two feature attribute maps are upsampled to obtain feature maps with a data dimension of / 4 relative to the original image, and then the concat layer is used to merge the channels of these three feature attributes. The merged features are sent to the HC, PC, and WC modules for feature extraction at the same time, and the extracted features are still merged by channels; the merged feature maps are sent to the two PC modules in turn, where the stride of the PC module is set to 1, indicating that the plane dimension of the feature attributes is not changed. Finally, the feature attributes are predicted. The prediction is a structure relative to the original image / 4. At this time, the L2 Loss function is used to train the model. The final output will use the activation function to classify the prediction results to obtain a probability map with 2 channels, representing the horizontal and vertical heat maps respectively.

[0056] This embodiment further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the method described in any one of the above technical solutions when running.

[0057] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method described in any one of the above technical solutions.

[0058] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for detecting shed damage, characterized in that: The following steps are involved: S1. Get the picture; S2, obtaining the regional coordinates of the shed; S3. Extracting the shed image and performing damage prediction to obtain a heat map of shed damage, specifically: performing feature extraction on the shed image and obtaining a heat map of shed damage points in the shed image using a shed prediction model; The shed prediction model is trained in advance, specifically including: Make labels: mark the sample images and generate peak maps of shed damage as labels. The peak maps include peak probability maps of damage points and peak probability maps of backgrounds. The pixel values of the peak maps are marked in the range of 0 to 1. The closer to the damage point, the larger the pixel value; Training model: Two-layer loss function is used for training; S4. Generate a peak map in the heat map, filter out points with peak values greater than a threshold, and obtain corresponding damage points of the shed skirt; The peak probability of any point p in the peak graph conforms to the following formula Among them, j represents the intersection point, p represents a point in the peak graph, l k Represents the intersection point in the image that has the greatest impact on point p, Dist(p,l k ) represents the distance between point p and point l k The distance between points; σ represents the decay rate of the control peak probability.

2. The method for detecting shed damage according to claim 1, wherein: Acquiring the regional coordinates of the shed specifically includes: using a target detection algorithm to detect the area where the shed is located, wherein the target detection algorithm uses a convolutional network to predict the target and obtain the position of the target.

3. The method for detecting shed damage according to claim 1, wherein: The training model specifically includes inputting a sample feature map, using σ as a transitivity parameter to measure the peak value, predicting the pixels in the sample feature map, outputting an activation function tanh(x) with a linear property between 0 and 1, and using the Euclidean distance as the loss function; in, Loss Function Where x and y represent the horizontal and vertical coordinates on the heat map respectively. The value of the point with x and y coordinates on the heat map representing the damage point. Indicates the label point corresponding to the label heat map, l, w, h are the length, width, and height of the data respectively.

4. The method for detecting shed damage according to any one of claims 1 to 3, characterized in that: The umbrella skirt prediction model includes a normal convolutional network, a deconvolutional network, a vertical convolutional network and a horizontal convolutional network, wherein the normal convolutional network downsamples the image, the deconvolutional network expands the plane dimension of the feature attributes, the vertical convolutional network extracts features from the vertical direction, and the horizontal convolutional network extracts features from the horizontal direction.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 4 when executed.

6. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 4.

Images