A power inspection image detection method, device and storage medium

Abstract

Embodiments of the present application disclose a power inspection image detection method, device and storage medium. The method comprises: acquiring a to-be-detected power inspection image, and performing at least four down-sampling feature extractions on the to-be-detected power inspection image to obtain corresponding image feature maps; performing feature fusion on the image feature maps to obtain fusion feature maps corresponding to the respective down-sampling; and determining a detection result of a power equipment in the to-be-detected power inspection image according to the fusion feature maps and a neural network model, wherein a prediction network in the neural network model comprises at least three prediction detection heads, each prediction detection head comprises at least three head feature extraction modules, and each head feature extraction module corresponds to a corresponding prediction result. Through the above technical solution, embodiments of the present application can detect both large targets and small targets in the to-be-detected power inspection image, improve the precision and accuracy of the detection of the power equipment with defects in the to-be-detected power inspection image, and have good real-time performance.

Description

Technical Field

[0001] This disclosure relates to the field of power grid operation, maintenance and repair technology, and in particular to a power grid inspection image detection method, equipment and storage medium. Background Technology

[0002] Image object detection is one of the fundamental tasks of computer vision, playing a crucial role in practical applications such as industrial inspection and facial recognition. In power equipment operation and maintenance, a large amount of inspection and monitoring video is acquired through inspection drones, inspection robots, and surveillance cameras, requiring the identification and location of equipment defects. Due to long shooting distances, poor viewing angles, and severe occlusion, defect areas often occupy a small proportion of the image, resulting in unclear imaging, incomplete structures, and small-scale characteristics, making it difficult to achieve high-precision detection using conventional object detection methods.

[0003] Object detection is typically modeled as the classification and regression of candidate regions. Depending on how candidate regions are generated, it is divided into two-stage detection and one-stage detection. In the former, candidate regions are predicted using a Region Proposal Network (RPN), while in the latter, predictions are generated by producing anchors of different sizes and aspect ratios at each location on a specific feature map. In this detection architecture, to regress more accurate object information, a detailed list of potential object locations needs to be compiled and the objects classified. However, object detectors consume significant computational resources during enumeration and classification. Furthermore, existing object detection methods typically perform object detection on low-resolution feature maps to obtain object location and category information. However, low-resolution feature maps lose spatial information and detailed features of the object, resulting in poor performance for small objects. Summary of the Invention

[0004] In view of this, the present invention provides a power inspection image detection method, device and storage medium that can take into account both large and small targets in the power inspection image to be detected. Without losing the detailed features of the target, it improves the accuracy and precision of detecting defective power equipment in the power inspection image to be detected, has good real-time performance and is more suitable for power inspection image target detection tasks.

[0005] According to one aspect of the present invention, an embodiment of the present invention provides a power line inspection image detection method, the method comprising:

[0006] Acquire the power inspection image to be detected, and extract the corresponding image feature map by performing at least four downsampling features on the power inspection image to be detected;

[0007] The image feature maps are fused to obtain the fused feature maps corresponding to each downsampling;

[0008] The detection results of power equipment in the power inspection image to be detected are determined based on the fused feature maps and the pre-trained neural network model. The prediction network in the neural network model includes at least three prediction detection heads, each of which includes at least three head feature extraction modules. Each head feature extraction module corresponds to a corresponding prediction result. Each prediction result includes: the center point position of the target equipment in the power inspection image to be detected, the bounding box width and height information, the center point offset information, the defect status of the target equipment, and the equipment type.

[0009] According to another aspect of the present invention, embodiments of the present invention also provide a power line inspection image detection device, the device comprising:

[0010] The downsampling feature extraction module is used to acquire the power inspection image to be detected and extract at least four downsampling features from the power inspection image to obtain the corresponding image feature map.

[0011] The feature fusion module is used to fuse the feature maps of each image to obtain the fused feature map corresponding to each downsampling.

[0012] The detection module is used to determine the detection result of the power equipment in the power inspection image to be detected based on the fused feature maps and the pre-trained neural network model. The prediction network in the neural network model includes at least three prediction detection heads, each prediction detection head includes at least three head feature extraction modules, each head feature extraction module corresponds to a corresponding prediction result, and each prediction result includes: the center point position of the target equipment in the power inspection image to be detected, the bounding box width and height information, the center point offset information, the defect status of the target equipment, and the equipment type.

[0013] According to another aspect of the present invention, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the power line inspection image detection method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the power line inspection image detection method according to any embodiment of the present invention.

[0018] The technical solution of this invention extracts downsampled feature maps at different scales and performs feature fusion to obtain fused feature maps corresponding to each downsampled feature map. Then, it uses at least three prediction detection heads of a neural network model to perform target detection on the fused feature maps at different resolutions. Each prediction detection head includes at least three head feature extraction modules, which can take into account both large and small targets in the power inspection image to be detected. Without losing the detailed features of the target, it improves the accuracy and precision of detecting defective power equipment in the power inspection image to be detected. It has good real-time performance and is more suitable for target detection tasks in power inspection images.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of a power line inspection image detection method provided in an embodiment of the present invention;

[0022] Figure 2 A flowchart of another power inspection image detection method provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of a prediction detection head in a neural network model according to an embodiment of the present invention;

[0024] Figure 4 This is an overall architecture diagram of a deep neural network in a power line inspection image detection method according to an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of a drag-and-drop annotation method for creating a label file, provided by an embodiment of the present invention.

[0026] Figure 6This is a schematic diagram illustrating the visualization result of overlaying a heat map of a target center point with the original image, as provided in an embodiment of the present invention.

[0027] Figure 7 This is a structural block diagram of a power line inspection image detection device according to an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] In one embodiment, Figure 1 This is a flowchart of a power inspection image detection method according to an embodiment of the present invention. This embodiment is applicable to the detection of defective power equipment in power inspection images. The method can be executed by a power inspection detection device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0032] like Figure 1 As shown, the power line inspection image detection method in this embodiment includes the following specific steps:

[0033] S110. Obtain the power inspection image to be detected, and extract the corresponding image feature map by performing at least four downsampling features on the power inspection image to be detected.

[0034] Among them, the power inspection images to be inspected refer to the inspection images that need to be inspected for power equipment. Of course, there may be one or more defective power equipment in the power inspection images to be inspected, or there may be no defective power equipment.

[0035] In this embodiment, downsampling can also be called downsampling, which can be understood as reducing the size of the power inspection image to be detected by a certain factor. In this embodiment, the power inspection image to be detected can be downsampled at least four times for feature extraction. The resolution after each downsampling is different. Downsampling can make the image conform to the size of the display area to obtain a corresponding thumbnail.

[0036] In this embodiment, the backbone network of the neural network model may include at least four parallel first feature extraction modules with different downsampling scales. Each first feature extraction module is equipped with a corresponding channel attention mechanism and a first adaptive feature fusion module. Then, the power inspection image to be detected is subjected to 4x downsampling feature extraction, 8x downsampling feature extraction, 16x downsampling feature extraction, and 32x downsampling feature extraction respectively to obtain the corresponding first image feature map. By adaptively assigning different weights to different channels of the feature map and performing corresponding adaptive feature fusion, the 4x downsampling image feature map, 8x downsampling image feature map, and 16x downsampling image feature map are obtained. After the corresponding feature extraction, the 4x downsampling image feature map, 8x downsampling image feature map, and 16x downsampling image feature map are passed to the neck network part of the neural network. In some embodiments, the power inspection image to be detected can also be transformed by corresponding conversion formulas to obtain the corresponding image feature map. This embodiment does not limit this.

[0037] S120. Perform feature fusion on each image feature map to obtain the fused feature map corresponding to each downsampling.

[0038] The fusion feature map can be understood as the fusion feature map obtained by adaptively fusing downsampled feature images at different scales. It should be noted that the fusion feature map may include, but is not limited to, the first fusion feature map corresponding to 4x downsampling, the second fusion feature map corresponding to 8x downsampling, and the third fusion feature map corresponding to 16x downsampling.

[0039] In this embodiment, the neck network of the neural network model includes a second adaptive feature fusion module with at least three parallel branches. This module performs a second adaptive weighted fusion on the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map, respectively, to obtain a first fused feature map corresponding to the 4x downsampled image, a second fused feature map corresponding to the 8x downsampled image, and a third fused feature map corresponding to the 16x downsampled image. This can be understood as setting corresponding weights for the image feature maps with different downsampled values, adding these weights together to obtain the fused image features. These fused feature maps form multiple branches and are transmitted to the prediction and detection head of the neural network model for subsequent processing. In some embodiments, a transformation algorithm can be used to decompose the image feature maps with different downsampled values ​​into high and low frequency coefficients. Then, different fusion strategies are applied to different coefficients, and fusion is completed hierarchically and directionally. Finally, an inverse transformation is used to achieve image fusion to obtain the fused feature maps corresponding to each downsampled image.

[0040] S130. Determine the detection results of power equipment in the power inspection image to be detected based on each fused feature map and the pre-trained neural network model.

[0041] In this embodiment, the prediction network in the neural network model includes at least three prediction detection heads. Each prediction detection head includes at least three head feature extraction modules, and each head feature extraction module corresponds to a specific prediction result. Each prediction result includes: the center point position of the target device in the power inspection image to be detected, the bounding box width and height information, the center point offset information, the defect status of the target device, and the device type. The target device can be understood as one or more power devices in the power inspection image to be detected that are defective or damaged.

[0042] In this embodiment, the output feature maps of the three branches of the neck network of the neural network model are respectively passed to at least three prediction detection heads for processing and prediction. Each prediction detection head includes at least three head feature extraction modules, and each head feature extraction module outputs a corresponding prediction result. In this embodiment, each fused feature map can be input into the corresponding head feature extraction module in the prediction detection head of the neural network to obtain the detection result corresponding to the fused feature map. Since there are multiple detection results, the non-maximum suppression method can be used to perform deduplication on each detection result to determine the target detection result with the highest probability among the detection results. The target detection result is used as the detection result of the power equipment in the power inspection image to be detected. In some embodiments, each fused feature map can also be converted into a one-dimensional vector fused feature through the global pooling layer of the neural network model. The diagnosis result of the power grid line defect is obtained by multiplying the optimal parameters of the convolution kernel of the fully connected layer in the neural network model with the one-dimensional vector fused feature. This embodiment does not impose any restrictions here.

[0043] The technical solution of this invention extracts downsampled feature maps at different scales and performs feature fusion to obtain fused feature maps corresponding to each downsampled feature map. Then, it uses at least three prediction detection heads of a neural network model to perform target detection on the fused feature maps at different resolutions. Each prediction detection head includes at least three head feature extraction modules, which can take into account both large and small targets in the power inspection image to be detected. Without losing the detailed features of the target, it improves the accuracy and precision of detecting defective power equipment in the power inspection image to be detected. It has good real-time performance and is more suitable for target detection tasks in power inspection images.

[0044] In one embodiment, the training process of the neural network model includes:

[0045] A training sample set of power inspection images is obtained, wherein the training sample set includes: a first set of power inspection images containing at least one equipment defect, and a second set of power inspection images not containing equipment defects; the first power inspection images in the first set of power inspection images correspond to corresponding first tag files; the second power inspection images in the second set of power inspection images correspond to corresponding second tag files; wherein the first tag file includes at least: coordinate information, width and height information, equipment type and defect details of the defective equipment; the first tag file is an empty file;

[0046] Multi-scale downsampling feature extraction is performed on the power inspection images corresponding to the first power inspection image set and the second power inspection image set to obtain the corresponding image feature maps;

[0047] Feature fusion is performed on the feature maps of each image to obtain fused feature maps downsampled at different scales;

[0048] The neural network model is trained using the backpropagation algorithm, the classification results of the fused feature maps obtained by the neural network model, and the label results until the loss function of the neural network model reaches its minimum, thus obtaining a trained power inspection image target detection neural network, which is then used to detect the power inspection image to be detected.

[0049] In this embodiment, a dataset is prepared for network training. The training data is divided into independent training, validation, and test sets. During the image preprocessing stage, if the amount of image data is insufficient, data augmentation operations can be performed on the power inspection images using one or more of the following image processing methods: random flipping, random cropping, image quality adjustment, adversarial processing, image blending, or adding white noise.

[0050] In this embodiment, before network training, it is necessary to set the hyperparameters for network training and initialize the network's backbone, neck, and prediction head. To ensure a smoother decrease in network loss during the early stages of training, pre-trained weights can be imported into the backbone feature extraction network. When network training begins, images from the training set are fed into the network in batches for training. The backpropagation algorithm is used to minimize the training set loss and optimize the model parameters. During training, at regular intervals, the training effect of the current network is validated using validation set data, facilitating the adjustment of the network model's hyperparameters and ensuring that the network does not suffer from severe overfitting. After several rounds of training, the loss curves of the training and validation sets tend to stabilize, training ends, and the network model file is saved.

[0051] It should be noted that, since the downsampling rates of the feature maps of each prediction head are different, labels corresponding to the three feature maps with different resolutions were set when training the three feature maps.

[0052] In this embodiment, a training sample set of power inspection images is obtained, wherein the training sample set includes: a first power inspection image set containing at least one equipment defect, and a second power inspection image set not containing equipment defects; the first power inspection image in the first power inspection image set corresponds to a corresponding first tag file; the second power inspection image in the second power inspection image set corresponds to a corresponding second tag file; wherein the first tag file includes at least: coordinate information, width and height information of the defective equipment, type of power equipment, and defect category; the first tag file is an empty file; multi-scale downsampling feature extraction is performed on the power inspection images corresponding to the first power inspection image set and the second power inspection image set respectively to obtain corresponding image feature maps; feature fusion is performed on each image feature map to obtain fused feature maps downsampled at different scales; the neural network model is trained by backpropagation algorithm, the classification results of each fused feature map obtained by the neural network model, and the tag results until the loss function of the neural network model reaches a minimum, to obtain a trained power inspection image target detection neural network, so as to detect the power inspection image to be detected by the trained power inspection image target detection neural network.

[0053] In some embodiments, the configuration of the tag file includes: when tagging a first power inspection image, drawing bounding boxes to label at least one defective device in the power inspection image to obtain bounding box labeling results for the defective device, wherein the bounding box labeling results include the coordinates of the upper left corner and lower right corner of the defective device within the bounding box, the width and height information of the defective device, the device type of the defective device, and the defect classification; determining the center point information of the defective device based on the bounding box labeling results, and determining a center point heatmap based on the center point information using a Gaussian distribution; wherein, in the center point heatmap, the brightness value is greater the closer to the center point; determining the center point offset information based on the center point heatmap; using the center point heatmap, center point offset information, width and height information, device type of the defective device, and defect classification as a first tag file; when tagging a second power inspection image, directly labeling the second tag file corresponding to the second power inspection image as an empty file.

[0054] In one embodiment, the method further includes:

[0055] During the training of the neural network model, the training effect is verified using a test sample set of power line inspection images every preset number of iterations. The verification of the neural network model includes:

[0056] Obtain a test sample set of power line inspection images;

[0057] The trained power inspection image target detection neural network was tested based on preset target detection evaluation indicators and power inspection images in the test sample set.

[0058] If the preset target detection evaluation index achieves the expected results, the trained power inspection image target detection neural network is used to test the power inspection image to be detected.

[0059] If the preset target detection evaluation index does not achieve the expected results, the pre-trained weights and hyperparameters of the pre-configured neural network are adjusted, and the power inspection image target detection neural network is retrained until the preset target detection evaluation index achieves the expected results.

[0060] In one embodiment, during the training of the neural network model, the training effect of the neural network model is verified every preset number of iterations using a test sample set of power inspection images. During the verification process, it is necessary to obtain a test sample set of power inspection images; test the trained power inspection image target detection neural network based on preset target detection evaluation indicators and the power inspection images in the test sample set; if the preset target detection evaluation indicators achieve the expected results, then the trained power inspection image target detection neural network is used to test the power inspection images to be detected; if the preset target detection evaluation indicators do not achieve the expected results, then the pre-configured pre-training weights and hyperparameters of the neural network are adjusted, and the power inspection image target detection neural network is retrained until the preset target detection evaluation indicators achieve the expected results. This can be understood as follows: During the network testing phase, the trained network model is imported and tested on a test set. Commonly used evaluation metrics in object detection tasks (such as map (mean accuracy, precision evaluation), accuracy, confusion matrix, precision, recall, average accuracy (AP), etc.) are used to evaluate the network output results. If the expected results are not achieved, the network hyperparameters are adjusted, and retraining is performed until the expected results are met. If the expected results are achieved, the network is deployed on the target computing device for application in real-world scenarios.

[0061] In one embodiment, Figure 2 This is a flowchart of another power inspection image detection method provided in an embodiment of the present invention. Based on the above embodiments, this embodiment further refines the process by extracting at least four downsampling features from the power inspection image to be detected to obtain corresponding image feature maps; fusing the image feature maps to obtain fused feature maps corresponding to each downsampling; and determining the detection results of power equipment in the power inspection image to be detected based on each fused feature map and a pre-trained neural network model.

[0062] like Figure 2 As shown, the power line inspection image detection method in this embodiment may specifically include the following steps:

[0063] S210. Obtain the power inspection image to be detected, and perform 4x downsampling feature extraction, 8x downsampling feature extraction, 16x downsampling feature extraction and 32x downsampling feature extraction on the power inspection image to be detected through each first feature extraction module to obtain the corresponding first image feature map.

[0064] In this embodiment, the backbone network of the neural network model includes at least four parallel first feature extraction modules with different downsampling scales. Each first feature extraction module is equipped with a corresponding channel attention mechanism and a first adaptive feature fusion module. The first feature extraction modules can extract features from the power inspection image to be detected by performing 4x downsampling, 8x downsampling, 16x downsampling, and 32x downsampling, respectively, to obtain the corresponding first image feature maps. This can be understood as the backbone network using a high-resolution convolutional neural network for basic feature extraction, which has four parallel branches. Different branches extract features downsampled at different scales. The length and width of the feature maps are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the length and width of the input image, respectively. In this embodiment, the image resolution is different for different downsampling scales. For example, the feature information of the 4x downsampling feature map is 40*40 pixels in length and width, with 128 channels; the 8x downsampling feature map is 80*80*64 pixels; and the 16x downsampling feature map is 160*160*32 pixels.

[0065] S220. Each first image feature map is assigned a corresponding weight value through the channel attention mechanism set by each first feature extraction module, and the first adaptive feature fusion module performs adaptive feature fusion on each first image feature map with the assigned weight value to obtain a 4x downsampled image feature map, an 8x downsampled image feature map and a 16x downsampled image feature map.

[0066] In this embodiment, the HRNet network is composed of stacked feature extraction modules. In order to fully extract the features of targets at different scales in the input image, each HRNet branch will stack different numbers of first feature extraction modules at different stages. Since different channels of the feature map contribute differently to the effective features of the network, a channel attention mechanism can be added after each feature extraction module to adaptively assign different weights to different channels of the feature map. That is, the channel attention mechanism set by each first feature extraction module assigns corresponding weight values ​​to each first image feature map, and the first adaptive feature fusion module performs adaptive feature fusion on each first image feature map with corresponding weight values ​​to obtain 4x downsampled image feature map, 8x downsampled image feature map and 16x downsampled image feature map.

[0067] S230. Use the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map as the input image feature map of the neck network in the neural network.

[0068] In this embodiment, the backbone network passes the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map to the neck network in the neural network. The neck network includes a second adaptive feature fusion module with at least three parallel branches, which can adaptively fuse the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map in parallel.

[0069] S240. The second adaptive feature fusion module adaptively weights and fuses the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map to obtain the first fused feature map corresponding to the 4x downsampled image, the second fused feature map corresponding to the 8x downsampled image, and the third fused feature map corresponding to the 16x downsampled image.

[0070] In this embodiment, the neck network of the neural network model includes a second adaptive feature fusion module with at least three parallel branches; the feature maps of each image are fused to obtain fused feature maps corresponding to different downsampling scales. Specifically, the second adaptive feature fusion module adaptively weights and fuses the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map to obtain the first fused feature map corresponding to the 4x downsampling, the second fused feature map corresponding to the 8x downsampling, and the third fused feature map corresponding to the 16x downsampling.

[0071] In this embodiment, the output of the backbone network is passed to the neck network with three branches. The three branches respectively use the second adaptive feature fusion module to adaptively weight and fuse the feature maps of the backbone network output by 4x downsampling, 8x downsampling and 16x downsampling before outputting them to form multi-scale feature output.

[0072] S250. Input the first fused feature map into the head feature extraction modules of the corresponding first prediction detection head in the neural network to obtain the first detection result corresponding to the first fused feature map.

[0073] The first prediction detection head serves as the prediction head for the first fused feature map, and includes a head feature extraction module with at least three branches. The first detection result is the detection result obtained by inputting the first fused feature map into the first prediction detection head.

[0074] In this embodiment, each prediction detection head in the neural network model includes a head feature extraction module with at least three branches. Each head feature extraction module consists of two convolutional layers, and batch normalization is performed after each convolutional layer. The output feature maps of the three branches of the neck network are respectively passed to the three prediction detection heads for processing and prediction. In this embodiment, the first fused feature map is input into the head feature extraction modules of the corresponding first prediction detection head in the neural network to obtain the first detection result corresponding to the first fused feature map. This result may include the center point position of the target device, the bounding box width and height, the center point offset, the defect status of the target device, and the device type. Of course, the device type can be understood as the device category of the defective device, which may include, but is not limited to, insulators, surge arresters, etc. For example, in an image, there is a device tower with many insulators, one of which has a defect. The device type is an insulator. The center point position, bounding box width and height, and center point offset of the defective insulator can be obtained.

[0075] In this embodiment, the center point is predicted on the feature map channel corresponding to each target category. The predicted center point and its vicinity are highlighted, while the brightness values ​​of other locations approach 0, forming a heat map. The number of channels in the heat map is the same as the number of target categories to be detected. The length and width of the target bounding box are predicted on the feature map of two channels, respectively, and the output value of the neuron at each pixel in the feature map is used as the predicted value. The offsets of each target center point in the X and Y directions are predicted on the feature map of two channels, respectively, and the output value of the neuron at each pixel in the feature map is used as the predicted value.

[0076] S260. Input the second fused feature map into the head feature extraction modules of the corresponding second prediction detection head in the neural network to obtain the second detection result corresponding to the second fused feature map.

[0077] The second prediction detection head serves as the prediction head for the second fused feature map. It includes a head feature extraction module with at least three branches, each consisting of two convolutional layers, followed by batch normalization. The second detection result is the result obtained by inputting the second fused feature map into the second prediction detection head.

[0078] In this embodiment, the second fused feature map is input into the head feature extraction modules of the corresponding second prediction detection head in the neural network to obtain the second detection result corresponding to the second fused feature map, which may include the center point position of the target device, the width and height of the bounding box, the center point offset, the defect status of the target device, and the device type.

[0079] S270. Input the third fusion feature map into the corresponding head feature extraction module of the third prediction detection head in the neural network to obtain the third detection result corresponding to the third fusion feature map.

[0080] The third prediction detection head serves as the prediction head for the third fused feature map. It includes a head feature extraction module with at least three branches, each consisting of two convolutional layers, followed by batch normalization. The third detection result is the result obtained by inputting the third fused feature map into the third prediction detection head.

[0081] In this embodiment, the third fused feature map is input into the corresponding third prediction detection head's head feature extraction modules in the neural network to obtain the third detection result corresponding to the third fused feature map. This result may include the center point position of the target device, the bounding box width and height, the center point offset, the defect status of the target device, and the device type. The defect status includes: the power equipment has a defect and the power equipment does not have a defect.

[0082] S280. The non-maximum suppression method is used to perform deduplication on the first detection result, the second detection result, and the third detection result to determine the target detection result with the highest probability among all detection results.

[0083] In this embodiment, a non-maximum suppression method is used to perform deduplication on the first detection result, the second detection result, and the third detection result to determine the target detection result with the highest probability among all detection results, and the target detection result is used as the detection result of the power equipment in the power inspection image to be detected.

[0084] S290. Use the target detection result as the detection result of the power equipment in the power inspection image to be detected.

[0085] In one embodiment, to facilitate a better understanding of the prediction detection head structure, Figure 3 This is a schematic diagram of the structure of a prediction detection head in a neural network model according to an embodiment of the present invention, as shown below. Figure 3 As shown, the fused feature map with 8x downsampling is input into the prediction detection head, and each head feature extraction module in the prediction detection head corresponds to a prediction result.

[0086] The technical solution of this invention involves using first feature extraction modules to extract features from the power inspection image to be detected by 4x downsampling, 8x downsampling, 16x downsampling, and 32x downsampling respectively, resulting in corresponding first image feature maps. Each first feature extraction module assigns a corresponding weight value to each first image feature map using a channel attention mechanism. A first adaptive feature fusion module then adaptively fuses these weighted first image feature maps to obtain 4x downsampling, 8x downsampling, and 16x downsampling image feature maps. Finally, a second adaptive feature fusion module combines the 4x downsampling, 8x downsampling, and 16x downsampling image feature maps. The sampled image feature maps are adaptively weighted and fused to obtain a first fused feature map corresponding to 4x downsampling, a second fused feature map corresponding to 8x downsampling, and a third fused feature map corresponding to 16x downsampling. Each prediction detection head includes a head feature extraction module with at least three branches, and each head feature extraction module consists of two convolutional layers. Based on each fused feature map and the corresponding prediction detection head, the detection result of the power inspection image to be detected is determined. This can further take into account the detection of large and small targets in the power inspection image to be detected. Without losing the target detail features, it improves the accuracy and precision of detecting defective power equipment in the power inspection image to be detected. It has good real-time performance and is more suitable for target detection tasks in power inspection images.

[0087] In one embodiment, to facilitate a better understanding of the overall architecture of the neural network for target detection in power line inspection images, Figure 4 This diagram illustrates the overall architecture of a deep neural network in a power line inspection detection method according to an embodiment of the present invention. The target detection neural network for power line inspection images consists of three parts: a backbone feature extraction network, a neck network, and a prediction head. It constructs a parallel multi-branch prediction structure, where each branch can extract features from feature maps of different resolutions at the network neck and detect targets on feature maps of different resolutions at the network head. Furthermore, the present invention adds a channel attention mechanism and an adaptive feature fusion module to the backbone feature extraction network to further enhance the extraction capability of key features. Compared to existing target detection networks, the power line inspection image target detection neural network proposed in this invention has higher accuracy in small target detection and better real-time performance, making it more suitable for power line inspection image target detection tasks.

[0088] In this embodiment, the backbone network uses a high-resolution convolutional neural network for basic feature extraction, and this network has four parallel branches. Different branches extract features at different scales, with the feature maps having lengths and widths of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image's length and width, respectively. After four stages of feature extraction, the output feature maps are passed to the neck network.

[0089] In this embodiment, to fully extract features of targets at different scales in the input image, each HRNet branch stacks different numbers of feature extraction modules at different stages, and fuses feature maps of different scales at the end of the second, third, and fourth stages. This invention replaces the feature fusion structure at the connection between the third and fourth stages in HRNet with an adaptive feature fusion module. Simultaneously, since different channels of the feature map contribute differently to the network's effective features, a channel attention mechanism is introduced, adaptively assigning different weights to different channels of the feature map. This channel attention mechanism is added after each feature extraction module.

[0090] In this embodiment, the output of the backbone network is passed to the neck network, which has three branches. These three branches use an adaptive feature fusion module to adaptively weight and fuse the feature maps downsampled by 4x, 8x, and 16x from the backbone network before outputting them, forming a multi-scale feature output.

[0091] In this embodiment, the output feature maps of the three branches of the neck network are respectively passed to three prediction heads for processing and prediction. Each prediction head includes three branches, and each branch has a head feature extraction module that processes the input feature map. Each head feature extraction module consists of two convolutional layers. At the end of the three branches, the center point position, bounding box width and height, and center point offset of the target to be detected are predicted based on the input feature map, thereby obtaining the predicted target bounding box.

[0092] In one embodiment, to facilitate a better understanding of label creation during neural network model training, Figure 5 This is a schematic diagram of a drag-and-drop annotation method for creating a label file, provided by an embodiment of the present invention. Figure 6 This is a schematic diagram illustrating the visualization result of overlaying a heatmap of a target center point with the original image, as provided in an embodiment of the present invention. Figure 5 and Figure 6 As shown, the coordinates of the center point of the target bounding box are calculated based on the coordinates of the top-left and bottom-right corners of the bounding box in the bounding box annotation results. A center point heatmap is generated based on a Gaussian distribution, with the brightness value increasing the closer to the center point. Figure 6As shown, a width and height information label for the observed target is generated. This label has the same length and width as the input image and two channels, storing the width and height information of the target bounding box, respectively. If a pixel on the label is not the target's center point, the width and height values ​​stored for that pixel are 0; if it is the target's center point, the width and height of the bounding box stored for that pixel are consistent with the target's width and height. A center point offset information label is generated. This label has the same length and width as the input image and two channels, storing the X and Y direction offset information of the target bounding box's center point, respectively. The center point offset is used to account for the accuracy loss caused by rounding when the label is downsampled to the same resolution as the network prediction image. If a pixel on the label is not the target's center point, the X and Y direction offset values ​​stored for that pixel are 0; if it is the target's center point, the offset value stored for that pixel is the target's center point position offset.

[0093] In one embodiment, Figure 7 This is a structural block diagram of a power line inspection image detection device according to an embodiment of the present invention. This device is suitable for detecting power equipment in power line inspection images and can be implemented in hardware or software. It can be configured in an electronic device to implement a power line inspection detection method according to an embodiment of the present invention.

[0094] like Figure 7 As shown, the device includes: a downsampling feature extraction module 710, a feature fusion module 720, and a detection module 730.

[0095] The downsampling feature extraction module 710 is used to acquire the power inspection image to be detected and to extract at least four downsampling features from the power inspection image to be detected to obtain the corresponding image feature map.

[0096] The feature fusion module 720 is used to fuse the feature maps of each image to obtain the fused feature map corresponding to each downsampling;

[0097] The detection module 730 is used to determine the detection result of the power equipment in the power inspection image to be detected based on the fused feature maps and the pre-trained neural network model. The prediction network in the neural network model includes at least three prediction detection heads, each prediction detection head includes at least three head feature extraction modules, each head feature extraction module corresponds to a corresponding prediction result, and each prediction result includes: the center point position of the target equipment in the power inspection image to be detected, the bounding box width and height information, the center point offset information, the defect status of the target equipment, and the equipment type.

[0098] In this embodiment of the invention, a downsampling feature extraction module extracts downsampling feature maps at different scales; a feature fusion module performs feature fusion to obtain fused feature maps corresponding to each downsampling; and a detection module uses at least three prediction detection heads of a neural network model to perform target detection on the fused feature maps at different resolutions. Each prediction detection head includes at least three head feature extraction modules, which can take into account both large and small targets in the power inspection image to be detected. Without losing the detailed features of the target, it improves the accuracy and precision of detecting defective power equipment in the power inspection image to be detected, has good real-time performance, and is more suitable for target detection tasks in power inspection images.

[0099] In one embodiment, the backbone network of the neural network model includes at least four parallel first feature extraction modules with different downsampling scales, each first feature extraction module having a corresponding channel attention mechanism, and a first adaptive feature fusion module; the downsampling feature extraction module 710 includes:

[0100] The extraction unit is used to extract the power inspection image to be detected by performing 4x downsampling feature extraction, 8x downsampling feature extraction, 16x downsampling feature extraction and 32x downsampling feature extraction respectively through each of the first feature extraction modules to obtain the corresponding first image feature map;

[0101] The determining unit is used to assign corresponding weight values ​​to each of the first image feature maps through the channel attention mechanism set by each of the first feature extraction modules, and to perform adaptive feature fusion on each of the first image feature maps with corresponding weight values ​​through the first adaptive feature fusion module to obtain image feature maps with 4x downsampling, 8x downsampling, and 16x downsampling.

[0102] The input feature map determination unit is used to use the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map as the input image feature map of the neck network in the neural network.

[0103] In one embodiment, the neck network of the neural network model includes a second adaptive feature fusion module with at least three parallel branches; the feature fusion module 720 includes:

[0104] The fusion unit is used to adaptively weight and fuse the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map respectively through the second adaptive feature fusion module to obtain the first fused feature map corresponding to the 4x downsampled image, the second fused feature map corresponding to the 8x downsampled image, and the third fused feature map corresponding to the 16x downsampled image.

[0105] In one embodiment, each of the prediction detection heads includes a head feature extraction module with at least three branches, each head feature extraction module consisting of two convolutional layers, and the detection module 730 includes:

[0106] The first detection unit is used to input the first fused feature map into each head feature extraction module in the corresponding first prediction detection head of the neural network to obtain the first detection result corresponding to the first fused feature map;

[0107] The second detection unit is used to input the second fused feature map into each head feature extraction module in the corresponding second prediction detection head of the neural network to obtain the second detection result corresponding to the second fused feature map;

[0108] The third detection unit is used to input the third fused feature map into the head feature extraction modules of the corresponding third prediction detection head in the neural network to obtain the third detection result corresponding to the third fused feature map; wherein, the first detection result, the second detection result, and the third detection result respectively include: the center point position of the target equipment in the power inspection image to be detected, the bounding box width and height information, the center point offset information, the defect status of the target equipment, and the equipment type; wherein, the defect status includes: the power equipment has equipment defects and the power equipment does not have equipment defects;

[0109] The first determining unit is used to perform a deduplication operation on the first detection result, the second detection result and the third detection result using a non-maximum suppression method, so as to determine the target detection result with the highest probability among the detection results.

[0110] The second determining unit uses the target detection result as the detection result of the power equipment in the power inspection image to be detected.

[0111] In one embodiment, the training process of the neural network model includes:

[0112] A training sample set of power line inspection images is obtained, wherein the training sample set includes: a first set of power line inspection images containing at least one equipment defect, and a second set of power line inspection images not containing equipment defects; the first power line inspection images in the first set of power line inspection images correspond to a corresponding first tag file; the second power line inspection images in the second set of power line inspection images correspond to a corresponding second tag file; wherein the first tag file includes at least: the coordinate information, width and height information, equipment type, and defect status of the defective equipment; the first tag file is an empty file;

[0113] Multi-scale downsampling feature extraction is performed on the power inspection images corresponding to the first power inspection image set and the second power inspection image set to obtain the corresponding image feature maps;

[0114] Feature fusion is performed on the image feature maps to obtain fused feature maps downsampled at different scales;

[0115] The neural network model is trained using the backpropagation algorithm, the classification results of the fused feature maps obtained by the neural network model, and the label results until the loss function of the neural network model reaches its minimum, thus obtaining a trained power inspection image target detection neural network, which is then used to detect the power inspection image to be detected.

[0116] In one embodiment, the configuration of the tag file includes:

[0117] When labeling the first power inspection image, at least one defective device in the power inspection image is bounded by a bounding box to obtain the bounding box annotation result of the defective device. The bounding box annotation result includes the coordinate information of the upper left corner and the lower right corner of the defective device within the bounding box, the width and height information of the defective device, the device type and defect classification of the defective device.

[0118] Based on the bounding box annotation results, the center point information of the defective device is determined, and a center point heat map is determined based on the center point information using a Gaussian distribution; wherein, in the center point heat map, the brightness value is greater the closer to the center point;

[0119] The center point offset information is determined based on the center point heat map.

[0120] The center point heat map, the center point offset information, the width and height information, the device type of the defective device, and the defect classification are used as the first tag file;

[0121] When labeling the second power inspection image, the second label file corresponding to the second power inspection image is directly labeled as an empty file.

[0122] After acquiring the training sample set of power line inspection images, the process also includes:

[0123] The power inspection images are subjected to data enhancement operations using one or more of the following image processing methods: random flipping, random cropping, image quality adjustment, adversarial processing, image mixing, or white noise addition.

[0124] In one embodiment, the device further includes:

[0125] A verification unit is used to verify the training effect of the neural network model using a test sample set of power inspection images at preset iteration intervals during the training process of the neural network model. The verification of the neural network model includes:

[0126] Obtain a test sample set of the power inspection images;

[0127] The trained power inspection image target detection neural network is tested based on preset target detection evaluation indicators and power inspection images in the test sample set;

[0128] If the preset target detection evaluation index reaches the expected result, the trained power inspection image target detection neural network is used to test the power inspection image to be detected;

[0129] If the preset target detection evaluation index does not achieve the expected result, the pre-training weights and hyperparameters of the pre-configured neural network are adjusted, and the power inspection image target detection neural network is retrained until the preset target detection evaluation index achieves the expected result.

[0130] The power line inspection image detection device provided in this embodiment of the invention can execute the power line inspection image detection method for financial systems provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0131] In one embodiment, Figure 8 This is a schematic diagram of an electronic device provided for an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0132] like Figure 8As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0133] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0134] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as power line inspection image detection methods.

[0135] In some embodiments, the power line inspection image detection processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the power line inspection image detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the power line inspection image detection method by any other suitable means (e.g., by means of firmware).

[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable power line inspection image detection device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0141] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0142] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0143] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting images during power line inspection, characterized in that, The method includes: Acquire the power inspection image to be detected, and perform downsampling feature extraction on the power inspection image at at least four scales to obtain the corresponding image feature map; The image feature maps are fused to obtain the fused feature maps corresponding to each downsampling; The detection results of power equipment in the power inspection image to be detected are determined based on the fused feature maps and the pre-trained neural network model. The prediction network in the neural network model includes at least three prediction detection heads, each of which includes at least three head feature extraction modules. Each head feature extraction module corresponds to a corresponding prediction result. Each prediction result includes: the center point position of the target equipment in the power inspection image to be detected, the bounding box width and height information, the center point offset information, the defect status of the target equipment, and the equipment type. The backbone network of the neural network model includes at least four parallel first feature extraction modules with different downsampling scales, each first feature extraction module is equipped with a corresponding channel attention mechanism, and a first adaptive feature fusion module; The step of extracting corresponding image feature maps by downsampling the power inspection image to be detected at at least four scales includes: The first feature extraction module performs 4x downsampling feature extraction, 8x downsampling feature extraction, 16x downsampling feature extraction and 32x downsampling feature extraction on the power inspection image to be detected to obtain the corresponding first image feature map. Each first image feature map is assigned a corresponding weight value by the channel attention mechanism set by each first feature extraction module, and the first adaptive feature fusion module performs adaptive feature fusion on each first image feature map with the corresponding weight value to obtain image feature maps at 4x downsampling, 8x downsampling, and 16x downsampling. The 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map are used as the input image feature map for the neck network in the neural network. The neck network of the neural network model includes a second adaptive feature fusion module with at least three parallel branches; the step of fusing the feature maps of each image to obtain the fused feature map corresponding to each downsampling includes: The second adaptive feature fusion module adaptively weights and fuses the 4x downsampled image feature map, the 8x downsampled image feature map, and the 16x downsampled image feature map to obtain the first fused feature map corresponding to the 4x downsampled image, the second fused feature map corresponding to the 8x downsampled image, and the third fused feature map corresponding to the 16x downsampled image.

2. The method according to claim 1, characterized in that, Each of the predicted detection heads includes a head feature extraction module with at least three branches, and each head feature extraction module consists of two convolutional layers. The step of determining the detection result of power equipment in the power inspection image based on the fused feature maps and a pre-trained neural network model includes: The first fused feature map is input into the head feature extraction modules of the corresponding first prediction detection head in the neural network to obtain the first detection result corresponding to the first fused feature map; The second fused feature map is input into the head feature extraction modules of the corresponding second prediction detection head in the neural network to obtain the second detection result corresponding to the second fused feature map; The third fused feature map is input into the head feature extraction modules of the corresponding third prediction detection head in the neural network to obtain the third detection result corresponding to the third fused feature map; wherein, the first detection result, the second detection result and the third detection result respectively include: the center point position of the target equipment in the power inspection image to be detected, the bounding box width and height information, the center point offset information, the defect status of the target equipment and the equipment type; wherein, the defect status includes: the power equipment has equipment defects and the power equipment does not have equipment defects; The non-maximum suppression method is used to perform a deduplication operation on the first detection result, the second detection result and the third detection result to determine the target detection result with the highest probability among all detection results; The target detection result is used as the detection result of the power equipment in the power inspection image to be detected.

3. The method according to claim 1, characterized in that, The training process of the neural network model includes: A training sample set of power inspection images is obtained, wherein the training sample set includes: a first set of power inspection images containing at least one equipment defect, and a second set of power inspection images not containing equipment defects; the first power inspection images in the first set of power inspection images correspond to a corresponding first tag file; the second power inspection images in the second set of power inspection images correspond to a corresponding second tag file; wherein the first tag file includes at least: the coordinate information, width and height information, equipment type, and defect status of the defective equipment; the second tag file is an empty file; Multi-scale downsampling feature extraction is performed on the power inspection images corresponding to the first power inspection image set and the second power inspection image set to obtain the corresponding image feature maps; Feature fusion is performed on the image feature maps to obtain fused feature maps downsampled at different scales; The neural network model is trained using the backpropagation algorithm, the classification results of the fused feature maps obtained by the neural network model, and the label results until the loss function of the neural network model reaches its minimum, thus obtaining a trained power inspection image target detection neural network, which is then used to detect the power inspection image to be detected.

4. The method according to claim 3, characterized in that, The configuration of the tag file includes: When labeling the first power inspection image, at least one defective device in the power inspection image is bounded by a bounding box to obtain the bounding box annotation result of the defective device. The bounding box annotation result includes the coordinate information of the upper left corner and the lower right corner of the defective device within the bounding box, the width and height information of the defective device, the device type and defect status of the defective device. Based on the bounding box annotation results, the center point information of the defective device is determined, and a center point heat map is determined based on the center point information using a Gaussian distribution; wherein, in the center point heat map, the brightness value is greater the closer to the center point; The center point offset information is determined based on the center point heat map. The center point heat map, the center point offset information, the width and height information, the equipment type and defect status of the defective equipment are used as the first tag file; When labeling the second power inspection image, the second label file corresponding to the second power inspection image is directly labeled as an empty file.

5. The method according to claim 3, characterized in that, After acquiring the training sample set of power line inspection images, the process further includes: The power inspection images are subjected to data enhancement operations using one or more of the following image processing methods: random flipping, random cropping, image quality adjustment, adversarial processing, image mixing, or white noise addition.

6. The method according to claim 3, characterized in that, The method further includes: During the training process of the neural network model, the training effect of the neural network model is verified every preset number of iterations using a test sample set of the power inspection images. The verification of the neural network model includes: Obtain a test sample set of the power inspection images; The trained power inspection image target detection neural network is tested based on preset target detection evaluation indicators and power inspection images in the test sample set; If the preset target detection evaluation index reaches the expected result, the trained power inspection image target detection neural network is used to test the power inspection image to be detected; If the preset target detection evaluation index does not achieve the expected result, the pre-training weights and hyperparameters of the pre-configured neural network are adjusted, and the power inspection image target detection neural network is retrained until the preset target detection evaluation index achieves the expected result.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the power inspection image detection method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the power line inspection image detection method according to any one of claims 1-6.

Images