Power equipment defect detection method and device, computer device, readable storage medium and program product

Abstract

The application relates to a power equipment defect detection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: using a drone to shoot power equipment, obtaining an original shooting image set and performing labeling and image enhancement processing to obtain a training sample set; inputting the training sample set into a defect detection model to be trained to obtain a defect detection result; determining a positioning loss, a classification loss, a boundary box regression loss and a defect region loss between the defect detection result and defect labeling information; determining an adaptive weight; determining a model total loss based on the adaptive weight, the positioning loss, the classification loss, the boundary box regression loss and the defect region loss; and performing parameter adjustment on the defect detection model to be trained based on the model total loss to obtain a trained defect detection model for defect detection of the power equipment. The method can accurately detect a transformer defect region.

Description

TECHNICAL FIELD

[0001] The present application relates to the electrical technical field, in particular to a power equipment defect detection method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the construction of smart grid and the increasing demand for power equipment automatic inspection, the traditional manual inspection method gradually cannot meet the efficient and accurate requirements. In recent years, computer vision technology has been widely used in image recognition and defect detection of power equipment, but due to the complex appearance of power equipment, the cluttered background and the various types of defects, the traditional image processing method still faces many challenges in positioning accuracy and defect detection.

[0003] Small defects on the surface of the equipment, such as cracks, corrosion, scratches, etc., usually appear as low-contrast local areas, and existing models are difficult to accurately detect these areas. SUMMARY

[0004] Therefore, it is necessary to provide a power equipment defect detection method, device, computer equipment, computer readable storage medium and computer program product capable of accurately detecting defect areas in view of the above technical problems.

[0005] In a first aspect, the present application provides a power equipment defect detection method, comprising:

[0006] Using a drone to shoot power equipment to obtain a set of original shooting images, labeling each original shooting image in the set of original shooting images to obtain defect labeling information corresponding to each original shooting image, and performing image enhancement processing on each original shooting image to obtain a device sample image corresponding to each original shooting image; the device sample image and the defect labeling information corresponding to each original shooting image are used as a group of training samples, and a plurality of groups of training samples constitute a training sample set;

[0007] Obtaining any group of training samples from the training sample set, inputting the device sample image in the current training sample into a defect detection model to be trained to obtain a defect detection result, determining the positioning loss, the classification loss, the bounding box regression loss and the defect area loss between the defect detection result and the defect labeling information in the current training sample, determining the adaptive weight, and determining the model total loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect area loss;

[0008] adjust the parameters of the defect detection model to be trained based on the model total loss, continue training the defect detection model with adjusted parameters based on the training sample set, until the model total loss calculated reaches a preset condition, and obtain a trained defect detection model; the trained defect detection model is used for defect detection on the power equipment.

[0009] In one of the embodiments, the adaptive weight is determined by:

[0010] obtaining the gradient of the model total loss calculated in the previous training, the gradient of the positioning loss calculated in the previous training, the weight of the positioning loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; determining the weight of the current positioning loss based on the gradient of the model total loss calculated in the previous training, the gradient of the positioning loss calculated in the previous training, and the weight of the positioning loss calculated in the previous training; determining the weight of the current classification loss based on the gradient of the model total loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determining the weight of the current bounding box regression loss based on the gradient of the model total loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determining the weight of the current defect region loss based on the gradient of the model total loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; and the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute the adaptive weight.

[0011] In one of the embodiments, the positioning loss, the classification loss, the bounding box regression loss, and the defect region loss between the defect detection result and the defect annotation information in the current training sample are determined by:

[0012] obtaining the bounding box output and the category output in the defect detection result; obtaining the real bounding box and the real category in the defect annotation information; determining the positioning loss, the bounding box regression loss, and the defect region loss based on the bounding box output and the real bounding box; and determining the classification loss based on the category output and the real category.

[0013] In one of the embodiments, the determining the model total loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss comprises:

[0014] The current positioning loss weight, the current classification loss weight, the current bounding box regression loss weight and the current defect region loss weight in the adaptive weight are obtained, and a weighted sum of the positioning loss, the classification loss, the bounding box regression loss and the defect region loss is determined based on the current positioning loss weight, the current classification loss weight, the current bounding box regression loss weight and the current defect region loss weight, to obtain the model total loss.

[0015] In one of the embodiments, the labeling each original photographed image in the original photographed image set comprises:

[0016] The transformer and the defect region on the original photographed image are labeled by a bounding box, and a class is labeled for each labeled bounding box, wherein the class is a transformer or a defect type.

[0017] In one of the embodiments, the image enhancement processing of each original photographed image comprises:

[0018] At least one of random rotation, scaling and noise addition is performed on each original photographed image.

[0019] In a second aspect, the present application further provides an electric power equipment defect detection device, comprising:

[0020] An acquisition module is configured to photograph electric power equipment using a UAV to obtain an original photographed image set, label each original photographed image in the original photographed image set to obtain defect labeling information corresponding to each original photographed image, and perform image enhancement processing on each original photographed image to obtain an equipment sample image corresponding to each original photographed image; the equipment sample image and the defect labeling information corresponding to each original photographed image are taken as a group of training samples, and a plurality of groups of training samples constitute a training sample set.

[0021] A training module is configured to obtain any group of training samples from the training sample set, input an equipment sample image in a current training sample into a defect detection model to be trained to obtain a defect detection result, determine a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect labeling information in the current training sample, determine an adaptive weight, and determine a model total loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss.

[0022] detecting a model total loss, adjusting parameters of the defect detection model to be trained based on the model total loss, continuing to train the defect detection model with the adjusted parameters based on the training sample set until the model total loss calculated reaches a preset condition, and obtaining a trained defect detection model; the trained defect detection model is used for defect detection on the power equipment.

[0023] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0024] The power equipment is photographed by using a UAV to obtain a set of original photographed images, each original photographed image in the set of original photographed images is labeled to obtain defect labeling information corresponding to each original photographed image, and each original photographed image is subjected to image enhancement processing to obtain a device sample image corresponding to each original photographed image; the device sample image and the defect labeling information corresponding to each original photographed image are taken as a group of training samples, and a plurality of groups of training samples constitute a training sample set;

[0025] Any group of training samples is obtained from the training sample set, a device sample image in a current training sample is input into a defect detection model to be trained to obtain a defect detection result, a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect labeling information in the current training sample are determined, an adaptive weight is determined, and a model total loss is determined based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss.

[0026] Parameters of the defect detection model to be trained are adjusted based on the model total loss, the defect detection model with the adjusted parameters is continuously trained based on the training sample set until the model total loss calculated reaches a preset condition, and a trained defect detection model is obtained; the trained defect detection model is used for defect detection on the power equipment.

[0027] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0028] The power equipment is photographed by using a UAV to obtain a set of original photographed images, each original photographed image in the set of original photographed images is labeled to obtain defect labeling information corresponding to each original photographed image, and each original photographed image is subjected to image enhancement processing to obtain a device sample image corresponding to each original photographed image; the device sample image and the defect labeling information corresponding to each original photographed image are taken as a group of training samples, and a plurality of groups of training samples constitute a training sample set;

[0029] obtaining any one group of training samples from the training sample set, inputting the equipment sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determining a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect annotation information in the current training sample; determining an adaptive weight; determining a model total loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss;

[0030] based on the model total loss, adjusting parameters of the defect detection model to be trained, continuing to train the defect detection model after the parameter adjustment based on the training sample set, until the model total loss calculated reaches a preset condition, obtaining a trained defect detection model; the trained defect detection model is used for defect detection of the power equipment.

[0031] In a fifth aspect, the present application also provides a computer program product, comprising a computer program which, when executed by a processor, implements the following steps:

[0032] using a UAV to take pictures of the power equipment to obtain an original picture set, annotating each original picture in the original picture set to obtain defect annotation information corresponding to each original picture, performing image enhancement processing on each original picture to obtain an equipment sample image corresponding to each original picture; the equipment sample image and the defect annotation information corresponding to each original picture are used as a group of training samples, and a plurality of groups of training samples constitute a training sample set;

[0033] obtaining any one group of training samples from the training sample set, inputting the equipment sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determining a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect annotation information in the current training sample; determining an adaptive weight; determining a model total loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss;

[0034] based on the model total loss, adjusting parameters of the defect detection model to be trained, continuing to train the defect detection model after the parameter adjustment based on the training sample set, until the model total loss calculated reaches a preset condition, obtaining a trained defect detection model; the trained defect detection model is used for defect detection of the power equipment.

[0035] The above-mentioned power equipment defect detection method, device, computer device, computer-readable storage medium and computer program product use a drone to photograph power equipment to obtain a set of original photographed images, annotate each original photographed image in the original photographed image set, obtain defect annotation information corresponding to each original photographed image, perform image enhancement processing on each original photographed image, and obtain a device sample image corresponding to each original photographed image; the device sample image and defect annotation information corresponding to each original photographed image are used as a set of training samples, and multiple sets of training samples constitute a training sample set; any set of training samples is obtained from the training sample set, and the device sample image in the current training sample is input into the defect detection model to be trained , obtaining a defect detection result; determining the positioning loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; determining an adaptive weight; determining a total model loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss; adjusting parameters of the defect detection model to be trained based on the total model loss, and continuing to train the defect detection model with the adjusted parameters based on the training sample set until the calculated total model loss meets a preset condition, thereby obtaining a trained defect detection model; and using the trained defect detection model to perform defect detection on power equipment. This method can more accurately detect defect areas in transformers. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 1 is a flow chart of a method for detecting defects in power equipment according to an embodiment;

[0038] Figure 2 is a detailed flow chart of a method for detecting defects in power equipment in one embodiment;

[0039] Figure 3 is a structural block diagram of a device for detecting defects in power equipment according to one embodiment;

[0040] Figure 4 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0042] In one embodiment, Figure 1 As shown, a method for detecting defects in power equipment is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0043] Step 102: Use a drone to photograph the power equipment to obtain a set of original photographed images, annotate each original photographed image in the set of original photographed images, obtain defect annotation information corresponding to each original photographed image, perform image enhancement processing on each original photographed image, and obtain a device sample image corresponding to each original photographed image; the device sample image and defect annotation information corresponding to each original photographed image are used as a group of training samples, and multiple groups of training samples constitute a training sample set.

[0044] Optionally, photographing the power equipment includes photographing equipment such as transmission lines and transformers. The original image set obtained by photographing may be an RGB image or an infrared image.

[0045] Among them, image enhancement processing is an image processing technology that aims to improve the visual quality of an image or highlight specific information in the image to meet the needs of different application scenarios.

[0046] Exemplarily, a drone is used to photograph equipment such as transmission lines and transformers along a preset route to obtain a set of original photographed images including RGB images and infrared images. LabelImg is used to label each original photographed image in the set to obtain defect annotation information corresponding to each original photographed image. Image enhancement processing is performed on each original photographed image to obtain a device sample image corresponding to each original photographed image. The device sample image and defect annotation information corresponding to each original photographed image are used as a group of training samples, and multiple groups of training samples constitute a training sample set.

[0047] Among them, LabelImg is a commonly used image annotation software.

[0048] Step 104, obtaining any group of training samples from the training sample set, inputting the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determining a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect annotation information in the current training sample; determining an adaptive weight; determining a model total loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss.

[0049] Optionally, the defect detection model can be YOLO v8, and the initial defect detection model is YOLO v8 pre-trained on the COCO dataset. The positioning loss can be a Smooth L1 loss; the classification loss can be a cross-entropy loss; the bounding box regression loss can be an IOU loss; and the defect region loss can be a Dice coefficient loss.

[0050] The adaptive weight includes the weight of each loss.

[0051] Step 106, adjusting the parameters of the defect detection model to be trained based on the model total loss, and continuing to train the defect detection model after the parameter adjustment based on the training sample set until the model total loss calculated reaches a preset condition to obtain a trained defect detection model; the trained defect detection model is used for defect detection of the power equipment.

[0052] The model total loss reaching the preset condition is that the current model total loss calculated is not less than the model total loss calculated in the previous training.

[0053] Optionally, the preset condition can be that the current model total loss is not less than the total loss of the previous round of training.

[0054] The power equipment defect detection method, device, computer equipment, computer readable storage medium and computer program product use a UAV to take pictures of power equipment, obtain a set of original pictures, label each original picture in the set of original pictures to obtain defect label information corresponding to each original picture, perform image enhancement processing on each original picture to obtain an equipment sample image corresponding to each original picture; the equipment sample image and the defect label information corresponding to each original picture are used as a group of training samples, and a plurality of groups of training samples constitute a training sample set; any group of training samples is obtained from the training sample set, the equipment sample image in the current training sample is input into a defect detection model to be trained to obtain a defect detection result; a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect label information in the current training sample are determined; an adaptive weight is determined; based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss, a model total loss is determined; the defect detection model to be trained is adjusted in parameters based on the model total loss, the defect detection model after parameter adjustment is continuously trained based on the training sample set until the model total loss calculated reaches a preset condition, and a trained defect detection model is obtained; the trained defect detection model is used for defect detection of power equipment. Through the method, the defect region of the transformer can be more accurately detected.

[0055] In one exemplary embodiment, the determining the adaptive weight comprises:

[0056] obtained by the previous training, a gradient of the positioning loss obtained by the previous training, a weight of the positioning loss obtained by the previous training, a gradient of the classification loss obtained by the previous training, a weight of the classification loss obtained by the previous training, a gradient of the bounding box regression loss obtained by the previous training, a weight of the bounding box regression loss obtained by the previous training, a gradient of the defect region loss obtained by the previous training, and a weight of the defect region loss obtained by the previous training; determining a weight of a current positioning loss based on the gradient of the model total loss obtained by the previous training, the gradient of the positioning loss obtained by the previous training, and the weight of the positioning loss obtained by the previous training; determining a weight of a current classification loss based on the gradient of the model total loss obtained by the previous training, the gradient of the classification loss obtained by the previous training, and the weight of the classification loss obtained by the previous training; determining a weight of a current bounding box regression loss based on the gradient of the model total loss obtained by the previous training, the gradient of the bounding box regression loss obtained by the previous training, and the weight of the bounding box regression loss obtained by the previous training; determining a weight of a current defect region loss based on the gradient of the model total loss obtained by the previous training, the gradient of the defect region loss obtained by the previous training, and the weight of the defect region loss obtained by the previous training; and the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute the adaptive weight.

[0057] wherein the previous training refers to that the model is trained for multiple rounds when iteratively trained, and the steps of each training are the same, only the parameters of the model are updated in real time.

[0058] For example, the gradient of the total loss of the model calculated in the previous training, the gradient of the positioning loss calculated in the previous training, the weight of the positioning loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training are obtained; based on the gradient of the total loss of the model calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training, the weight of the current classification loss is determined; based on the gradient of the total loss of the model calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training, the weight of the current bounding box regression loss is determined; based on the gradient of the total loss of the model calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training, the weight of the current defect region loss is determined; the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute the adaptive weight; the calculation formula of the above calculation of different loss weights is as follows:

[0059]

[0060] wherein i represents different losses, is the weight of the current i loss, is the weight of the i loss calculated in the previous training, is the gradient of the total loss of the model calculated in the previous training, is the gradient of the i loss calculated in the previous training, is an adjustment step, which controls the rate of weight adjustment.

[0061] In the embodiment, by calculating the weights of different losses, the weights can be dynamically adjusted according to the requirements of different tasks to ensure that the model has good performance in positioning and defect detection.

[0062] In an exemplary embodiment, the determination of the positioning loss, the classification loss, the bounding box regression loss, and the defect region loss between the defect detection result and the defect annotation information in the current training sample comprises:

[0063] The bounding box output and the category output in the defect detection result are obtained, and the real bounding box and the real category in the defect annotation information are obtained. A positioning loss, a bounding box regression loss and a defect region loss are determined based on the bounding box output and the real bounding box. A classification loss is determined based on the category output and the real category.

[0064] For example, the bounding box output and the category output in the defect detection result are obtained, and the real bounding box and the real category in the defect annotation information are obtained. A positioning loss is calculated based on the bounding box output and the real bounding box , and the formula is as follows:

[0065]

[0066] wherein, is the real coordinate of the i th pixel in the real bounding box, is the predicted coordinate in the bounding box output, and N is the number of pixels in the bounding box output.

[0067]

[0068] A bounding box regression loss is calculated based on the bounding box output and the real bounding box , and the calculation formula is as follows:

[0069]

[0070] wherein, is the bounding box region in the bounding box output, is the bounding box region in the real bounding box, represents the intersection, represents the union. A defect region loss is calculated based on the bounding box output and the real bounding box , and the calculation formula is as follows:

[0071]

[0072] wherein, is the defect region in the bounding box output, is the defect region in the real bounding box, and i is the pixel in the bounding box output. N is the number of pixels in the bounding box output. A classification loss is calculated based on the category output and the real category, and the calculation formula is as follows:

[0073]

[0074] wherein, is the label of the real category, is the predicted category probability, and N is the number of all categories.

[0075] In this embodiment, the positioning loss and the bounding box regression loss are calculated, which can significantly improve the positioning accuracy of the power equipment, the defect area loss enhances the detection ability of the micro defect, especially in the small defect detection on the surface of the equipment, and the classification loss optimizes the classification accuracy of all the bounding boxes.

[0076] In an exemplary embodiment, the model total loss is determined based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect area loss.

[0077] The current positioning loss weight, the current classification loss weight, the current bounding box regression loss weight and the current defect area loss weight in the adaptive weight are obtained, and the weighted sum of the positioning loss, the classification loss, the bounding box regression loss and the defect area loss is determined based on the current positioning loss weight, the current classification loss weight, the current bounding box regression loss weight and the current defect area loss weight, to obtain the model total loss.

[0078] For example, the current positioning loss weight, the current classification loss weight, the current bounding box regression loss weight and the current defect area loss weight in the adaptive weight are obtained, and the weighted sum of the positioning loss, the classification loss, the bounding box regression loss and the defect area loss is calculated based on the current positioning loss weight, the current classification loss weight, the current bounding box regression loss weight and the current defect area loss weight, to obtain the model total loss, and the calculation formula is as follows:

[0079]

[0080] wherein, is the weight of the positioning loss, is the positioning loss, is the weight of the classification loss, is the classification loss, is the weight of the bounding box regression loss, is the bounding box regression loss, is the weight of the defect area loss, is the defect area loss.

[0081] In this embodiment, the weighted sum of the positioning loss, the classification loss, the bounding box regression loss and the defect area loss is calculated by the dynamically adjusted adaptive loss to obtain the model total loss, which can conveniently control the direction of model optimization.

[0082] In an exemplary embodiment, the labeling of each original shooting image in the original shooting image set comprises:

[0083] For each original shooting image in the original shooting image set, the transformer and the defect area on the original shooting image are annotated with a bounding box, and each annotated bounding box is annotated with a category, which is a transformer or a defect type.

[0084] Optionally, the defect type can be a crack, corrosion, scratch, etc.

[0085] For example, for each original shooting image in the original shooting image set, the transformer and the defect area on the original shooting image are annotated with a bounding box, and each annotated bounding box is annotated with a category, which is a transformer or a defect type, and the defect type includes a crack, corrosion, scratch, etc.

[0086] In this embodiment, by annotating the data with the transformer and the defect area, the model can effectively learn the features of the defects.

[0087] In an example embodiment, the image enhancement processing of each original shooting image includes:

[0088] At least one of random rotation, scaling, and adding noise is performed on each original shooting image.

[0089] Optionally, the image enhancement can also be grayscale processing and filtering processing, etc.

[0090] For example, random rotation, scaling, adding noise, grayscale processing, and filtering processing, etc. are sequentially performed on each original shooting image.

[0091] In this embodiment, by performing different image enhancement processing on the image data, a variety of images can be obtained, which is conducive to the network learning of the features of multiple images.

[0092] In an example embodiment, assuming that there is an image of a power equipment, after model detection, the following data is obtained: the real bounding box of the power equipment: [x1, x2, x3, x4] = [30, 50, 100, 150], the predicted bounding box [x_1, x_2, x_3, x_4] = [30, 50, 100, 150], the category label: "transformer", the predicted category probability: 0.9, and the Dice coefficient of the defect area: 0.8. The process of calculating the total loss of the model is as follows: first, calculate the localization loss, the classification loss, the bounding box regression loss, and the defect area loss. :

[0093]

[0094]

[0095] 2. Calculate the classification loss :

[0096]

[0097] 3. Calculate bounding box regression loss :

[0098]

[0099] 4. Calculate the loss of defective area:

[0100]

[0101] Then, calculate the total model loss (set up =1, =1, =1, =2):

[0102]

[0103] Therefore, the total model loss is 4.505.

[0104] In an exemplary embodiment, a method for detecting defects in power equipment is provided. Figure 2 As shown, it includes: using a drone to shoot power transmission lines, transformers and other equipment according to a preset route to obtain a set of original captured images including RGB images and infrared images; for each original captured image in the original captured image set, using LabelImg to annotate the transformer and defect area on the original captured image with a bounding box; annotating each annotated bounding box with a category, wherein the category is transformer or defect type, and the defect type includes: cracks, corrosion, scratches, etc.; obtaining defect annotation information corresponding to each original captured image; performing random rotation, scaling, adding noise, grayscale processing and filtering on each original captured image in turn to obtain a device sample image corresponding to each original captured image; the device sample image and defect annotation information corresponding to each original captured image are used as a group of training samples, and multiple groups of training samples constitute a training sample set. Obtain any group of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained, and obtain the defect detection result; obtain the bounding box output and category output in the defect detection result; obtain the true bounding box and true category in the defect annotation information; calculate the positioning loss based on the bounding box output and the true bounding box , the formula is as follows:

[0105]

[0106] in, is the true coordinate of the i-th pixel in the ground-truth bounding box, is the predicted coordinate in the bounding box output, and N is the number of pixels in the region.

[0107]

[0108] The bounding box regression loss is calculated based on the bounding box output and the real bounding box , and the calculation formula is as follows:

[0109]

[0110] wherein, is the bounding box region in the bounding box output, is the bounding box region in the real bounding box, represents the intersection, represents the union. The defect region loss is calculated based on the bounding box output and the real bounding box , and the calculation formula is as follows:

[0111]

[0112] wherein, is the defect region in the bounding box output, is the defect region in the real bounding box. The classification loss is calculated based on the category output and the real category, and the calculation formula is as follows:

[0113]

[0114] wherein, is the label of the real category, is a predicted class probability, and N is the number of all classes. The gradient of the total loss of the model calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training are obtained. Based on the gradient of the total loss of the model calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training, the weight of the current classification loss is determined. Based on the gradient of the total loss of the model calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training, the weight of the current bounding box regression loss is determined. Based on the gradient of the total loss of the model calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training, the weight of the current defect region loss is determined. The weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute adaptive weights. The calculation formula of the above calculation of different loss weights is as follows:

[0115]

[0116] wherein i represents different losses, is the weight of the current i loss, is the weight of the i loss calculated in the previous training, is the gradient of the total loss of the model calculated in the previous training, is the gradient of the i loss calculated in the previous training, is an adjustment step, which controls the rate of weight adjustment. The weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss in the adaptive weights are obtained. Based on the weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss, the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect region loss is calculated to obtain the total loss of the model, and the calculation formula is as follows:

[0117]

[0118] wherein, is the weight of the localization loss, is the localization loss, is the weight of the classification loss, is a classification loss, is a weight of a bounding box regression loss, is a bounding box regression loss, is a weight of a defect region loss, is a defect region loss. Based on the model total loss, the defect detection model to be trained is adjusted in parameters, the defect detection model adjusted in parameters is continuously trained based on the training sample set, until the model total loss calculated reaches a preset condition, and a trained defect detection model is obtained; the trained defect detection model is used for defect detection on the power equipment.

[0119] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless explicitly stated herein, there is no strict sequence limitation for the execution of these steps, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0120] In an exemplary embodiment, as shown in Figure 3 a power equipment defect detection device is provided, comprising: an acquisition module 301, a training module 302 and a detection module 303, wherein:

[0121] The acquisition module is used to use a UAV to shoot a power equipment, obtain a set of original shooting images, label each original shooting image in the set of original shooting images to obtain defect labeling information corresponding to each original shooting image, perform image enhancement processing on each original shooting image to obtain an equipment sample image corresponding to each original shooting image; the equipment sample image corresponding to each original shooting image and the defect labeling information are taken as a group of training samples, and a plurality of groups of training samples constitute a training sample set;

[0122] The training module is used to obtain any group of training samples from the training sample set, input an equipment sample image in a current training sample into a defect detection model to be trained to obtain a defect detection result; determine a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect labeling information in the current training sample; determine an adaptive weight; based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss, determine a model total loss;

[0123] detecting a model total loss based on the defect detection model, and continuing to train the defect detection model based on the training sample set until the model total loss reaches a preset condition, to obtain a trained defect detection model; the trained defect detection model is used for defect detection on the power equipment.

[0124] In one of the embodiments, the training module is further configured to:

[0125] obtain a gradient of the model total loss calculated in the previous training, a gradient of the positioning loss calculated in the previous training, a weight of the positioning loss calculated in the previous training, a gradient of the classification loss calculated in the previous training, a weight of the classification loss calculated in the previous training, a gradient of the bounding box regression loss calculated in the previous training, a weight of the bounding box regression loss calculated in the previous training, a gradient of the defect region loss calculated in the previous training, and a weight of the defect region loss calculated in the previous training; determine a current weight of the positioning loss based on the gradient of the model total loss calculated in the previous training, the gradient of the positioning loss calculated in the previous training, and the weight of the positioning loss calculated in the previous training; determine a current weight of the classification loss based on the gradient of the model total loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determine a current weight of the bounding box regression loss based on the gradient of the model total loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determine a current weight of the defect region loss based on the gradient of the model total loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; and the current weight of the positioning loss, the current weight of the classification loss, the current weight of the bounding box regression loss, and the current weight of the defect region loss together constitute the adaptive weight.

[0126] In one of the embodiments, the training module is further configured to:

[0127] obtain the bounding box output and the category output in the defect detection result, obtain the real bounding box and the real category in the defect annotation information, determine the positioning loss, the bounding box regression loss, and the defect region loss based on the bounding box output and the real bounding box, and determine the classification loss based on the category output and the real category.

[0128] In one of the embodiments, the training module is further configured to:

[0129] obtaining a weight of a current positioning loss, a weight of a current classification loss, a weight of a current bounding box regression loss and a weight of a current defect region loss in adaptive weights; determining a weighted sum of the positioning loss, the classification loss, the bounding box regression loss and the defect region loss based on the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss and the weight of the current defect region loss, to obtain a model total loss.

[0130] In one of the embodiments, the obtaining module is further configured to:

[0131] For each original photograph in the original photograph set, the transformer and the defect region on the original photograph are marked with a bounding box, and each marked bounding box is marked with a category, which is a transformer or a defect type.

[0132] In one of the embodiments, the obtaining module is further configured to:

[0133] Each original photograph is processed with at least one of random rotation, scaling and noise addition.

[0134] The above-mentioned modules in the power equipment defect detection device can be realized by software, hardware and combinations thereof in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to the above-mentioned modules by the processor.

[0135] In one exemplary embodiment, a computer device is provided, which can be a server, and the internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store original photograph set data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a power equipment defect detection method.

[0136] Those skilled in the art can understand that, Figure 4The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0137] In one exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0138] An unmanned aerial vehicle is used to take pictures of power equipment to obtain a set of original pictures, each original picture is labeled to obtain corresponding defect label information, and each original picture is subjected to image enhancement processing to obtain a corresponding equipment sample picture; the equipment sample picture and the defect label information corresponding to each original picture are taken as a set of training samples, and a plurality of sets of training samples constitute a training sample set;

[0139] Any set of training samples is obtained from the training sample set, an equipment sample picture in the current training sample is input into a defect detection model to be trained to obtain a defect detection result, a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect label information in the current training sample are determined, an adaptive weight is determined, and based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss, a total loss of the model is determined;

[0140] The parameter of the defect detection model to be trained is adjusted based on the total loss of the model, the defect detection model after the parameter adjustment is continuously trained based on the training sample set, until the total loss of the model calculated reaches a preset condition, and a trained defect detection model is obtained; the trained defect detection model is used for defect detection of power equipment.

[0141] In one embodiment, the processor further implements the following steps when executing the computer program:

[0142] obtaining a gradient of a model total loss calculated in the previous training, a gradient of a positioning loss calculated in the previous training, a weight of the positioning loss calculated in the previous training, a gradient of a classification loss calculated in the previous training, a weight of the classification loss calculated in the previous training, a gradient of a bounding box regression loss calculated in the previous training, a weight of the bounding box regression loss calculated in the previous training, a gradient of a defect region loss calculated in the previous training, and a weight of the defect region loss calculated in the previous training; determining a weight of a current positioning loss based on the gradient of the model total loss calculated in the previous training, the gradient of the positioning loss calculated in the previous training, and the weight of the positioning loss calculated in the previous training; determining a weight of a current classification loss based on the gradient of the model total loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determining a weight of a current bounding box regression loss based on the gradient of the model total loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determining a weight of a current defect region loss based on the gradient of the model total loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; and the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute adaptive weights.

[0143] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0144] obtaining a bounding box output and a category output in the defect detection result; obtaining a real bounding box and a real category in the defect annotation information; determining a positioning loss, a bounding box regression loss, and a defect region loss based on the bounding box output and the real bounding box; and determining a classification loss based on the category output and the real category.

[0145] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0146] obtaining a weight of a current positioning loss, a weight of a current classification loss, a weight of a current bounding box regression loss, and a weight of a current defect region loss in the adaptive weights; and determining a weighted sum of the positioning loss, the classification loss, the bounding box regression loss, and the defect region loss based on the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss, to obtain a model total loss.

[0147] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0148] For each original photographed image in the set of original photographed images, a transformer and a defect region on the original photographed image are annotated with a bounding box, and each bounding box is annotated with a category, which is a transformer or a defect type.

[0149] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0150] At least one of random rotation, scaling, and adding noise is performed on each original photographed image.

[0151] In one embodiment, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the following steps:

[0152] The power equipment is photographed using a drone to obtain a set of original photographed images, each original photographed image in the set of original photographed images is annotated to obtain corresponding defect annotation information of each original photographed image, and each original photographed image is subjected to image enhancement processing to obtain a corresponding equipment sample image of each original photographed image; the equipment sample image and the defect annotation information corresponding to each original photographed image serve as a set of training samples, and a plurality of sets of training samples constitute a training sample set.

[0153] Any set of training samples is obtained from the training sample set, an equipment sample image in the current set of training samples is input into a defect detection model to be trained to obtain a defect detection result, a positioning loss, a classification loss, a bounding box regression loss, and a defect region loss between the defect detection result and the defect annotation information in the current set of training samples are determined, an adaptive weight is determined, and based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss, and the defect region loss, a total loss of the model is determined.

[0154] The defect detection model to be trained is subjected to parameter adjustment based on the total loss of the model, the defect detection model after parameter adjustment is continuously trained based on the training sample set until the total loss of the model calculated reaches a preset condition, and a trained defect detection model is obtained; the trained defect detection model is used for defect detection of power equipment.

[0155] In one embodiment, the computer program, when executed by the processor, also implements the following steps:

[0156] obtaining a gradient of a model total loss calculated in the previous training, a gradient of a positioning loss calculated in the previous training, a weight of the positioning loss calculated in the previous training, a gradient of a classification loss calculated in the previous training, a weight of the classification loss calculated in the previous training, a gradient of a bounding box regression loss calculated in the previous training, a weight of the bounding box regression loss calculated in the previous training, a gradient of a defect region loss calculated in the previous training, and a weight of the defect region loss calculated in the previous training; determining a weight of a current positioning loss based on the gradient of the model total loss calculated in the previous training, the gradient of the positioning loss calculated in the previous training, and the weight of the positioning loss calculated in the previous training; determining a weight of a current classification loss based on the gradient of the model total loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determining a weight of a current bounding box regression loss based on the gradient of the model total loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determining a weight of a current defect region loss based on the gradient of the model total loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; and the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute adaptive weights.

[0157] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0158] obtaining a bounding box output and a category output in the defect detection result; obtaining a real bounding box and a real category in the defect annotation information; determining a positioning loss, a bounding box regression loss, and a defect region loss based on the bounding box output and the real bounding box; and determining a classification loss based on the category output and the real category.

[0159] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0160] obtaining a weight of a current positioning loss, a weight of a current classification loss, a weight of a current bounding box regression loss, and a weight of a current defect region loss in the adaptive weights; and determining a weighted sum of the positioning loss, the classification loss, the bounding box regression loss, and the defect region loss based on the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss, to obtain a model total loss.

[0161] In one embodiment, the computer program, which is executed by the processor, further implements the following steps:

[0162] For each original photograph in the set of original photographs, a bounding box is labeled on the transformer and the defect region on the original photograph, and a class is labeled for each labeled bounding box, the class being a transformer or a defect type.

[0163] In one embodiment, the computer program, which is executed by the processor, further implements the following steps:

[0164] At least one of random rotation, scaling and adding noise is performed on each original photograph.

[0165] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by the processor, implements the following steps:

[0166] The power equipment is photographed using a drone to obtain a set of original photographs, each original photograph in the set of original photographs is labeled to obtain corresponding defect labeling information for each original photograph, and each original photograph is subjected to image enhancement processing to obtain a corresponding equipment sample image for each original photograph; the equipment sample image and the defect labeling information corresponding to each original photograph serve as a group of training samples, and a plurality of groups of training samples constitute a training sample set;

[0167] Any group of training samples is obtained from the training sample set, the equipment sample image in the current training sample is input into the defect detection model to be trained to obtain a defect detection result, a positioning loss, a classification loss, a bounding box regression loss and a defect region loss between the defect detection result and the defect labeling information in the current training sample are determined, an adaptive weight is determined, and based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect region loss, a model total loss is determined.

[0168] The parameter of the defect detection model to be trained is adjusted based on the model total loss, the defect detection model after the parameter adjustment is continuously trained based on the training sample set until the model total loss calculated reaches a preset condition, and a trained defect detection model is obtained; the trained defect detection model is used for defect detection of power equipment.

[0169] In one embodiment, the computer program, which is executed by the processor, further implements the following steps:

[0170] obtaining a gradient of a total model loss calculated in a previous training, a gradient of a localization loss calculated in the previous training, a weight of the localization loss calculated in the previous training, a gradient of a classification loss calculated in the previous training, a weight of the classification loss calculated in the previous training, a gradient of a bounding box regression loss calculated in the previous training, a weight of the bounding box regression loss calculated in the previous training, a gradient of a defect region loss calculated in the previous training, and a weight of the defect region loss calculated in the previous training; determining a weight of a current localization loss based on the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, and the weight of the localization loss calculated in the previous training; determining a weight of a current classification loss based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determining a weight of a current bounding box regression loss based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determining a weight of a current defect region loss based on the gradient of the total model loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; and the weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute adaptive weights.

[0171] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0172] obtaining a bounding box output and a class output in the defect detection result; obtaining a real bounding box and a real class in the defect annotation information; determining a localization loss, a bounding box regression loss, and a defect region loss based on the bounding box output and the real bounding box; and determining a classification loss based on the class output and the real class.

[0173] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0174] obtaining a weight of a current localization loss, a weight of a current classification loss, a weight of a current bounding box regression loss, and a weight of a current defect region loss in the adaptive weights; and determining a weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect region loss based on the weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss, to obtain a total model loss.

[0175] In one embodiment, the computer program which when executed by the processor further implements the following steps:

[0176] For each original photographed image in the original photographed image set, a transformer and a defect region on the original photographed image are labeled with a bounding box, and each labeled bounding box is labeled with a category, the category being a transformer or a defect type.

[0177] In one embodiment, the computer program which when executed by the processor further implements the following steps:

[0178] At least one of random rotation, scaling and adding noise is performed on each original photographed image.

[0179] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0180] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0181] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for detecting defects in electric power equipment, characterized in that: The method comprises: Using a drone to photograph power equipment, obtaining a set of original photographed images, annotating each original photographed image in the set of original photographed images, obtaining defect annotation information corresponding to each original photographed image, and performing image enhancement processing on each original photographed image to obtain a device sample image corresponding to each original photographed image; the device sample image corresponding to each original photographed image and the defect annotation information are used as a set of training samples, and multiple sets of training samples constitute a training sample set; Obtain any group of training samples from the training sample set, input the device sample images in the current training samples into the defect detection model to be trained, and obtain defect detection results; determine the positioning loss, classification loss, bounding box regression loss, and defect area loss between the defect detection results and the defect annotation information in the current training samples; determine adaptive weights; determine the total model loss based on the adaptive weights, the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss; determining the adaptive weights includes: determining weights of different losses, and determining the adaptive weights based on the weights of the different losses; the different losses include: positioning loss, classification loss, bounding box regression loss, and defect area loss; the weight calculation formulas for the different losses are as follows: Among them, i represents different losses, is the weight of the current i loss, is the weight of the i loss calculated in the previous training, is the gradient of the total loss of the model calculated in the previous training, is the gradient of the i loss calculated in the previous training, It is to adjust the step size and control the rate of weight adjustment; The parameters of the defect detection model to be trained are adjusted based on the total loss of the model, and the defect detection model with adjusted parameters is continuously trained based on the training sample set until the calculated total loss of the model reaches a preset condition, thereby obtaining a trained defect detection model; the trained defect detection model is used to perform defect detection on power equipment.

2. The method according to claim 1, characterized in that The determining of the adaptive weight comprises: Obtain the gradient of the total model loss calculated from the previous training, the gradient of the positioning loss calculated from the previous training, the weight of the positioning loss calculated from the previous training, the gradient of the classification loss calculated from the previous training, the weight of the classification loss calculated from the previous training, the gradient of the bounding box regression loss calculated from the previous training, the weight of the bounding box regression loss calculated from the previous training, the gradient of the defect area loss calculated from the previous training, and the weight of the defect area loss calculated from the previous training; Determining the weight of the current positioning loss based on the gradient of the total model loss calculated from the previous training, the gradient of the positioning loss calculated from the previous training, and the weight of the positioning loss calculated from the previous training; Determine the weight of the current classification loss based on the gradient of the total model loss calculated from the previous training, the gradient of the classification loss calculated from the previous training, and the weight of the classification loss calculated from the previous training; Determining a weight of a current bounding box regression loss based on the gradient of the total model loss calculated from the previous training, the gradient of the bounding box regression loss calculated from the previous training, and the weight of the bounding box regression loss calculated from the previous training; Determine the weight of the current defective area loss based on the gradient of the total model loss calculated from the previous training, the gradient of the defective area loss calculated from the previous training, and the weight of the defective area loss calculated from the previous training; The weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss and the weight of the current defect area loss together constitute the adaptive weight.

3. The method according to claim 1, characterized in that The determining of the positioning loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample includes: Obtaining the bounding box output and category output in the defect detection result; obtaining the true bounding box and true category in the defect annotation information; Determining a positioning loss, a bounding box regression loss, and a defect area loss based on the bounding box output and the true bounding box; Based on the class output and the true class, a classification loss is determined.

4. The method according to claim 2, characterized in that The determining of the total model loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss comprises: Get the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss in the adaptive weights; Based on the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss, a weighted sum of the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss is determined to obtain a total model loss.

5. The method according to claim 1, wherein The labeling of each original image in the original image set includes: For each original captured image in the original captured image set, a transformer and a defect area on the original captured image are annotated with a bounding box, and a category is annotated for each annotated bounding box, where the category is a transformer or a defect type.

6. The method according to claim 1, characterized in that The image enhancement processing is performed on each original captured image, comprising: At least one of randomly rotating, scaling, and adding noise is performed on each original captured image.

7. A power equipment defect detection device, characterized in that: The device comprises: An acquisition module is configured to use a drone to photograph power equipment to obtain a set of original images, annotate each original image in the set of original images, obtain defect annotation information corresponding to each original image, and perform image enhancement processing on each original image to obtain a sample image of the equipment corresponding to each original image; the sample image of the equipment corresponding to each original image and the defect annotation information are used as a set of training samples, and multiple sets of training samples constitute a training sample set; A training module is configured to obtain any set of training samples from the training sample set, input the device sample images in the current training sample into the defect detection model to be trained, and obtain a defect detection result; determine the positioning loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; determine an adaptive weight; determine the total model loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss; determining the adaptive weight includes: determining weights of different losses, and determining the adaptive weight based on the weights of different losses; the different losses include: positioning loss, classification loss, bounding box regression loss, and defect area loss; the weight calculation formula of the different losses is as follows: Among them, i represents different losses, is the weight of the current i loss, is the weight of the i loss calculated in the previous training, is the gradient of the total loss of the model calculated in the previous training, is the gradient of the i loss calculated in the previous training, It is to adjust the step size and control the rate of weight adjustment; A detection module is used to adjust the parameters of the defect detection model to be trained based on the total loss of the model, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total loss of the model reaches a preset condition, thereby obtaining a trained defect detection model; the trained defect detection model is used to perform defect detection on power equipment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images