Power equipment defect detection method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a power equipment defect detection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: shooting power equipment by using an unmanned aerial vehicle to obtain an original shot image set, and carrying out labeling and image enhancement processing to obtain a training sample set; inputting the training sample set into a defect detection model to be trained to obtain a defect detection result; determining positioning loss, classification loss, bounding box regression loss and defect area loss between the defect detection result and the defect labeling information; determining an adaptive weight; based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect area loss, determining the total loss of the model; and performing parameter adjustment on the to-be-trained defect detection model based on the total model loss to obtain a trained defect detection model for performing defect detection on the power equipment. By adopting the method, the transformer defect area can be accurately detected.

Description

Technical Field

[0001] This application relates to the field of electrical technology, and in particular, to a method, device, computer device, computer-readable storage medium, and computer program product for detecting defects in power equipment. Background Art

[0002] With the construction of smart grids and the increasing demand for automated inspection of power equipment, traditional manual inspection methods can no longer meet the requirements of high efficiency and accuracy. In recent years, computer vision technology has been widely used in image recognition and defect detection of power equipment. However, due to the complex appearance of power equipment, cluttered backgrounds, and diverse defect types, traditional image processing methods still face many challenges in terms of positioning accuracy and defect detection.

[0003] Tiny defects on the surface of the equipment, such as cracks, corrosion, scratches, etc., usually appear as low-contrast local areas, and existing models are difficult to accurately detect these areas. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for detecting defects in power equipment that can accurately detect defect areas in response to the above technical problems.

[0005] In a first aspect, this application provides a method for detecting defects in power equipment, including:

[0006] Using a drone to capture images of power equipment to obtain an original captured image set, annotating each original captured image in the original captured image set to obtain defect annotation information corresponding to each original captured image, and performing image enhancement processing on each original captured image to obtain a device sample image corresponding to each original captured image; the device sample image and defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples form a training sample set;

[0007] Obtaining any set of training samples from the training sample set, inputting the device sample image in the current training sample into a defect detection model to be trained to obtain a defect detection result; determining the positioning loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; determining an adaptive weight; and determining the total model loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss;

[0008] Adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition, so as to obtain a trained defect detection model; the trained defect detection model is used for defect detection of power equipment.

[0009] In one embodiment, the determining the adaptive weights includes:

[0010] Obtain the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; determine the current weight of the localization loss based on the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, and the weight of the localization loss calculated in the previous training; determine the current weight of the classification loss based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determine the current weight of the bounding box regression loss based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determine the current weight of the defect region loss based on the gradient of the total model loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; the current weights of the localization loss, the classification loss, the bounding box regression loss, and the defect region loss together constitute the adaptive weights.

[0011] In one embodiment, the determining the localization loss, the classification loss, the bounding box regression loss, and the defect region loss between the defect detection result and the defect annotation information in the current training sample includes:

[0012] Obtain the bounding box output and the class output in the defect detection result; obtain the ground truth bounding box and the ground truth class in the defect annotation information; determine the localization loss, the bounding box regression loss, and the defect region loss based on the bounding box output and the ground truth bounding box; determine the classification loss based on the class output and the ground truth class.

[0013] In one embodiment, determining the total model loss based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss includes:

[0014] Obtain the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss in the adaptive weight; based on the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss, determine the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect area loss to obtain the total model loss.

[0015] In one embodiment, annotating each original captured image in the original captured image set includes:

[0016] For each original captured image in the original captured image set, perform bounding box annotation on the transformer and the defect area on the original captured image, and annotate the category for each annotated bounding box, where the category is the transformer or the defect type.

[0017] In one embodiment, performing image enhancement processing on each original captured image includes:

[0018] Perform at least one of random rotation, scaling, and adding noise on each original captured image.

[0019] In a second aspect, the present application also provides a power equipment defect detection device, including:

[0020] An acquisition module, configured to use a drone to capture power equipment to obtain an original captured image set, annotate each original captured image in the original captured image set to obtain defect annotation information corresponding to each original captured image, perform image enhancement processing on each original captured image to obtain a device sample image corresponding to each original captured image; the device sample image and the defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples constitute a training sample set;

[0021] A training module, configured to obtain any set of training samples from the training sample set, input the device sample image in the current training sample into a to-be-trained defect detection model to obtain a defect detection result; determine the localization loss, the classification loss, the bounding box regression loss, and the defect area loss between the defect detection result and the defect annotation information in the current training sample; determine the adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss, determine the total model loss;

[0022] A detection module, configured to adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition, so as to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0023] In a third aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0024] Use a drone to take pictures of power equipment to obtain an original set of captured images, annotate each original captured image in the original set of captured images to obtain defect annotation information corresponding to each original captured image, and perform image enhancement processing on each original captured image to obtain a device sample image corresponding to each original captured image; the device sample image and the defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples constitute a training sample set;

[0025] Obtain any set of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect region loss between the defect detection result and the defect annotation information in the current training sample; determine an adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect region loss, determine the total model loss;

[0026] Adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition, so as to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0027] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0028] Use a drone to take pictures of power equipment to obtain an original set of captured images, annotate each original captured image in the original set of captured images to obtain defect annotation information corresponding to each original captured image, and perform image enhancement processing on each original captured image to obtain a device sample image corresponding to each original captured image; the device sample image and the defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples constitute a training sample set;

[0029] Obtain any set of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained, and obtain the defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect region loss between the defect detection result and the defect annotation information in the current training sample; determine the adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect region loss, determine the total model loss;

[0030] Adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0031] In a fifth aspect, the present application also provides a computer program product, including a computer program, which when executed by a processor implements the following steps:

[0032] Use a drone to photograph power equipment to obtain an original set of photographed images, annotate each original photographed image in the original set of photographed images to obtain the defect annotation information corresponding to each original photographed image, and perform image enhancement processing on each original photographed image to obtain the device sample image corresponding to each original photographed image; the device sample image and the defect annotation information corresponding to each original photographed image are used as a set of training samples, and multiple sets of training samples form a training sample set;

[0033] Obtain any set of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained, and obtain the defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect region loss between the defect detection result and the defect annotation information in the current training sample; determine the adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect region loss, determine the total model loss;

[0034] Adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0035] The above-mentioned power equipment defect detection method, device, computer device, computer-readable storage medium, and computer program product use a drone to photograph power equipment to obtain an original set of photographed images, annotate each original photographed image in the original set of photographed images to obtain defect annotation information corresponding to each original photographed image, perform image enhancement processing on each original photographed image to obtain a device sample image corresponding to each original photographed image; the device sample image and defect annotation information corresponding to each original photographed image are used as a set of training samples, and multiple sets of training samples form a training sample set; any set of training samples is obtained from the training sample set, the device sample image in the current training sample is input into the defect detection model to be trained to obtain a defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; determine the adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss, determine the total model loss; based on the total model loss, adjust the parameters of the defect detection model to be trained, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment. Through this method, the defect area of the transformer can be detected more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 It is a schematic flowchart of a power equipment defect detection method in an embodiment;

[0038] Figure 2 It is a detailed flowchart of a power equipment defect detection method in an embodiment;

[0039] Figure 3 It is a structural block diagram of a power equipment defect detection device in an embodiment;

[0040] Figure 4 It is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] To make the objectives, technical solutions, and advantages of this application clearer and more understandable, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.

[0042] In one embodiment, as Figure 1 shown, a method for detecting defects in power equipment is provided. In this embodiment, an example is given where this method is applied to a terminal. It can be understood that this method can also be applied to a server or a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0043] Step 102: Use a drone to capture images of power equipment to obtain an original captured image set. Label each original captured image in the original captured image set to obtain the defect annotation information corresponding to each original captured image. Perform image enhancement processing on each original captured image to obtain the device sample image corresponding to each original captured image; the device sample image and the defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples form a training sample set.

[0044] Optionally, capturing images of power equipment includes: capturing images of equipment such as transmission lines and transformers. The original image set obtained by capturing can be an RGB image or an infrared image.

[0045] Among them, image enhancement processing is an image processing technology aimed at improving the visual quality of an image or highlighting specific information in the image to meet the requirements of different application scenarios.

[0046] Exemplarily, use a drone to capture images of equipment such as transmission lines and transformers according to a preset flight path to obtain an original captured image set including RGB images and infrared images. Use LabelImg to label each original captured image in the original captured image set to obtain the defect annotation information corresponding to each original captured image. Perform image enhancement processing on each original captured image to obtain the device sample image corresponding to each original captured image; the device sample image and the defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples form a training sample set.

[0047] Among them, LabelImg is a commonly used image annotation software.

[0048] Step 104: Obtain any set of training samples from the training sample set. Input the device sample images in the current training samples into the defect detection model to be trained to obtain defect detection results. Determine the localization loss, classification loss, bounding box regression loss, and defect area loss between the defect detection results and the defect annotation information in the current training samples. Determine the adaptive weights. Based on the adaptive weights, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss, determine the total model loss.

[0049] Optionally, the defect detection model can be YOLO v8, and the initial defect detection model is YOLO v8 pre-trained on the COCO dataset. The localization loss can be Smooth L1 loss; the classification loss can be cross-entropy loss; the bounding box regression loss can be IOU loss; the defect area loss can be Dice coefficient loss.

[0050] Among them, the adaptive weights include the weights of each loss.

[0051] Step 106: Adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss meets the preset conditions to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0052] Among them, the situation where the total model loss meets the preset conditions is when the currently calculated total model loss is not less than the total model loss calculated in the previous training.

[0053] Optionally, the preset condition can be that the current total model loss is not less than the total loss of the previous round of training.

[0054] The above-mentioned power equipment defect detection method, device, computer equipment, computer-readable storage medium and computer program product use a drone to photograph power equipment to obtain an original set of photographed images, annotate each original photographed image in the original set of photographed images to obtain defect annotation information corresponding to each original photographed image, perform image enhancement processing on each original photographed image to obtain a device sample image corresponding to each original photographed image; the device sample image and defect annotation information corresponding to each original photographed image are used as a set of training samples, and multiple sets of training samples constitute a training sample set; obtain any set of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determine the positioning loss, classification loss, bounding box regression loss and defect area loss between the defect detection result and the defect annotation information in the current training sample; determine an adaptive weight; based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss and the defect area loss, determine the total model loss; based on the total model loss, adjust the parameters of the defect detection model to be trained, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss meets a preset condition to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment. Through this method, the defect area of the transformer can be detected more accurately.

[0055] In an exemplary embodiment, the determining the adaptive weight includes:

[0056] Obtain the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; based on the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, and the weight of the localization loss calculated in the previous training, determine the weight of the current localization loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training, determine the weight of the current classification loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training, determine the weight of the current bounding box regression loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training, determine the weight of the current defect region loss; the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect region loss together constitute the adaptive weights.

[0057] Among them, the previous training means that when the model is iteratively trained, it will be trained for multiple rounds, and the steps of each training are the same, only the parameters of the model are updated in real time.

[0058] Exemplarily, obtain the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; determine the weight of the current classification loss based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determine the weight of the current bounding box regression loss based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determine the weight of the current defect region loss based on the gradient of the total model loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; the weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute the adaptive weights; the calculation formulas for calculating the weights of different losses are as follows:

[0059]

[0060] where i represents different losses, is the weight of the current i-th loss, is the weight of the i-th loss calculated in the previous training, is the gradient of the total model loss calculated in the previous training, is the gradient of the i-th loss calculated in the previous training, is the adjustment step size, which controls the rate of weight adjustment.

[0061] In this embodiment, by calculating the weights of different losses, the weights can be dynamically adjusted according to the requirements of different tasks to ensure that the model has good performance in localization and defect detection.

[0062] In an exemplary embodiment, the determining the localization loss, classification loss, bounding box regression loss, and defect region loss between the defect detection result and the defect annotation information in the current training sample includes:

[0063] Obtain the bounding box output and class output in the defect detection result; obtain the ground truth bounding box and ground truth class in the defect annotation information; determine the localization loss, bounding box regression loss, and defect region loss based on the bounding box output and the ground truth bounding box; determine the classification loss based on the class output and the ground truth class.

[0064] Exemplarily, obtain the bounding box output and class output in the defect detection result; obtain the ground truth bounding box and ground truth class in the defect annotation information; calculate the localization loss based on the bounding box output and the ground truth bounding box , the formula is as follows:

[0065]

[0066] where is the ground truth coordinate of the i-th pixel in the ground truth bounding box, is the predicted coordinate in the bounding box output, and N is the number of pixels in the bounding box output.

[0067]

[0068] Calculate the bounding box regression loss based on the bounding box output and the ground truth bounding box , and the calculation formula is as follows:

[0069]

[0070] where is the bounding box area in the bounding box output, is the bounding box area in the ground truth bounding box, represents the intersection, represents the union. Calculate the defect region loss based on the bounding box output and the ground truth bounding box , and the calculation formula is as follows:

[0071]

[0072] where is the defect region in the bounding box output, is the defect region in the ground truth bounding box, i is the pixel in the bounding box output, and N is the number of pixels in the bounding box output. Calculate the classification loss based on the class output and the ground truth class, and the calculation formula is as follows:

[0073]

[0074] where is the label of the ground truth class, is the predicted class probability, and N is the number of all classes.

[0075] In this embodiment, by calculating the positioning loss and the bounding box regression loss, the positioning accuracy of power equipment can be significantly improved. The defect area loss enhances the detection ability of tiny defects, especially outstanding in the detection of small-range defects on the equipment surface. The classification loss optimizes the classification accuracy of all bounding boxes.

[0076] In an exemplary embodiment, determining the total model loss based on the adaptive weight, the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss includes:

[0077] Obtain the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss in the adaptive weight; based on the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss, determine the weighted sum of the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss to obtain the total model loss.

[0078] Exemplarily, obtain the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss in the adaptive weight; based on the weight of the current positioning loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss, calculate the weighted sum of the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss to obtain the total model loss. The calculation formula is as follows:

[0079]

[0080] Where, is the weight of the positioning loss, is the positioning loss, is the weight of the classification loss, is the classification loss, is the weight of the bounding box regression loss, is the bounding box regression loss, is the weight of the defect area loss, is the defect area loss.

[0081] In this embodiment, by calculating the weighted sum of the positioning loss, the classification loss, the bounding box regression loss, and the defect area loss through the adaptively adjusted adaptive loss to obtain the total model loss, the direction of model optimization can be conveniently controlled.

[0082] In an exemplary embodiment, annotating each original captured image in the original captured image set includes:

[0083] For each original captured image in the original captured image set, perform bounding box annotation on the transformer and defect regions on the original captured image, and label the category for each annotated bounding box, where the category is the transformer or the defect type.

[0084] Optionally, the defect type can be: crack, corrosion, scratch, etc.

[0085] Exemplarily, for each original captured image in the original captured image set, perform bounding box annotation on the transformer and defect regions on the original captured image, and label the category for each annotated bounding box, where the category is the transformer or the defect type, and the defect types include: crack, corrosion, scratch, etc.

[0086] In this embodiment, by annotating the transformer and defect regions in the data, the model can effectively learn the features of the defects.

[0087] In an exemplary embodiment, the image enhancement processing for each original captured image includes:

[0088] Performing at least one of random rotation, scaling, and adding noise on each original captured image.

[0089] Optionally, the image enhancement can also be: grayscale processing, filtering processing, etc.

[0090] Exemplarily, perform random rotation, scaling, adding noise, grayscale processing, filtering processing, etc. on each original captured image in sequence.

[0091] In this embodiment, by performing different image enhancement processing on the image data, a variety of images can be obtained, which is beneficial for the network to learn the features of multiple images.

[0092] In an exemplary embodiment, assume there is an image of a power device. After being detected by the model, the following data is obtained: the true bounding box of the power device: [x1, x2, x3, x4] = [30, 50, 100, 150], the predicted bounding box [x_1, x_2, x_3, x_4] = [30, 50, 100, 150], the class label: "transformer", the predicted class probability: 0.9, and the Dice coefficient of the defect region: 0.8. The process of calculating the total loss of the model is as follows: First, calculate the localization loss, classification loss, bounding box regression loss, and defect region loss. 1. Calculate the localization loss :

[0093]

[0094]

[0095] 2. Calculate the classification loss :

[0096]

[0097] 3. Calculate the bounding box regression loss :

[0098]

[0099] 4. Calculate the defect area loss:

[0100]

[0101] Then, calculate the total loss of the model (Set = 1, = 1, = 1, = 2):

[0102]

[0103] Therefore, the total loss of the model is 4.505.

[0104] In an exemplary embodiment, a method for detecting defects in power equipment, as Figure 2 shown, includes: using a drone to capture images of equipment such as transmission lines and transformers according to a preset route to obtain an original set of captured images including RGB images and infrared images. For each original captured image in the original set of captured images, use LabelImg to perform bounding box annotation on the transformers and defect areas on the original captured image, and label the category for each annotated bounding box. The category is a transformer or a defect type, and the defect types include: cracks, corrosion, scratches, etc., to obtain the defect annotation information corresponding to each original captured image. Perform random rotation, scaling, adding noise, grayscale processing, and filtering processing on each original captured image in sequence to obtain the equipment sample image corresponding to each original captured image; The equipment sample image and defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples form a training sample set. Obtain any set of training samples from the training sample set, input the equipment sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; obtain the bounding box output and category output in the defect detection result; obtain the true bounding box and true category in the defect annotation information; calculate the localization loss , and the formula is as follows:

[0105]

[0106] Wherein, is the true coordinate of the i-th pixel in the true bounding box, are the predicted coordinates in the bounding box output, and N is the number of pixels in the region.

[0107]

[0108] Calculate the bounding box regression loss based on the bounding box output and the ground truth bounding box , and the calculation formula is as follows:

[0109]

[0110] where is the bounding box area in the bounding box output, is the bounding box area in the ground truth bounding box, represents the intersection, represents the union. Calculate the defect area loss based on the bounding box output and the ground truth bounding box , and the calculation formula is as follows:

[0111]

[0112] where is the defect area in the bounding box output, is the defect area in the ground truth bounding box. Calculate the classification loss based on the class output and the ground truth class, and the calculation formula is as follows:

[0113]

[0114] where is the label of the ground truth class, is the predicted class probability, and N is the number of all classes. Obtain the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect area loss calculated in the previous training, and the weight of the defect area loss calculated in the previous training; based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training, determine the weight of the current classification loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training, determine the weight of the current bounding box regression loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the defect area loss calculated in the previous training, and the weight of the defect area loss calculated in the previous training, determine the weight of the current defect area loss; the weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss together constitute the adaptive weight; the calculation formulas for the weights of different losses are as follows:

[0115]

[0116] where i represents different losses, is the weight of the current i-th loss, is the weight of the i-th loss calculated in the previous training, is the gradient of the total model loss calculated in the previous training, is the gradient of the i-th loss calculated in the previous training, is the adjustment step size, which controls the rate of weight adjustment. Obtain the weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss in the adaptive weight; based on the weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect area loss, calculate the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect area loss to obtain the total model loss. The calculation formula is as follows:

[0117]

[0118] where, is the weight of the localization loss, is the localization loss, is the weight of the classification loss, is the classification loss, is the weight of the bounding box regression loss, is the bounding box regression loss, is the weight of the defect area loss, is the defect area loss. The parameters of the defect detection model to be trained are adjusted based on the total model loss, and the defect detection model after parameter adjustment is continuously trained based on the training sample set until the calculated total model loss reaches the preset condition, and a trained defect detection model is obtained; the trained defect detection model is used to detect defects in power equipment.

[0119] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the indications of the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages, and these steps or stages do not necessarily have to be executed at the same moment, but can be executed at different moments, and the execution order of these steps or stages does not necessarily have to be sequential, but can be rotated or alternated with at least a part of other steps or steps or stages in other steps.

[0120] In an exemplary embodiment, as Figure 3 shown, a power equipment defect detection device is provided, including: an acquisition module 301, a training module 302, and a detection module 303, where:

[0121] The acquisition module is used to use a drone to photograph a power equipment, obtain an original photographed image set, annotate each original photographed image in the original photographed image set to obtain defect annotation information corresponding to each original photographed image, and perform image enhancement processing on each original photographed image to obtain a device sample image corresponding to each original photographed image; the device sample image and defect annotation information corresponding to each original photographed image are used as a set of training samples, and multiple sets of training samples constitute a training sample set;

[0122] The training module is used to obtain any set of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; determine an adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss, determine the total model loss;

[0123] A detection module, configured to adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition, so as to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0124] In one embodiment, the training module is further configured to:

[0125] Obtain the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; determine the current weight of the localization loss based on the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, and the weight of the localization loss calculated in the previous training; determine the current weight of the classification loss based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determine the current weight of the bounding box regression loss based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determine the current weight of the defect region loss based on the gradient of the total model loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; the current weights of the localization loss, the classification loss, the bounding box regression loss, and the defect region loss together constitute the adaptive weights.

[0126] In one embodiment, the training module is further configured to:

[0127] Obtain the bounding box output and the class output in the defect detection result; obtain the true bounding box and the true class in the defect annotation information; determine the localization loss, the bounding box regression loss, and the defect region loss based on the bounding box output and the true bounding box; determine the classification loss based on the class output and the true class.

[0128] In one embodiment, the training module is further configured to:

[0129] Obtain the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss in the adaptive weights; based on the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss, determine the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect area loss to obtain the total model loss.

[0130] In one embodiment, the obtaining module is further configured to:

[0131] For each original captured image in the original captured image set, perform bounding box annotation on the transformers and defect areas on the original captured image, and annotate each labeled bounding box with a category, where the category is a transformer or a defect type.

[0132] In one embodiment, the obtaining module is further configured to:

[0133] Perform at least one of random rotation, scaling, and adding noise on each original captured image.

[0134] Each module in the above power equipment defect detection device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0135] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 4 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data of the original captured image set. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a power equipment defect detection method.

[0136] Those skilled in the art can understand, Figure 4The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0137] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0138] Use a drone to photograph power equipment to obtain an original set of photographed images. Annotate each original photographed image in the original set of photographed images to obtain defect annotation information corresponding to each original photographed image. Perform image enhancement processing on each original photographed image to obtain a device sample image corresponding to each original photographed image; The device sample image and defect annotation information corresponding to each original photographed image are used as a set of training samples, and multiple sets of training samples constitute a training sample set;

[0139] Obtain any set of training samples from the training sample set. Input the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; Determine the localization loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; Determine the adaptive weight; Based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss, determine the total model loss;

[0140] Based on the total model loss, adjust the parameters of the defect detection model to be trained. Based on the training sample set, continue to train the defect detection model with adjusted parameters until the calculated total model loss reaches a preset condition to obtain a trained defect detection model; The trained defect detection model is used to detect defects in power equipment.

[0141] In an embodiment, when the processor executes the computer program, the following steps are also implemented:

[0142] Obtain the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; determine the weight of the current localization loss based on the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, and the weight of the localization loss calculated in the previous training; determine the weight of the current classification loss based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training; determine the weight of the current bounding box regression loss based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training; determine the weight of the current defect region loss based on the gradient of the total model loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training; the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect region loss together constitute the adaptive weights.

[0143] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0144] Obtain the bounding box output and the class output in the defect detection result; obtain the ground truth bounding box and the ground truth class in the defect annotation information; determine the localization loss, the bounding box regression loss, and the defect region loss based on the bounding box output and the ground truth bounding box; determine the classification loss based on the class output and the ground truth class.

[0145] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0146] Obtain the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect region loss in the adaptive weights; determine the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect region loss based on the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect region loss to obtain the total model loss.

[0147] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0148] For each original captured image in the original captured image set, perform bounding box annotation on the transformer and the defect area on the original captured image, and annotate the category for each annotated bounding box, where the category is the transformer or the defect type.

[0149] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0150] Perform at least one of random rotation, scaling, and adding noise on each original captured image.

[0151] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0152] Use a drone to capture power equipment to obtain an original captured image set, annotate each original captured image in the original captured image set to obtain defect annotation information corresponding to each original captured image, perform image enhancement processing on each original captured image to obtain a device sample image corresponding to each original captured image; the device sample image and the defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples constitute a training sample set;

[0153] Obtain any set of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; determine the adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss, determine the total model loss;

[0154] Based on the total model loss, adjust the parameters of the defect detection model to be trained, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0155] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0156] Obtain the gradients of the total model loss calculated in the previous training, the gradients of the localization loss calculated in the previous training, the weights of the localization loss calculated in the previous training, the gradients of the classification loss calculated in the previous training, the weights of the classification loss calculated in the previous training, the gradients of the bounding box regression loss calculated in the previous training, the weights of the bounding box regression loss calculated in the previous training, the gradients of the defect area loss calculated in the previous training, and the weights of the defect area loss calculated in the previous training; Based on the gradients of the total model loss calculated in the previous training, the gradients of the localization loss calculated in the previous training, and the weights of the localization loss calculated in the previous training, determine the weight of the current localization loss; Based on the gradients of the total model loss calculated in the previous training, the gradients of the classification loss calculated in the previous training, and the weights of the classification loss calculated in the previous training, determine the weight of the current classification loss; Based on the gradients of the total model loss calculated in the previous training, the gradients of the bounding box regression loss calculated in the previous training, and the weights of the bounding box regression loss calculated in the previous training, determine the weight of the current bounding box regression loss; Based on the gradients of the total model loss calculated in the previous training, the gradients of the defect area loss calculated in the previous training, and the weights of the defect area loss calculated in the previous training, determine the weight of the current defect area loss; The weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss together constitute the adaptive weights.

[0157] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0158] Obtain the bounding box output and the class output in the defect detection result; Obtain the ground truth bounding box and the ground truth class in the defect annotation information; Based on the bounding box output and the ground truth bounding box, determine the localization loss, the bounding box regression loss, and the defect area loss; Based on the class output and the ground truth class, determine the classification loss.

[0159] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0160] Obtain the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss in the adaptive weights; Based on the weights of the current localization loss, the weights of the current classification loss, the weights of the current bounding box regression loss, and the weights of the current defect area loss, determine the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect area loss to obtain the total model loss.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0162] For each original captured image in the original captured image set, perform bounding box annotation on the transformer and defect areas on the original captured image, and label each annotated bounding box with a category, where the category is a transformer or a defect type.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0164] Perform at least one of random rotation, scaling, and adding noise to each original captured image.

[0165] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the following steps:

[0166] Use a drone to capture power equipment to obtain an original captured image set, annotate each original captured image in the original captured image set to obtain defect annotation information corresponding to each original captured image, perform image enhancement processing on each original captured image to obtain a device sample image corresponding to each original captured image; the device sample image and defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples form a training sample set;

[0167] Obtain any set of training samples from the training sample set, input the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect area loss between the defect detection result and the defect annotation information in the current training sample; determine the adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect area loss, determine the total model loss;

[0168] Based on the total model loss, adjust the parameters of the defect detection model to be trained, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0170] Obtain the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect area loss calculated in the previous training, and the weight of the defect area loss calculated in the previous training; based on the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, and the weight of the localization loss calculated in the previous training, determine the weight of the current localization loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training, determine the weight of the current classification loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training, determine the weight of the current bounding box regression loss; based on the gradient of the total model loss calculated in the previous training, the gradient of the defect area loss calculated in the previous training, and the weight of the defect area loss calculated in the previous training, determine the weight of the current defect area loss; the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss together constitute the adaptive weights.

[0171] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0172] Obtain the bounding box output and the class output in the defect detection result; obtain the ground truth bounding box and the ground truth class in the defect annotation information; based on the bounding box output and the ground truth bounding box, determine the localization loss, the bounding box regression loss, and the defect area loss; based on the class output and the ground truth class, determine the classification loss.

[0173] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0174] Obtain the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss in the adaptive weights; based on the weights of the current localization loss, the current classification loss, the current bounding box regression loss, and the current defect area loss, determine the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect area loss to obtain the total model loss.

[0175] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0176] For each original captured image in the original captured image set, perform bounding box annotation on the transformers and defect regions on the original captured image, and label each bounding box with a category, where the category is a transformer or a defect type.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0178] Perform at least one of random rotation, scaling, and adding noise to each original captured image.

[0179] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, Resistive Random Access Memory (ReRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene memory, etc. Volatile memory can include Random Access Memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, Artificial Intelligence (AI) processors, etc., without limitation.

[0180] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope recorded in this application.

[0181] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for detecting defects in power equipment, characterized in that, The method includes: Using a drone to photograph power equipment to obtain an original set of photographed images, annotating each original photographed image in the original set of photographed images to obtain defect annotation information corresponding to each original photographed image, and performing image enhancement processing on each original photographed image to obtain a device sample image corresponding to each original photographed image; The device sample image and defect annotation information corresponding to each original photographed image are used as a set of training samples, and multiple sets of training samples form a training sample set. Obtaining any set of training samples from the training sample set, inputting the device sample image in the current training sample into the defect detection model to be trained to obtain a defect detection result; determining the localization loss, classification loss, bounding box regression loss, and defect region loss between the defect detection result and the defect annotation information in the current training sample; determining an adaptive weight; based on the adaptive weight, the localization loss, the classification loss, the bounding box regression loss, and the defect region loss, determining the total model loss. Based on the total model loss, adjusting the parameters of the defect detection model to be trained, and continuing to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss reaches a preset condition to obtain a trained defect detection model; The trained defect detection model is used to detect defects in power equipment.

2. The method according to claim 1, characterized in that, The determining of the adaptive weight includes: Obtaining the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, the weight of the localization loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, the weight of the classification loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, the weight of the bounding box regression loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training. Based on the gradient of the total model loss calculated in the previous training, the gradient of the localization loss calculated in the previous training, and the weight of the localization loss calculated in the previous training, determining the weight of the current localization loss. Based on the gradient of the total model loss calculated in the previous training, the gradient of the classification loss calculated in the previous training, and the weight of the classification loss calculated in the previous training, determining the weight of the current classification loss. Based on the gradient of the total model loss calculated in the previous training, the gradient of the bounding box regression loss calculated in the previous training, and the weight of the bounding box regression loss calculated in the previous training, determining the weight of the current bounding box regression loss. Based on the gradient of the total model loss calculated in the previous training, the gradient of the defect region loss calculated in the previous training, and the weight of the defect region loss calculated in the previous training, determining the weight of the current defect region loss. The weight of the current localization loss, the weight of the current classification loss, the weight of the current bounding box regression loss, and the weight of the current defect region loss together constitute the adaptive weight.

3. The method according to claim 1, characterized in that, Determining the localization loss, classification loss, bounding box regression loss, and defect region loss between the defect detection result and the defect annotation information in the current training sample includes: Obtaining the bounding box output and class output in the defect detection result; obtaining the ground truth bounding box and ground truth class in the defect annotation information; Determining the localization loss, bounding box regression loss, and defect region loss based on the bounding box output and the ground truth bounding box; Determining the classification loss based on the class output and the ground truth class.

4. The method according to claim 2, characterized in that, Determining the total model loss based on the adaptive weights, the localization loss, the classification loss, the bounding box regression loss, and the defect region loss includes: Obtaining the weights of the current localization loss, current classification loss, current bounding box regression loss, and current defect region loss in the adaptive weights; Determining the weighted sum of the localization loss, the classification loss, the bounding box regression loss, and the defect region loss based on the weights of the current localization loss, current classification loss, current bounding box regression loss, and current defect region loss to obtain the total model loss.

5. The method according to claim 1, characterized in that, Annotating each original captured image in the original captured image set includes: For each original captured image in the original captured image set, performing bounding box annotation on the transformer and defect regions on the original captured image, and annotating the class for each annotated bounding box, where the class is the transformer or the defect type.

6. The method according to claim 1, characterized in that, Performing image enhancement processing on each original captured image includes: Performing at least one of random rotation, scaling, and adding noise on each original captured image.

7. A device for detecting defects in power equipment, characterized in that, The device includes: An acquisition module, configured to use a drone to capture power equipment to obtain an original captured image set, annotate each original captured image in the original captured image set to obtain defect annotation information corresponding to each original captured image, perform image enhancement processing on each original captured image to obtain a device sample image corresponding to each original captured image; the device sample image and the defect annotation information corresponding to each original captured image are used as a set of training samples, and multiple sets of training samples constitute a training sample set; A training module, configured to obtain any set of training samples from the training sample set, input the device sample image in the current training sample into a defect detection model to be trained to obtain a defect detection result; determine the localization loss, classification loss, bounding box regression loss, and defect region loss between the defect detection result and the defect annotation information in the current training sample; determine adaptive weights; determine the total model loss based on the adaptive weights, the localization loss, the classification loss, the bounding box regression loss, and the defect region loss; A detection module, configured to adjust the parameters of the defect detection model to be trained based on the total model loss, and continue to train the defect detection model with adjusted parameters based on the training sample set until the calculated total model loss meets a preset condition to obtain a trained defect detection model; the trained defect detection model is used to detect defects in power equipment.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images