Insulator condition detection method and insulator condition detection model training method

Abstract

This application discloses an insulator condition detection method and an insulator condition detection model training method. A target insulator localization model is trained on a first dataset, where each training sample includes an insulator image and its location information. A target insulator classification model is trained on a second dataset, where each training sample includes a first bounding box image and its label information, where the label information indicates whether the insulator displayed in the first bounding box image is a defect-free insulator or a defective insulator. The target insulator localization model achieves good training results based on a large number of defect-free insulator samples. The target insulator classification model has a relatively simple network structure and achieves good training results even with a smaller number of defect-free and defective insulator samples. Therefore, using both models for insulator condition detection can ensure detection accuracy even with a limited number of defective insulator samples.

Description

Technical Field

[0001] This application relates to the field of insulator detection, specifically to an insulator condition detection method and an insulator condition detection model training method. Background Technology

[0002] Insulators are crucial components of overhead transmission lines and an essential part of power grid equipment. They provide electrical insulation and mechanical fixation within the transmission line. Therefore, damage to insulators caused by various internal or external factors, especially electrical damage, can lead to insulator defects, degrading their performance and even affecting the operation of the entire transmission line. Thus, real-time defect detection of each insulator to ensure accurate functional operation is of paramount importance to power companies.

[0003] Currently, a single neural network model is used to simultaneously identify the location of insulators in the image under test and classify the identified insulators. Obviously, to complete both of these operations at the same time, the neural network model would be very complex. Furthermore, due to the small number of image samples of defective insulators, the training effect of this neural network model is generally poor, resulting in a low detection rate and accuracy of insulator defects in the image, which cannot meet the detection requirements. Summary of the Invention

[0004] This application provides an insulator condition detection method and an insulator condition detection model training method to improve the speed and accuracy of insulator condition detection.

[0005] The first aspect of this application provides a method for training an insulator state detection model, the method comprising:

[0006] Obtain a first dataset, wherein each training sample in the first dataset includes an image of an insulator containing defective insulators and / or a defect-free insulator, and the location information of the insulators in the insulator images;

[0007] An initial insulator localization model is obtained. The first dataset is input into the initial insulator localization model to obtain the position information output by the initial insulator localization model. Training is stopped when the relationship between the output position information and the position information of the insulator in the insulator image meets the convergence condition, and the target insulator localization model is obtained.

[0008] Obtain a second dataset. Each training sample in the second dataset includes a first bounding box image and label information for the first bounding box image. Each first bounding box image displays a single insulator. The label information of the first bounding box image is used to indicate whether the insulator displayed in the first bounding box image is a defect-free insulator or a defective insulator.

[0009] An initial insulator classification model is obtained. The second dataset is input into the initial insulator classification model to obtain the label information output by the initial insulator classification model. Training is stopped when the relationship between the output label information and the label information of the first bounding box image meets the convergence condition, and the target insulator classification model is obtained.

[0010] The target insulator positioning model is used to identify the position information of the insulator in the target image, and the target insulator classification model is used to classify the second frame image obtained based on the position information of the insulator in the target image. The classification result is used to indicate whether the insulator in the second frame image is a defect-free insulator or a defective insulator.

[0011] A second aspect of this application provides an insulator condition detection method, the method comprising:

[0012] Acquire the target image showing insulators to be detected;

[0013] A pre-trained target insulator localization model is obtained. The target insulator localization model is trained on a first dataset by a machine learning algorithm. Each training sample in the first dataset includes an insulator image containing defective insulators and / or defect-free insulators, as well as the location information of the insulators in the insulator image.

[0014] A pre-trained target insulator classification model is obtained. The target insulator classification model is trained on a second dataset by a machine learning algorithm. Each training sample in the second dataset includes a first frame image and label information of the first frame image. Each first frame image displays a single insulator. The label information of the first frame image is used to indicate whether the insulator displayed in the first frame image is a defect-free insulator or a defective insulator.

[0015] The target image is input into the target insulator positioning model to obtain the position information output by the target insulator positioning model. Based on the position information output by the target insulator positioning model, a frame surrounding a single insulator is determined, and the frame surrounding the single insulator is cropped to obtain a frame image.

[0016] The frame image is input into the target insulator classification model to obtain the label information output by the target insulator classification model. The label information output by the target insulator classification model is used to indicate whether the insulator displayed in the frame image is a defect-free insulator or a defective insulator.

[0017] A third aspect of this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods of the first and / or second aspects described above.

[0018] A fourth aspect of this application provides a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods of the first and / or second aspects described above.

[0019] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0020] The target insulator localization model is trained on a first dataset using a machine learning algorithm. Each training sample in the first dataset includes images of insulators containing defective and / or defect-free insulators, along with the insulator's location information within the images. The target insulator classification model is trained on a second dataset using a machine learning algorithm. Each training sample in the second dataset includes a first bounding box image and its label information. The label information of the first bounding box image indicates whether the insulator displayed in the image is defect-free or defective. Since there are a large number of defect-free insulator samples, the target insulator localization model can achieve good training results by training on a large number of defect-free insulator sample images and a smaller number of defective insulator sample images. The target insulator classification model, because it only performs a classification task, has a relatively simple network structure and can also achieve good training results by training on a large number of defect-free insulator sample images and a smaller number of defective insulator sample images. Therefore, both models can be used for insulator condition detection, ensuring both accuracy and speed, and meeting the detection requirements. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the insulator state detection model training method in an embodiment of this application;

[0022] Figure 2 This is another flowchart illustrating the insulator state detection model training method in this application embodiment;

[0023] Figure 3 This is a schematic diagram of an insulator image in an embodiment of this application;

[0024] Figure 4This is a schematic diagram of the Extended-ELAN (E-ELAN) structure in the YOLOv7 algorithm model in this application embodiment;

[0025] Figure 5 In the embodiments of this application, by Figure 3 A schematic diagram of a frame-shaped image surrounding a single insulator obtained by cropping an image of an insulator shown;

[0026] Figure 6 This is a schematic diagram of the Bottleneck structure in the ResNet50 model in an embodiment of this application;

[0027] Figure 7 This is a flowchart illustrating the insulator condition detection method in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of a computer device in an embodiment of this application. Detailed Implementation

[0029] This application provides an insulator condition detection method and an insulator condition detection model training method to improve the speed and accuracy of insulator condition detection.

[0030] The training method for the insulator state detection model in the embodiments of this application is described below:

[0031] Please see Figure 1 One embodiment of the insulator state detection model training method in this application includes:

[0032] 101. Obtain a first dataset, wherein each training sample in the first dataset includes an image of an insulator containing defective insulators and / or a defect-free insulator, and the location information of the insulators in the insulator images;

[0033] The method in this embodiment can be applied to computer devices, which can be any device with data processing capabilities, such as terminals, servers, and other computer devices.

[0034] In this embodiment, the insulator condition detection model is used to identify and detect the condition of insulators, that is, to detect whether there are defects in the insulators. Defects in insulators can include cracks caused by poor product quality, electrical damage caused by various factors, etc. The insulator condition detection model consists of two models: an insulator location model and an insulator classification model. Therefore, the computer equipment trains these two models respectively, and after training, these two models can be used to identify insulator defects.

[0035] The first dataset is used to train the insulator localization model. In each training sample, the insulator image shows one or more insulators, which may include defective insulators and / or defect-free insulators. The insulator images can be obtained by capturing images of the insulators with a camera, such as by using a drone carrying a camera to photograph the insulators in outdoor overhead transmission lines.

[0036] After obtaining the insulator image, personnel can mark the position of each insulator in the insulator image, such as using a frame of any shape to mark the position of the insulator in the insulator image. Then, the computer equipment determines the position information of the insulator in the insulator image based on the position marked by the personnel. This position information can be represented by coordinates, or it can be represented by the direction and distance of the position marked by the personnel relative to a reference point in the insulator image. There is no limitation here.

[0037] 102. Obtain the initial insulator localization model. Input the first dataset into the initial insulator localization model to obtain the position information output by the initial insulator localization model. Stop training when the relationship between the output position information and the position information of the insulator in the insulator image meets the convergence condition, and obtain the target insulator localization model.

[0038] The initial insulator positioning model can be pre-set with hyperparameters by personnel. The computer equipment acquires the initial insulator positioning model and trains it using the first dataset based on a deep learning algorithm. The first dataset is input into the initial insulator positioning model to obtain the position information output by the initial insulator positioning model. Training stops when the relationship between the position information output by the initial insulator positioning model and the position information of the insulator in the insulator image meets the convergence condition, thus obtaining the target insulator positioning model.

[0039] The convergence condition can be that the loss value tends to stabilize, the number of iterations reaches a preset number, or the change in the model weights between any two model iterations is less than a preset threshold. This embodiment does not limit the conditions for meeting the convergence condition.

[0040] 103. Obtain the second dataset. Each training sample in the second dataset includes a first frame image and the label information of the first frame image. Each first frame image displays a single insulator. The label information of the first frame image is used to indicate whether the insulator displayed in the first frame image is a defect-free insulator or a defective insulator.

[0041] The second dataset is used to train the insulator classification model. In each training sample, the first frame image shows a single insulator, and the label information of the first frame image is used to indicate whether the insulator shown in the first frame image is a defect-free insulator or a defective insulator.

[0042] 104. Obtain an initial insulator classification model. Input the second dataset into the initial insulator classification model to obtain the label information output by the initial insulator classification model. Stop training when the relationship between the output label information and the label information of the first bounding box image meets the convergence condition, and obtain the target insulator classification model.

[0043] The initial insulator classification model can be pre-set with hyperparameters by personnel. The computer equipment acquires the initial insulator classification model and trains it using a second dataset based on a deep learning algorithm. The second dataset is input into the initial insulator classification model to obtain the label information output by the initial insulator classification model. Training stops when the relationship between the output label information and the label information of the first bounding box image meets the convergence condition, thus obtaining the target insulator classification model.

[0044] The convergence conditions are similar to those for the initial insulator positioning model training, and will not be repeated here.

[0045] After obtaining the target insulator location model and the target insulator classification model, the target insulator location model can be used to identify the position information of the insulators in the target image to be detected, and the target insulator classification model can be used to classify the second frame image extracted based on the position information of the insulators in the target image. The classification result is used to indicate whether the insulator in the second frame image is a defect-free insulator or a defective insulator. Therefore, after completing the identification of the position information of each insulator displayed in the target image and the classification of the second frame image corresponding to the position information of each insulator, the classification result of each insulator displayed in the target image can be output, thus completing the state detection of the insulators.

[0046] In this embodiment, the target insulator localization model is trained using a machine learning algorithm on a first dataset. Each training sample in the first dataset includes images of insulators containing defective and / or defect-free insulators, along with the location information of the insulators within those images. The target insulator classification model is trained using a machine learning algorithm on a second dataset. Each training sample in the second dataset includes a first bounding box image and its label information. The label information of the first bounding box image indicates whether the insulator displayed in the first bounding box image is defect-free or defective. Since there are a large number of defect-free insulator samples, the target insulator localization model can achieve good training results by training on a large number of defect-free insulator sample images and a smaller number of defective insulator sample images. Similarly, the target insulator classification model, which only performs classification tasks, has a relatively simple network structure and can also achieve good training results by training on a large number of defect-free insulator sample images and a smaller number of defective insulator sample images. Therefore, using both models for insulator condition detection can ensure both accuracy and speed, meeting the detection requirements.

[0047] The following will be discussed in the preceding text. Figure 1 Based on the illustrated embodiments, embodiments of this application will be described in further detail. Please refer to [link to relevant documentation]. Figure 2 Another embodiment of the insulator state detection model training method in this application includes:

[0048] 201. Obtain a first dataset, wherein each training sample in the first dataset includes an image of an insulator containing defective insulators and / or an insulator without defects, and the location information of the insulators in the insulator images;

[0049] In this embodiment, the position information of the insulator in the insulator image is determined based on the position of the insulator marked by the personnel. For example, as Figure 3 As shown, the insulator image displays multiple insulators and transmission lines, which can be captured by a camera. Personnel can mark the positions of the insulators in the image, as shown in the figure, with the positions of the insulators indicated by a rectangular dashed frame. Then, computer equipment determines the position information of the insulators in the image based on the positions marked by the personnel. For example, a coordinate system can be established, and the coordinates of any reference point of the rectangular dashed frame can be determined in the coordinate system. For example, the line containing the left side of the insulator image can be used as the y-axis, and the line containing the bottom side can be used as the x-axis. The position information of the insulator can be represented by [xmin, ymin, xmax, ymax], where [xmin, ymin] are the coordinates of the lower left corner vertex of the circumscribed rectangular dashed frame, and [xmax, ymax] are the coordinates of the upper right corner vertex of the circumscribed rectangular dashed frame.

[0050] Of course, the position information of the insulator in the insulator image can also be represented by the direction and distance of the position marked by the personnel relative to a preset reference point in the insulator image. This embodiment does not limit the way the position information of the insulator in the insulator image is represented.

[0051] 202. Obtain the initial insulator localization model. Input the first dataset into the initial insulator localization model to obtain the position information output by the initial insulator localization model. Stop training when the relationship between the output position information and the position information of the insulator in the insulator image meets the convergence condition, and obtain the target insulator localization model.

[0052] In this embodiment, the initial insulator localization model can be any neural network model. Preferably, the initial insulator localization model can be a neural network model built based on the YOLOv7 algorithm. The YOLO algorithm is a high-performance object detection algorithm that can perform object detection in real time. Unlike traditional object detection algorithms that evaluate different slices of a test image using a classifier, the YOLO algorithm transforms object detection into a regression problem, treating the entire test image as the detection target and performing a test only once for each image. In this process, the YOLO algorithm divides the image into a grid. If the center of an object falls into a certain grid, the object is detected through that grid, and the class probability and bounding box of each object are predicted, ultimately achieving the detection of all objects. Because YOLO utilizes global information of the input image during object detection, the YOLO algorithm has excellent object detection performance. The YOLOv7 algorithm model is the latest YOLO series model. Based on YOLOv4, this model uses bag-of-freebies, thereby improving detection accuracy without increasing inference cost; it also introduces the Extended-ELAN (E-ELAN) structure, which is as follows: Figure 4 As shown, it utilizes expand, shuffle, and mergecardinality to continuously enhance the network's learning ability without disrupting the original gradient path, thereby improving the overall performance of YOLOv7. Simultaneously, YOLOv7's model scaling can generate models of different scales to meet diverse model requirements, facilitating use on various devices and enhancing its practicality. In short, YOLOv7 is a highly suitable and high-performance target detection algorithm for insulator detection.

[0053] When training the initial insulator positioning model, the first dataset used for model training can be either a training set or a combination of a training set and a validation set. Therefore, in a preferred embodiment, the first dataset includes a first training set and a first validation set. Each training sample in the first training set and the first validation set includes an insulator image and the position information of the insulators within the insulator image. During training, the first training set is input into the initial insulator positioning model to obtain the position information output by the initial insulator positioning model. Training stops when the relationship between the output position information and the position information of the insulators in the insulator image meets the convergence condition. After stopping training on the first training set, the first validation set is input into the initial insulator positioning model to obtain the position information output by the initial insulator positioning model. Training stops when the relationship between the output position information and the position information of the insulators in the insulator image meets the convergence condition. The hyperparameters of the initial insulator positioning model are adjusted, and the step of inputting the first training set into the initial insulator positioning model is repeated until M training iterations of the initial insulator positioning model are completed based on the first training set and the first validation set, where M is a positive integer greater than 1.

[0054] For example, when the initial insulator positioning model is a YOLOv7 model, its training process is as follows:

[0055] 1) The computer equipment should be configured according to the user manual of the YOLOv7 official open-source code repository under the operation of personnel;

[0056] 2) Place the first dataset in the specified directory;

[0057] 3) In the configuration file of the initial insulator localization model, set the network training hyperparameters, the path of the first dataset, etc., and set the number of target categories of the model to 1, that is, treat defective insulators and defect-free insulators as the same target category and do not distinguish them.

[0058] 4) Input the first training set into the initial insulator positioning model for iterative training. Calculate the loss value during the iterative training process and represent the change of the loss value through a visual loss value curve. The loss value of the current model on the first training set can be observed at certain iterations. Finally, the model with a stable loss value under the current hyperparameters is saved.

[0059] 5) After stopping training on the first training set, the first validation set is input into the initial insulator positioning model for iterative training. During the iterative training process, the loss value is calculated and the change of the loss value is represented by a visual loss value curve. The loss value of the current model on the first validation set can be observed at certain iterations. Finally, the model with the loss value tending to be stable under the current hyperparameters is saved, and the training ends.

[0060] 6) The computer equipment adjusts the hyperparameters of the initial insulator positioning model under the operation of the personnel, and returns to step 4 to perform the next model iteration training until the initial insulator positioning model has been trained M times.

[0061] After M training iterations, with the model's weights and hyperparameters saved after each iteration, the training effectiveness of these M iterations can be evaluated to select the model with the best training performance.

[0062] Therefore, in a preferred embodiment, when training the initial insulator positioning model for each time based on the first training set and the first validation set, the evaluation index value of the training effect of the initial insulator positioning model is determined according to the position information of the insulator in the insulator image and the position information output by the initial insulator positioning model; wherein, if the evaluation index value corresponding to the i-th training is the best among the index values ​​of N evaluation indices, then the target insulator positioning model obtained by the i-th training is used to identify the position information of the insulator in the target image, 1≤i≤M, where i is a positive integer.

[0063] For example, if the initial insulator positioning model has been trained 10 times, and the corresponding evaluation index values ​​have been calculated for each of the 10 training sessions, and the evaluation index value corresponding to the 6th training session is the best, then the target insulator positioning model obtained from the 6th training session can be used to identify the position information of the insulator in the target image to be detected.

[0064] Evaluation metrics for the model include commonly used metrics in object detection such as mAP, Recall, F1-score, and Precision. mAP stands for Mean Average Precision, where Average Precision is the average precision of each detection result across all result types, expressed as AP. Mean is the average across all detection types, calculated over AP. Recall reflects the percentage of correctly predicted object detections out of the original labeled object count. Precision represents the percentage of correctly predicted object detections out of the total predicted object count; in other words, how many predictions are accurate. F1-score is a final evaluation metric, a harmonic averaging of precision and recall: 2 / F1score = 1 / Precision + 1 / Recall. It reflects the overall prediction performance in the current detection environment. FPS stands for Frames Per Second. Higher FPS indicates a faster object detection speed in images, and it's used to evaluate the detection rate of power grid equipment, a crucial indicator for achieving real-time online detection.

[0065] After training the target insulator positioning model using the training and validation sets, a test set can be used to validate the model's final performance indicators. Therefore, the computer device can also acquire a first test set, where each sample includes images of insulators containing defective and / or defect-free insulators, along with the positional information of the insulators within those images. The first test set is then input into the target insulator positioning model to obtain the output positional information. Based on this output positional information and the positional information of the insulators in the images within the first test set, the performance index values ​​of the target insulator positioning model are determined.

[0066] 203. Obtain the second dataset. Each training sample in the second dataset includes a first frame image and the label information of the first frame image. Each first frame image displays a single insulator. The label information of the first frame image is used to indicate whether the insulator displayed in the first frame image is a defect-free insulator or a defective insulator.

[0067] In this embodiment, each training sample in the first dataset also includes label information for each insulator in the insulator image. The label information of the insulator in the insulator image is used to indicate whether the insulator in the insulator image is a defect-free insulator or a defective insulator. This label information can be pre-annotated by personnel. For example, the entire first dataset can be labeled using the PASCALVOC general object detection annotation format, and an XML file can be output. The labeled insulator attributes include: the insulator category, such as the label for a defective insulator can be "Defect_Insulator" (the English equivalent of "Defective Insulator"), and the label for a defect-free insulator can be "Normal_Insulator" (the English equivalent of "Normal Insulator").

[0068] Therefore, the computer device can sequentially determine the bounding box surrounding a single insulator based on each position information of the insulator in the insulator image, and crop the bounding box surrounding the single insulator to obtain a first frame image. The first frame image and the label information of the insulator displayed therein constitute a set of training samples for the second dataset.

[0069] by Figure 3 Taking the insulator image shown as an example, the computer equipment determines the rectangular dashed frame based on the coordinate information of various reference points (such as the upper right corner and lower left corner) of the rectangular dashed frame pre-marked by the operator. This rectangular dashed frame surrounds a single insulator, and then the rectangular dashed frame is extracted to obtain the image shown. Figure 5The frame image shown can be used to train an initial insulator classification model because the insulators displayed in the frame image have corresponding pre-labeled information.

[0070] Depend on Figure 5 As can be seen, the first frame image shows insulators in most areas. Compared with the original insulator image, it removes a large amount of background and other elements (such as transmission lines) that are not related to insulator classification. It eliminates the interference of image background and irrelevant elements, making it easier to train the insulator classification model and perform actual insulator classification tasks.

[0071] 204. Obtain an initial insulator classification model. Input the second dataset into the initial insulator classification model to obtain the label information output by the initial insulator classification model. Stop training when the relationship between the output label information and the label information of the first bounding box image meets the convergence condition, and obtain the target insulator classification model.

[0072] In this embodiment, the initial insulator classification model can be any neural network model. Preferably, the initial insulator classification model can be a neural network model built based on the ResNet50 algorithm. ResNet50 is one of the classic ResNet models, which solves the vanishing gradient problem and the exploding gradient problem in convolutional neural networks through its core component, Bottleneck. The structure of Bottleneck is as follows: Figure 6 As shown, by using ResNet50 as the basis for the insulator classification model, the final trained model can obtain more ideal insulator state classification results when classifying insulators.

[0073] When training the initial insulator classification model, the second dataset used for model training can be either entirely a training set or a combination of a training set and a validation set. Therefore, in a preferred embodiment, the second dataset includes a second training set and a second validation set. Each training sample in the second training set and the second validation set includes a first bounding box image and its label information. During model training, the second training set is input into the initial insulator classification model to obtain the label information output by the initial insulator classification model. Training stops when the relationship between the output label information and the label information of the first bounding box image satisfies the convergence condition. After stopping training on the second training set, the second validation set is input into the initial insulator classification model to obtain the label information output by the initial insulator classification model. Training stops when the relationship between the output label information and the label information of the first bounding box image satisfies the convergence condition. The hyperparameters of the initial insulator classification model are adjusted, and the step of inputting the second training set into the initial insulator classification model is repeated until N training iterations of the initial insulator classification model are completed based on the second training set and the second validation set, where N is a positive integer greater than 1.

[0074] For example, when the initial insulator classification model is a ResNet50 model, its training process is as follows:

[0075] 1) The computer equipment is configured according to the user manual of the ResNet50 official open-source code repository under the operation of personnel;

[0076] 2) Place the second dataset in the specified directory;

[0077] 3) In the configuration file of the initial insulator classification model, set the network training hyperparameters, the path of the second dataset, etc., and set the number of target categories of the model to 2, namely defective insulators and defect-free insulators;

[0078] 4) Input the second training set into the initial insulator classification model for iterative training. Calculate the loss value during the iterative training process and represent the change of the loss value through a visual loss value curve. The loss value of the current model on the second training set can be observed at certain iterations. Finally, save the model whose loss value tends to be stable under the current hyperparameters.

[0079] 5) After stopping training on the second training set, input the second validation set into the initial insulator classification model for iterative training. Calculate the loss value during the iterative training process and represent the change in the loss value through a visual loss value curve. The loss value of the current model on the second validation set can be observed at certain iterations. Finally, save the model whose loss value tends to be stable under the current hyperparameters and end the training.

[0080] 6) The computer equipment adjusts the hyperparameters of the initial insulator classification model under the operation of the personnel, and returns to step 4 to perform the next model iteration training until the N training cycles of the initial insulator classification model are completed.

[0081] After N training iterations, with the model's weights and hyperparameters saved after each training iteration, the training effectiveness of these N iterations can be evaluated to select the model with the best training performance.

[0082] Therefore, in a preferred embodiment, when training the initial insulator classification model for each iteration based on the second training set and the second validation set, the evaluation index value of the training effect of the initial insulator classification model is determined according to the label information of the first frame image and the label information output by the initial insulator classification model; wherein, if the evaluation index value corresponding to the j-th training is the best among the N evaluation index values, then the target insulator localization model obtained from the j-th training is used to classify the second frame image, 1≤j≤N, where j is a positive integer.

[0083] For example, if the initial insulator classification model has been trained 8 times, and the corresponding evaluation index values ​​have been calculated for each of the 8 training sessions, and the evaluation index value corresponding to the 5th training session is the best, then the target insulator classification model obtained from the 5th training session can be used to classify the second bounding box image obtained from the target image to be detected.

[0084] The evaluation metric can be a commonly used evaluation metric in the field of classification models, such as mAP, Recall, F1-score, and Precision.

[0085] After training the target insulator classification model on the training and validation sets, a test set can be used to validate the model's final performance metrics. Therefore, the computer can also acquire a second test set, where each sample includes a first bounding box image and its label information. The second test set is then input into the target insulator classification model to obtain the label information output by the model. Based on this label information and the label information of the first bounding box images in the second test set, the performance metrics of the target insulator classification model are determined.

[0086] In this embodiment, an electrical damage defect identification model for distribution network insulators is proposed. Unlike the end-to-end insulator defect identification process that often involves deep convolutional neural networks (end-to-end means using a single model to directly and simultaneously identify and classify insulator locations), this application uses a non-end-to-end identification model. The algorithm for identifying damaged insulators is divided into two steps: the insulator localization model performs insulator target detection on the input image, and the insulator classification model classifies the insulator state. Ultimately, this can improve the detection rate without sacrificing detection accuracy.

[0087] Meanwhile, this invention makes good use of YOLOv7, the most cutting-edge target detection technology in the field of deep learning, and combines it with the traditional and efficient classification algorithm ResNet50 to propose a new method that is both practical and promising for the real-time identification of electrical damage defects in distribution network insulators.

[0088] The training process of the target insulator location model and the target insulator classification model has been introduced. The following section will further describe the insulator condition detection method based on these target insulator location and classification models. Please refer to [link to relevant documentation]. Figure 7 One embodiment of the insulator condition detection method in this application includes:

[0089] 701. Obtain the target image showing insulators to be detected;

[0090] In this embodiment, the target image can be obtained by a drone carrying a camera inspecting and photographing an outdoor power transmission line, and then the camera transmits the target image to a computer device. This computer device can be any device with data processing capabilities, such as a terminal, server, or a data processing component of the drone.

[0091] 702. Obtain a pre-trained target insulator positioning model, wherein the target insulator positioning model is trained on a first dataset by a machine learning algorithm, and each training sample in the first dataset includes an insulator image containing defective insulators and / or defect-free insulators and the position information of the insulators in the insulator image;

[0092] 703. Obtain a pre-trained target insulator classification model. The target insulator classification model is trained on a second dataset by a machine learning algorithm. Each training sample in the second dataset includes a first frame image and the label information of the first frame image. Each first frame image displays a single insulator. The label information of the first frame image is used to indicate whether the insulator displayed in the first frame image is a defect-free insulator or a defective insulator.

[0093] The training process for the target insulator localization model and the target insulator classification model is similar to that described above. Figure 1 and Figure 2 The training process described in the illustrated embodiment will not be repeated here.

[0094] The computer equipment can deploy trained target insulator localization and classification models. For example, the ONNX RuntimeBackend of the NVIDIA Triton inference framework is used for model deployment. This requires converting all neural network models to ONNX format. The conversion process for ResNet50 networks is implemented using the MMDeploy library, while the conversion of YOLOv7 networks is performed using conversion tools provided in its official code repository, ultimately enabling low-parameter deployment of the models.

[0095] When efficient model deployment and real-time inference are implemented on NVIDIA Jetson Orin embedded devices, online real-time target detection and insulator state classification in target images can be achieved on drones, improving detection speed and accuracy.

[0096] 704. Input the target image into the target insulator positioning model to obtain the position information output by the target insulator positioning model, determine the frame surrounding a single insulator based on the position information output by the target insulator positioning model, and crop the frame surrounding the single insulator to obtain a frame image;

[0097] The computer equipment can input the target image into the target insulator positioning model to obtain the position information output by the target insulator positioning model. Based on the position information output by the target insulator positioning model, the frame surrounding a single insulator is determined, and the frame surrounding the single insulator is cropped to obtain a frame image.

[0098] 705. Input the frame image into the target insulator classification model to obtain the label information output by the target insulator classification model. The label information output by the target insulator classification model is used to indicate whether the insulator displayed in the frame image is a defect-free insulator or a defective insulator.

[0099] The computer equipment then inputs the frame image into the target insulator classification model to obtain the label information output by the target insulator classification model. The label information output by the target insulator classification model is used to indicate whether the insulator displayed in the frame image is a defect-free insulator or a defective insulator, thereby obtaining the classification result of the state of each insulator displayed in the target image.

[0100] In this embodiment, the target insulator localization model is trained using a machine learning algorithm on a first dataset. Each training sample in the first dataset includes images of insulators containing defective and / or defect-free insulators, along with the location information of the insulators within those images. The target insulator classification model is trained using a machine learning algorithm on a second dataset. Each training sample in the second dataset includes a first bounding box image and its label information. The label information of the first bounding box image indicates whether the insulator displayed in the first bounding box image is defect-free or defective. Since there are a large number of defect-free insulator samples, the target insulator localization model can achieve good training results by training on a large number of defect-free insulator sample images and a smaller number of defective insulator sample images. Similarly, the target insulator classification model, which only performs classification tasks, has a relatively simple network structure and can also achieve good training results by training on a large number of defect-free insulator sample images and a smaller number of defective insulator sample images. Therefore, using both models for insulator condition detection can ensure both accuracy and speed, meeting the detection requirements.

[0101] The computer device in the embodiments of this application is described below. Please refer to [link / reference]. Figure 8 One embodiment of the computer device in this application includes:

[0102] The computer device 800 may include one or more central processing units (CPUs) 801 and a memory 805, in which one or more applications or data are stored.

[0103] The memory 805 can be volatile or persistent storage. The program stored in the memory 805 can include one or more modules, each module including a series of instruction operations on the computer device. Furthermore, the central processing unit 801 can be configured to communicate with the memory 805 and execute the series of instruction operations in the memory 805 on the computer device 800.

[0104] The computer device 800 may also include one or more power supplies 802, one or more wired or wireless network interfaces 803, one or more input / output interfaces 804, and / or one or more operating systems, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, etc.

[0105] The central processing unit 801 can perform the aforementioned... Figures 1 to 2 , Figure 7 The specific operations performed by the computer device in the illustrated embodiment will not be described in detail here.

[0106] This application also provides a computer storage medium, one embodiment of which includes: the computer storage medium storing instructions, which, when executed on a computer, cause the computer to perform the aforementioned... Figures 1 to 2 , Figure 7 The operations performed by the computer device in the illustrated embodiment.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for training a state detection model of an insulator, characterized in that, The method comprises: obtaining a first data set, each training sample in the first data set comprising an insulator image containing a defective insulator and / or a non-defective insulator and position information of the insulator in the insulator image; obtaining an initial insulator positioning model, inputting the first data set into the initial insulator positioning model to obtain position information output by the initial insulator positioning model, stopping training when a relationship between the output position information and the position information of the insulator in the insulator image satisfies a convergence condition, and obtaining a target insulator positioning model; obtaining a second data set, each training sample in the second data set comprising a first frame-shaped image and label information of the first frame-shaped image, each first frame-shaped image displaying a single insulator, and the label information of the first frame-shaped image being used to indicate whether the insulator displayed in the first frame-shaped image is a non-defective insulator or a defective insulator; obtaining an initial insulator classification model, inputting the second data set into the initial insulator classification model to obtain label information output by the initial insulator classification model, stopping training when a relationship between the output label information and the label information of the first frame-shaped image satisfies a convergence condition, and obtaining a target insulator classification model; wherein the target insulator positioning model is used to identify position information of an insulator in a target image, and the target insulator classification model is used to classify a second frame-shaped image obtained by cutting based on the position information of the insulator in the target image, and a classification result is used to indicate whether the insulator in the second frame-shaped image is a non-defective insulator or a defective insulator.

2. The method of claim 1, wherein, The first data set comprises a first training set and a first validation set, and each training sample in the first training set and the first validation set comprises the insulator image and position information of the insulator in the insulator image; The inputting of the first data set into the initial insulator positioning model comprises: inputting the first training set into the initial insulator positioning model to obtain position information output by the initial insulator positioning model, and stopping training when a relationship between the output position information and the position information of the insulator in the insulator image satisfies a convergence condition; after stopping training on the first training set, inputting the first validation set into the initial insulator positioning model to obtain position information output by the initial insulator positioning model, and stopping training when a relationship between the output position information and the position information of the insulator in the insulator image satisfies a convergence condition; adjusting a hyperparameter of the initial insulator positioning model, and returning to perform the inputting of the first training set into the initial insulator positioning model until M times of training on the initial insulator positioning model based on the first training set and the first validation set are completed, M being a positive integer greater than 1.

3. The method of claim 2, wherein, The method further comprises: determining an index value of an evaluation index of a training effect of the initial insulator positioning model according to the position information of the insulator in the insulator image and the position information output by the initial insulator positioning model, when each training of the initial insulator positioning model is completed based on the first training set and the first verification set; wherein if the index value of the evaluation index corresponding to the i-th training is optimal among the index values of the N evaluation indexes, the target insulator positioning model obtained by the i-th training is used to identify the position information of the insulator in the target image, 1≤i≤M, i is a positive integer.

4. The method of claim 1, wherein, Each set of training samples in the first data set further includes label information of each insulator in the insulator image, and the label information of the insulator in the insulator image is used to indicate whether the insulator in the insulator image is a defect-free insulator or a defective insulator. The method further includes: determining a frame surrounding a single insulator based on each position information of the insulator in the insulator image in turn, and obtaining the first frame-shaped image by intercepting the frame surrounding the single insulator; The first frame-shaped image and the displayed label information of the insulator constitute a set of training samples of the second data set.

5. The method of claim 1, wherein, The second data set includes a second training set and a second verification set, and each set of training samples in the second training set and the second verification set includes the first frame-shaped image and label information of the first frame-shaped image; The method further includes: inputting the second training set into the initial insulator classification model to obtain label information output by the initial insulator classification model, and stopping training when a relationship between the output label information and the label information of the first frame-shaped image satisfies a convergence condition; after stopping training on the second training set, inputting the second verification set into the initial insulator classification model to obtain label information output by the initial insulator classification model, and stopping training when a relationship between the output label information and the label information of the first frame-shaped image satisfies a convergence condition; adjusting the hyperparameters of the initial insulator classification model, and returning to execute the inputting of the second training set into the initial insulator classification model until N times of training of the initial insulator classification model are completed based on the second training set and the second verification set, N being a positive integer greater than 1.

6. The method of claim 5, wherein, The method further includes: determining an index value of an evaluation index of a training effect of the initial insulator positioning model according to the position information of the insulator in the insulator image and the position information output by the initial insulator positioning model, when each training of the initial insulator positioning model is completed based on the first training set and the first verification set; wherein if the index value of the evaluation index corresponding to the j-th training is optimal among the index values of the N evaluation indexes, the target insulator positioning model obtained by the j-th training is used to classify the second frame-shaped image, 1≤j≤N, j is a positive integer.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: obtaining a first test set, each group of samples in the first test set comprising an insulator image containing a defective insulator and / or a non-defective insulator and position information of an insulator in the insulator image; inputting the first test set into the target insulator positioning model to obtain position information output by the target insulator positioning model, and determining an index value of a performance index of the target insulator positioning model according to the position information output by the target insulator positioning model and the position information of the insulator in the insulator image in the first test set.

8. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: obtaining a second test set, each group of samples in the second test set comprising the first frame-shaped image and label information of the first frame-shaped image; inputting the second test set into the target insulator classification model to obtain label information output by the target insulator classification model, and determining an index value of a performance index of the target insulator classification model according to the label information output by the target insulator classification model and the label information of the first frame-shaped image in the second test set.

9. A method of detecting a state of an insulator, characterized by, The method comprises: obtaining a target image to be detected displaying an insulator; obtaining a target insulator positioning model trained in advance, the target insulator positioning model being trained by a machine learning algorithm on a first data set, each group of training samples in the first data set comprising an insulator image containing a defective insulator and / or a non-defective insulator and position information of an insulator in the insulator image; obtaining a target insulator classification model trained in advance, the target insulator classification model being trained by a machine learning algorithm on a second data set, each group of training samples in the second data set comprising a first frame-shaped image and label information of the first frame-shaped image, each first frame-shaped image displaying a single insulator, and the label information of the first frame-shaped image being used to indicate whether the insulator displayed by the first frame-shaped image is a non-defective insulator or a defective insulator; inputting the target image into the target insulator positioning model to obtain position information output by the target insulator positioning model, determining a frame surrounding a single insulator based on the position information output by the target insulator positioning model, and cutting the frame surrounding the single insulator to obtain a frame-shaped image; inputting the frame-shaped image into the target insulator classification model to obtain label information output by the target insulator classification model, the label information output by the target insulator classification model being used to indicate whether the insulator displayed by the frame-shaped image is a non-defective insulator or a defective insulator. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. The processor implements the method of any one of claims 1 to 9 when executing the computer program.

11. A computer storage medium, characterized in that The computer storage medium stores instructions, which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 9.

Images