Substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack

Abstract

The application relates to a substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack. The method comprises the following steps: constructing a substation equipment defect detection special dataset; improving the original YOLOv5 detection algorithm; replacing the conventional convolution in the YOLOv5 network model with a variable convolution, and adding a context enhancement module to the YOLOv5 network model; training a substation equipment defect detection model based on the improved YOLOv5 detection algorithm; performing target detection on the substation by using the substation equipment defect detection model, and tracking the substation equipment defects by using ByteTrack. The conventional convolution in the YOLOv5 network model is replaced with a variable convolution, a context enhancement module is added to the YOLOv5 network model, and the original YOLOv5 detection algorithm is improved, so that the calculation amount in the model training process is significantly reduced without reducing the detection precision, the improved YOLOv5 network model is more lightweight, and the economy and engineering practicability of the intelligent inspection of the substation are effectively realized.

Description

TECHNICAL FIELD

[0001] The embodiment of the present disclosure relates to the technical field of computer vision, in particular to a substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack. BACKGROUND

[0002] The substation is an important node connecting the backbone network and the distribution network, and is the core hub of the smart grid and the key to the stable operation of the power system. With the continuous increase of engineering electrical equipment, the substation plays an increasingly important role in the entire power system. However, with the increase of voltage level and transmission capacity, the scale of the substation is continuously expanding, and the number of substation equipment is continuously increasing. The risk coefficient of traditional manual on-site inspection is getting higher and higher, and the misdiagnosis and missed diagnosis rate is getting higher and higher.

[0003] With the development of computer vision technology and intelligent robot technology, power equipment maintenance gradually develops from traditional manual troubleshooting and remote "four remote" monitoring to intelligent inspection new way. High-precision unmanned aerial vehicles are applied to substation inspection to achieve efficient and autonomous accurate inspection. The R-CNN series algorithm has high detection accuracy, but due to the use of a two-stage structure, its detection speed cannot meet the needs of actual engineering applications, and it is criticized. In order to break this deadlock, it is the trend of the times to design a faster target detector. In 2016, Joseph Redmon et al. proposed a target detection algorithm named You Only Look Once, which is the YOLO series algorithm widely concerned by scholars in various fields in the future. The YOLO series algorithm adopts a single-stage network structure, so the detection speed can meet the needs of substation intelligent inspection, and with the continuous updating of the YOLO series, its detection accuracy is continuously improved, and it is the most popular target detection algorithm at present.

[0004] The YOLOv5 algorithm needs a large amount of data set, computing resource and time in the process of training model. However, at present, there are very few public data sets of substation equipment defect images, and the on-board computing power of the unmanned aerial vehicles and robots used for intelligent inspection in the substation is low, and most of the substations lack large computers. All the above reasons seriously hinder the development of intelligent inspection of substations and even power systems. Therefore, under the premise of not affecting the detection accuracy of substation equipment defects, how to optimize YOLOv5 to make its network as light as possible and reduce the intelligent inspection transformation cost of the substation has become a problem to be solved.

[0005] Therefore, it is necessary to improve one or more problems in the related technical solutions.

[0006] It is to be understood that the information disclosed in the Background section is merely for the purpose of enhancing the understanding of the background of the present disclosure, and thus can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0007] The purpose of the embodiments of the present disclosure is to provide a substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack, thereby at least partially overcoming one or more problems caused by the limitations and defects of the related art.

[0008] The present application provides a substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack, the method comprising the following steps:

[0009] Constructing a substation equipment defect detection special dataset;

[0010] Improving the original YOLOv5 detection algorithm: replacing the conventional convolution in the YOLOv5 network model with a variable convolution, and adding a context enhancement module to the YOLOv5 network model;

[0011] Training a substation equipment defect detection model based on the improved YOLOv5 detection algorithm;

[0012] Using the substation equipment defect detection model to detect targets in the substation, and using ByteTrack to track substation equipment defects.

[0013] In the present application, the construction of the substation equipment defect detection special dataset comprises:

[0014] Collecting substation equipment defect pictures and storing the collected defect pictures in a first folder;

[0015] Creating a text file in the first folder for storing the class names for labeling the defect pictures;

[0016] Creating a second folder in the first folder for storing the label files for labeling the defect pictures, the label files corresponding to the defect pictures stored in the first folder;

[0017] Converting the format of the label file to text format, and dividing the defect pictures in the first folder and the label files in the second folder into a training set and a test set according to a predetermined ratio.

[0018] In the present application, the original YOLOv5 detection algorithm is improved, including:

[0019] Replace the conventional convolution in the C3 module of the 8th layer in the original YOLOv5 network model with a variable convolution DCN to obtain a C3_DCN module;

[0020] Replace the C3 of the 13th layer, the 20th layer and the 23rd layer in the original YOLOv5 network model with a context-aware cross-level fusion network C2F module;

[0021] Add a context enhancement CAM module between the 22nd layer and the 23rd layer in the original YOLOv5 network model;

[0022] Replace the loss function L CIou of the original YOLOv5 network model with a modified L Sigmiod-Focal-CIoU ;

[0023] An improved YOLOv5 detection algorithm is obtained.

[0024] In the application, the calculation formula of the variable convolution DCN is:

[0025]

[0026] Wherein, x is a feature map, w is a convolution kernel parameter, p0 is the coordinate of the center of the convolution kernel relative to the top left corner of the input feature map, p n is the offset of the R elements of the convolution kernel relative to the center of the convolution kernel, Δp n is a position offset vector, and Δp n The expression of Δp

[0027] {Δp n |n=1,2,…N},N=|R|

[0028] |R|={(-1,-1),(-1,0),…,(0,1),(1,1)}.

[0029] In the application, the context-aware cross-level fusion network C2F module comprises a convolution 1, a Split module, N Bottleneck modules and a Concatl module connected in sequence, wherein N is greater than or equal to 2.

[0030] The input end of the convolution 1 serves as the input end of the context-aware cross-level fusion network C2F module, the output end of the convolution 1 is connected with the input end of the Split module, the output end of the Split module is connected with the input end of the first Bottleneck module, the N Bottleneck modules are connected in sequence, the output end of the Nth Bottleneck module is connected with the input end of the Concatl module, and the output end of the Concatl module serves as the output end of the context-aware cross-level fusion network C2F module.

[0031] In the application, the context-enhanced CAM module comprises a convolution a, a convolution b, a convolution c, a convolution d, a convolution e, a convolution f and a Concat2 module;

[0032] The input end of the convolution a, the convolution b and the convolution c is the input end of the context-enhanced CAM module, the output end of the convolution a is connected with the input end of the convolution d, the output end of the convolution b is connected with the input end of the convolution e, and the output end of the convolution c is connected with the input end of the convolution f; the output ends of the convolution d, the convolution e and the convolution f are connected as the input end of the Concat2 module; and the output end of the Concat2 module is the output end of the context-enhanced CAM module.

[0033] In the application, the convolution a, the convolution b and the convolution c are respectively a convolution with a dilated convolution rate of 1, 3 and 5 and a kernel size of 3*3; and the convolution d, the convolution e and the convolution f are a convolution with a kernel size of 1*1.

[0034] In the application, the loss function is improved as follows:

[0035] The loss function L of the original YOLOv5 network total is expressed as:

[0036] L total =L box +L obj +L cls

[0037] Wherein, L box is a positioning loss, L obj is a confidence loss, and L cls is a classification loss.

[0038]

[0039] Wherein, λ Iou is a weight coefficient of the positioning loss L box , and L CIoU is a positioning loss function used by default in the training process of the original YOLOv5 network model, and its expression is:

[0040]

[0041] Wherein, B is a predicted bounding box, B gt is a real bounding box, ρ represents the Euclidean distance between the center point of the predicted bounding box B and the center point, c represents the diagonal distance of the minimum closed region C that can be simultaneously obtained, α is a weight coefficient, and V is a width-height ratio penalty term, and its expression is:

[0042]

[0043] Wherein, w gtwidth of the real bounding box B gt height of the real bounding box B gt width of the real bounding box B gt width of the real bounding box B CIoU height of the real bounding box B CIoU height of the real bounding box B

[0044] On the basis of the original YOLOv5 network loss function L CIoU L CIoU is combined with Focal Loss, and a loss function L Focal-CIoU is obtained, and its expression is:

[0045] L Focal-CIoU = IoU γ L CIoU

[0046] Wherein, Y is a parameter for controlling the degree of abnormal value suppression;

[0047] On this basis, the weight coefficient of L EIoU in L Focal-CIoU is improved by using a Sigmoid function, and an improved loss function L Sigmiod-Focal-CIoU is obtained, and its expression is:

[0048] L Sigmiod-Focal-CIoU = sigma (IoU γ ) L CIoU

[0049] Wherein, sigma is a sigmoid function, and its expression is:

[0050]

[0051] L Sigmiod-Focal-CIoU is used to replace the default L CIoU in the original YOLOv5 as the positioning loss function L box , and an improved YOLOv5 detection algorithm is obtained.

[0052] In the application, the step of training the substation equipment detection model based on the improved YOLOv5 detection algorithm is as follows:

[0053] The training parameters are set as follows: the training round epochs is set to 200, the picture batch size batch-size is set to 16, the input picture size img-size is set to (640x640), the maximum working core number is set to 8, and the initial learning rate is 0.01;

[0054] The improved YOLOv5 network structure is used for training a training set in a substation equipment special data set, a trained model is used for performance test on a test set in the substation equipment special data set, and a substation equipment detection model best.pt with the highest detection accuracy in the training process and a substation equipment detection model last.pt obtained through the last training are obtained.

[0055] In the application, the step of tracking the substation equipment defects by using ByteTrack is as follows:

[0056] The positions, categories and confidence of all targets in the detection result of the substation equipment detection model best.pt are stored in the form of a detection frame, and are input into the ByteTrack target tracking algorithm, and the array of the detection frame is as follows:

[0057] Bbox = [x, y, w, h, cls, conf]

[0058] Wherein, x and y are the horizontal and vertical coordinates of the center point of the detection frame, w and h are the width and height of the detection frame respectively, cls is the category of the target, and conf is the confidence of the target.

[0059] Detection frame classification: if the confidence conf of the detection frame is greater than 0.6, the detection frame is put into the high-score detection frame set B H If the confidence value conf of the detection frame is less than 0.6, the detection frame is put into the low-score detection frame set B L .

[0060] First association matching: the high-score detection frame set B H is matched with the track set T by using the Hungarian algorithm; for the successfully matched track, the Kalman filter is updated and put into the current frame track set TN; the track that cannot be successfully matched is put into the first association unmatched track set T R The track that cannot be successfully matched is put into the first association unmatched detection frame set B R .

[0061] Second association matching: the low-score detection frame set B L is secondly associated with the first association unmatched track set T R by using the Hungarian algorithm; the low-score detection frame set B L is matched with the second association unmatched track set TR; the track that cannot be successfully matched is put into the lost track set T D The low-score detection frame that cannot be successfully matched is deleted, and for the successfully matched track, the Kalman filter is updated and put into the current frame track set T N .

[0062] Track creation, deletion and merging: for the first association failed to match the detection frame set B R , create a new track for it, and merge into the current frame track set; for the track in T D , if it exceeds 30 frames, it is considered as a lost track and deleted; return the track set of the current frame as the existing track set of the next frame image, output the track set, and perform Kalman filter prediction track T.

[0063] The technical solution provided by the application can include the following beneficial effects:

[0064] The application replaces the conventional convolution in the YOLOv5 network model with a variable convolution, adds a context enhancement module to the YOLOv5 network model, and improves the original YOLOv5 detection algorithm, thereby significantly reducing the computational amount in the model training process without reducing the detection accuracy, making the improved YOLOv5 network model more lightweight, and effectively realizing the economy and engineering practicability of the intelligent inspection of the substation.

[0065] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0066] The drawings herein are incorporated into the specification and form part of the specification, show embodiments consistent with the present disclosure, and together with the specification serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained from these drawings without creative labor for those skilled in the art.

[0067] Figure 1 A flowchart of a substation equipment defect detection and tracking method based on an improved YOLOv5+ByteTrack in an exemplary embodiment of the present disclosure is shown;

[0068] Figure 2 A flowchart of constructing a substation equipment defect detection special data set in an exemplary embodiment of the present disclosure is shown;

[0069] Figure 3 A flowchart of improving the original YOLOv5 detection algorithm in an exemplary embodiment of the present disclosure is shown;

[0070] Figure 4 A YOLOv5 network structure diagram in the related art is shown;

[0071] Figure 5 A context-aware cross-level fusion network C2F module structure diagram in an exemplary embodiment of the present disclosure is shown;

[0072] Figure 6 A context-enhanced CAM module structure diagram in the example embodiment of the present disclosure is shown;

[0073] Figure 7 An improved YOLOv5 network structure diagram in the example embodiment of the present disclosure is shown;

[0074] Figure 8 A ByteTrack real-time tracking algorithm flowchart in the example embodiment of the present disclosure is shown;

[0075] Figure 9 A structural schematic diagram of an electronic device in the example embodiment of the present disclosure is shown;

[0076] Figure 10 A structural schematic diagram of a program product for implementing a substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack in the example embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0077] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these examples are provided so that this disclosure will be thorough and complete, and will fully convey the gist of the example implementations to those skilled in the art. Features described in the description, examples, or claims can be combined in any suitable manner in one or more implementations.

[0078] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments described herein, and together with the description serve to explain principles of the present disclosure. Features, elements, and / or properties that are the same in the various embodiments are labeled with the same reference numbers throughout the drawings, and descriptions of these parts will not be repeated. Some of the block diagrams in the drawings are functional entities, which do not necessarily have to correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0079] In the present example implementation, a substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack is first provided, as shown in Figure 1 The method can include the following steps:

[0080] Step S100, a substation equipment defect detection special dataset is constructed.

[0081] Step S200, improving the original YOLOv5 detection algorithm: replacing the conventional convolution in the YOLOv5 network model with a variable convolution, and adding a context enhancement module to the YOLOv5 network model.

[0082] Step S300, training a substation equipment defect detection model based on the improved YOLOv5 detection algorithm.

[0083] Step S400, using the substation equipment defect detection model to detect the target of the substation, and using ByteTrack to track the substation equipment defects.

[0084] The present application improves the original YOLOv5 detection algorithm by replacing the conventional convolution in the YOLOv5 network model with a variable convolution, and adding a context enhancement module to the YOLOv5 network model. Without reducing the detection accuracy, the calculation amount in the model training process is significantly reduced, making the improved YOLOv5 network model more lightweight, and effectively realizing the economy and engineering practicability of the substation intelligent inspection.

[0085] The steps of the above method in the present example embodiment will be described in more detail below.

[0086] First, please refer to Figure 2 The specific steps of step S100 are as follows:

[0087] Step S101, collect substation equipment defect pictures and store the collected defect pictures in a first folder. Specifically, the unmanned aerial vehicle can be used for intelligent inspection of substation equipment to collect substation outdoor equipment defect pictures. The collected substation equipment defect pictures are named according to the format of PASCALVOC2007, a folder named VOC2007 (first folder) is created, and a folder named JPEGImages is created in VOC2007 to store the substation equipment defect pictures.

[0088] Step S102, create a text file in the first folder for storing the class names for labeling the defect pictures. Create a txt file named predefined_classes.txt in VOC2007 (first folder) to store the class names to be labeled.

[0089] Step S103, a second folder is created in the first folder for storing a label file for labeling the defect picture, and the label file corresponds to the defect picture stored in the first folder. Specifically, a folder named Annotations (second folder) is created in VOC2007 (first folder) to store the labeled label file, and the labelimg tool is used to make labels for the substation equipment defect pictures to generate xml format label files corresponding to the substation equipment defect pictures stored in the JPEGImages folder.

[0090] Step S104, the format of the label file is converted into a text format, and the defect pictures in the first folder and the label files in the second folder are divided into a training set and a test set according to a preset ratio. Specifically, the format conversion script is used to convert the label file from xml format to txt format, and the substation equipment defect pictures stored in the JPEGImages folder and the label files stored in the Annotations folder are divided into a training set and a test set according to an 8:2 ratio.

[0091] As shown in Table 1, in the embodiment, 11 device scenes and 14 device defects are selected as the detection and tracking targets of the embodiment, including table damage, insulator damage, oil leakage, breather damage, cabinet door closure anomaly, foreign matter, cover plate damage or loss, meter reading anomaly, oil level state anomaly, breather anomaly and pressure plate state anomaly, a total of 8769 substation equipment defect pictures are stored in the JPEGImages folder. The labelimg tool is used to make labels for each substation equipment defect picture in the JPEGImages folder to obtain xml format label files stored in the Annotations folder. The format conversion script is used to convert the label file from xml format to txt format, and the substation equipment defect pictures stored in the JPEGImages folder and the label files stored in the Annotations folder are divided into a training set and a test set according to an 8:2 ratio. Then, in the substation equipment defect detection special dataset constructed in the embodiment, the training set contains 7015 pictures and their one-to-one corresponding label files, and the test set contains 1753 pictures and their one-to-one corresponding label files.

[0092] Table 1: Substation equipment defect detection special dataset collected in the embodiment of the application

[0093]

[0094]

[0095] Secondly, please refer toFigure 3 The specific steps of step S200 are as follows:

[0096] Step S201, replace the conventional convolution in the C3 module of the 8th layer in the original YOLOv5 network model with a variable convolution DCN to obtain a C3_DCN module. The calculation formula of the variable convolution DCN is:

[0097]

[0098] Wherein, x is a feature map, w is a convolution kernel parameter, p0 is the coordinate of the center of the convolution kernel relative to the top left corner of the input feature map, p n is the offset of the R elements of the convolution kernel relative to the center of the convolution kernel, Δp n is a position offset vector, Δp n The expression of Δp

[0099] {Δp n |n=1,2,…N},N=|R|

[0100] |R|={(-1,-1),(-1,0),…,(0,1),(1,1)}.

[0101] Step S202, replace the C3 of the 13th layer, the 20th layer and the 23rd layer in the original YOLOv5 network model (see Figure 4 ) with a context-aware cross-level fusion network C2F module. Specifically, the context-aware cross-level fusion network C2F module includes a convolution 1, a Split module, N Bottleneck modules and a Concat1 module connected as shown in Figure 5 .

[0102] The input end of the convolution 1 serves as the input end of the context-aware cross-level fusion network C2F module, the output end of the convolution 1 is connected with the input end of the Split module, the output end of the Split module is connected with the input end of the first Bottleneck, the N Bottleneck modules are connected in a loop, the output end of the Nth Bottleneck is connected with the input end of the Concat1 module; the output end of the Concat1 module serves as the output end of the context-aware cross-level fusion network C2F module.

[0103] Step S203, add a context enhancement CAM module between the 22nd layer and the 23rd layer in the original YOLOv5 network model. The context enhancement CAM module includes a convolution a, a convolution b, a convolution c, a convolution d, a convolution e, a convolution f and a Concat2 module, as shown in Figure 6 .

[0104] The convolution a, the convolution b and the convolution c are respectively a convolution with a kernel size of 3*3 and a dilated convolution rate of 1, 3 and 5; and the convolution d, the convolution e and the convolution f are a convolution with a kernel size of 1*1.

[0105] The input end of the convolution a, the convolution b and the convolution c is the input end of the context enhanced CAM module, the output end of the convolution a is connected with the input end of the convolution d, the output end of the convolution b is connected with the input end of the convolution e, and the output end of the convolution c is connected with the input end of the convolution f; the output ends of the convolution d, the convolution e and the convolution f are connected as the input ends of the Concat2 module; and the output end of the Concat2 module is the output end of the context enhanced CAM module.

[0106] In step S204, the loss function L in the original YOLOv5 network model is replaced by L CIoU . Sigmiod-Focal-CIoU The expression of the loss function L in the original YOLOv5 network is as follows: total

[0107] L total = L box + L obj + L cls

[0108] Wherein, L box is a positioning loss, L obj is a confidence loss, and L cls is a classification loss.

[0109]

[0110] Wherein, λ IoU is a weight coefficient of the positioning loss L box , and L CIoU is a positioning loss function by default in the training process of the original YOLOv5 network model, and the expression is as follows:

[0111]

[0112] Wherein, B is a predicted bounding box, B gt is a real bounding box, ρ represents the Euclidean distance between the center point of the predicted bounding box B and the center point, c represents the diagonal distance of the minimum closed region C that can be simultaneously obtained, α is a weight coefficient, and V is a width-height ratio penalty term, and the expression is as follows:

[0113]

[0114] Wherein, w gt is the width of the real bounding box B gt , and h gt is the height of the real bounding box B gt ​the height of the prediction bounding box B, w is the width of the prediction bounding box B, and h is the height of the prediction bounding box B.

[0115] On the basis of the original YOLOv5 network loss function L CIoU , L CIoU is combined with Focal Loss to obtain a loss function L Focal-CIoU , and the expression is as follows:

[0116] L Focal-CIoU = IoU γ L CIoU

[0117] wherein Y is a parameter for controlling the degree of abnormal value suppression, and the embodiment suggests that Y = 0.5.

[0118] On this basis, the weight coefficient of L Focal-CIoU in L EIoU is improved by using a Sigmoid function to obtain an improved loss function L Sigmiod-Focal-CIoU , and the expression is as follows:

[0119] L Sigmiod-Focal-CIoU = σ(IoU Υ )L CIoU

[0120] wherein σ is a sigmoid function, and the expression is as follows:

[0121]

[0122] L Sigmiod-Focal-CIoU is used to replace the default L CIoU in the original YOLOv5 to calculate the positioning loss L box as the loss function.

[0123] Step S205, an improved YOLOv5 detection algorithm is obtained, and the framework of the improved YOLOv5 detection algorithm is as shown in Figure 7 .

[0124] It should be understood that the step S300 can include the following specific steps:

[0125] Step S301, the training parameters are set as follows: the training round epochs is set to 200, the picture batch size batch-size is set to 16, the input picture size img-size is set to (640x640), the maximum number of working cores is set to 8, and the initial learning rate is set to 0.01.

[0126] Step S302, the improved YOLOv5 network structure is used for training the training set in the substation equipment special data set, the trained model is used for performance test on the test set in the substation equipment special data set, and the substation equipment detection model best.pt with the highest detection accuracy in the training process and the substation equipment detection model last.pt obtained by the last training are obtained.

[0127] On the basis of the above steps, the substation equipment defect detection model best.pt based on the improved YOLOv5 detection algorithm obtained by step 300 is used for target detection on the substation unmanned aerial vehicle inspection video, the target detection result is used as the input of the ByteTrack target tracking algorithm, the ByteTrack tracks the substation equipment defects, and the tracking algorithm flow is combined Figure 8 The specific steps are as follows:

[0128] Step S401, the positions, categories and confidence of all targets in the detection result of the substation equipment defect detection model best.pt are stored in the form of a detection frame, and are input into the ByteTrack target tracking algorithm, and the array form of the detection frame is:

[0129] Bbox=[x,y,w,h,cls,conf]

[0130] Wherein, x and y are the center point horizontal and vertical coordinates of the detection frame, w and h are the width and height of the detection frame, cls is the category of the target, and conf is the confidence of the target.

[0131] Step S402, detection frame classification: if the detection frame confidence conf is greater than 0.6, the detection frame is put into the high-score detection frame set B H , and if the detection frame confidence value conf is less than 0.6, the detection frame is put into the low-score detection frame set B L .

[0132] Step S403, first association matching: the high-score detection frame set B H and the track set T are matched by using the Hungarian algorithm. For the successfully matched track, the Kalman filter is updated, and is put into the current frame track set T N . The track that cannot be successfully matched is put into the first association unmatched track set T R , and the high-score detection frame that cannot be successfully matched is put into the first association unmatched detection frame set B R .

[0133] Step S404, second association matching: the low-score detection frame set B L is matched with the first association unmatched track set T RSecond association matching is performed on the low-score bounding box set B using the Hungarian algorithm L and the track set T that failed to match in the second association R Matching is performed. The tracks that failed to match successfully are put into the lost track set T D The low-score bounding boxes that failed to match successfully are deleted, and for the tracks that matched successfully, their Kalman filter is updated and put into the current frame track set T N .

[0134] Step S405, track creation, deletion and merging: for the bounding boxes in the bounding box set B that failed to match in the first association R , a new track is created for them and merged into the current frame track set. For the tracks in T D , if they exceed 30 frames, they are considered to be lost tracks and are deleted. Return the track set of the current frame as the existing track set of the next frame image, output the track set, and perform Kalman filter prediction on the track T.

[0135] The original YOLOv5 and the optimized YOLOv5 of the application are trained using the substation multi-target special dataset constructed in step S100, and the parameters are set in accordance with step S301. The optimization method of the application focuses on three target parameters: first, the average precision of defect detection, i.e., the mAP value; second, the total loss value of prediction, i.e., L total ; and third, the amount of calculation during model training, i.e., parameters. As shown in Table 2, the comparison of the performance of the trained models of the optimized YOLOv5 of the application and the original YOLOv5 shows that the optimized YOLOv5 of the application has an average precision mAP of defect detection improved by 2.5%, a total loss value L total of prediction reduced by 43.72%, and the amount of calculation of model training decreased by 29.7%, which verifies the effectiveness of the substation equipment defect detection and tracking method based on the improved YOLOv5 proposed by the application. Without affecting the accuracy of substation equipment defect detection, the amount of calculation of model training and the total loss value of model prediction are reduced by optimizing YOLOv5, thereby reducing the cost of substation intelligent inspection transformation.

[0136] Table 2 Comparison of algorithm performance

[0137]

[0138]

[0139] As for the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments relating to the method, and will not be described in detail here.

[0140] It should be noted that although several modules of the system for action execution are mentioned in the foregoing detailed description, such a division is not mandatory. Indeed, according to embodiments of the application, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into modules embodied by several modules. The components shown as modules can or can not be physical units, i.e. can be located in one place or can be distributed over several network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the inventive scheme. Those skilled in the art can understand and implement without creative labor.

[0141] Referring to Figure 9 The electronic device 300 according to the embodiment of the present application further comprises a bus 330 connecting different platform systems.

[0142] The memory 310 can include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 211 and / or a cache memory 312, and can further include a read-only memory (ROM) 313.

[0143] The memory 310 further stores a computer program, which can be executed by the processor 320 to enable the processor 320 to perform the steps of the substation equipment defect detection and tracking method based on the improved YOLOv5+ByteTrack according to any one of the embodiments of the present application, the specific implementation and the achieved technical effects of which are consistent with the implementation described in the above embodiment of the substation equipment defect detection and tracking method based on the improved YOLOv5+ByteTrack, and some contents will not be repeated.

[0144] The memory 310 can further include utility 314 having at least one program module 315, such as an operating system, one or more application programs, other program modules, and program data, each of which or some combination of which can include implementation of a network environment.

[0145] Correspondingly, the processor 320 can execute the above computer program and can execute the utility 314.

[0146] The bus 330 can represent one or more of several bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures.

[0147] The electronic device 300 can also communicate with one or more external devices 340 such as a keyboard, a pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with the electronic device 300; and / or any devices (e.g., a router, a modem, a network card, etc.) that enable the electronic device 300 to communicate with one or more other computing devices. Such communication can occur via the input / output interface 350. Still yet, the electronic device 300 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or the Internet) via the network adapter 360. The network adapter 360 can communicate with the other components of the electronic device 300 via the bus 330. It should be understood that, although not shown explicitly, other hardware and / or software components could be used in conjunction with the electronic device 300. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0148] The embodiment of the present application further provides a computer readable storage medium for storing a computer program, the computer program being executed to implement the steps of the substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack in the embodiment of the present application, and the specific implementation manners and the achieved technical effects are consistent with those described in the embodiment of the substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack, and some contents will not be described herein.

[0149] Figure 10A program product 400 for implementing the substation equipment defect detection and tracking method based on the improved YOLOv5+ByteTrack provided by the embodiment is shown, which can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product 400 of the present application is not limited to this, and in the present application, the readable storage medium can be any tangible medium containing or storing a program, which can be used or combined with an instruction execution system, device or apparatus. The program product 400 can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0150] The computer readable storage medium can include a data signal carried in a baseband or propagated as a carrier wave in a propagated data signal, in which the readable program code is carried. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium can also be any readable medium that can send, propagate or transmit a program for use by or in connection with an instruction execution system, device or apparatus. The program code contained on the readable storage medium can be transmitted in any suitable medium, including but not limited to wireless, wired, optical, cable, RF, or any suitable combination of the above. The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, etc., and a conventional procedural programming language such as C language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as a separate software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, through the Internet by connecting to an Internet service provider).

[0151] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection of the present application.

Claims

1. A substation equipment defect detection and tracking method based on improved YOLOv5+ByteTrack, characterized in that, The method comprises the following steps: Construct a special dataset for substation equipment defect detection; The original YOLOv5 detection algorithm is improved: the regular convolution in the YOLOv5 network model is replaced with a variable convolution, and a context enhancement module is added to the YOLOv5 network model; wherein, comprising: replacing the regular convolution in the C3 module of the 8th layer in the original YOLOv5 network model with a variable convolution DCN to obtain a C3_DCN module; replacing the C3 of the 13th layer, the 20th layer and the 23rd layer in the original YOLOv5 network model with a context-aware cross-level fusion network C2F module; adding a context enhancement CAM module between the 22nd layer and the 23rd layer in the original YOLOv5 network model; replacing the loss function of the original YOLOv5 network model with ; obtaining the improved YOLOv5 detection algorithm; The context enhanced CAM module comprises a convolution a, a convolution b, a convolution c, a convolution d, a convolution e, a convolution f and a Concat2 module; The input end of the convolution a, the convolution b and the convolution c is the input end of the context enhanced CAM module, the output end of the convolution a is connected with the input end of the convolution d, the output end of the convolution b is connected with the input end of the convolution e, and the output end of the convolution c is connected with the input end of the convolution f; the output ends of the convolution d, the convolution e and the convolution f are connected as the input end of the Concat2 module; the output end of the Concat2 module is the output end of the context enhanced CAM module; Train a substation equipment defect detection model based on the improved YOLOv5 detection algorithm; Use the substation equipment defect detection model to detect the target of the substation, and use ByteTrack to track the substation equipment defects.

2. The method of claim 1, wherein, The special dataset for substation equipment defect detection is constructed, comprising: Collect substation equipment defect pictures, and store the collected defect pictures in a first folder; Create a text file in the first folder for storing the class names for labeling the defect pictures; Create a second folder in the first folder for storing a label file for labeling the defect pictures, and the label file corresponds to the defect pictures stored in the first folder; Convert the format of the label file into a text format, and divide the defect pictures in the first folder and the label file in the second folder into a training set and a test set according to a preset ratio.

3. The method of claim 1, wherein, The calculation formula of the variable convolution DCN is: wherein, is a feature map, is a convolution kernel parameter, is a coordinate of the center of the convolution kernel relative to the top-left corner of the input feature map, is a convolution kernel is an offset of the element relative to the center of the convolution kernel, is a position offset vector, is expressed as: 。 4. The method of claim 1, wherein, The context-aware cross-level fusion network C2F module comprises a convolution 1, a Split module, N Bottleneck modules and a Concat1 module connected in sequence, wherein N is greater than or equal to 2; The input end of the convolution 1 is the input end of the context-aware cross-level fusion network C2F module, the output end of the convolution a is connected with the input end of the Split module, the output end of the Split module is connected with the input end of the first Bottleneck module, the N Bottleneck modules are connected in sequence, the output end of the Nth Bottleneck module is connected with the input end of the Concat1 module, and the output end of the Concat1 module is the output end of the context-aware cross-level fusion network C2F module.

5. The method of claim 1, wherein, The convolution a, the convolution b and the convolution c are dilated convolution with a dilated rate of 1, 3 and 5 respectively and a kernel size of 3*3; and the convolution d, the convolution e and the convolution f are convolution with a kernel size of 1*1.

6. The method of claim 1, wherein, The loss function is improved as follows: Loss function of the original YOLOv5 network The expression is: wherein, is a localization loss, is a confidence loss, is a classification loss; wherein, is a weight coefficient of the positioning loss , is a default positioning loss function for the original YOLOv5 network model in the training process, and its expression is: wherein, is the predicted bounding box, is the real bounding box, represents the predicted bounding box is the Euclidean distance between the center points, represents the diagonal distance of the minimum closed region C that can be simultaneously, is the weight coefficient, is the aspect ratio penalty term, and its expression is: in, For real borders width, For real borders height, For predicting bounding boxes width, For predicting bounding boxes Height; On the basis of the original YOLOv5 network loss function , the Focal Loss is combined to obtain the loss function , and its expression is as follows: wherein is a parameter to control the degree of outlier suppression; On this basis, the weight coefficient of the loss function is improved by using the Sigmoid function, and an improved loss function is obtained The expression of the improved loss function is as follows: ​​ wherein is a sigmoid function expressed as: Use Instead of the default Computing method as positioning loss function , get improved YOLOv5 detection algorithm.

7. The method of claim 1, wherein, The steps of training the substation equipment detection model based on the improved YOLOv5 detection algorithm are as follows: The training parameters are set as follows: the training rounds epochs are set to 200, the picture batch size batch-size is 16, the input picture size img-size is set to , the maximum number of working cores is set to 8, and the initial learning rate is 0.

01. The improved YOLOv5 network model is used for training of a training set in a substation equipment special data set, a trained model is used for performance test of a test set in the substation equipment special data set, and a substation equipment detection model best.pt with the highest detection accuracy in the training process and a substation equipment detection model last.pt obtained through the last training are obtained.

8. The method of claim 7, wherein, The step of tracking the substation equipment defects by using ByteTrack is as follows: The positions, categories and confidence of all targets in the detection result of the substation equipment detection model best.pt are stored in the form of a detection frame, and are input into the ByteTrack target tracking algorithm, and the array form of the detection frame is as follows: wherein, is a horizontal coordinate of a center point of the detection frame, is a width and height of the detection frame, is a class of the target, is a confidence of the target; Bounding box classification: if the bounding box confidence is greater than 0.6, then place the bounding box in the high scoring bounding box set B H where the bounding box confidence value is less than 0.6, then place the bounding box in the low scoring bounding box set B L where the bounding box confidence value is​ First association matching: using the Hungarian algorithm to match the set of high-scoring bounding boxes B H The trajectory is matched against the trajectory set T; for a successfully matched trajectory, its Kalman filter is updated and it is added to the current frame trajectory set T. N In the middle; trajectories that failed to match are added to the first set of unmatched trajectories T. R In the first set of failed-match detection boxes, high-scoring detection boxes that could not be matched are placed into the first set of failed-match detection boxes B. R middle; Second association matching: match low-score bounding box set B L Trajectory set T that failed to match with the first association R Second association matching is performed, and the low-score bounding box set B is matched with the trajectory set T by using the Hungarian algorithm L Trajectory set T that failed to match with the second association R Matching is performed; trajectories that failed to successfully match are put into a lost trajectory set T D The low-score bounding box that failed to successfully match is deleted, and for trajectories that successfully matched, their Kalman filter is updated and put into the current frame trajectory set T N ​ Track creation, deletion and merging: for the 1st association failed to match the detection box set B in T R , create a new track for it and merge into the current frame track set; for the track in T D , if it exceeds 30 frames, it is considered as a lost track and deleted; return the current frame track set as the existing track set of the next frame image, output the track set, and perform Kalman filter prediction track T.

Images