Detection Method, Device, Computer Equipment, and Storage Medium for Transmission Line

Abstract

The present application relates to a detection method, device, computer device, storage medium, and computer program product for a transmission line. The method includes: obtaining a to-be-detected image obtained by photographing a target transmission line. Detecting a suspension clamp on the to-be-detected image through a trained first detection model to obtain suspension clamp position information, and based on the suspension clamp position information, dividing a target image including a suspension clamp area from the to-be-detected image. In this way, based on the trained first detection model, the area corresponding to the to-be-detected image is accurately located in the suspension clamp area. Identifying the state of a target flat washer located in the suspension clamp area in the target image through a trained second detection model to obtain the state of the target flat washer. Therefore, the trained second detection model can accurately determine the state of the target flat washer, greatly increasing the detection accuracy for ultra-small targets.

Description

Technical Field

[0001] The present application relates to the technical field of image detection, and particularly to a detection method, device, computer device, storage medium and computer program product for transmission lines. Background Art

[0002] With the rapid development of the power system, the scale of transmission lines is also getting larger and larger. To fix and support transmission lines, a large number of suspension clamps are often used to fix transmission lines. Among them, metal parts such as flat washers need to be installed at the joints of suspension clamps.

[0003] For transmission lines that are exposed outdoors for a long time, once damaged by harsh environments or due to the negligence of construction workers, problems such as the detachment of flat washers on suspension clamps may occur, greatly affecting the normal use of transmission lines.

[0004] In the prior art, to ensure the normal use of transmission lines, a YOLO (you only look once) model is often used to detect relevant parts in transmission lines. However, when detecting ultra-small targets such as flat washers, there are often a large number of missed detections and false detections, resulting in low detection accuracy for ultra-small targets. Summary of the Invention

[0005] Based on this, it is necessary to provide a detection method, device, computer device, computer-readable storage medium and computer program product for transmission lines in view of the above technical problems.

[0006] In a first aspect, the present application provides a detection method for transmission lines. The method includes:

[0007] Obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes a suspension clamp and a flat washer;

[0008] Detect the suspension clamp in the to-be-detected image through a trained first detection model to obtain suspension clamp position information, and based on the suspension clamp position information, divide a target image including a suspension clamp area from the to-be-detected image;

[0009] Identify the state of a target flat washer located in the suspension clamp area in the target image through a trained second detection model to obtain the state of the target flat washer.

[0010] In a second aspect, the present application further provides a detection device for transmission lines. The device includes:

[0011] An obtaining module, configured to obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes a suspension clamp and a flat washer;

[0012] A detection module, configured to detect the suspension clamp on the to-be-detected image through a trained first detection model, obtain the position information of the suspension clamp, and based on the position information of the suspension clamp, divide a target image including the area of the suspension clamp from the to-be-detected image;

[0013] An identification module, configured to identify the state of the target flat washer located in the area of the suspension clamp in the target image through a trained second detection model, and obtain the state of the target flat washer.

[0014] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented:

[0015] Obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes a suspension clamp and a flat washer;

[0016] Detect the suspension clamp on the to-be-detected image through a trained first detection model, obtain the position information of the suspension clamp, and based on the position information of the suspension clamp, divide a target image including the area of the suspension clamp from the to-be-detected image;

[0017] Identify the state of the target flat washer located in the area of the suspension clamp in the target image through a trained second detection model, and obtain the state of the target flat washer.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0019] Obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes a suspension clamp and a flat washer;

[0020] Detect the suspension clamp on the to-be-detected image through a trained first detection model, obtain the position information of the suspension clamp, and based on the position information of the suspension clamp, divide a target image including the area of the suspension clamp from the to-be-detected image;

[0021] Identify the state of the target flat washer located in the area of the suspension clamp in the target image through a trained second detection model, and obtain the state of the target flat washer.

[0022] Fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, which when executed by a processor, implements the following steps:

[0023] Obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes suspension clamps and flat washers;

[0024] Detect the suspension clamps in the to-be-detected image through a trained first detection model to obtain suspension clamp position information, and based on the suspension clamp position information, divide a target image including a suspension clamp area from the to-be-detected image;

[0025] Identify the state of the target flat washers located in the suspension clamp area in the target image through a trained second detection model to obtain the state of the target flat washers.

[0026] The above detection method, device, computer device, storage medium and computer program product for transmission lines obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes suspension clamps and flat washers. Detect the suspension clamps in the to-be-detected image through a trained first detection model to obtain suspension clamp position information, and based on the suspension clamp position information, divide a target image including a suspension clamp area from the to-be-detected image. In this way, based on the trained first detection model, the area corresponding to the to-be-detected image can be accurately located to the suspension clamp area, reducing the image information irrelevant to the suspension clamps and further highlighting the image information of the area where the suspension clamps are located. Identify the state of the target flat washers located in the suspension clamp area in the target image through a trained second detection model to obtain the state of the target flat washers. Therefore, by using the trained second detection model to detect the target image including the suspension clamp area, the target flat washers in the suspension clamp area can be quickly and accurately determined, and the state of the target flat washers can be accurately identified, thereby greatly increasing the detection accuracy of ultra-small targets. Description of the Drawings

[0027] Figure 1 It is an application environment diagram of the detection method for transmission lines in an embodiment;

[0028] Figure 2 It is a schematic flowchart of the detection method for transmission lines in an embodiment;

[0029] Figure 3 It is a schematic flowchart of the detection method for transmission lines in another embodiment;

[0030] Figure 4Schematic flowchart of steps for determining a pre-trained first detection model in an embodiment;

[0031] Figure 5 Schematic flowchart of steps for determining a pre-trained second detection model in an embodiment;

[0032] Figure 6 Schematic flowchart of steps for determining the stacking training in an embodiment;

[0033] Figure 7 Schematic flowchart of a detection method for a transmission line in another embodiment;

[0034] Figure 8 Block diagram of the structure of a detection device for a transmission line in an embodiment;

[0035] Figure 9 Block diagram of the structure of a detection device for a transmission line in another embodiment;

[0036] Figure 10 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0037] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] The detection method for a transmission line provided by an embodiment of the present application can be applied to an application environment as Figure 1 shown. Among them, the unmanned aerial vehicle 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers. The unmanned aerial vehicle 102 sends the captured image to be detected to the server 104. The server 104 obtains the image to be detected obtained by photographing the target transmission line, and the target transmission line includes suspension clamps and flat washers. The server 104 detects the suspension clamps in the image to be detected through the trained first detection model, obtains the position information of the suspension clamps, and based on the position information of the suspension clamps, divides the target image including the area of the suspension clamps from the image to be detected. The server 104 identifies the state of the target flat washer located in the area of the suspension clamps in the target image through the trained second detection model, and obtains the state of the target flat washer. Among them, the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0039] In one embodiment, as Figure 2As shown, a detection method for a transmission line is provided. Taking the application of this method to a computer device as an example for illustration, the computer device can be Figure 1 the server or the terminal in Figure 1 . Among them, the terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The detection method for the transmission line includes the following steps:

[0040] Step S202: Obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes suspension clamps and flat washers.

[0041] Among them, the transmission line is a high-voltage power line that steps up the voltage from a power plant or a substation and transmits the power to a step-down substation. The suspension clamp is a fitting that suspends the conductor (ground wire) to a suspension insulator string (group) or a fitting string (group). The suspension clamp is used to fix the conductor on the insulator string of a straight-line tower, or to hang a lightning rod on a straight-line tower. It can also be used to support the transposed conductor on a transposition tower and fix the jumper wire on a tension-angle tower. The flat washer is usually a thin sheet of various shapes, which is used to reduce friction, prevent leakage, isolate, prevent loosening, or disperse pressure. Among them, the flat washer is installed on the suspension clamp.

[0042] Specifically, the computer device obtains the to-be-detected image obtained by photographing the target transmission line. The to-be-detected image contains the information of the suspension clamps and flat washers in the target line. The target transmission line includes suspension clamps and flat washers. The to-be-detected image can be obtained by a machine with flight and photographing functions photographing the target transmission line in a wide-angle situation. The machine can be a drone.

[0043] Step S204: Use the trained first detection model to detect the suspension clamps in the to-be-detected image, obtain the suspension clamp position information, and based on the suspension clamp position information, divide a target image including the suspension clamp area from the to-be-detected image.

[0044] Among them, the trained first detection model is the YOLOv5 model (You Only Look Once version 5). The YOLO model is a target detection model and can be implemented based on the one-stage algorithm. The trained first detection model is used to determine the suspension clamps in the target transmission line. Among them, the trained first detection model consists of four parts, namely the first input end, the first BackBone (benchmark network), the first Neck (intermediate layer), and the first Head (output layer).

[0045] Specifically, the computer device obtains the trained first detection model, and uses the trained first detection model to detect the suspension clamp in the image to be detected, and determines the coordinate information of the suspension clamp in the image to be detected. Based on the coordinate information of the suspension clamp, the computer device divides a high-definition target image including the area of the suspension clamp from the image to be detected. Among them, the trained first detection model is constructed based on multiple sliding windows of the suspension clamp, and the multiple sliding windows of the suspension clamp are obtained by clustering the sizes of the sample suspension clamps in multiple sample annotation images.

[0046] Step S206: Use the trained second detection model to identify the state of the target flat washer located in the area of the suspension clamp in the target image, and obtain the state of the target flat washer.

[0047] Among them, the trained second detection model is the YOLOv5 model (the 5th version of You Only Look Once), and this model is a model for target detection based on the one-stage algorithm. The trained second detection model is used to detect the state of the target flat washer. The states are divided into the state of missing flat washer and the state of normal flat washer. The trained second detection model consists of four parts, namely the second input end, the second BackBone (benchmark network), the second Neck (intermediate layer), and the second Head (output layer).

[0048] Specifically, the computer device obtains the trained second detection model, and inputs the target image into the trained second detection model to identify the state of the target flat washer located in the area of the suspension clamp in the target image, and obtain the state of the target flat washer. Among them, the trained second detection model is constructed based on multiple sliding windows of the flat washer, and the multiple sliding windows of the flat washer are obtained by clustering the sizes of the flat washers in multiple sample annotation images.

[0049] For example, the computer device obtains the trained second YOLOv5 model, and uses the trained second YOLOv5 model to identify the state of the target flat washer located in the area of the suspension clamp in the target image, and obtain the state of the target flat washer. If the state of the target flat washer is the state of missing flat washer, the computer device sends an alarm message to instruct the technician to conduct a patrol; if the state of the target flat washer is the state of normal flat washer, the computer device obtains the next image to be detected corresponding to the next target transmission line, and detects the state of the flat washer in the next target transmission line based on the first YOLOv5 model and the second YOLOv5 model.

[0050] In the above detection method for transmission lines, by obtaining a to-be-detected image captured from a target transmission line, the target transmission line includes suspension clamps and flat washers. Through the trained first detection model, the to-be-detected image is detected for suspension clamps to obtain the position information of the suspension clamps, and based on the position information of the suspension clamps, a target image including the area of the suspension clamps is divided from the to-be-detected image. In this way, based on the first detection model, the area corresponding to the to-be-detected image can be accurately located to the area of the suspension clamps, reducing the image information unrelated to the suspension clamps and further highlighting the image information of the area where the suspension clamps are located. Through the trained second detection model, the state of the target flat washers located in the area of the suspension clamps in the target image is identified to obtain the state of the target flat washers. Therefore, by detecting the target image including the area of the suspension clamps through the second detection model, the target flat washers in the area of the suspension clamps can be determined quickly and accurately, and the state of the target flat washers can be accurately identified. Thus, the detection accuracy for ultra-small targets is greatly increased.

[0051] In one embodiment, as Figure 3 shown, the method includes:

[0052] Step S302, obtain a plurality of sample images of the transmission line. For each sample image, label the sample position of the sample suspension clamp and the sample state of the sample flat washer in the corresponding sample image to obtain a first data set composed of a plurality of sample labeled images.

[0053] Specifically, the computer device obtains a plurality of sample images captured from the transmission line. For each sample image, the computer device labels the sample position of the sample suspension clamp and the sample state of the sample flat washer in the corresponding sample image to obtain each labeled sample image. The computer device combines the plurality of sample labeled images to obtain a first data set. Among them, each labeled sample image contains two labels, namely, sample position label and sample state label. The sample position label is used to represent the coordinate information of the sample suspension clamp, and the sample state label is used to represent the state of whether the sample flat washer is missing.

[0054] For example, for sample image A, the sample image A corresponds to an image of a certain line segment in the transmission line. The sample image A contains the transmission line segment corresponding to the certain line segment, the sample suspension clamp and sample flat washer in the line segment, and the environmental information corresponding to the line segment (such as the background of the line segment is a mountain). The computer device labels the sample suspension clamp in the sample image A with a first character and labels the sample flat washer in the sample image A with a second character to obtain the labeled sample image a.

[0055] Step S304: Based on the multiple sample annotation images in the first dataset, by clustering the sizes of the sample suspension clamps in the multiple sample annotation images, multiple sliding windows of the suspension clamps are determined, and by clustering the sizes of the sample flat washers in the multiple sample annotation images, multiple sliding windows of the flat washers are determined.

[0056] Specifically, the computer device, based on the multiple sample annotation images in the first dataset, clusters the sizes of the sample suspension clamps in the multiple sample annotation images based on the first K-means clustering algorithm to determine multiple sliding windows of the suspension clamps, and determines the hyperparameters corresponding to the suspension clamps based on the multiple sliding windows of the suspension clamps. Clusters the sizes of the sample flat washers in the multiple sample annotation images based on the second K-means clustering algorithm to determine multiple sliding windows of the flat washers, and determines the hyperparameters corresponding to the flat washers based on the multiple sliding windows of the flat washers.

[0057] Among them, the hyperparameters corresponding to the suspension clamps include multiple sliding windows of the suspension clamps, the size of the first input image, the size of the first batch, the first initial learning rate, the first initialized network parameters, etc. The hyperparameters corresponding to the flat washers include multiple sliding windows of the flat washers, the size of the second input image, the size of the second batch, the second initial learning rate, the second initialized network parameters. Among them, both the first initialized network parameters and the second initialized network parameters can be obtained through the coco pre-trained weights.

[0058] For example, the computer device annotates images based on multiple samples in the first dataset, clusters the sizes of the sample suspension clamps in the multiple sample-annotated images based on the first K-means clustering algorithm, and obtains 9 anchors with different sizes and aspect ratios corresponding to the suspension clamps. Among them, the 9 anchors contain three different scales of tensors for identifying small targets, medium targets, and large targets. The computer device determines the hyperparameters corresponding to the suspension clamps based on the 9 anchors corresponding to the suspension clamps, the first input image with a size of 608*608, the first batch size of 8, the first initial learning rate of 0.001, and the first initialized network parameters. The computer device clusters the sizes of the sample flat washers in the multiple sample-annotated images based on the second K-means clustering algorithm and obtains 9 anchors with different sizes and aspect ratios corresponding to the flat washers. Among them, the 9 anchors contain three different scales of tensors for identifying small targets, medium targets, and large targets. The computer device determines the hyperparameters corresponding to the flat washers based on the 9 anchors corresponding to the flat washers, the second input image with a size of 608*608, the second batch size of 8, the second initial learning rate of 0.001, and the second initialized network parameters. Among them, the anchor corresponding to the suspension clamp corresponds to the sliding window of the suspension clamp. Among them, the anchor corresponding to the flat washer corresponds to the sliding window of the flat washer

[0059] Step S306: Based on the multiple sliding windows of the suspension clamp, construct an initial first detection model, and pre-train the initial first detection model based on the first dataset to obtain a pre-trained first detection model.

[0060] Specifically, the computer device determines the hyperparameters corresponding to the suspension clamp based on the multiple sliding windows of the suspension clamp, and constructs an initial first detection model composed of a first input end, a first Backbone, a first Neck, and a first head based on the hyperparameters corresponding to the suspension clamp. The computer device pre-trains the initial first detection model based on the first dataset to obtain a pre-trained first detection model.

[0061] For example, the computer device determines the hyperparameters corresponding to the suspension line clamp based on the 9 anchors of the suspension line clamp, the first input image with a size of 608*608, the first batch size of 8, the first initial learning rate of 0.001, and the first initialization network parameters. The computer device constructs an initial first detection model consisting of a first input end, a first Backbone, a first Neck, and a first head based on the hyperparameters corresponding to the suspension line clamp. The computer device performs preprocessing and data enhancement on the first data set based on the first input end to obtain a processed first data set, and the processed first data set contains multiple processed new images. The processed first data set is subjected to data feature extraction based on the first Backbone (reference network) to obtain an extracted first data set, and the extracted first data set contains multiple feature maps. The computer device identifies the extracted first data set based on the first Neck (middle layer), inputs three tensors of different scales, and identifies the borders and categories of the three tensors of different scales based on the first Head (output layer), and obtains the borders and corresponding categories of the target objects in the first data set. The computer device determines the difference between the detected border and the real border through GIOU_loss, and adjusts the model parameters to obtain the pre-trained first detection model. The specific calculation formula is as follows:

[0062]

[0063] Where a is the detection box and b is the true box. IOU is the intersection over union ratio, that is, the ratio of the intersection and union of the detection box and the true box, and C is the minimum circumscribed matrix that can enclose a and b. Based on the pre-trained first detection model, the image is used for target recognition. Each image will generate 22743 pre-selected boxes. Each pre-selected box contains 5+n values, which are the center point coordinates x, y and width and height w, h of the pre-selected box, as well as the confidence c and the probability of n categories. First, according to the value of the confidence c, some low-confidence pre-selected boxes are filtered out. In this model, the confidence threshold is 0.5. The retained pre-selected boxes are screened by DIOU-NMS to screen out too many repeated pre-selected boxes. The remaining pre-selected boxes are used as the final detection results.

[0064] Step S308, constructing an initial second detection model based on multiple sliding windows of the flat washer, performing image cropping processing on the first data set to obtain a second data set, and training the initial second detection model based on the second data set to obtain a pre-trained second detection model.

[0065] Specifically, the computer device determines the hyperparameters corresponding to the flat washer based on the multiple sliding windows of the flat washer, and constructs an initial second detection model consisting of a second input end, a second backbone, a second neck, and a second head based on the hyperparameters corresponding to the flat washer. The computer device performs image cropping processing on the first data set to obtain a second data set. The computer device pre-trains the initial second detection model based on the second data set to obtain a pre-trained second detection model.

[0066] For example, the computer device determines the hyperparameters corresponding to the flat washer based on the 9 anchors of the flat washer, the second input image with a size of 608*608, the second batch size of 8, the second initial learning rate of 0.001, and the second initialization network parameters. The computer device constructs an initial second detection model consisting of a second input end, a second Backbone, a second Neck, and a second head based on the hyperparameters corresponding to the flat washer. The computer device performs image cropping processing on the first data set to obtain a second data set. The computer device performs preprocessing and data enhancement on the second data set based on the second input end to obtain a processed second data set, and the processed second data set contains multiple processed new images. Based on the second Backbone (reference network), the processed second data set is subjected to data feature extraction to obtain an extracted second data set, and the extracted second data set contains multiple feature maps. The computer device identifies the extracted second data set based on the second Neck (intermediate layer), inputs three tensors of different scales, and identifies the borders and categories of the three tensors of different scales based on the second Head (output layer), and obtains the borders and corresponding categories of the target objects in the second data set. The computer device determines the difference between the detected border and the real border through GIOU_loss, and adjusts the model parameters to obtain a pre-trained second detection model. Among them, based on the pre-trained second detection model, the image is recognized as a target, and each image will generate 22743 pre-selected boxes. Each pre-selected box contains 5+n values, which are the center point coordinates x, y and width and height w, h of the pre-selected box, as well as the confidence c and the probability of n categories. First, according to the value of confidence c, some low-confidence pre-selected boxes are filtered out. In this model, the confidence threshold is 0.5. The retained pre-selected boxes are screened by DIOU-NMS to screen out too many repeated pre-selected boxes. The remaining pre-selected boxes are used as the final detection results.

[0067] Step S310, obtaining a second verification set from the second data set, and based on the second verification set, superimposing training is performed on the pre-trained first detection model and the pre-trained second detection model to obtain a trained first detection model and a trained second detection model.

[0068] Specifically, the computer device obtains a second validation set from the second data set, processes the second validation set through a pre-trained first detection model to obtain a pre-trained first result, and processes the pre-trained first result through a pre-trained second detection model to obtain a trained first detection model and a trained second detection model.

[0069] In this embodiment, based on the sample images of each transmission line, the sample positions of the sample suspension clamps and the sample states of the sample flat washers in the corresponding sample images are marked to obtain a first data set composed of multiple sample marked images. By clustering the sizes of the sample suspension clamps in the multiple sample marked images, multiple sliding windows of the suspension clamps are determined, and by clustering the sizes of the sample flat washers in the multiple sample marked images, multiple sliding windows of the flat washers are determined. Based on the multiple sliding windows of the suspension clamps, an initial first detection model is constructed, and the initial first detection model is pre-trained based on the first data set to obtain a pre-trained first detection model capable of detecting suspension clamps from sample images. Based on the multiple sliding windows of the flat washers, an initial second detection model is constructed, and the first data set is subjected to image cropping processing to obtain a second data set. Based on the second data set, the initial second detection model is trained to obtain a pre-trained second detection model capable of quickly detecting the state of the flat washers in the suspension clamps. A second validation set is obtained from the second data set, and based on the second validation set, the pre-trained first detection model and the pre-trained second detection model are subjected to stacked training to obtain a trained first detection model and a trained second detection model. In this way, through stacked training, the accuracy of the trained first detection model for detecting the suspension line area and the accuracy of the trained second detection model for detecting the state of the flat washers are further improved.

[0070] In one embodiment, as Figure 4 shown, the first data set consists of a first training set, a first test set, and a first validation set. The pre-training of the initial first detection model based on the first data set to obtain a pre-trained first detection model includes:

[0071] Step S402, pre-train the initial first detection model based on multiple first training sample marked images in the first training set to obtain a first detection model to be tested in the current cycle, and test the first detection model to be tested based on the first test set to obtain the theoretical test positions of multiple test sample suspension clamps.

[0072] Specifically, the computer device pre-trains the initial first detection model based on multiple first training sample annotation images in the first training set to obtain the first detection model to be tested in the current cycle. The computer device processes the first test set through the first detection model to be tested in the current cycle to obtain the theoretical test coordinate information of multiple test sample suspension clamps in the current cycle. Here, the cycle corresponds to the training round.

[0073] Step S404: Based on multiple first test sample annotation images in the first test set, determine the actual test positions of multiple test sample suspension clamps, and based on the actual test positions of the multiple test sample suspension clamps and the theoretical test positions of the multiple test sample suspension clamps, obtain multiple first test evaluation index results. The first test evaluation index results include the first test recall rate, the first test accuracy, and the first test mean average precision.

[0074] Among them, the recall rate is the same as the recall ratio, which is used to characterize the correct rate of positive samples. The accuracy is used to characterize the probability of correct prediction. The mean average precision (mAP) is used to measure the performance of the object detection algorithm.

[0075] Specifically, the computer device determines multiple first test sample annotation images based on the first test set. For each first test sample annotation image, based on the test sample position annotation in the corresponding first test sample annotation image, determine the theoretical test positions of multiple test sample suspension clamps. The computer device compares the actual test positions and the theoretical test positions of the same test sample suspension clamps to determine the first test recall rate, the first test accuracy, and the first test mean average precision.

[0076] Step S406: Based on the multiple first test evaluation index results, adjust the parameters of the first detection model to be tested, enter the loop iteration of the next cycle, and return to the step of testing the first detection model to be tested based on the first test set to continue execution. Until multiple first test evaluation index results all meet the index conditions, then determine the first detection model to be tested corresponding to the current cycle as the first intermediate detection model, and use the multiple first test evaluation index results in the current cycle as the first target evaluation threshold of the corresponding evaluation index.

[0077] Specifically, the computer device adjusts the parameters of the first detection model to be tested based on the results of multiple first test evaluation metrics, enters the next cycle of iterative loop, and returns to the step of testing the first detection model to be tested based on the first test set to continue execution, and determines the results of multiple first test evaluation metrics corresponding to each cycle. The computer device determines whether the results of multiple first test evaluation metrics meet the metric conditions based on the results of multiple first test evaluation metrics corresponding to each cycle. If the results of multiple first test evaluation metrics all meet the metric conditions, it determines that the first detection model to be tested corresponding to the current cycle is the first intermediate detection model, and uses the results of multiple first test evaluation metrics in the current cycle as the first target evaluation thresholds for the corresponding evaluation metrics.

[0078] For example, if the current cycle is the 30th cycle, if the difference between the first test recall rate in the current cycle and the first test recall rate in the previous cycle is small, and the difference between the first test accuracy in the current cycle and the first test accuracy in the previous cycle is small, and the difference between the first test average precision in the current cycle and the first test average precision in the previous cycle is small, that is, the change rates of the first test recall rate, the first test accuracy, and the first test average precision in the current cycle all tend to be unchanged, it is determined that the results of multiple first test metrics in the current cycle all meet the metric conditions, and it is determined that the first detection model to be tested corresponding to the current cycle is the first intermediate detection model. The first test recall rate in the current cycle is used as the first target evaluation threshold for the first evaluation metric, the first test accuracy in the current cycle is used as the first target evaluation threshold for the second evaluation metric, and the first test average precision in the current cycle is used as the first target evaluation threshold for the third evaluation metric. The first evaluation metric corresponds to the recall rate metric, the second evaluation metric corresponds to the accuracy metric, and the third evaluation metric corresponds to the average precision metric.

[0079] Step S408, process the first validation set through the first intermediate detection model to obtain multiple first validation evaluation metric results. If each first validation evaluation metric result matches the first target evaluation threshold of the corresponding evaluation metric, it is determined that the first intermediate detection model is the pre-trained first detection model.

[0080] Specifically, the computer device processes the first validation set through the first intermediate detection model, obtains multiple first validation evaluation index results, and compares each first validation evaluation index result with the first target evaluation threshold of the corresponding evaluation index. If the difference between a first validation evaluation index result and the first target evaluation threshold of the corresponding evaluation index is within the difference range, it is determined that the first validation evaluation index result matches the first target evaluation threshold of the corresponding evaluation index. If each first validation evaluation index result matches the first target evaluation threshold of the corresponding evaluation index, it is determined that the first intermediate detection model is the pre-trained first detection model.

[0081] In this embodiment, the initial first detection model is pre-trained through multiple first training sample annotation images in the first training set to obtain the first detection model to be tested in the current cycle. Based on the first test set, the first detection model to be tested is tested to obtain the theoretical test positions of multiple test sample suspension clamps. Based on the actual test positions of the multiple test sample suspension clamps and the theoretical test positions of the multiple test sample suspension clamps, multiple first test evaluation index results are obtained. Based on the multiple first test evaluation index results, the parameters of the first detection model to be tested are adjusted, and the loop iteration enters the next cycle, and returns to the step of testing the first detection model to be tested based on the first test set and continues to execute until multiple first test evaluation index results all meet the index conditions. Then it is determined that the first detection model to be tested corresponding to the current cycle is the first intermediate detection model, and the multiple first test evaluation index results in the current cycle are used as the first target evaluation thresholds of the corresponding evaluation indexes. The first intermediate detection model processes the first validation set to obtain multiple first validation evaluation index results. If each first validation evaluation index result matches the first target evaluation threshold of the corresponding evaluation index, it is determined that the first intermediate detection model is the pre-trained first detection model. In this way, based on the first training set and the first test set, a first intermediate detection model that can detect suspension clamps from sample images can be obtained. Then, through the verification of the first validation set, it is ensured that the detection effect of the pre-trained first detection model determined by the first intermediate detection model is more ideal and the accuracy is higher.

[0082] In one embodiment, as Figure 5 shown, the second data set includes a second training set, a second test set, and a second validation set. Based on the second data set, training the initial second detection model to obtain a pre-trained second detection model includes:

[0083] Step 502: Based on the multiple second training sample annotation images in the second training set, pre-train the initial second detection model to obtain the second detection model to be tested in the current cycle. Then, based on the second test set, test the second detection model to be tested to obtain the theoretical test states of multiple test sample flat washers.

[0084] Specifically, the computer device pre-trains the initial second detection model based on the multiple second training sample annotation images in the second training set to obtain the second detection model to be tested in the current cycle. The computer device processes the second test set through the second detection model to be tested in the current cycle to obtain the theoretical test states of multiple test sample flat washers in the current cycle. This state can be the state of missing flat washers or the normal state of flat washers.

[0085] Step 504: Based on the multiple second test sample annotation images in the second test set, determine the actual test states of multiple test sample flat washers. Then, based on the actual test states of multiple test sample flat washers and the theoretical test states of multiple test sample flat washers, obtain multiple second test evaluation index results, where the second test evaluation index results include the second test recall rate, the second test accuracy, and the second test mean average precision.

[0086] Among them, the recall rate is the recall rate, which is used to characterize the correct rate of positive samples. The accuracy is used to characterize the probability of correct prediction. The mean average precision (mAP) is used to measure the performance of the object detection algorithm.

[0087] Specifically, the computer device determines multiple second test sample annotation images based on the second test set. For each second test sample annotation image, based on the test sample state annotation in the corresponding second test sample annotation image, determine the actual test states of multiple test sample flat washers. The computer device compares the actual test states and the theoretical test states of flat washers belonging to the same test sample to determine the second test recall rate, the second test accuracy, and the second test mean average precision.

[0088] Step 506: Based on the multiple second test evaluation index results, adjust the parameters of the second detection model to be tested, enter the loop iteration of the next cycle, and return to the step of testing the second detection model to be tested based on the second test set to continue execution. Until multiple second test evaluation index results all meet the index conditions, determine the second detection model to be tested corresponding to the current cycle as the second intermediate detection model, and use the multiple second test evaluation index results in the current cycle as the second target evaluation thresholds of the corresponding evaluation indexes.

[0089] Specifically, based on the results of multiple second test evaluation metrics, the computer device adjusts the parameters of the second detection model to be tested, enters the loop iteration of the next cycle, and returns to the step of testing the second detection model to be tested based on the second test set to continue execution, determining the results of multiple second test evaluation metrics corresponding to each cycle. The computer device determines whether the results of multiple second test evaluation metrics meet the metric conditions based on the results of multiple second test evaluation metrics corresponding to each cycle. If the results of multiple second test evaluation metrics all meet the metric conditions, it is determined that the second detection model to be tested corresponding to the current cycle is the second intermediate detection model, and the results of multiple second test evaluation metrics in the current cycle are used as the second target evaluation thresholds for the corresponding evaluation metrics.

[0090] For example, if the current cycle is the 40th cycle, if the difference between the second test recall rate in the current cycle and the second test recall rate in the previous cycle is small, and the difference between the second test accuracy in the current cycle and the second test accuracy in the previous cycle is small, and the difference between the second test mean average precision in the current cycle and the second test mean average precision in the previous cycle is small, that is, the change rates of the second test recall rate, the second test accuracy, and the second test mean average precision in the current cycle all tend to be unchanged, it is determined that the results of multiple second test metrics in the current cycle all meet the metric conditions, and it is determined that the second detection model to be tested corresponding to the current cycle is the second intermediate detection model. The second test recall rate in the current cycle is used as the second target evaluation threshold for the first evaluation metric, the second test accuracy in the current cycle is used as the second target evaluation threshold for the second evaluation metric, and the second test mean average precision in the current cycle is used as the second target evaluation threshold for the third evaluation metric. The first evaluation metric corresponds to the recall rate metric, the second evaluation metric corresponds to the accuracy metric, and the third evaluation metric corresponds to the mean average precision metric.

[0091] Step 508, process the second validation set through the second intermediate detection model to obtain the results of multiple second validation evaluation metrics. If each result of the second validation evaluation metric matches the second target evaluation threshold of the corresponding evaluation metric, it is determined that the second intermediate detection model is the second detection model that has been pre-trained.

[0092] Specifically, the computer device processes the second validation set through the second intermediate detection model, obtains multiple second validation evaluation index results, and compares each second validation evaluation index result with the second target evaluation threshold of the corresponding evaluation index. If the difference between a second validation evaluation index result and the second target evaluation threshold of the corresponding evaluation index is within the difference range, it is determined that the second validation evaluation index result matches the second target evaluation threshold of the corresponding evaluation index. If each second validation evaluation index result matches the second target evaluation threshold of the corresponding evaluation index, it is determined that the second intermediate detection model is the pre-trained second detection model.

[0093] In this embodiment, the initial second detection model is pre-trained through multiple second training sample annotation images in the second training set to obtain the second detection model to be tested in the current cycle. Based on the second test set, the second detection model to be tested is tested to obtain the theoretical test states of multiple test sample flat washers. Based on the actual test states of the multiple test sample flat washers and the theoretical test states of the multiple test sample flat washers, multiple second test evaluation index results are obtained. Based on the multiple second test evaluation index results, the parameters of the second detection model to be tested are adjusted, and the loop iteration enters the next cycle, and returns to the step of testing the second detection model to be tested based on the second test set to continue execution until multiple second test evaluation index results all meet the index conditions. Then it is determined that the second detection model to be tested corresponding to the current cycle is the second intermediate detection model, and the multiple second test evaluation index results in the current cycle are used as the second target evaluation thresholds of the corresponding evaluation indexes. Through the second intermediate detection model, the second validation set is processed to obtain multiple second validation evaluation index results. If each second validation evaluation index result matches the second target evaluation threshold of the corresponding evaluation index, it is determined that the second intermediate detection model is the pre-trained second detection model. In this way, based on the second training set and the second test set, a second intermediate detection model that can detect the state of ultra-small target flat washers can be obtained. Then, based on the verification of the second validation set, it is ensured that the recognition effect of the pre-trained second detection model determined by the second intermediate detection model is more ideal and the accuracy is higher.

[0094] In one embodiment, the first data set consists of a first training set, a first test set, and a first validation set. Image cropping processing is performed on the first data set to obtain a second data set, including: intercepting the images containing suspension clamps in the multiple first training sample labeled images in the first training set to obtain a second training set. Intercepting the images containing suspension clamps in the multiple first test sample labeled images in the first test set to obtain a second test set. Intercepting the images containing suspension clamps in the multiple first validation sample labeled images in the first validation set to obtain a second validation set. The second training set, the second test set, and the second validation set constitute the second data set.

[0095] Specifically, for the multiple first training sample labeled images in the first training set, the computer device determines the images containing suspension clamps in each first training sample labeled image and intercepts them through cropping to obtain a second training set. For the multiple first test sample labeled images in the second test set, the computer device determines the images containing suspension clamps in each second test sample labeled image and intercepts them through cropping to obtain a second test set. For the multiple first validation sample labeled images in the second validation set, the computer device determines the images containing suspension clamps in each second validation sample labeled image and intercepts them through cropping to obtain a second validation set. The computer device constitutes the second data set with the second training set, the second test set, and the second validation set.

[0096] In this embodiment, the images containing suspension clamps in the multiple first training sample labeled images in the first training set are intercepted to obtain a second training set. The images containing suspension clamps in the multiple first test sample labeled images in the first test set are intercepted to obtain a second test set. The images containing suspension clamps in the multiple first validation sample labeled images in the first validation set are intercepted to obtain a second validation set. The second training set, the second test set, and the second validation set constitute the second data set. In this way, the second test set is obtained by intercepting the first data set, highlighting the area where the small target suspension clamp is located, so that the flat washers in the area where the suspension clamp is located can be strengthened, which can greatly increase the accuracy of model training and is conducive to obtaining a pre-trained second detection model that can quickly detect the state of the flat washers in the suspension clamp subsequently.

[0097] In one embodiment, as Figure 6 shown, based on the second validation set, the pre-trained first detection model and the pre-trained second detection model are superimposed and trained to obtain a trained first detection model and a trained second detection model, including:

[0098] Step S602: Process the labeled images of multiple second validation samples in the second validation set through a pre-trained first detection model to obtain multiple first sample target images carrying the position information of the suspension clamp.

[0099] Specifically, the computer device obtains the pre-trained first detection model and detects the suspension clamp through the pre-trained first detection model for the labeled images of multiple second validation samples in the second validation set to determine the suspension clamp coordinate information in the labeled images of multiple second validation samples. For each labeled image of a second validation sample, the computer device divides, based on the suspension clamp coordinate information in the labeled images of multiple second validation samples, a first sample target image including the area of the suspension clamp from the corresponding labeled image of the second validation sample.

[0100] Step S604: Detect the suspension clamp for multiple first sample target images through a pre-trained second detection model to obtain the sample flat washer status corresponding to each first sample target image.

[0101] Specifically, the computer device obtains the pre-trained second detection model. For each first sample target image, the computer device identifies the status of the flat washer within the area of the suspension clamp in the corresponding first sample target image based on the pre-trained second detection model to obtain the sample flat washer status corresponding to each first sample target image.

[0102] Step S606: Determine the actual verification status of multiple verification sample flat washers based on the labeled images of multiple second validation samples in the second validation set.

[0103] Specifically, the computer device obtains the labeled images of multiple second validation samples in the second validation set and determines the actual verification status of multiple verification sample flat washers based on the verification sample status labels in each labeled image of the second validation sample.

[0104] Step S608: Determine the results of multiple third verification evaluation metrics based on the actual verification status of multiple verification sample flat washers and the status of multiple sample flat washers. If each result of the third verification evaluation metric meets the threshold evaluation conditions of the corresponding evaluation metric, determine the pre-trained first detection model as the trained first detection model and the pre-trained second detection model as the trained second detection model.

[0105] Specifically, the computer device determines the results of multiple third verification evaluation indicators based on the actual verification status of multiple verification sample flat washers and the status of the sample flat washers corresponding to each of the first sample target images. The computer device obtains the threshold evaluation parameters of each evaluation indicator. If each result of the third verification evaluation indicator is greater than the threshold evaluation parameter of the corresponding evaluation indicator, it determines that the pre-trained first detection model is the trained first detection model and the pre-trained second detection model is the trained second detection model.

[0106] In this embodiment, through the pre-trained first detection model, multiple second verification sample annotation images in the second verification set are processed to obtain multiple first sample target images carrying the position information of the suspension clamp. Through the pre-trained second detection model, the suspension clamps are detected for the multiple first sample target images to obtain the status of the sample flat washers corresponding to each of the first sample target images. Based on the multiple second verification sample annotation images in the second verification set, the actual verification status of the multiple verification sample flat washers is determined. Based on the actual verification status of the multiple verification sample flat washers and the status of the multiple sample flat washers, the results of multiple third verification evaluation indicators are determined. If each result of the third verification evaluation indicator meets the threshold evaluation conditions of the corresponding evaluation indicator, it determines that the pre-trained first detection model is the trained first detection model and the pre-trained second detection model is the trained second detection model. In this way, through the stacked training, the accuracy of the trained first detection model for detecting the suspension line area and the accuracy of the trained second detection model for detecting the status of the flat washer are further improved.

[0107] To more clearly understand the training processes of the trained first detection model and the trained second detection model, a more detailed embodiment is provided for description. As Figure 7As shown, step S1: The drone collects pictures of the transmission line. The computer device obtains the pictures of the transmission line, marks the positions of the suspension clamps and the states of the flat washers, and constructs the first data set. Specifically, step S1 includes: using the drone to collect pictures along the transmission line, the computer device obtains the pictures along the transmission line, and uses the marking tool to mark the positions of the suspension clamps in the pictures and the positions where the flat washers on the suspension clamps should be (i.e., the hanging shafts of the suspension clamps and the nuts above the hanging plates), and mark the states of the flat washers. The states of the flat washers can be the state of missing flat washers and the state of normal flat washers. The computer device selects the pictures containing the suspension clamps as the first data set D, and the first data set D is divided into a first training set (i.e., Train_D), a first test set (i.e., Test_D), and a first verification set (i.e., Verification_D) according to 8:1:1. The computer device respectively crops the suspension clamp position images in the first training set, the first test set, and the first verification set to form a second training set (i.e., Train_D1), a second test set (i.e., Test_D1), and a second verification set (i.e., Verification_D1).

[0108] Step S2: The computer device clusters the sizes of the suspension clamps and the flat washers respectively according to the first data set. Specifically: The computer device uses the K-means clustering algorithm to cluster the sizes of the suspension clamps and the flat washers respectively, and obtains 8 anchors with different sizes and aspect ratios (i.e., multiple sliding windows corresponding to the suspension clamps and multiple sliding windows corresponding to the flat washers).

[0109] Step S3: The computer device constructs the YOLOv5 model for suspension clamps and the YOLOv5 model for the state of flat washers, and based on the 9 anchors of the suspension clamp, the first input image with a size of 608*608, the first batch size of 8, the first initial learning rate of 0.001, and the first initialized network parameters, determines the hyperparameters corresponding to the suspension clamp (i.e., sets the relevant parameters corresponding to the suspension clamp), and based on the 9 anchors of the flat washer, the second input image with a size of 608*608, the second batch size of 8, the second initial learning rate of 0.001, and the second initialized network parameters, determines the hyperparameters corresponding to the flat washer (i.e., sets the relevant parameters corresponding to the flat washer). The computer device trains based on the first dataset and tests the effect to determine the pre-trained first detection model, and trains based on the second dataset and tests the effect to determine the pre-trained second detection model. Among them, the YOLO model consists of four parts, namely the input end, BackBone (benchmark network), Neck (intermediate layer), and Head (output layer). For the input end, it mainly preprocesses and augments the input data. Among them, the preprocessing includes scaling the input image to meet the size required by the network and normalizing the image to improve the training speed and network accuracy. The data augmentation includes randomly erasing, translating, rotating, mirroring, randomly adding noise, using Mix-up, Mosaic, and SAT to augment the data. Among them, Mosaic data augmentation utilizes four annotated pictures and splices the four pictures to obtain a new picture containing the annotations of the four pictures. Processing the input based on this input end not only makes the detected object relatively smaller but also enriches the background of the detected object, effectively improving the model's detection ability for small targets. For the BackBone (benchmark network), it mainly extracts data features from the input image, that is, the input image first undergoes downsampling without information loss through the Facus module, and the CSPDarknet with a large receptive field deep network structure is used to extract data features. The CSPDarknet adds a CSP module on the basis of Darknet to solve the problem of repeated gradient information during the optimization of large networks. The CSP module divides the feature map of the basic layer into two parts for processing, enabling the gradient flow to propagate through different network paths, reducing the repeated calculation of gradients. Then they are merged through a cross-layer splicing structure, ensuring the model accuracy and reducing the computational amount. At the same time, this benchmark network uses a smooth Mish activation function to avoid the hard zero boundary of ReLU, allowing more information to penetrate deep into the neural network, improving the accuracy and generalization ability of the model. In addition, Dropblock is used interspersed in the model to discard the features of adjacent regions, overcoming the shortcoming that Dropout has an insignificant effect on convolutional layers, preventing overfitting, and thus improving the generalization ability of the model.For the Neck (middle layer), the data features extracted from the BackBone are further extracted, and according to the size of the object to be recognized, three tensors of different scales are output, which are used to recognize small objects, medium objects, and large objects respectively. In this layer, by adding the CSP2 module, SPP module, and FPN+PAN structure, diverse features are further fused and extracted. Among them, CSP2 draws on the CSP module and replaces the residual structure in CSP with a convolutional structure, further enhancing the network's ability to fuse features. Among them, SPP (Spatial Pyramid Pooling Networks) performs max pooling using pooling kernels of sizes 1*1, 5*5, 9*9, and 13*13, and then stitches the results together to perform multi-scale fusion of features, more effectively increasing the receptive field of the backbone features and significantly separating the context features. Among them, FPN (Feature Pyramid Networks) feature pyramid network constructs a feature pyramid by deconvolving and magnifying the original feature map layer by layer from top to bottom, which is conducive to the network extracting semantic features of objects of different sizes, enabling the model to recognize the same object of different sizes and scales. PAN (Path Aggregation Network) draws on the PANet algorithm in the field of image segmentation, extracts features from bottom to top, and aggregates the parameters of different layers from FPN by stitching, thereby enhancing the network's localization ability. For the output layer, for the three different sizes, the bounding boxes of the recognized objects and the corresponding categories are output.

[0110] Specifically, for the pre-trained first detection model, the computer device determines the hyperparameters corresponding to the suspension clamp based on 9 anchors of the suspension clamp, the first input image with a size of 608*608, the first batch size of 8, the first initial learning rate of 0.001, and the first initialized network parameters. The computer device constructs an initial first detection model composed of a first input end, a first Backbone, a first Neck, and a first head based on the hyperparameters corresponding to the suspension clamp. The computer device tests the initial first detection model based on the first dataset, analyzes evaluation metrics such as its recall rate and accuracy, and performs parameter adjustment and other processing methods for the missed detection and false alarm situations that occur in the test set until an ideal effect is achieved. Specifically: The computer device pre-trains the initial first detection model based on multiple first training sample annotation images in the first training set to obtain the first detection model to be tested in the current cycle. The computer device processes the first test set through the first detection model to be tested in the current cycle to obtain the theoretical test coordinate information of multiple test sample suspension clamps in the current cycle. The computer device determines multiple first test sample annotation images based on the first test set. For each first test sample annotation image, based on the test sample position annotation in the corresponding first test sample annotation image, it determines the theoretical test positions of multiple test sample suspension clamps. The computer device compares the actual test positions and the theoretical test positions of the suspension clamps belonging to the same test sample to determine the first test recall rate, the first test accuracy, and the first test mean average precision. The computer device adjusts the parameters of the first detection model to be tested based on the results of multiple first test evaluation metrics, enters the loop iteration of the next cycle, and returns to the step of testing the first detection model to be tested based on the first test set to continue execution, and determines the results of multiple first test evaluation metrics corresponding to each cycle. The computer device determines whether the results of multiple first test evaluation metrics meet the metric conditions based on the results of multiple first test evaluation metrics corresponding to each cycle. If the results of multiple first test evaluation metrics all meet the metric conditions, it determines that the first detection model to be tested corresponding to the current cycle is the first intermediate detection model, and uses the results of multiple first test evaluation metrics in the current cycle as the first target evaluation thresholds for the corresponding evaluation metrics. The computer device processes the first validation set through the first intermediate detection model to obtain multiple first validation evaluation metric results, and compares each first validation evaluation metric result with the first target evaluation threshold of the corresponding evaluation metric. If there is a difference between a first validation evaluation metric result and the first target evaluation threshold of the corresponding evaluation metric within the difference range, it is determined that the first validation evaluation metric result matches the first target evaluation threshold of the corresponding evaluation metric. If each first validation evaluation metric result matches the first target evaluation threshold of the corresponding evaluation metric, it is determined that the first intermediate detection model is the pre-trained first detection model.

[0111] For the pre-trained second detection model, based on the 9 anchors of the flat washer, the second input image with a size of 608*608, the second batch size of 8, the second initial learning rate of 0.001, and the second initialized network parameters, determine the hyperparameters corresponding to the flat washer. The computer device constructs an initial second detection model consisting of a second input end, a second Backbone, a second Neck, and a second head based on the hyperparameters corresponding to the flat washer. The computer device performs image cropping processing on the first dataset to obtain a second dataset. The computer device tests the initial second detection model based on the second dataset, analyzes evaluation metrics such as its recall rate and accuracy, and performs parameter adjustment and other processing methods for missed detections and false alarms in the test set until an ideal effect is achieved. Specifically: The computer device pre-trains the initial second detection model based on multiple second training sample annotation images in the second training set to obtain the second detection model to be tested in the current cycle. The computer device processes the second test set through the second detection model to be tested in the current cycle to obtain the theoretical test status of multiple test sample flat washers in the current cycle. This status can be the missing state of the flat washer or the normal state of the flat washer. The computer device determines multiple second test sample annotation images based on the second test set. For each second test sample annotation image, based on the test sample status annotation in the corresponding second test sample annotation image, determine the actual test status of multiple test sample flat washers. The computer device compares the actual test status and the theoretical test status of the flat washers belonging to the same test sample to determine the second test recall rate, the second test accuracy, and the second test mean average precision. The computer device adjusts the parameters of the second detection model to be tested based on the results of multiple second test evaluation metrics, enters the loop iteration of the next cycle, and returns to the step of testing the second detection model to be tested based on the second test set to continue execution, and determines the results of multiple second test evaluation metrics corresponding to each cycle. The computer device determines whether the results of multiple second test evaluation metrics meet the metric conditions based on the results of multiple second test evaluation metrics corresponding to each cycle. If the results of multiple second test evaluation metrics all meet the metric conditions, then determine the second detection model to be tested corresponding to the current cycle as the second intermediate detection model, and use the results of multiple second test evaluation metrics in the current cycle as the second target evaluation threshold for the corresponding evaluation metric. The computer device processes the second validation set through the second intermediate detection model to obtain multiple second validation evaluation metric results, and compares each second validation evaluation metric result with the second target evaluation threshold of the corresponding evaluation metric. If there is a difference between a second validation evaluation metric result and the second target evaluation threshold of the corresponding evaluation metric within the difference range, then determine that the second validation evaluation metric result matches the second target evaluation threshold of the corresponding evaluation metric.If the results of each second verification evaluation index match the second target evaluation threshold of the corresponding evaluation index, determine that the second intermediate detection model is the pre-trained second detection model.

[0112] Step S4: Two models are stacked to detect the preprocessed verification set (i.e., the second verification set) data, that is, the pre-trained first detection model and the pre-trained second detection model are stacked together for testing. Specifically, the computer device uses the pre-trained first detection model to detect the suspension clamps in multiple second verification sample annotation images in the second verification set, and determines the coordinate information of the suspension clamps in the multiple second verification sample annotation images. For each second verification sample annotation image, the computer device divides a first sample target image including the area of the suspension clamp from the corresponding second verification sample annotation image based on the coordinate information of the suspension clamps in the multiple second verification sample annotation images. For each first sample target image, the computer device uses the pre-trained second detection model to identify the state of the flat washers located in the area of the suspension clamp in the corresponding first sample target image, and obtains the sample flat washer state corresponding to each first sample target image.

[0113] Step S5: Adjust the model parameters or the data set (the first data set or the second data set) according to the detection results to achieve an ideal effect. Specifically, the computer device determines the actual verification state of the multiple verification sample flat washers based on the verification sample state annotation in each second verification sample annotation image. The computer device determines the results of multiple third verification evaluation indexes based on the actual verification state of the multiple verification sample flat washers and the sample flat washer state corresponding to each of the first sample target images. The computer device obtains the threshold evaluation parameters of each evaluation index. If the results of each third verification evaluation index are greater than the threshold evaluation parameters of the corresponding evaluation index, determine that the pre-trained first detection model is the trained first detection model and the pre-trained second detection model is the trained second detection model. If there is at least one result of the third verification evaluation index that is less than the threshold evaluation parameter of the corresponding evaluation index, adjust the pre-trained first detection model and the pre-trained second detection model, or the first data set and the second data set, so that the results of each adjusted third verification evaluation index are greater than the threshold evaluation parameters of the corresponding evaluation index.

[0114] In this embodiment, first, based on the trained first detection model, the large area of the suspension clamp is located to obtain a high-definition large image of the suspension clamp, and then the state of the flat washer in the suspension clamp area is detected by using the trained second detection model. Therefore, through the superimposed detection of the trained first detection model and the trained second detection model, the detection recall rate and accuracy of the flat washer can be greatly improved, that is, the detection accuracy of ultra-small targets is greatly increased. That is, through the two-step detection scheme, the disadvantage of poor detection ability of a single algorithm for ultra-small targets is overcome, thereby improving the efficiency of daily operation management and maintenance of the substation and reducing the probability of accidents. In addition, based on the drone shooting technology, unmanned detection is realized, which greatly saves manpower, overcomes the potential personal safety risks of inspection personnel, and greatly improves the inspection efficiency. At the same time, both the trained first detection model and the trained second detection model adopt the YOLOv5 model. The YOLOv5 model has a smaller model scale, low computing power requirements for computer devices, and is convenient to be deployed on devices. The YOLOv5 algorithm has a fast operation speed and can achieve the effect of real-time detection, further improving the detection efficiency.

[0115] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiment of the present application also provides a detection device for a transmission line for implementing the detection method of the transmission line involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the detection device for the transmission line provided below can refer to the limitations on the detection method of the transmission line in the above text, and will not be repeated here.

[0117] In one embodiment, as Figure 8 shown, a detection device for a transmission line is provided. The device 800 includes: an acquisition module 802, a detection module 804, and an identification module 806, where:

[0118] An acquisition module 802, configured to acquire a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes suspension clamps and flat washers.

[0119] A detection module 804, configured to detect the suspension clamps in the to-be-detected image through a trained first detection model, obtain suspension clamp position information, and based on the suspension clamp position information, divide a target image including a suspension clamp area from the to-be-detected image.

[0120] An identification module 806, configured to identify the state of a target flat washer located in the suspension clamp area in the target image through a trained second detection model, and obtain the state of the target flat washer.

[0121] In one embodiment, as Figure 9 shown, the apparatus 800 includes an annotation module 808, a clustering module 810, a first pre-training module 812, a second pre-training module 814, and a superimposed training module 816, where:

[0122] The annotation module 808 is configured to acquire a plurality of sample images of a transmission line, and for each sample image, annotate the sample position of the sample suspension clamp and the sample state of the sample flat washer in the corresponding sample image, so as to obtain a first data set composed of a plurality of sample annotated images.

[0123] The clustering module 810 is configured to, based on the plurality of sample annotated images in the first data set, determine a plurality of sliding windows of the suspension clamp by clustering the sizes of the sample suspension clamps in the plurality of sample annotated images, and determine a plurality of sliding windows of the flat washer by clustering the sizes of the sample flat washers in the plurality of sample annotated images.

[0124] The first pre-training module 812 is configured to construct an initial first detection model based on the plurality of sliding windows of the suspension clamp, and pre-train the initial first detection model based on the first data set to obtain a pre-trained first detection model.

[0125] The second pre-training module 814 is configured to construct an initial second detection model based on the plurality of sliding windows of the flat washer, perform image cropping processing on the first data set to obtain a second data set, and train the initial second detection model based on the second data set to obtain a pre-trained second detection model;

[0126] The superimposed training module 816 is configured to obtain a second validation set from the second data set, and perform superimposed training on the pre-trained first detection model and the pre-trained second detection model based on the second validation set to obtain a trained first detection model and a trained second detection model.

[0127] In one embodiment, the first pre-training module 812 is configured to pre-train the initial first detection model based on the images labeled with multiple first training samples in the first training set to obtain the first detection model to be tested in the current cycle, and test the first detection model to be tested based on the first test set to obtain the theoretical test positions of multiple test sample suspension clamps. Based on the multiple first test sample labeled images in the first test set, determine the actual test positions of the multiple test sample suspension clamps, and based on the actual test positions of the multiple test sample suspension clamps and the theoretical test positions of the multiple test sample suspension clamps, obtain multiple first test evaluation index results, where the first test evaluation index results include the first test recall rate, the first test accuracy, and the first test mean average precision. Based on the multiple first test evaluation index results, adjust the parameters of the first detection model to be tested, enter the loop iteration of the next cycle, and return to the step of testing the first detection model to be tested based on the first test set to continue execution until the multiple first test evaluation index results all meet the index conditions, then determine the first detection model to be tested corresponding to the current cycle as the first intermediate detection model, and use the multiple first test evaluation index results in the current cycle as the first target evaluation thresholds of the corresponding evaluation indexes. Process the first validation set through the first intermediate detection model to obtain multiple first validation evaluation index results. If each first validation evaluation index result matches the first target evaluation threshold of the corresponding evaluation index, determine that the first intermediate detection model is the pre-trained first detection model.

[0128] In one embodiment, the second pre-training module 814 is configured to pre-train the initial second detection model based on the multiple second training sample labeled images in the second training set to obtain the second detection model to be tested in the current cycle, and test the second detection model to be tested based on the second test set to obtain the theoretical test states of multiple test sample flat washers. Based on the multiple second test sample labeled images in the second test set, determine the actual test states of the multiple test sample flat washers, and based on the actual test states of the multiple test sample flat washers and the theoretical test states of the multiple test sample flat washers, obtain multiple second test evaluation index results, where the second test evaluation index results include the second test recall rate, the second test accuracy, and the second test mean average precision. Based on the multiple second test evaluation index results, adjust the parameters of the second detection model to be tested, enter the loop iteration of the next cycle, and return to the step of testing the second detection model to be tested based on the second test set to continue execution. Until multiple second test evaluation index results all meet the index conditions, determine the second detection model to be tested corresponding to the current cycle as the second intermediate detection model, and use the multiple second test evaluation index results in the current cycle as the second target evaluation thresholds of the corresponding evaluation indexes. Process the second validation set through the second intermediate detection model to obtain multiple second validation evaluation index results. If each second validation evaluation index result matches the second target evaluation threshold of the corresponding evaluation index, determine the second intermediate detection model as the pre-trained second detection model.

[0129] In one embodiment, the second pre-training module 814 is configured to intercept the images containing suspension clamps from the multiple first training sample labeled images in the first training set to obtain the second training set. Intercept the images containing suspension clamps from the multiple first test sample labeled images in the first test set to obtain the second test set. Intercept the images containing suspension clamps from the multiple first validation sample labeled images in the first validation set to obtain the second validation set. The second training set, the second test set, and the second validation set form the second data set.

[0130] In one embodiment, the superimposed training module 816 is configured to process multiple second verification sample annotation images in the second verification set through a pre-trained first detection model to obtain multiple first sample target images carrying the position information of the suspension clamp. Detect the suspension clamp for multiple first sample target images through a pre-trained second detection model to obtain the sample flat washer states corresponding to each first sample target image. Determine the actual verification states of multiple verification sample flat washers based on the multiple second verification sample annotation images in the second verification set. Determine multiple third verification evaluation index results based on the actual verification states of the multiple verification sample flat washers and the multiple sample flat washer states. If each third verification evaluation index result meets the threshold evaluation conditions of the corresponding evaluation index, determine that the pre-trained first detection model is the trained first detection model and the pre-trained second detection model is the trained second detection model.

[0131] Each module in the above detection device for transmission lines can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0132] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 10 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store detection data of transmission lines. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a detection method for transmission lines.

[0133] Those skilled in the art can understand that Figure 10 the structure shown in

[0134] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0135] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0136] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0138] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0139] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0140] The above-described embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A detection method for a transmission line, characterized in that, The method includes: Obtain a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes suspension clamps and flat washers; Use a trained first detection model to detect the suspension clamps in the to-be-detected image, obtain the suspension clamp position information, and based on the suspension clamp position information, divide a target image including the suspension clamp area from the to-be-detected image; Use a trained second detection model to identify the state of the target flat washers located in the suspension clamp area in the target image, and obtain the state of the target flat washers; wherein, the training steps of the first detection model and the second detection model include: obtain a second validation set from a second dataset, and use a pre-trained first detection model to process multiple second validation sample annotation images in the second validation set to obtain multiple first sample target images carrying suspension clamp position information; use a pre-trained second detection model to detect the suspension clamps in the multiple first sample target images to obtain the sample flat washer states corresponding to each first sample target image; based on the multiple second validation sample annotation images in the second validation set, determine the actual validation states of multiple validation sample flat washers; based on the actual validation states of the multiple validation sample flat washers and the multiple sample flat washer states, determine the results of multiple third validation evaluation indicators. If each result of the third validation evaluation indicator meets the threshold evaluation conditions of the corresponding evaluation indicator, then determine that the pre-trained first detection model is the trained first detection model and the pre-trained second detection model is the trained second detection model.

2. The method according to claim 1, characterized in that, The method includes: Obtain multiple sample images of a transmission line. For each sample image, label the sample position of the sample suspension clamp and the sample state of the sample flat washer in the corresponding sample image to obtain a first dataset composed of multiple sample annotation images; Based on the multiple sample annotation images in the first dataset, determine multiple sliding windows of the suspension clamps by clustering the sizes of the sample suspension clamps in the multiple sample annotation images, and determine multiple sliding windows of the flat washers by clustering the sizes of the sample flat washers in the multiple sample annotation images; Based on the multiple sliding windows of the suspension clamps, construct an initial first detection model, and pre-train the initial first detection model based on the first dataset to obtain a pre-trained first detection model; Based on the multiple sliding windows of the flat washers, construct an initial second detection model, perform image cropping processing on the first dataset to obtain a second dataset, and based on the second dataset, train the initial second detection model to obtain a pre-trained second detection model.

3. The method according to claim 2, wherein The first dataset is composed of a first training set, a first test set, and a first validation set. The pre-training of the initial first detection model based on the first dataset to obtain a pre-trained first detection model includes: Pre-train the initial first detection model based on multiple first training sample annotation images in the first training set to obtain the first detection model to be tested in the current cycle, and test the first detection model to be tested based on the first test set to obtain the theoretical test positions of multiple test sample suspension clamps; Determine the actual test positions of multiple test sample suspension clamps based on multiple first test sample annotation images in the first test set, and obtain multiple first test evaluation index results based on the actual test positions of the multiple test sample suspension clamps and the theoretical test positions of the multiple test sample suspension clamps. The first test evaluation index results include the first test recall rate, the first test accuracy, and the first test mean average precision; Adjust the parameters of the first detection model to be tested based on the multiple first test evaluation index results, enter the loop iteration of the next cycle, and return to the step of testing the first detection model to be tested based on the first test set to continue execution. Until multiple first test evaluation index results all meet the index conditions, determine the first detection model to be tested corresponding to the current cycle as the first intermediate detection model, and use the multiple first test evaluation index results in the current cycle as the first target evaluation thresholds of the corresponding evaluation indexes; Process the first validation set through the first intermediate detection model to obtain multiple first validation evaluation index results. If each first validation evaluation index result matches the first target evaluation threshold of the corresponding evaluation index, determine the first intermediate detection model as the pre-trained first detection model.

4. The method according to claim 2, characterized in that, The second data set includes a second training set, a second test set, and a second validation set. Training the initial second detection model based on the second data set to obtain a pre-trained second detection model includes: Pre-train the initial second detection model based on multiple second training sample annotation images in the second training set to obtain the second detection model to be tested in the current cycle, and test the second detection model to be tested based on the second test set to obtain the theoretical test states of multiple test sample flat washers; Determine the actual test states of multiple test sample flat washers based on multiple second test sample annotation images in the second test set, and obtain multiple second test evaluation index results based on the actual test states of the multiple test sample flat washers and the theoretical test states of the multiple test sample flat washers. The second test evaluation index results include the second test recall rate, the second test accuracy, and the second test mean average precision; Based on the results of the multiple second test evaluation metrics, adjust the parameters of the second detection model to be tested, enter the next cycle of iterative loop, and return to the step of testing the second detection model to be tested based on the second test set and continue to execute until the results of the multiple second test evaluation metrics all meet the metric conditions. Then, determine the second detection model to be tested corresponding to the current cycle as the second intermediate detection model, and use the results of the multiple second test evaluation metrics in the current cycle as the second target evaluation thresholds for the corresponding evaluation metrics. Process the second validation set through the second intermediate detection model to obtain multiple second validation evaluation metric results. If each of the second validation evaluation metric results matches the second target evaluation threshold of the corresponding evaluation metric, determine the second intermediate detection model as the pre-trained second detection model.

5. The method according to claim 2, characterized in that, The first data set consists of a first training set, a first test set, and a first validation set. The image cropping process on the first data set to obtain a second data set includes: Intercept the images containing suspension clamps in the multiple first training sample labeled images in the first training set to obtain a second training set; Intercept the images containing suspension clamps in the multiple first test sample labeled images in the first test set to obtain a second test set; Intercept the images containing suspension clamps in the multiple first validation sample labeled images in the first validation set to obtain a second validation set; Construct a second data set from the second training set, the second test set, and the second validation set.

6. A detection device for a transmission line, characterized in that, The device includes: An acquisition module, configured to acquire a to-be-detected image obtained by photographing a target transmission line, where the target transmission line includes suspension clamps and flat washers; A detection module, configured to detect suspension clamps in the to-be-detected image through a trained first detection model to obtain suspension clamp position information, and based on the suspension clamp position information, divide a target image including a suspension clamp area from the to-be-detected image. An identification module, configured to identify the state of the target flat washer located in the suspension clamp area in the target image through the second detection model that has completed training, so as to obtain the state of the target flat washer; wherein, the device further includes a superimposed training module, configured to obtain a second validation set from the second data set, and process multiple second validation sample labeled images in the second validation set through the pre-trained first detection model to obtain multiple first sample target images carrying suspension clamp position information; detect the suspension clamps for the multiple first sample target images through the pre-trained second detection model to obtain the sample flat washer states corresponding to the respective first sample target images; determine the actual validation states of multiple validation sample flat washers based on the multiple second validation sample labeled images in the second validation set; determine multiple third validation evaluation index results based on the actual validation states of the multiple validation sample flat washers and the multiple sample flat washer states. If each third validation evaluation index result meets the threshold evaluation conditions of the corresponding evaluation index, determine that the pre-trained first detection model is the trained first detection model and the pre-trained second detection model is the trained second detection model.

7. The device according to claim 6, characterized in that, The device further includes a labeling module, a clustering module, a first pre-training module, and a second pre-training module. The labeling module is configured to obtain multiple sample images of the transmission line, and for each sample image, label the sample position of the sample suspension clamp and the sample state of the sample flat washer in the corresponding sample image to obtain a first data set composed of multiple sample labeled images; the clustering module is configured to, based on the multiple sample labeled images in the first data set, determine multiple sliding windows of the suspension clamp by clustering the sizes of the sample suspension clamps in the multiple sample labeled images, and determine multiple sliding windows of the flat washer by clustering the sizes of the sample flat washers in the multiple sample labeled images. The first pre-training module is configured to construct an initial first detection model based on the multiple sliding windows of the suspension clamp, and pre-train the initial first detection model based on the first data set to obtain a pre-trained first detection model. The second pre-training module is configured to construct an initial second detection model based on the multiple sliding windows of the flat washer, perform image cropping processing on the first data set to obtain a second data set, and train the initial second detection model based on the second data set to obtain a pre-trained second detection model.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that, When this computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 5.

Images