Power transmission line inspection method and device, computer program product and inspection system

Abstract

The invention provides a power transmission line inspection method and device, a computer program product and an inspection system. The method comprises the following steps: acquiring image data of a power transmission line, the image data being acquired by an unmanned aerial vehicle; an image analysis model is constructed, the image analysis model is obtained by training through a YOLOv8 algorithm by using multiple groups of training data, and each group of training data in the multiple groups of training data comprises historical image data acquired in a historical time period and historical inspection results corresponding to the historical image data; the historical inspection result is used for representing whether an obstacle exists in the historical image data; and inputting the image data to an image analysis model to obtain an inspection result corresponding to the image data, the inspection result being used for representing whether there is an obstacle in the image data. According to the scheme, the problem of low inspection efficiency of the power transmission line in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a transmission line inspection method, device, computer program product and inspection system. Background Art

[0002] In modern power transmission networks, obstacle detection around tower bases is an important part of ensuring the safe operation of transmission lines. Obstacles around tower bases, such as trees and buildings, may affect the safe operation of transmission lines, increase the risk of electric shock and fire, and may collapse and damage power poles in stormy weather. Traditional detection methods mainly rely on manual inspections, which are not only inefficient, but also easily affected by environmental factors, resulting in missed detections and false detections. Summary of the invention

[0003] The main purpose of the present application is to provide a transmission line inspection method, device, computer program product and inspection system to at least solve the problem of low inspection efficiency of transmission lines in the prior art.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a transmission line inspection method is provided, comprising: obtaining image data of the transmission line, wherein the image data is collected by a drone; constructing an image analysis model, wherein the image analysis model is trained by a YOLOv8 algorithm using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical image data obtained within a historical time period and historical inspection results corresponding to the historical image data, and the historical inspection results are used to characterize whether there are obstacles in the historical image data; inputting the image data into the image analysis model to obtain the inspection results corresponding to the image data, wherein the inspection results are used to characterize whether there are obstacles in the image data.

[0005] Optionally, acquiring image data of a transmission line includes: controlling the UAV to inspect and collect images along the transmission line to obtain initial image data; preprocessing the initial image data to obtain the image data, wherein the preprocessing method includes one or more of data enhancement, sample delineation and data cleaning.

[0006] Optionally, after constructing the image analysis model, the method further includes: optimizing the image analysis model to obtain an optimized image analysis model, wherein the optimization method includes one or more of pruning, distillation and model quantization; using the optimized image analysis model to analyze the image data to obtain a target inspection result corresponding to the image data.

[0007] Optionally, when the optimization method includes pruning, the image analysis model is optimized to obtain an optimized image analysis model, including: obtaining the L1 norms of all filter layers in the image analysis model; and pruning all the filter layers in turn according to the size of the L1 norm to obtain the optimized image analysis model.

[0008] Optionally, when the optimization method includes distillation, the image analysis model is optimized to obtain an optimized image analysis model, including: performing feature extraction on the image analysis model obtained by the Nth training, and calculating a first probability distribution for the extracted features, where N≥1; performing feature extraction on the image analysis model obtained by the N+1th training, and calculating a second probability distribution for the extracted features; calculating the KL divergence according to the first probability distribution and the second probability distribution to obtain a distillation loss value; performing weighted averaging on the distillation loss value, the classification loss value, and the target box loss value to obtain a total loss value; and using the total loss value for back propagation to update the weights of the image analysis model during training to obtain the optimized image analysis model.

[0009] Optionally, when the optimization method includes pruning and model quantization, the image analysis model is optimized to obtain an optimized image analysis model, including: performing unstructured pruning on the image analysis model to obtain the pruned image analysis model; converting the first type of data in the pruned image analysis model into the second type of data to obtain the optimized image analysis model, wherein the space occupied by the first type of data is larger than the space occupied by the second type of data.

[0010] Optionally, after inputting the image data into the image analysis model to obtain the inspection result corresponding to the image data, the method further includes: generating an alarm message when the inspection result indicates that the image data includes an obstacle; and sending the image data including the obstacle and the alarm message to the target terminal.

[0011] According to another aspect of the present application, there is provided a transmission line inspection device, comprising: an acquisition unit, for acquiring image data of the transmission line, wherein the image data is collected by a drone; a construction unit, for constructing an image analysis model, wherein the image analysis model is trained by a YOLOv8 algorithm using multiple sets of training data, each set of the multiple sets of training data includes historical image data acquired within a historical time period and historical inspection results corresponding to the historical image data, wherein the historical inspection results are used to characterize whether there are obstacles in the historical image data; a processing unit, for inputting the image data into the image analysis model to obtain the inspection results corresponding to the image data, wherein the inspection results are used to characterize whether there are obstacles in the image data.

[0012] According to another aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the transmission line inspection methods.

[0013] According to another aspect of the present application, there is provided an inspection system, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the inspection methods of the transmission lines.

[0014] By applying the technical solution of the present application, inspections can be carried out by drones. Image data of transmission lines can be collected by drones, and a model can be built based on the YOLOv8 algorithm to automatically identify objects in the image. The YOLOv8 algorithm has a high accuracy rate for target detection, can adapt well to complex and changing environments, maintain high detection accuracy, and can greatly improve inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for performing a transmission line inspection method provided in an embodiment of the present application is shown;

[0017] Figure 2 A schematic diagram of a process flow of a transmission line inspection method provided according to an embodiment of the present application is shown;

[0018] Figure 3 A schematic diagram of the training process for lightweight processing of the model is shown;

[0019] Figure 4 A schematic diagram of a two-stage self-distillation is shown;

[0020] Figure 5 A schematic flow chart of another inspection method for a power transmission line is shown;

[0021] Figure 6 A structural block diagram of a transmission line inspection device provided according to an embodiment of the present application is shown.

[0022] The above drawings include the following reference numerals:

[0023] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0028] Deep learning technology enables computers to learn high-dimensional feature patterns of objects under massive data samples through complex multi-layer neural networks and accurately identify objects. Deep learning technology can process large data samples and complete tasks including image recognition, large language models and speech recognition. With the development of drone technology, real-time automated detection systems for drones based on image processing have gradually emerged, combining the flexibility and high maneuverability of drones with high-performance lightweight data processing equipment, allowing drones to quickly process data while acquiring data close to the data generation area.

[0029] Common deep learning target detection algorithms are now divided into one-stage and two-stage. One-stage algorithms are faster than two-stage algorithms, but have lower accuracy. After extracting target features, they are directly input into the detector to obtain the target location and category. YOLO is a classic one-stage algorithm. As an efficient target detection algorithm, the YOLO (You Only Look Once) series of algorithms are widely used in various target detection tasks due to their fast detection speed and high accuracy. As a more advanced version of the series, YOLOv8 further optimizes the network structure and detection performance. However, YOLOv8 still has problems with high model complexity and high computing resource consumption in practical applications, which limits its application in resource-constrained environments.

[0030] Knowledge distillation is a technique that transfers knowledge from a large and complex model (teacher model) to a smaller model (student model). Its purpose is to reduce the computational complexity and storage requirements of the model while maintaining model performance, making it more suitable for running on resource-constrained devices. If self-distillation is used for training, there is no need to train a large and complex teacher model. Instead, the student model is used to distill itself, further improving efficiency.

[0031] Edge computing is to provide edge intelligent services at the edge of the network close to the source of objects or data through a distributed open platform that integrates network, computing, storage, and application core capabilities. Simply put, edge computing is to analyze the data collected from the terminal directly in the local device or network close to the data generation, without transmitting the data to the cloud data processing center. Compared with cloud computing, edge computing has a faster processing speed, can analyze data in real time, and only transmits useful data to reduce redundancy. Edge computing technology is well suited to the needs of drone detection and provides reliable technical support for the detection of hidden dangers in the power transmission network.

[0032] As introduced in the background technology, the inspection efficiency of power transmission lines in the prior art is low. To solve the above problems, the embodiments of the present application provide a transmission line inspection method, device, computer program product and inspection system.

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a transmission line inspection method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] In this embodiment, a method for inspecting a power transmission line running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 2 FIG. 1 is a flow chart of a method for inspecting a power transmission line according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0038] Step S201, acquiring image data of the power transmission line, wherein the image data is acquired by using a drone;

[0039] Specifically, the drone flies along a predetermined path of the transmission line. During the flight, the camera on the drone continuously captures images of the transmission line and its surroundings. The image data collected by the drone includes, but is not limited to, high-definition images of the transmission line, images of the environment around the tower base, and images of possible obstacles (such as trees, buildings, etc.).

[0040] Step S202, constructing an image analysis model, wherein the image analysis model is trained by a YOLOv8 algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical image data acquired within a historical time period and historical inspection results corresponding to the historical image data, and the historical inspection results are used to characterize whether there are obstacles in the historical image data;

[0041] Specifically, the image analysis model is trained using the YOLOv8 algorithm. The YOLOv8 algorithm is an object detection algorithm that can simultaneously predict the location and category of an object in an image. During the training process, the model learns the features of obstacles in historical inspection results so that obstacles can be accurately detected on new image data. Historical inspection results are obtained after machine analysis of historical image data to determine whether there are obstacles in the image and their type and location. Historical inspection results can be detailed obstacle annotation information, including the bounding box and category label of the obstacle.

[0042] Step S203: input the image data into the image analysis model to obtain an inspection result corresponding to the image data, wherein the inspection result is used to characterize whether there is an obstacle in the image data.

[0043] Specifically, the model analyzes the input image data, identifies and locates obstacles, and the output is the detection result of obstacles in the image, including the type of obstacle, location (usually represented by a bounding box), and whether it exists.

[0044] Through this embodiment, inspections can be carried out by drones, and image data of transmission lines can be collected by drones. Then, a model is built based on the YOLOv8 algorithm to automatically identify objects in the image. The YOLOv8 algorithm has a high accuracy rate for target detection, can adapt well to complex and changing environments, maintain high detection accuracy, and can greatly improve inspection efficiency.

[0045] In the specific implementation process, the image data of the transmission line can be obtained through the following steps: controlling the above-mentioned UAV to inspect and collect images along the above-mentioned transmission line to obtain initial image data; preprocessing the above-mentioned initial image data to obtain the above-mentioned image data, wherein the preprocessing method includes one or more of data enhancement, sample delineation and data cleaning.

[0046] In this solution, the drone is controlled to fly along the path of the transmission line. At the same time, the camera it carries is used to photograph the line environment and collect a large amount of original initial image data. By preprocessing the initial image data, the obtained image data can be guaranteed to be more accurate.

[0047] Specifically, the solution of the present application can collect image data around the transmission tower base based on a multi-platform drone. The image is preprocessed, data enhanced, and sampled to produce a transmission network tower base obstacle drone image dataset.

[0048] The target image is acquired at different heights and flight attitudes by using drone sensors. In the present invention, the image of obstacles in the area near the tower base of the power transmission network is acquired by taking aerial photography along the power transmission network at a certain height by using drones. The acquired image is subjected to feature enhancement processing. For target detection tasks, data enhancement methods include rotation, cropping, random masking, and grid masking. Grid masking is to perform masking processing on the image by setting the grid size and grid spacing. Sample annotation is performed on the processed data. In this embodiment, the LabelImg plug-in is used to annotate the image and output it as a sample in YOLO format.

[0049] Data augmentation is a technique that adds diversity to a training dataset in order to improve the generalization and robustness of the model. In the field of image processing, data augmentation usually involves performing various transformations on the original image, such as rotation, translation, scaling, flipping, brightness and contrast adjustment, etc., to generate new training samples. Through data augmentation, different shooting conditions or perspectives can be simulated, allowing the model to learn more comprehensive features and perform better on new, unseen images.

[0050] Sample delineation refers to labeling the target objects in the dataset, which is a necessary step for training deep learning models, especially target detection models. In image datasets, bounding boxes are usually used to outline the location of the target object and mark the category of the object. For example, in tower base obstacle detection, it is necessary to draw the bounding box of the obstacle on each image manually or semi-automatically and mark the type of obstacle, such as trees, buildings, etc. These labeled data are the key to the model learning to identify and locate targets.

[0051] Data cleaning refers to preprocessing the data set before data analysis or model training to remove or correct incomplete, erroneous, irrelevant or inconsistently formatted data. In image data processing, data cleaning may include removing blurry, improperly exposed or images without target objects, correcting image metadata, unifying image size and format, etc. Data cleaning is an important step to ensure data quality and improve model training efficiency and accuracy.

[0052] Specifically, data augmentation refers to generating new images through methods such as rotation, scaling, flipping, and brightness adjustment to increase the diversity of training data and improve the generalization ability of the model.

[0053] Specifically, sample delineation refers to labeling objects in an image, such as drawing a bounding box on the image to mark the location of obstacles and labeling the categories of obstacles, such as trees, bird nests, buildings, etc.

[0054] Specifically, data cleaning refers to removing unclear, repeated, or task-irrelevant images to ensure that the model only learns meaningful data and improves training efficiency.

[0055] Specifically, the flight path of the drone can be set at the ground control station to ensure that the drone can fly along the transmission line and cover all important nodes. When the drone is flying, it can continuously shoot at a resolution of 3840×2160 to record images of the transmission line and its surrounding environment, including the tower base, conductors and the surrounding environment. Of course, the resolution can be resampled to 640×640 during post-processing.

[0056] Specifically, the collected raw image data is processed through data augmentation, including image rotation, random flipping, brightness and contrast adjustment, and partial occlusion of the image using a grid mask to simulate different lighting conditions and perspective changes, thus increasing the robustness of the model.

[0057] Specifically, use tools such as LabelImg to annotate the obstacles in the enhanced image. Specifically, draw a bounding box for each obstacle and annotate its category (for example, a tree is marked as 1 and a building is marked as 2). These labeled data will be used as input for training the YOLOv8 model.

[0058] Specifically, the dataset is cleaned to remove images that are too blurry, completely occluded, or do not contain any target objects, in order to reduce the interference of useless data and ensure that the model learns high-quality samples.

[0059] During the specific implementation process, after constructing the image analysis model, the method further includes the following steps: optimizing the image analysis model to obtain an optimized image analysis model, wherein the optimization method includes one or more of pruning, distillation and model quantization; using the optimized image analysis model to analyze the image data to obtain the target inspection results corresponding to the image data.

[0060] In this solution, by optimizing the image analysis model, the complexity and computing requirements of the model can be reduced, while maintaining or improving its accuracy in detecting obstacles as much as possible. The optimization methods can include the following: Pruning: By removing unimportant neurons or connections in the model, the number of model parameters is reduced, thereby reducing computing costs and increasing running speed. Distillation: Using knowledge distillation technology, a smaller model (student model) learns the features and decisions of a larger model (teacher model), so that the student model can have a faster running speed and smaller size while maintaining high accuracy. Model quantization: Convert the model's weights and calculation processes from floating point numbers to fixed point numbers to reduce memory consumption and speed up the model's running speed. The preprocessed image data is input into the optimized image analysis model, and the model will perform target detection on the image, identify obstacles in the image, and provide detailed information such as their type and location, so as to obtain more accurate target inspection results.

[0061] Specifically, an optimized image analysis model can be constructed, wherein the optimized image analysis model is obtained by training using multiple sets of training data, each of which includes historical image data acquired within a historical time period and historical inspection results corresponding to the historical image data, and the historical inspection results are used to characterize whether there are obstacles in the historical image data. The image data is input into the optimized image analysis model to obtain the inspection results corresponding to the image data, wherein the inspection results are used to characterize whether there are obstacles in the image data.

[0062] Specifically, the YOLOv8 model is pruned structurally, using the L1 norm to measure the importance of filters in each layer and removing the least important filters to reduce the number of model parameters and computational burden and improve the model's operating efficiency. The pruning ratio is set to 30% to ensure a balance between model size and performance.

[0063] Specifically, the lightweight YOLOv8 model is further optimized using a two-stage self-distillation method. After training the lightweight model in the first stage, the model is used as the teacher model to perform distillation training on other models (or the next iteration of the model). By calculating the KL divergence of channel-scale distillation (CWD), the student model learns the internal knowledge of the teacher model and improves the detection accuracy of the model.

[0064] Specifically, after distillation and pruning optimization, the model is subjected to unstructured pruning and model quantization. Unstructured pruning further removes connections with smaller weights in the model, while model quantization converts weights from 32-bit floating point numbers to 8-bit fixed point numbers, reducing storage requirements and accelerating model processing.

[0065] Specifically, the optimized image analysis model is deployed in an edge computing box and connected to the real-time data stream from the drone. The edge computing box has computing, storage and network functions, which can quickly process the image data transmitted by the drone near the location where the data is generated, reducing the delay and bandwidth requirements of data transmission.

[0066] Specifically, the image data collected by the drone in real time during the inspection process is directly transmitted to the optimized image analysis model in the edge computing box. The model quickly analyzes the image and outputs the target inspection results, that is, the type, location and size of obstacles in the image. These results are fed back to the operator and the monitoring center in real time via the wireless network, facilitating rapid response and handling of potential line safety hazards.

[0067] Specifically, based on the real-time feedback of the targeted inspection results, operators can immediately mark and deal with obstacles on the transmission lines, such as adjusting the flight path of the drone to avoid obstacles, or notifying the ground team to manually inspect and remove specific obstacles. In addition, the inspection results can also be used to evaluate the operating status of the transmission lines and predict potential maintenance needs.

[0068] In the specific implementation process, when the optimization method includes pruning, the above-mentioned image analysis model is optimized to obtain an optimized image analysis model, which can be achieved through the following steps: obtaining the L1 norm of all filter layers in the above-mentioned image analysis model; according to the size of the above-mentioned L1 norm, all the above-mentioned filter layers are pruned in turn to obtain the above-mentioned optimized image analysis model.

[0069] In this scheme, the L1 norm is a norm that measures the sum of the absolute values ​​of all elements in a vector or matrix, and is used to evaluate the importance of each filter in the filter layer. Generally, filters with smaller L1 norms contribute less to the output of the model and can be regarded as "unimportant" filters. Based on the calculated L1 norm, the filters are sorted by the size of the norm. Starting with the filter with the smallest L1 norm, they are pruned and removed in sequence until a predetermined pruning ratio is reached, such as 30%. The number of model parameters is reduced after pruning, but the model runs faster.

[0070] Specifically, based on the YOLOv8 detection algorithm with efficient target detection capability and combined with lightweight module design, the YOLOv8 original network (i.e., image analysis model) is structured pruned to obtain a lightweight target detection model (i.e., optimized image analysis model).

[0071] Specifically, Figure 3As shown in the figure, the YOLOv8 initial model is structured pruned to reduce the model complexity and obtain a lightweight model. The core idea of ​​YOLO is to transform target detection into a regression problem, using the entire image as the input of the network, and only passing through a neural network to obtain the position of the bounding box and its category.

[0072] Specifically, in the solution of the present application, a YOLOv8s model with both detection speed and accuracy is used for pruning. The input image size of YOLOv8s is 640×640, and it will be resampled before input. The backbone part of the network extracts features through multiple convolutional layers, uses the C2F layer to branch the extracted features, divides the extracted feature channels into two, and directly splices half of the features, and changes the features of the other half through multiple convolutional layers before splicing. Finally, the extracted features are spliced ​​with multi-scale features through the SPPF layer. The head part will detect the features extracted from the 4th, 6th, and 9th layers of the network as three feature scales, upsample the small-scale features and splice them with large-scale features to enrich the feature information, and convert the features to three scales of 80×80, 40×40, and 20×20. The detector part mainly calculates the loss, firstly distributes positive and negative samples, and selects positive samples in YOLOv8 according to the weighted scores of classification and regression. The loss is divided into two parts, one is the classification loss, and the other is the bounding box loss. The bounding box loss includes DFL (distribution focal loss) and CIoU (Complete Intersection over Union). To calculate the classification loss, the number of feature channels is first converted to the same number of category channels, and then the features of the three scales are converted into one-dimensional vectors for splicing to obtain a matrix of 8400× the number of categories, which represents the category score of a bounding box predicted by each pixel in the feature map, and finally the cross entropy is calculated with the true category. The bounding box loss converts the features into 4 one-dimensional vectors to represent the distance from the 4 edges of the bounding box to the center of the grid, and then calculates the intersection and union ratio and DFL. YOLOv8 controls the size of the network through the depth and width parameters, which mainly affect the depth of the C2F layer. The depth and width of the YOLOv8s model are 0.33 and 0.5, respectively, with a total of 225 layers.

[0073] Specifically, the solution of this application mainly uses the Torch-pruning pruning library to optimize the model at the filter layer. First, build a network dependency graph to determine the relationship between the layers, then select the layer to be pruned, and use the L1 norm to measure the importance of the filter in the layer. The L1 norm is the sum of the absolute values ​​of each element in the vector, and the calculation formula is as follows:

[0074]

[0075] x represents the L1 norm, n represents the number of elements, x i Represents an element.

[0076] Specifically, the pruning ratio can be set, the filters of the layers to be pruned are arranged according to the norm size, and the weights and mask values ​​of the filters to be pruned are set to zero. Then the filter is removed. Finally, the layers associated with the removed filters are pruned according to the determined dependency groups. After pruning, fine-tuning is required to restore the performance of the model, and the lightweight model is trained using a specific data set until the model accuracy is close to the original model accuracy.

[0077] Specifically, this solution is to train the pruned lightweight network. The present invention produces a UAV transmission network tower base obstacle dataset. The obstacle images around the transmission tower base are obtained by using UAVs and processed into a sample dataset through various processes.

[0078] Specifically, the YOLOv8 initial model is structured pruned to reduce the model complexity and obtain a lightweight model. In 2016, a one-stage object detection network YOLO was proposed. Its detection speed is very fast, it can process 45 frames per second, and can easily run in real time. The core idea of ​​YOLO is to transform object detection into a regression problem, using the entire image as the input of the network, and only through a neural network to obtain the position of the bounding box and its category.

[0079] In some embodiments, when the optimization method includes distillation, the image analysis model is optimized to obtain an optimized image analysis model, which can be specifically achieved through the following steps: extracting features from the image analysis model obtained by the Nth training, and calculating a first probability distribution for the extracted features, where N≥1; extracting features from the image analysis model obtained by the N+1th training, and calculating a second probability distribution for the extracted features; calculating the KL divergence based on the first probability distribution and the second probability distribution to obtain a distillation loss value; taking the weighted average of the distillation loss value, the classification loss value and the target box loss value to obtain a total loss value; and using the total loss value to perform back propagation to update the weights of the image analysis model during training to obtain the optimized image analysis model.

[0080] In this scheme, a two-stage self-distillation method is used, KL divergence is used to calculate the distillation loss, and back propagation is used to update the weights to achieve the purpose of model optimization. In this way, the model can learn the knowledge obtained from its previous training in each round of iteration, thereby improving the detection accuracy and efficiency, and further improving the real-time and reliability of transmission line obstacle detection.

[0081] Specifically, the above scheme in this scheme is a two-stage self-distillation method, which is as follows: after one iterative training using the pruned model, the model is used as the teacher model input feature extraction network to extract features as soft targets, and the loss is calculated with the features of the next trained model, so that the student model learns the knowledge of the teacher model.

[0082] Specifically, the lightweight model is trained using a two-stage distillation method and the training loss is calculated using CWD heterogeneity. Figure 3 As shown in Figure 2, the lightweight model is trained by self-distillation in two stages. Figure 4 As shown. The biggest feature of this method is to use the model obtained from the last training to distill the model currently being trained, that is, to let the model learn the knowledge obtained from its last training. First, use the model to extract features, and use the softmax function to obtain the category probability distribution for each channel of the extracted feature map, which is used as a soft target for distillation during the next training. The KL divergence (Kullback-Leibler divergence) is calculated using the probability distribution of the feature map extracted in this training and the probability distribution of the feature map in the last training as the distillation loss for this training. The calculation formula of KL divergence is as follows:

[0083]

[0084] p(x) and q(x) represent the probability distribution of the teacher model and the student model, namely the first probability distribution and the second probability distribution, respectively. This method of calculating the distillation loss is called channel-scale distillation (CWD).

[0085] Then use the current model to perform detection and calculate the category loss, detection box intersection loss and DFL loss with the real label, namely Distribution Focal Loss, which is a loss function used in target detection tasks and is mainly used in the bounding box regression process. Its main function is to correct the error of the model when predicting the object bounding box. The calculation formula is as follows:

[0086] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(y i -y)log(S i+1 )),

[0087] S i and S i+1Respectively represent the probability of the bounding box being i and i+1, i and i+1 represent the range of the bounding box, y∈{0,1}, y i and i+1 is the adjustment factor, y i ≤y≤y i+1 For details, please refer to "Target Detection Paper Reading: GFL Algorithm Notes".

[0088] All losses are weighted summed up, and the weights are updated by backpropagation. As the number of iterations increases, the weight of the distillation loss will gradually increase.

[0089] Specifically, in order to improve training efficiency and model accuracy, this solution uses a two-stage distillation method to train a lightweight YOLOv8 model. The model obtained from the previous training is used as a soft target and the KL divergence is calculated with the model obtained from the current training as the distillation loss. This method does not require the additional training of a large teacher model to distill the original model, and is also applicable to shallow networks, making full use of the model trained each time. In order to enable the student model to fully learn the knowledge in the teacher model, the CWD (channel-scale distillation) method is used to calculate the loss. The training samples are input into the teacher model and the student model, and the feature maps of the training samples are extracted respectively. The feature maps are input into the softmax function to obtain the feature probability distribution of the teacher model and the student model, and the KL divergence of the probability distribution of the teacher model and the student model is calculated as the distillation loss. This method of calculating the loss pays more attention to the significant areas of each channel and makes full use of the information of each channel. The student model is guided to learn areas with significant activation values.

[0090] Specifically, compared with the prior art, this solution uses a two-stage self-distillation method to train the model, maintains the layer structure of the original network, maintains the stability of the entire model, and avoids the increase of weight parameters in the model affecting the model's operating speed. This method is simple and easy to operate. Using the model obtained from the previous training for distillation, compared with other distillation methods, there is no need to additionally train the teacher model, which speeds up training efficiency. During the training process, the model not only learns the knowledge of the real label, but also learns the knowledge in the teacher model, effectively improving the detection accuracy. The CWD structure is used to calculate the distillation loss. This method makes full use of the information of each channel, pays more attention to the salient areas of each channel, and guides the student model to imitate these salient features.

[0091] Specifically, in the Nth iteration of model training, this version of the model is used to extract features from the input image. The extracted features are converted into probability distributions on each channel through the softmax function. This distribution can be understood as the model's confidence level for each object category in the features, i.e., the first probability distribution. In the N+1th iteration of the model, feature extraction is also performed to obtain new features and calculate the second probability distribution. This is another version of the model during the learning process, which is used to compare with the model of the Nth iteration.

[0092] Specifically, KL divergence (Kullback-Leibler divergence) is a statistic that measures the difference between two probability distributions and is used to evaluate the gap between the knowledge of the Nth iteration model and the N+1th iteration model. By calculating the KL divergence, the distillation loss value can be obtained, which is the key loss function in the self-distillation process and is used to guide the student model (N+1th training model) to learn from the teacher model (Nth training model).

[0093] Specifically, different types of loss function values ​​are weighted averaged to form a comprehensive loss value, which will guide the weight update of the model to optimize the performance of the model.

[0094] Distillation loss values ​​usually appear in the training scenario of knowledge distillation. In the context of the present invention, the distillation loss value comes from the self-distillation training process, that is, the difference between the feature map or output probability distribution between the model obtained by the Nth training (which can be regarded as the teacher model) and the model obtained by the N+1th training (student model). This loss function encourages the student model to learn and imitate the output of the teacher model, even if the teacher model and the student model are versions of the same model at different training stages. The distillation loss value can be calculated using methods such as KL divergence (Kullback-Leibler divergence). Distillation loss values ​​usually appear in the training scenario of knowledge distillation. Knowledge distillation is a model compression technique that reduces the complexity and computing resource requirements of the small model while maintaining or approaching the performance of the large model by allowing a smaller model (student model) to learn the output of a larger model (teacher model). The distillation loss value measures the difference between the output of the student model and the output of the teacher model, and is usually calculated using KL divergence (Kullback-Leibler divergence). KL divergence is a statistical distance metric that calculates the difference between two probability distributions. In knowledge distillation, the output of the teacher model is converted into a probability distribution through the softmax function, and then the KL divergence is calculated with the output probability distribution of the student model to obtain the distillation loss value.

[0095] The classification loss value measures the performance of the model on the classification task. In the object detection task, the model not only needs to locate the object, but also needs to identify which category the object belongs to. The classification loss is usually calculated using the cross-entropy loss, which measures the difference between the probability distribution predicted by the model and the true category. The classification loss value is a loss function that measures the prediction accuracy of the model on the classification task. In the classification problem, the goal of the model is to predict the category of the input data, and the classification loss value directly reflects the difference between the model's prediction result and the true label. The most commonly used classification loss function is the cross-entropy loss. The cross-entropy loss function measures the distance between the probability distribution predicted by the model and the probability distribution of the true category. It encourages the probability distribution predicted by the model to be as consistent as possible with the probability distribution of the true category to improve classification accuracy.

[0096] The target box loss value is used to evaluate the accuracy of the model in target positioning. In target detection, the bounding box predicted by the model needs to be close to the bounding box of the real target. The target box loss can be decomposed into CIoU loss (Complete Intersection over Union Loss), etc., which is used to evaluate the position and size differences between the predicted box and the real box. The target box loss value is particularly important in the target detection task. It is used to evaluate the position and size errors between the target box predicted by the model (i.e., the bounding box, which represents the position of the object in the image) and the real target box. The target box loss usually consists of position loss and size loss. The position loss is used to measure the distance between the center of the predicted box and the center of the real box, while the size loss measures the difference between the width and height of the predicted box and the width and height of the real box. Common target box loss functions include IoU loss (Intersection over Union Loss), CIoU loss (Complete Intersection over Union Loss), etc. These loss functions encourage the target box predicted by the model to overlap with the actual target box as much as possible, thereby improving the accuracy of target positioning.

[0097] These loss values ​​are weighted and averaged to obtain a comprehensive indicator - the total loss value. Weighting can be based on the number of model iterations, confidence or other important factors to reflect the loss contribution of model training at different stages. Finally, the total loss value is used as the gradient to update the model weights through the back-propagation algorithm. This process continues until the model's performance reaches the expected level or the number of iterations reaches the set upper limit. Through self-distillation training, the model can continuously optimize its weights and structure, ultimately obtaining a more efficient and accurate optimized image analysis model.

[0098] Specifically, the weights can be dynamically adjusted based on the increase in the number of model iterations, such as using the weighted averaged sum of the losses to perform back propagation and update the model weights to guide the model to learn deeper features and decisions.

[0099] During the self-distillation training process, the model will undergo multiple iterations and weight updates. After completing the self-distillation training, the model is fine-tuned and trained using standard detection losses (such as classification loss and bounding box loss) to ensure the detection performance of the model. Finally, the model is further optimized through unstructured pruning and quantization to reduce the demand for computing resources and improve operating efficiency.

[0100] In addition, regarding the self-distillation training of the lightweight model, the present invention uses a two-stage distillation method to train the model, that is, the model obtained from the previous training is used to distill the currently trained model, and the CWD method is used to calculate the distillation loss. In this method, in addition to using the two-stage distillation method to implement model self-distillation training, it can also be trained by the synchronous distillation method. Synchronous distillation uses two identical models with independent weights as pseudo-twin networks, and uses a mutual learning strategy in the two models to allow the two models to learn from each other, serving as teachers and students to each other, so that the similarity of the two models after enhancement of the same samples ensures their consistency.

[0101] Specifically, after an iterative training using the pruned model, the model is used as the teacher model to distill the model for the next training, so that the student model can learn the knowledge of the teacher model. The loss of the entire model training consists of three parts: the category cross entropy loss between the model prediction category and the true category, the intersection loss and DFL loss between the prediction box coordinates and the actual target position, and the KL divergence of the student model prediction results and the teacher model prediction results calculated by the CWD method, weighted by a certain ratio. The weight of the loss between the teacher model and the teacher model will gradually increase with the increase of epochs.

[0102] Specifically, the KL divergence between the prediction results of the student model and the teacher model calculated by the CWD method is used to obtain the probability distribution of the teacher model and the student model through the softmax function, and then the KL divergence between the prediction probabilities of the teacher model and the student model is calculated as the loss, so that the student network imitates the teacher network in the foreground salient area. KL divergence is used to measure the similarity between two probability distributions. When the KL divergence is larger, the difference between the two is greater; when the KL divergence is smaller, the difference between the two is smaller. If the two are the same, the KL divergence should be 0.

[0103] In some embodiments, when the optimization method includes pruning and model quantization, the image analysis model is optimized to obtain an optimized image analysis model, which can be specifically achieved through the following steps: performing unstructured pruning on the image analysis model to obtain the pruned image analysis model; converting the first type of data in the pruned image analysis model into the second type of data to obtain the optimized image analysis model, wherein the space occupied by the first type of data is larger than the space occupied by the second type of data.

[0104] In this solution, unstructured pruning and model quantization can be used to optimize the image analysis model, which can not only reduce the computational complexity and resource requirements of the model, but also enable the model to maintain a high detection accuracy.

[0105] Specifically, unstructured pruning and model quantization are performed on the distilled model to further improve the model detection efficiency. Figure 3 As shown, in order to make the model more streamlined, unstructured pruning and model quantization are continued to be used to optimize the model. Unstructured pruning is different from structured pruning, and it mainly prunes the model at the connection layer. However, similar to structured pruning, the importance of weights is evaluated by a certain indicator, and the weight connections with less influence are removed to reduce the complexity of the model. In this embodiment, the pruning ratio is set to 30%, the absolute value of the weight is used as the evaluation criterion of importance, the weights are sorted by absolute value, and the 30% of the weights with the lowest importance are set to 0. The sparse matrix of weights obtained by unstructured pruning does not have an obvious effect on speeding up the operation of ordinary algorithms, and needs to be coordinated with a sparse matrix calculation module.

[0106] Model quantization is to replace data types with low-space data types to speed up the operation. These two methods have little impact on the accuracy of the model. They can further speed up the model operation after pruning and distillation.

[0107] Specifically, the trained model and lightweight algorithm can be configured in the environment of the edge detection box, so that the drone can obtain the ability of real-time obstacle detection.

[0108] This step is as follows Figure 3 As shown, the model is deployed on the edge computing platform. In the present invention, the lightweight algorithm is configured and the model is deployed in the environment of the edge box. The edge box is connected to the drone sensor, and the data obtained by the drone is processed in real time in the edge box. The processing results are screened to detect the area with obstacles around the tower base. Useful results are transmitted to the operator in a timely manner, and risks are handled in a timely manner.

[0109] Specifically, unstructured pruning is a pruning method at the connection level. The norm of each weight in the weight matrix is ​​calculated to determine the importance of the weight. The weights are arranged by importance, and the weight values ​​less than the threshold are replaced with 0. Quantization mainly converts the data type that occupies a large amount of weight space into a data type that occupies a small amount of space.

[0110] Specifically, Edge Box is a distributed open platform for edge computing that integrates network, computing, storage, and application core capabilities. The aforementioned algorithms and models are compiled on this platform to process the image data obtained by drones in real time.

[0111] Edge computing technology provides edge intelligence services at the edge of the network close to the source of objects or data through a distributed open platform that integrates network, computing, storage, and application core capabilities. It can directly analyze the data in local devices or networks close to where the data is generated, eliminating the need to transmit the data to a cloud data processing center.

[0112] Specifically, this solution identifies and monitors potential obstacles that may endanger transmission lines, which is the key to ensuring the safe operation of transmission lines. By deploying a lightweight target detection network on the edge box, the data obtained by the drone can be processed in real time. In addition, by accurately identifying and monitoring obstacles around the tower base of the transmission network, it can improve inspection efficiency, optimize resource allocation, reduce maintenance costs, and ensure the stable operation of the transmission line.

[0113] Specifically, unstructured pruning is different from structured pruning in that it mainly targets the weights of neurons in the convolutional layer, removing those weights that contribute less to the model output, rather than just the filters of the entire layer. This pruning method is more sophisticated and can further reduce the number of model parameters without destroying the structural integrity of the model. After pruning, the number of parameters of the model is significantly reduced, and the computational burden is reduced.

[0114] Specifically, the process of converting the model's weight data from high precision (such as 32-bit floating point numbers) to low precision (such as 8-bit fixed point numbers) is called model quantization. Through model quantization, the model's memory usage and computing time can be reduced at runtime, and the model's operating efficiency on edge devices can be improved. The quantized model is called an optimized image analysis model.

[0115] For example, by analyzing the weights of the convolutional layers in the model, the importance of each weight is evaluated using methods such as the L1 norm or Taylor first-order approximation. The least important 30% of weights are removed, but their positions and connections in the model structure are retained to form a pruned image analysis model. Note that the pruned model needs to be fine-tuned to recover the performance loss that may be caused by pruning.

[0116] For the pruned model, model quantization is performed to convert the weights from 32-bit floating point numbers to 8-bit fixed point numbers. This process can use uniform quantization or non-uniform quantization, where uniform quantization divides the weight range evenly, while non-uniform quantization performs more flexible division based on the distribution of weights.

[0117] In some embodiments, after the image data is input into the image analysis model to obtain the inspection results corresponding to the image data, the method further includes the following steps: when the inspection results indicate that the image data includes obstacles, generating an alarm message; and sending the image data including the obstacle and the alarm message to the target terminal.

[0118] In this solution, when an obstacle in the transmission line is detected, an alarm message is generated, and the alarm message together with the image data is sent to the target terminal, so that the staff of the target terminal can be informed in time to carry out subsequent maintenance of the transmission line.

[0119] Specifically, when the optimized image analysis model performs real-time analysis on the image data collected by the drone, if an obstacle is detected in the image, that is, the detection result output by the model contains information such as the location and category of the obstacle, an alarm message will be generated immediately. The alarm message usually contains the type, location, size and possible threat level of the obstacle, so that the operator or monitoring center can quickly understand the specific situation of the obstacle.

[0120] Once the alarm information is generated, it will be sent to the target terminal together with the original image data containing the obstacle, such as the operator's mobile device (mobile phone, tablet), the computer in the monitoring center or other designated receiving devices. The image data and alarm information are transmitted through wireless communication networks such as 4G, 5G or Wi-Fi to ensure that the information can be delivered to the recipient in a timely and accurate manner.

[0121] Specifically, current deep learning methods focus on improving network structure and modules to enhance network performance. This approach often increases the number of model parameters, making the model more complex, increasing the amount of computation, and slowing down processing speed. This model is not conducive to deployment on mobile processors with weaker performance. At the same time, the current obstacle detection solution is to wait for the drone to transmit data back before processing the data, rather than processing the data in real time near the data acquisition area while the drone is acquiring the data. This results in the transmission of more redundant data, which takes up storage space and wastes computing resources.

[0122] In view of the shortcomings of the above-mentioned prior art, the scheme of the present application uses a YOLOv8 deep learning algorithm to automatically detect the input image. The YOLOv8 algorithm has a faster data processing speed and less complexity. The pruning algorithm removes smaller weights, further reduces the model parameters, and improves the operation speed to achieve the purpose of lightweight model. At the same time, the present invention adds a two-stage self-distillation method to distill the model during the model training process, learns knowledge from itself, and uses the pruned model for one iterative training. After that, the model is used as a teacher model to distill the model for the next training, so that the student model learns the knowledge of the teacher model without the need to train a complex teacher model, thereby improving the training efficiency. In addition, the present invention deploys the algorithm and model on an edge processing platform, that is, a distributed open platform for edge computing that integrates network, computing, storage, and application core capabilities. The platform is linked with the drone sensor to process the data obtained by the drone sensor in real time, and directly mark the obstacle positions near the tower base scanned by the drone along the way.

[0123] In summary, in the scheme of the present application, it is planned to use a drone platform to collect image data around the transmission tower base through a drone, and then preprocess, data enhance, and sample delineate the image to produce a drone image dataset of tower base obstacles; based on the YOLOv8 algorithm with efficient target detection capability, combined with lightweight model design, the YOLOv8 original network is structured pruned to obtain a lightweight target detection model; the lightweight model is self-distilled using a two-stage distillation method, and the training loss is calculated using CWD heterogeneous computing; the trained model is unstructured pruned and quantized to further improve the model detection efficiency; finally, the trained model and lightweight algorithm are configured in an edge computing environment to enable the drone to obtain the ability to detect obstacles in real time.

[0124] In summary, in this embodiment, data is collected through drone aerial photography, and the YOLOv8 algorithm is improved in combination with lightweight module design, and the lightweight algorithm is applied to drone real-time transmission network tower base obstacle detection. First, the drone platform is used to collect image data around the transmission tower base, and then the image is preprocessed, data enhanced, and sampled to produce a tower base obstacle drone image data set; then, based on the YOLOv8 algorithm with efficient target detection capabilities, combined with lightweight model design, the YOLOv8 original network is structured pruned to obtain a lightweight target detection model; the lightweight model is self-distilled using a two-stage distillation method, and CWD heterogeneous calculation training loss is used; then, the trained model is unstructured pruned and quantized to further improve the model detection efficiency; finally, the trained model and lightweight algorithm are configured in an edge computing box environment. Compared with existing classification methods, the method provided in this example can effectively improve the speed of obstacle detection.

[0125] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the power transmission line inspection method of the present application will be described in detail below in combination with specific embodiments.

[0126] This embodiment relates to a specific inspection method for a power transmission line, such as Figure 5 As shown, the following steps are included:

[0127] Step S1: Collect image data around the transmission tower base based on the multi-platform UAV. Perform image preprocessing, data enhancement, and sample mapping to produce a UAV image dataset of transmission network tower base obstacles;

[0128] Step S2: Based on the YOLOv8 detection algorithm with efficient target detection capability and combined with lightweight module design, the YOLOv8 original network is structured pruned to obtain a lightweight target detection model;

[0129] Step S3: Use the two-stage distillation method to perform self-distillation training on the lightweight model, and use CWD heterogeneous calculation of training loss;

[0130] Step S4: Perform unstructured pruning and model quantization on the distilled model to further improve the model detection efficiency;

[0131] Step S5: Configure the trained model and lightweight algorithm in the environment of the edge detection box to enable the drone to obtain real-time obstacle detection capabilities.

[0132] The embodiment of the present application also provides a patrol device for a power transmission line. It should be noted that the patrol device for a power transmission line of the embodiment of the present application can be used to execute the patrol method for a power transmission line provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0133] The following is an introduction to the inspection device for power transmission lines provided in an embodiment of the present application.

[0134] Figure 6 1 is a structural block diagram of a transmission line inspection device according to an embodiment of the present application. Figure 6 As shown, the device comprises:

[0135] An acquisition unit 10 is used to acquire image data of the power transmission line, wherein the image data is acquired by using a drone;

[0136] A construction unit 20 is used to construct an image analysis model, wherein the image analysis model is trained by a YOLOv8 algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical image data acquired within a historical time period and historical inspection results corresponding to the historical image data, and the historical inspection results are used to characterize whether there are obstacles in the historical image data;

[0137] The processing unit 30 is used to input the above image data into the above image analysis model to obtain the inspection result corresponding to the above image data, wherein the above inspection result is used to indicate whether there is an obstacle in the above image data.

[0138] During the specific implementation process, the acquisition unit includes a control module and a preprocessing module. The control module is used to control the above-mentioned UAV to patrol and collect images along the above-mentioned transmission line to obtain initial image data; the preprocessing module is used to preprocess the above-mentioned initial image data to obtain the above-mentioned image data, wherein the preprocessing method includes one or more of data enhancement, sample delineation and data cleaning.

[0139] In this solution, the drone is controlled to fly along the path of the transmission line. At the same time, the camera it carries is used to photograph the line environment and collect a large amount of original initial image data. By preprocessing the initial image data, the obtained image data can be guaranteed to be more accurate.

[0140] During the specific implementation process, the above-mentioned device also includes an optimization unit and an analysis unit. The optimization unit is used to optimize the above-mentioned image analysis model after constructing the image analysis model to obtain an optimized image analysis model, wherein the optimization method includes one or more of pruning, distillation and model quantization; the analysis unit is used to use the above-mentioned optimized image analysis model to analyze the above-mentioned image data to obtain the target inspection results corresponding to the above-mentioned image data.

[0141] In this solution, by optimizing the image analysis model, the complexity and computing requirements of the model can be reduced, while maintaining or improving its accuracy in detecting obstacles as much as possible. The optimization methods can include the following: Pruning: By removing unimportant neurons or connections in the model, the number of model parameters is reduced, thereby reducing computing costs and increasing running speed. Distillation: Using knowledge distillation technology, a smaller model (student model) learns the features and decisions of a larger model (teacher model), so that the student model can have a faster running speed and smaller size while maintaining high accuracy. Model quantization: Convert the model's weights and calculation processes from floating point numbers to fixed point numbers to reduce memory consumption and speed up the model's running speed. The preprocessed image data is input into the optimized image analysis model, and the model will perform target detection on the image, identify obstacles in the image, and provide detailed information such as their type and location, so as to obtain more accurate target inspection results.

[0142] During the specific implementation process, when the optimization method includes pruning, the optimization unit includes an acquisition module and a first pruning module. The acquisition module is used to obtain the L1 norm of all filter layers in the above-mentioned image analysis model; the first pruning module is used to prune all the above-mentioned filter layers in turn according to the size of the above-mentioned L1 norm to obtain the above-mentioned optimized image analysis model.

[0143] In this scheme, the L1 norm is a norm that measures the sum of the absolute values ​​of all elements in a vector or matrix, and is used to evaluate the importance of each filter in the filter layer. Generally, filters with smaller L1 norms contribute less to the output of the model and can be regarded as "unimportant" filters. Based on the calculated L1 norm, the filters are sorted by the size of the norm. Starting with the filter with the smallest L1 norm, they are pruned and removed in sequence until a predetermined pruning ratio is reached, such as 30%. The number of model parameters is reduced after pruning, but the model runs faster.

[0144] In some embodiments, when the optimization method includes distillation, the optimization unit includes a first extraction module, a second extraction module, a calculation module, a weighted average module and an update module, the first extraction module is used to perform feature extraction on the image analysis model obtained by the Nth training, and calculate a first probability distribution for the extracted features, wherein N≥1; the second extraction module is used to perform feature extraction on the image analysis model obtained by the N+1th training, and calculate a second probability distribution for the extracted features; the calculation module is used to calculate the KL divergence according to the first probability distribution and the second probability distribution to obtain a distillation loss value; the weighted average module is used to weighted average the distillation loss value, the classification loss value and the target box loss value to obtain a total loss value; the update module is used to use the total loss value to perform back propagation so as to update the weights of the image analysis model during training to obtain the optimized image analysis model.

[0145] In this scheme, a two-stage self-distillation method is used, KL divergence is used to calculate the distillation loss, and back propagation is used to update the weights to achieve the purpose of model optimization. In this way, the model can learn the knowledge obtained from its previous training in each round of iteration, thereby improving the detection accuracy and efficiency, and further improving the real-time and reliability of transmission line obstacle detection.

[0146] In some embodiments, when the optimization method includes pruning and model quantization, the optimization unit includes a second pruning module and a conversion module, the second pruning module is used to perform unstructured pruning on the above-mentioned image analysis model to obtain the pruned image analysis model; the conversion module is used to convert the first type of data in the above-mentioned image analysis model after pruned into the second type of data to obtain the above-mentioned optimized image analysis model, wherein the space occupied by the above-mentioned first type of data is larger than the space occupied by the above-mentioned second type of data.

[0147] In this solution, unstructured pruning and model quantization can be used to optimize the image analysis model, which can not only reduce the computational complexity and resource requirements of the model, but also enable the model to maintain a high detection accuracy.

[0148] In some embodiments, the above-mentioned device also includes a generating unit and a sending unit. The generating unit is used to generate an alarm message after inputting the above-mentioned image data into the above-mentioned image analysis model to obtain the inspection result corresponding to the above-mentioned image data, when the above-mentioned inspection result indicates that the above-mentioned image data includes an obstacle; the sending unit is used to send the above-mentioned image data including the obstacle and the above-mentioned alarm message to the target terminal.

[0149] In this solution, when an obstacle in the transmission line is detected, an alarm message is generated, and the alarm message together with the image data is sent to the target terminal, so that the staff of the target terminal can be informed in time to carry out subsequent maintenance of the transmission line.

[0150] The above-mentioned inspection device for power transmission lines includes a processor and a memory. The above-mentioned acquisition unit, construction unit and processing unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0151] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be provided, and the problem of low inspection efficiency of the power transmission line in the prior art can be solved by adjusting kernel parameters.

[0152] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0153] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the transmission line inspection method.

[0154] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the inspection method of the power transmission line is executed when the program is run.

[0155] An embodiment of the present invention provides a device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the steps of the inspection method of the power transmission line are implemented. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0156] A computer program product includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the power transmission line inspection method in each embodiment of the present application are implemented.

[0157] The present application also provides an inspection system, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the above-mentioned inspection methods of the transmission lines.

[0158] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0159] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0161] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0163] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0164] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0165] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0166] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0167] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for inspecting a power transmission line, characterized in that: include: Acquire image data of the power transmission line, wherein the image data is collected by using a drone; Constructing an image analysis model, wherein the image analysis model is trained by a YOLOv8 algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical image data acquired within a historical time period and historical inspection results corresponding to the historical image data, and the historical inspection results are used to characterize whether there are obstacles in the historical image data; The image data is input into the image analysis model to obtain an inspection result corresponding to the image data, wherein the inspection result is used to characterize whether there is an obstacle in the image data.

2. The method according to claim 1, characterized in that Acquire image data of power transmission lines, including: Controlling the UAV to inspect along the power transmission line and collect images to obtain initial image data; The initial image data is preprocessed to obtain the image data, wherein the preprocessing method includes one or more of data enhancement, sample delineation and data cleaning.

3. The method according to claim 1, characterized in that After constructing the image analysis model, the method further includes: Optimizing the image analysis model to obtain an optimized image analysis model, wherein the optimization method includes one or more of pruning, distillation and model quantization; The image data is analyzed using the optimized image analysis model to obtain target inspection results corresponding to the image data.

4. The method according to claim 3, characterized in that When the optimization method includes pruning, the image analysis model is optimized to obtain an optimized image analysis model, including: Obtaining the L1 norms of all filter layers in the image analysis model; According to the size of the L1 norm, all the filter layers are pruned in sequence to obtain the optimized image analysis model.

5. The method according to claim 3, characterized in that: When the optimization method includes distillation, the image analysis model is optimized to obtain an optimized image analysis model, including: Performing feature extraction on the image analysis model obtained by the Nth training, and calculating a first probability distribution for the extracted features, wherein N≥1; Performing feature extraction on the image analysis model obtained through the N+1th training, and calculating a second probability distribution for the extracted features; Calculate the KL divergence according to the first probability distribution and the second probability distribution to obtain a distillation loss value; Taking a weighted average of the distillation loss value, the classification loss value, and the target box loss value to obtain a total loss value; The loss sum value is used for back propagation to update the weights of the image analysis model during training to obtain the optimized image analysis model.

6. The method according to claim 3, characterized in that When the optimization method includes pruning and model quantization, the image analysis model is optimized to obtain an optimized image analysis model, including: Performing unstructured pruning processing on the image analysis model to obtain the pruned image analysis model; The first type of data in the pruned image analysis model is converted into a second type of data to obtain the optimized image analysis model, wherein the space occupied by the first type of data is larger than the space occupied by the second type of data.

7. The method according to any one of claims 1 to 6, characterized in that: After the image data is input into the image analysis model to obtain the inspection result corresponding to the image data, the method further includes: If the inspection result indicates that the image data includes an obstacle, generating an alarm message; The image data including the obstacle and the alarm information are sent to a target terminal.

8. A transmission line inspection device, characterized in that: include: An acquisition unit, used to acquire image data of the power transmission line, wherein the image data is acquired by using a drone; A construction unit, configured to construct an image analysis model, wherein the image analysis model is trained by a YOLOv8 algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical image data acquired within a historical time period and historical inspection results corresponding to the historical image data, and the historical inspection results are used to characterize whether there are obstacles in the historical image data; A processing unit is used to input the image data into the image analysis model to obtain an inspection result corresponding to the image data, wherein the inspection result is used to characterize whether there is an obstacle in the image data.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the power transmission line inspection method according to any one of claims 1 to 7 are implemented.

10. A patrol inspection system, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the inspection method of the transmission line described in any one of claims 1 to 7.

Images