A Hidden Danger Detection Method for Transmission Lines under Wind Disaster Scenarios Based on Model Lightweighting

Abstract

The present invention provides a hidden danger detection method for transmission lines in a wind disaster scenario based on model lightweighting, including: constructing a hidden danger detection super network including a backbone super network and a feature fusion super network, and selectively removing unimportant candidate operations through intelligent pruning based on importance parameters to optimize the super network structure. Secondly, for the weights and activation values of each layer, a candidate set of numerical precisions is defined, and random sampling and dynamic adjustment of numerical precisions are performed through the Gumbel-Softmax mechanism to ensure that the precision configuration of each layer of neural network can be adaptively optimized during the training process. The backbone super network and the feature fusion super network are streamlined through a greedy algorithm to obtain a lightweight hidden danger detection model that meets the requirements of precision and computational complexity. The method of the present invention can effectively reduce the computational overhead, improve the inference efficiency of the hidden danger detection model, and is applicable to resource-constrained embedded devices and edge computing scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of transmission line detection, and specifically relates to a method for detecting hidden dangers of transmission lines in a wind disaster scenario based on model lightweighting. Background Technique

[0002] With the rapid development of the power system, the safe operation of transmission lines has become an important link in ensuring power supply. Transmission line inspection is a key measure to ensure line safety. It can timely detect and handle potential hazards that may cause power supply interruption, such as broken strands of conductors, foreign objects hanging on the line, loose fittings, etc. If these potential hazards are not detected and repaired in time, they may trigger safety accidents, causing serious economic losses and social impacts.

[0003] Traditional methods for inspecting transmission lines mainly rely on manual patrols and ground vehicles. These methods not only take a long time and are costly, but also greatly increase the safety risks of manual inspections and have extremely low efficiency under harsh weather conditions, such as wind disasters and heavy rains. With the development of technology, unmanned aerial vehicle (UAV) inspection technology has gradually become an important means for inspecting transmission lines due to its advantages of high efficiency and low cost. UAVs can quickly reach inaccessible areas and conduct a comprehensive inspection of transmission lines from the air, greatly improving the efficiency and safety of inspections.

[0004] However, in extreme weather conditions such as wind disasters, UAV inspections face many challenges. First, strong winds can cause the stability of UAVs to decline, affecting operation safety and the quality of image acquisition. The violent shaking of conductors caused by increasing wind speed, as well as the vibration of the UAV itself, often make the collected images appear blurred and have smear, seriously affecting the clarity of the images and the subsequent image processing effect. In addition, adverse weather conditions such as rainfall, snow, and wind can also cause image blurring, or even completely obscure important components of the transmission line, making it difficult to accurately identify potential hazards.

[0005] In addition, communication facilities in a wind disaster may be damaged, affecting the real-time data transmission between the UAV and the control center. Even if the UAV successfully acquires image data, it may not be able to transmit it back to the analysis center in time due to communication obstacles, delaying the processing time of potential hazards. For the UAV side, its computing resources are usually limited and it is difficult to process complex image analysis and potential hazard identification algorithms, which further increases the difficulty of implementing effective potential hazard detection in complex environments. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a method for detecting hidden dangers of transmission lines in a wind disaster scenario based on model lightweighting.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] On the one hand, the present invention provides a method for detecting hidden dangers of transmission lines in a wind disaster scenario based on model lightweighting, including the following steps:

[0009] Construct a hidden danger detection super network, set model accuracy constraints and computational complexity constraints, construct a candidate operation set for the hidden danger detection super network, set importance parameters for the candidate operation set of the hidden danger detection super network, and perform intelligent pruning on the hidden danger detection super network based on the candidate operation set of the hidden danger detection super network and its importance parameters under the model accuracy constraints and computational complexity constraints;

[0010] For the weights and activation values of each layer of the neural network in the pruned hidden danger detection super network, define a candidate set of numerical precisions, and dynamically adjust the numerical precision of the hidden danger detection super network according to the model accuracy constraints and computational complexity constraints to obtain a lightweight hidden danger detection model;

[0011] Deploy the lightweight hidden danger detection model to the drone edge side. The drone collects hidden danger image data of the transmission line. The hidden danger image data includes broken strand pictures, hanging foreign object pictures, damaged tower pictures and other hidden danger images. Input the hidden danger image data into the lightweight hidden danger detection model to achieve the detection of hidden dangers of the transmission line in a wind disaster scenario.

[0012] Further, the hidden danger detection super network includes a backbone super network and a feature fusion super network;

[0013] The construction of the candidate operation set for the hidden danger detection super network, setting importance parameters for the candidate operation set of the hidden danger detection super network, and performing intelligent pruning on the hidden danger detection super network based on the candidate operation set of the hidden danger detection super network and its importance parameters under the model accuracy constraints and computational complexity constraints specifically include:

[0014] Construct a candidate operation set for the backbone super network, and perform intelligent pruning on the backbone super network based on the candidate operation set under the model accuracy constraints and computational complexity constraints;

[0015] Construct a candidate operation set for the feature fusion super network, and perform intelligent pruning on the feature fusion super network based on the candidate operation set under the model accuracy constraints and computational complexity constraints.

[0016] Further, the model accuracy constraint is L(A, W) ≤ Lmax, where A represents the model architecture, W represents the network parameters trained under the given model architecture, L(A, W) is the error function of the model on the validation set, and Lmax is the preset maximum allowable error;

[0017] The computational complexity constraint is C(A) ≤ Cmax. C(A) includes the inference latency, floating-point operations per second and the number of parameters of the model. Cmax is the preset maximum allowable computational complexity, determined based on the computing resources of the drone edge side.

[0018] Furthermore, a candidate operation set for constructing the skeletal super network is built, and the skeletal super network is intelligently pruned based on the candidate operation set under the constraints of model accuracy and computational complexity, which specifically includes:

[0019] Step A1: Build a candidate operation set for the skeletal super network;

[0020] Step A2: Assign an importance parameter to each candidate operation in the candidate operation set of the skeletal super network α i , α i indicating the contribution degree of the i-th candidate operation to the network performance;

[0021] Step A3: Initialize the network parameters W of the skeletal super network, and initialize the importance parameter of each candidate operation to 1;

[0022] Step A4: Perform forward propagation using the current network structure and network parameters W, calculate the output of each node, and for each candidate operation o i , its output is:

[0023] ,

[0024] where o i (X; W) is the calculation result of the i-th candidate operation based on the shared network parameters W, α i is the importance parameter of the i-th candidate operation, and X is the input feature of the current node;

[0025] Step A5: Sum the outputs of all candidate operations with weights to obtain the final output of the current layer:

[0026] ,

[0027] where n is the number of candidate operations in the current layer;

[0028] Step A6: Calculate the error of the model through the loss function L(A, W), fix the importance parameter, and update the network parameters W through backpropagation;

[0029] Step A7: Fix the network parameters W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) according to the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through backpropagation, and update the importance parameter using gradient descent;

[0030] Step A8: Repeat Step A6 and Step A7 until the network parameters W and the importance parameters of the skeletal super network converge;

[0031] Step A9: After the training is completed, based on the value of the importance parameter, the skeletal super network is streamlined through a greedy algorithm under the model accuracy constraint L(A,W) and the computational complexity constraint C(A) to obtain the streamlined skeletal super network.

[0032] Furthermore, the skeletal super network includes downsampling blocks and regular blocks, and the candidate operation set of the skeletal super network includes a downsampling block operation candidate set, a regular block operation candidate set, and a connection method candidate set between blocks;

[0033] The downsampling block operation candidate set includes 3x3 convolution, 5x5 convolution, 3x3 max pooling, 3x3 average pooling, and 3x3 dilated convolution;

[0034] The regular block operation candidate set includes 3x3 convolution, 1x1 convolution, depthwise separable convolution, and 3x3 max pooling;

[0035] The connection method candidate set between blocks includes a direct connection operation and a convolution operation.

[0036] Furthermore, the streamlining of the skeletal super network through a greedy algorithm based on the value of the importance parameter under the model accuracy constraint and the computational complexity constraint to obtain the streamlined skeletal super network specifically includes:

[0037] Select the candidate operation with the smallest importance parameter in the candidate operation set of the skeletal super network for the first removal operation. Check whether the updated network still meets the set accuracy threshold and computational complexity constraint. If removing this operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, restore this operation; otherwise, continue with the removal operation. After traversing all candidate operations with importance parameters less than θ, output the streamlined skeletal super network;

[0038] The first removal operation includes removing the candidate operations of the module and removing the candidate operations of the connection method between blocks;

[0039] When the candidate operation of the removed module is the last candidate operation of the removed module, perform the following operations:

[0040] Disconnect the input and output connections of this module, delete the connections of subsequent modules to this module, and delete this module and its related computational nodes;

[0041] When the candidate operation of the connection method between blocks being removed is the last candidate operation, perform the following operations: Disconnect this connection, delete the subsequent modules and related nodes that depend on this connection, and delete this connection edge.

[0042] Further, constructing a candidate operation set for the feature fusion super network, and intelligently pruning the feature fusion super network based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including:

[0043] Step B1: Construct a candidate operation set for the feature fusion super network. The feature fusion super network includes multiple fusion blocks. Each node in a fusion block represents a feature map, and nodes are connected by edges. A feature map is connected to all its predecessor nodes. The candidate operation set of the feature fusion super network includes an edge candidate operation set; the edge candidate operation set includes 1x1 convolution, 3x3 convolution, 5x5 convolution, and 3x3 dilated convolution;

[0044] Step B2: Assign an importance parameter β to each edge e indicating the importance of the edge, and assign an importance parameter β to each candidate operation in the edge candidate operation set of each edge o indicating the importance of the candidate operation. The calculation process of the feature map in each fusion block is expressed as:

[0045] ,

[0046] where, z j is the output feature map of the j-th fusion block in the feature fusion super network, represents the output after applying the candidate operation o to the feature map x i of the i-th predecessor node, where r o is the output channel expansion rate corresponding to the candidate operation o, represents the importance parameter of the edge (i, j), represents the importance parameter of the candidate operation o for the edge (i, j);

[0047] Step B3: Calculate the error of the model through the loss function L(A, W) based on the output feature map. Fix the importance parameter and update the network parameter W of each fusion block in the feature fusion super network through backpropagation; <{}

[0048] Step B4: Fix the network parameter W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) based on the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through backpropagation, and update the importance parameter using gradient descent;

[0049] Step B5: Repeat Step B3 and Step B4 until the network parameter W and the importance parameter of the feature fusion super network converge;

[0050] Step B6: After training, prune the feature fusion super network based on the value of the importance parameter under the constraints of model accuracy and computational complexity to obtain the pruned feature fusion super network.

[0051] Furthermore, the feature fusion hypernetwork is streamlined based on the model accuracy constraint and the computational complexity constraint through a greedy algorithm according to the value of the importance parameter, and the streamlined feature fusion hypernetwork is obtained, which specifically includes:

[0052] According to the edge importance parameter β of each edge e Perform preliminary pruning on the edges in the feature fusion hypernetwork, select the edge with the smallest importance for removal. If removing this operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, then restore this operation; otherwise, continue with the removal operation. When all the importance parameters β e After traversing the edges with β less than θ, select the candidate operation with the largest importance parameter β in the candidate operation set of the remaining edges o as the operation of this edge to obtain the streamlined feature fusion hypernetwork.

[0053] Furthermore, for the weights and activation values of each layer of the neural network in the pruned model, define a numerical precision candidate set, and dynamically adjust the numerical precision of the model according to the model accuracy constraint and the computational complexity constraint, which specifically includes:

[0054] For the weights and activation values of each layer of the neural network in the pruned hidden danger detection model, define an optional numerical precision candidate set P = { p 1 , p 2 、 …, p n}, where p i represents the i-th precision configuration, including FP32, FP16, INT8, and mixed precision;

[0055] Introduce a learnable importance parameter γ for the candidate precisions in the numerical precision candidate set of each layer. The dimension of the importance parameter γ is the same as the number of candidate precisions in the numerical precision candidate set;

[0056] In the forward propagation during the training phase, use Gumbel-Softmax to randomly sample the numerical precision of each layer of the neural network to obtain the importance parameter γ of the numerical precision candidate set of the neural network each time;

[0057] Only activate the candidate precision with the largest importance parameter γ sampled each time as the precision configuration of the current neural network layer;

[0058] According to the model accuracy constraint and the computational complexity constraint, fix the current importance parameter γ, update the network parameter 𝑊 through the conventional backpropagation process. After freezing the network parameter 𝑊, use the gradient estimation of Gumbel-Softmax to update the importance parameter γ of the numerical accuracy candidate set;

[0059] Repeat the above process until both the network parameter 𝑊 and the importance parameter γ converge. Take the candidate accuracy with the largest importance parameter γ in the numerical accuracy candidate sets of each layer of the neural network as the accuracy configuration of the current neural network layer, and obtain the lightweight hidden danger detection model.

[0060] Furthermore, use Gumbel-Softmax to randomly sample the numerical accuracy of each layer of the neural network, and obtain the importance parameter γ of the numerical accuracy candidate set of the neural network each time. The formula is:

[0061] ,

[0062] where, is the importance parameter of the i-th accuracy configuration in the t-th training, is the importance parameter of the i-th accuracy configuration in the (t - 1)-th training, g i is the Gumbel noise, and τ is the temperature coefficient.

[0063] Compared with the prior art, the present invention has the following advantages:

[0064] (1) The present invention adopts the intelligent pruning of the model structure and the dynamic regulation of the numerical accuracy under double constraint conditions. On the premise of ensuring the hidden danger detection accuracy, by optimizing the computational complexity, the lightweight model is successfully realized, which can quickly and efficiently perform hidden danger detection on the UAV terminal side, significantly reducing the consumption of computing resources and improving the working efficiency of the UAV.

[0065] (2) The present invention uses the Gumbel-Softmax method to dynamically adjust the numerical accuracy of each layer of the neural network, calculates the importance parameter through the temperature coefficient and the Gumbel noise, realizes the flexible configuration of the numerical accuracy, enables the model to adjust the accuracy according to actual needs, thereby balancing the computing resources and accuracy requirements, and further improving the efficiency and adaptability of the detection model.

[0066] (3) In the process of streamlining the backbone super network and the feature fusion super network, the present invention adopts the greedy algorithm. Based on the model accuracy and computational complexity constraints, it streamlines unnecessary candidate operations, reduces the computational complexity, and at the same time ensures the accuracy requirements of the model, optimizing the computational efficiency of the model, and is suitable for the UAV terminal side deployment with limited resources. Description of the Drawings

[0067] Figure 1 This is the method flow chart of the present invention. Specific embodiments

[0068] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0069] Embodiment 1.

[0070] As Figure 1 shown, on the one hand, the present invention provides a hidden danger detection method for transmission lines under wind disaster scenarios based on model lightweighting, including the following steps:

[0071] Construct a hidden danger detection super network, set model accuracy constraints and computational complexity constraints, construct a candidate operation set for the hidden danger detection super network, set importance parameters for the candidate operation set of the hidden danger detection super network, and perform intelligent pruning on the hidden danger detection super network based on the candidate operation set of the hidden danger detection super network and its importance parameters under the model accuracy constraints and computational complexity constraints;

[0072] For the weights and activation values of each layer of the neural network in the pruned hidden danger detection super network, define a numerical accuracy candidate set, and dynamically adjust the numerical accuracy of the hidden danger detection super network according to the model accuracy constraints and computational complexity constraints to obtain a lightweight hidden danger detection model;

[0073] Deploy the lightweight hidden danger detection model to the drone side. The drone collects hidden danger image data of the transmission line. The hidden danger image data includes broken strand pictures, hanging foreign object pictures, damaged tower pictures and other hidden danger images. Input the hidden danger image data into the lightweight hidden danger detection model to realize the detection of hidden dangers of the transmission line under wind disaster scenarios.

[0074] Furthermore, the hidden danger detection super network includes a backbone super network and a feature fusion super network;

[0075] The construction of the candidate operation set for the hidden danger detection super network, setting importance parameters for the candidate operation set of the hidden danger detection super network, and performing intelligent pruning on the hidden danger detection super network based on the candidate operation set of the hidden danger detection super network and its importance parameters under the model accuracy constraints and computational complexity constraints specifically include:

[0076] Construct a candidate operation set for the backbone super network, and perform intelligent pruning on the backbone super network based on the candidate operation set under the model accuracy constraints and computational complexity constraints;

[0077] Construct a candidate operation set for the feature fusion supernetwork, and perform intelligent pruning on the feature fusion supernetwork based on the candidate operation set under the constraints of model accuracy and computational complexity.

[0078] The hazard detection supernetwork is the basic network structure of the entire model, including two main parts:

[0079] Skeleton supernetwork: The skeleton supernetwork is the basic framework of the network, containing some core operations (such as convolution operations, pooling operations, etc.) for processing input data.

[0080] Feature fusion supernetwork: The feature fusion supernetwork is mainly used to extract information from different feature maps and combine this information to improve the model's representation ability, usually used to enhance the model's feature learning and fusion effects.

[0081] Set the constraints of model accuracy and computational complexity;

[0082] In actual deployment, it is necessary to balance the model's accuracy (i.e., the accuracy of the detection results) and computational complexity (i.e., the computational resources required for model inference). The purpose of this step is to ensure that the model meets the accuracy requirements without exceeding the limit of computational resources. Specifically, two constraints are set:

[0083] Accuracy constraint: Ensure that the error of the model does not exceed the predetermined maximum allowable error (Lmax), that is, the error of the model output cannot be too large to ensure the performance of the model.

[0084] Computational complexity constraint: Ensure that the computational complexity of the model (including inference latency, floating-point operations per second, number of parameters, etc.) does not exceed the preset limit (Cmax). This can prevent the model from being too large to run on devices such as drones during actual deployment.

[0085] Construct a candidate operation set for the skeleton supernetwork, and perform intelligent pruning on the skeleton supernetwork based on the candidate operation set under the constraints of model accuracy and computational complexity;

[0086] For each layer in the skeleton supernetwork, there are multiple selectable operations. For example, the convolutional layer may have convolutional kernels of different sizes, and the pooling layer may have different pooling methods, etc. The candidate operation set is the set of all these selectable operations. According to the accuracy and computational complexity constraints, the most suitable operations are selected from the candidate operation set for retention, and other operations are removed. The process of intelligent pruning optimizes the model structure by automatically selecting which operations have a greater impact on model accuracy and which have a greater impact on computational complexity.

[0087] Construct a candidate operation set for the feature fusion supernetwork, and perform intelligent pruning on the feature fusion supernetwork based on the candidate operation set under the constraints of model accuracy and computational complexity;

[0088] Each layer of the feature fusion network also has multiple operations to choose from, usually different types of convolutional operations (such as 1x1 convolution, 3x3 convolution, etc.). The selection of these operations will affect the effect of the model when fusing different feature maps. Similar to the intelligent pruning of the backbone supernetwork, the feature fusion network also performs intelligent pruning according to the constraints of accuracy and computational complexity, selects the optimal operations for fusion, and removes unnecessary operations.

[0089] For the weights and activation values of each layer of the neural network in the pruned hazard detection supernetwork, define a candidate set of numerical precisions, and dynamically adjust the numerical precision of the hazard detection supernetwork according to the model accuracy constraint and computational complexity constraint to obtain a lightweight hazard detection model.

[0090] Different numerical precisions (such as FP32, FP16, INT8, mixed precision, etc.) will affect the computational efficiency and accuracy of the model. The weights and activation values of each layer of the neural network can choose different precision configurations. By introducing an importance parameter (such as γ), the numerical precision of each layer of the network is adjusted. During the training process, by randomly sampling the precision of each layer (using the Gumbel-Softmax method), the network automatically selects the most suitable precision configuration. This process helps to reduce the computational burden while ensuring accuracy, enabling the model to run more efficiently.

[0091] Furthermore, the model accuracy constraint is L(A, W) ≤ Lmax, where A represents the model architecture, W represents the network parameters trained under the given model architecture, L(A, W) is the error function of the model on the validation set, and Lmax is the preset maximum allowable error;

[0092] The computational complexity constraint is C(A) ≤ Cmax, C(A) includes the inference latency, floating-point operations per second, and the number of parameters of the model, and Cmax is a preset value determined based on the computing resources on the drone side.

[0093] Furthermore, the candidate operation set of the backbone supernetwork is constructed, and the backbone supernetwork is intelligently pruned based on the candidate operation set under the model accuracy constraint and computational complexity constraint, specifically including:

[0094] Step A1: Construct the candidate operation set of the backbone supernetwork;

[0095] Step A2: Assign an importance parameter to each candidate operation in the candidate operation set of the backbone supernetwork α i , α i indicating the contribution degree of the i-th candidate operation to the network performance;

[0096] Step A3: Initialize the network parameters W of the skeletal supernetwork, and initialize the importance parameter of each candidate operation to 1;

[0097] For each candidate operation, an importance parameter is assigned α i , which represents the contribution degree of this operation to the network performance. Simply put, the importance parameter α i measures the impact of this operation on the final model performance. The initial value of the importance parameter is 1, which means that it is assumed that all candidate operations initially have the same contribution degree to the network performance;

[0098] Step A4: Use the current network structure and network parameters W for forward propagation, calculate the output of each node. For each candidate operation o i , its output is:

[0099] ,

[0100] where, o i (X; W) is the calculation result of the i-th candidate operation based on the shared network parameter W, α i is the importance parameter of the i-th candidate operation, and X is the input feature of the current node;

[0101] Step A5: Weight and sum the outputs of all candidate operations to obtain the final output of the current layer:

[0102] ,

[0103] where, n is the number of candidate operations in the current layer;

[0104] Step A6: Calculate the error of the model through the loss function L(A, W) according to the current output, fix the importance parameter, and update the network parameter W through backpropagation;

[0105] Step A7: Fix the network parameter W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) according to the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through backpropagation, and update the importance parameter using gradient descent;

[0106] Step A8: Repeat Step A6 and Step A7 until the network parameter W and the importance parameter of the skeletal supernetwork converge;

[0107] Furthermore, the skeletal supernetwork includes downsampling blocks and regular blocks, and the candidate operation set of the skeletal supernetwork includes a downsampling block operation candidate set, a regular block operation candidate set, and a connection method candidate set between blocks;

[0108] The candidate set of the downsampling block operations includes 3x3 convolution, 5x5 convolution, 3x3 max pooling, 3x3 average pooling, and 3x3 dilated convolution;

[0109] The candidate set of the conventional block operations includes 3x3 convolution, 1x1 convolution, depthwise separable convolution, and 3x3 max pooling;

[0110] The candidate set of the connection methods between blocks includes direct connection operation and convolution operation.

[0111] Furthermore, the skeleton supernetwork is pruned based on the model accuracy constraint and computational complexity constraint through a greedy algorithm according to the value of the importance parameter to obtain the pruned skeleton supernetwork, which specifically includes:

[0112] Select the candidate operation with the smallest importance parameter in the candidate operation set of the skeleton supernetwork for removal, and check whether the updated network still meets the set accuracy threshold and computational complexity constraint. If removing this operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, then restore this operation; otherwise, continue with the removal operation. After traversing all candidate operations with importance parameters less than θ, output the pruned skeleton supernetwork;

[0113] The removal of candidate operations includes the candidate operations of removing modules and the connection methods between blocks;

[0114] When the candidate operation of the removed module is the last candidate operation, perform the following operations:

[0115] Disconnect the input and output connections of this module, delete the connections of subsequent modules to this module, and delete this module and its related computational nodes;

[0116] When the candidate operation of the connection method between blocks is the last candidate operation, perform the following operations: Disconnect this connection, delete the subsequent modules and related nodes that depend on this connection, and delete this connection edge.

[0117] The importance parameter is the contribution degree of each candidate operation to the network performance (accuracy), and a smaller value indicates that the operation has a smaller impact on the network. Therefore, the greedy algorithm will first select those operations with smaller contribution degrees and preferentially remove them to reduce the computational amount. After each removal of a candidate operation, it is necessary to check the performance of the updated network;

[0118] The core objective of this part is to gradually remove unimportant operations and connections according to the importance parameters of each candidate operation through a greedy algorithm, streamline the network structure, and thus reduce the computational complexity. The whole process follows the following principles: only remove those operations that contribute less to the accuracy, ensuring that the network accuracy is not lower than the set threshold; after removal, check whether the computational complexity constraint conditions are met to ensure that the network still has sufficient computational efficiency; when removing operations, ensure the connectivity of the network and the consistency of the structure, avoiding an incoherent network structure.

[0119] Through these steps, the finally obtained streamlined skeleton supernetwork can minimize the computational amount as much as possible under the premise of meeting the accuracy and computational complexity constraints, and is suitable for application in resource-constrained environments.

[0120] After training, based on the value of the importance parameter, the skeleton supernetwork is streamlined through a greedy algorithm under the model accuracy constraint L(A, W) and the computational complexity constraint C(A) to obtain the streamlined skeleton supernetwork.

[0121] Furthermore, constructing a candidate operation set for the feature fusion supernetwork and intelligently pruning the feature fusion supernetwork based on the candidate operation set under the model accuracy constraint and the computational complexity constraint specifically includes:

[0122] Step B1: Construct a candidate operation set for the feature fusion supernetwork. The feature fusion supernetwork includes multiple fusion blocks, each node in the fusion block represents a feature map, nodes are connected by edges, and a feature map is connected to all its predecessor nodes. The candidate operation set of the feature fusion supernetwork includes an edge candidate operation set; the edge candidate operation set includes 1x1 convolution, 3x3 convolution, 5x5 convolution, 3x3 dilated convolution;

[0123] Step B2: Assign an importance parameter β to each edge e , representing the importance of the edge, and assign an importance parameter β to each candidate operation in the edge candidate operation set on each edge o , representing the importance of the candidate operation. The calculation process of the feature maps in each fusion block is expressed as:

[0124] ,

[0125] where, z j the output feature map of the j-th fusion block in the feature fusion supernetwork, represents the output after applying the candidate operation o to the feature map x i of the i-th predecessor node, where r o is the output channel expansion rate corresponding to the candidate operation o, represents the importance parameter of the edge (i,j), It represents the importance parameter of the candidate operation o for the edge (i,j);

[0126] Step B3: Calculate the error of the model through the loss function L(A,W) based on the output feature map. Fix the importance parameter and update the network parameters W of each fusion block in the feature fusion supernetwork through backpropagation;

[0127] Step B4: Fix the network parameters W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A,W) according to the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through backpropagation, and update the importance parameter using gradient descent;

[0128] Step B5: Repeat Step B3 and Step B4 until the network parameters W and the importance parameter of the feature fusion supernetwork converge;

[0129] Step B6: After training, streamline the feature fusion supernetwork based on the value of the importance parameter under the constraints of model accuracy and computational complexity to obtain the streamlined feature fusion supernetwork.

[0130] Furthermore, the streamlining of the feature fusion supernetwork based on the value of the importance parameter through the greedy algorithm under the constraints of model accuracy and computational complexity to obtain the streamlined feature fusion supernetwork specifically includes:

[0131] According to the edge importance parameter β of each edge e Perform preliminary pruning on the edges in the feature fusion supernetwork, select the edge with the smallest importance for removal. If removing this operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, then restore this operation; otherwise, continue with the removal operation. When all the importance parameters β e of the edges less than θ have been traversed, select the candidate operation with the largest importance parameter β o in the candidate operation set of the remaining edges as the operation for this edge to obtain the streamlined feature fusion supernetwork.

[0132] According to the β of each edge e value, select the edge with the smallest importance for removal. The removal order is from smallest to largest according to the edge importance parameter. After removing each edge, check whether the network still meets the computational complexity constraint Cmax and the accuracy threshold requirements. If removing this edge causes the computational complexity to exceed Cmax or the accuracy to be lower than the set threshold, then restore this edge, that is, cancel the removal operation. If removing this edge does not affect the accuracy or computational complexity, then retain this removal operation and continue to process the next edge. Repeat the above process until all edges with importance parameters β e less than the preset threshold θ have been processed. For each remaining edge, select the importance parameter β in the edge candidate operation seto The largest operation serves as the operation for that edge. Each operation in the candidate operation set has an importance parameter β o , indicating the degree of contribution of this operation to that edge. Through the above steps, all unimportant edges are removed, and the remaining edges together with the most important operation corresponding to each edge constitute the refined feature fusion hypernetwork. This refined network can keep the model's accuracy and computational complexity within an acceptable range while reducing redundant computations.

[0133] Furthermore, for the weights and activation values of each layer of the neural network in the pruned model, define a set of candidate numerical precisions, and dynamically adjust the numerical precision of the model according to the model accuracy constraint and computational complexity constraint, specifically including:

[0134] For the weights and activation values of each layer of the neural network in the pruned hazard detection model, define an optional set of candidate numerical precisions P = {}, where represents the i-th precision configuration, including FP32, FP16, INT8, and mixed precision;

[0135] Introduce a learnable importance parameter γ for the candidate precisions in the set of candidate numerical precisions for each layer. The dimension of the importance parameter γ is the same as the number of candidate precisions in the set of candidate numerical precisions;

[0136] In the forward propagation during the training phase, use Gumbel-Softmax to randomly sample the numerical precision of each layer of the neural network to obtain the importance parameter γ of the set of candidate numerical precisions of the neural network each time;

[0137] Only activate the candidate precision with the largest importance parameter γ sampled each time in the forward propagation as the precision configuration of the current neural network layer;

[0138] Gumbel-Softmax is a smooth approximation method that allows random sampling among discrete options and can perform gradient estimation during backpropagation. Through this method, the model can dynamically select the most appropriate numerical precision during training. Obtain the importance parameter γ: Each time of sampling, Gumbel-Softmax will perform random sampling based on the γ parameters of each precision configuration in the set of candidate numerical precisions of each layer to generate a probability distribution among different precision configurations. Select the precision configuration corresponding to the largest importance parameter: Each time of forward propagation, only activate the precision configuration with the largest importance parameter γ as the precision of the current neural network layer.

[0139] According to the model accuracy constraint and the computational complexity constraint, fix the current importance parameter γ, update the network parameter 𝑊 through the conventional backpropagation process. After freezing the network parameter 𝑊, use the gradient estimation of Gumbel-Softmax to update the importance parameter γ of the numerical accuracy candidate set;

[0140] Repeat the above process until both the network parameter 𝑊 and the importance parameter γ converge. Take the candidate accuracy with the largest importance parameter γ in the numerical accuracy candidate sets of each layer of the neural network as the accuracy configuration of the current neural network layer, and obtain the lightweight hidden danger detection model.

[0141] Furthermore, use Gumbel-Softmax to randomly sample the numerical accuracy of each layer of the neural network, and obtain the importance parameter γ of the numerical accuracy candidate set of the neural network each time. The formula is:

[0142] ,

[0143] where, is the importance parameter of the i-th accuracy configuration in the t-th training, is the importance parameter of the i-th accuracy configuration in the (t - 1)-th training, g i is the Gumbel noise, and τ is the temperature coefficient.

[0144] Embodiment 2.

[0145] This embodiment provides an electronic device, including a memory and a processor. A computer program is stored on the memory, and when the processor executes the program, it implements the method for detecting hidden dangers of transmission lines in a wind disaster scenario based on model lightweight as described in any one of the above.

[0146] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for detecting hidden dangers of transmission lines in a wind disaster scenario based on model lightweight as described in any one of the above.

[0147] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0148] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models, characterized in that: The following steps are involved: Construct a hidden danger detection supernetwork, set model accuracy constraints and computational complexity constraints, construct a candidate operation set for the hidden danger detection supernetwork, set importance parameters for the candidate operation set of the hidden danger detection supernetwork, and intelligently trim the hidden danger detection supernetwork based on the candidate operation set of the hidden danger detection supernetwork and its importance parameters under the constraints of model accuracy and computational complexity. Based on the weights and activation values of each layer of the neural network in the pruned hidden danger detection hypernetwork, a candidate set of numerical precision is defined. Based on the model precision constraints and computational complexity constraints, the numerical precision of the hidden danger detection hypernetwork is dynamically adjusted to obtain a lightweight hidden danger detection model. Specifically, the following steps are performed: For the weights and activation values of each layer of the neural network in the pruned hidden danger detection model, define an optional numerical precision candidate set P = { p 1 、 p 2 、 …、 p n },in p i Indicates the i-th precision configuration, including FP32, FP16, INT8, and mixed precision; Introducing a learnable importance parameter γ for the candidate precisions in the numerical precision candidate set of each layer, where the dimension of the importance parameter γ is the same as the number of candidate precisions in the numerical precision candidate set; In the forward propagation of the training phase, Gumbel-Softmax is used to randomly sample the numerical accuracy of each layer of the neural network to obtain the importance parameter γ of the numerical accuracy candidate set of each neural network; Each forward propagation only activates the candidate accuracy with the largest importance parameter γ obtained by sampling as the accuracy configuration of the current neural network layer; According to the model accuracy constraints and computational complexity constraints, the current importance parameter γ is fixed and the network parameters are updated through the conventional back propagation process. W , in freezing the network parameters W Finally, the gradient estimation of Gumbel-Softmax is used to update the importance parameter γ of the numerical precision candidate set; Repeat the above process until the network parameters W When the importance parameter γ converges, the candidate accuracy with the largest importance parameter γ in the candidate set of numerical accuracy of each layer of the neural network is used as the accuracy configuration of the current neural network layer to obtain a lightweight hidden danger detection model; The lightweight hidden danger detection model is deployed on the drone side, and the drone collects hidden danger image data of the transmission line. The hidden danger image data includes pictures of broken strands, pictures of foreign objects hanging on the wires, pictures of damaged poles and towers, and other hidden danger images. The hidden danger image data is input into the lightweight hidden danger detection model to realize the detection of transmission line hidden dangers in wind disaster scenarios.

2. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 1 is characterized in that: The hidden danger detection super network includes a skeleton super network and a feature fusion super network; The method of constructing a candidate operation set of the hidden danger detection supernetwork, setting importance parameters for the candidate operation set of the hidden danger detection supernetwork, and intelligently tailoring the hidden danger detection supernetwork based on the candidate operation set of the hidden danger detection supernetwork and its importance parameters under the constraints of model accuracy and computational complexity specifically includes: Construct a candidate operation set of the skeleton super-network, and intelligently trim the skeleton super-network based on the candidate operation set under the constraints of model accuracy and computational complexity; A candidate operation set of the feature fusion super-network is constructed, and the feature fusion super-network is intelligently trimmed based on the candidate operation set under the constraints of model accuracy and computational complexity.

3. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 1 is characterized in that: The model accuracy constraint is L(A, W) ≤ Lmax, where A represents the model architecture, W represents the network parameters trained under the given model architecture, L(A, W) is the error function of the model on the validation set, and Lmax is the preset maximum allowable error; The computational complexity constraint is C(A)≤Cmax, where C(A) includes the model's inference delay, floating-point operations per second, and parameter count. Cmax is the preset maximum allowable computational complexity, which is determined based on the computing resources on the drone side.

4. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 2 is characterized in that: The candidate operation set of the skeleton super-network is constructed, and the skeleton super-network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step A1: Construct a candidate operation set of the skeleton hypernetwork; Step A2: Assign an importance parameter to each candidate operation in the candidate operation set of the skeleton hypernetwork α i , α i Indicates the contribution of the i-th candidate operation to network performance; Step A3: Initialize the network parameter W of the skeleton hypernetwork and initialize the importance parameter of each candidate operation to 1; Step A4: Use the current network structure and network parameters W to perform forward propagation and calculate the output of each node. For each candidate operation o i , which outputs: ， Among them, i (X; W) is the calculation result of the i-th candidate operation based on the shared network parameters W, α i is the importance parameter of the i-th candidate operation, and X is the input feature of the current node; Step A5: Take the weighted sum of the outputs of all candidate operations to get the final output of the current layer: ， Where n is the number of candidate operations in the current layer; Step A6: Calculate the model error based on the current output through the loss function L(A, W), fix the importance parameters, and update the network parameters W through back propagation; Step A7: Fix the network parameters W and perform forward propagation to calculate the output of each candidate operation. Calculate the loss function L(A, W) based on the model output and the target output. Calculate the gradient of the loss function with respect to the importance parameter through backpropagation and update the importance parameter using gradient descent. Step A8: Repeat steps A6 and A7 until the network parameter W and importance parameter of the skeleton hypernetwork converge; Step A9: After training is completed, the skeleton super network is simplified based on the values of the importance parameters using a greedy algorithm under the model accuracy constraint L(A, W) and the computational complexity constraint C(A) to obtain the simplified skeleton super network.

5. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 4 is characterized in that: The skeleton super network includes downsampling blocks and regular blocks, and the candidate operation set of the skeleton super network includes a downsampling block operation candidate set, a regular block operation candidate set, and a connection mode candidate set between blocks; The candidate set of downsampling block operations includes 3x3 convolution, 5x5 convolution, 3x3 maximum pooling, 3x3 average pooling, and 3x3 dilated convolution; The conventional block operation candidate set includes 3x3 convolution, 1x1 convolution, depthwise separable convolution, and 3x3 maximum pooling; The candidate set of connection modes between the blocks includes direct connection operation and convolution operation.

6. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 4 is characterized in that: The method of simplifying the skeleton super-network based on the values of the importance parameters by a greedy algorithm under the constraints of model accuracy and computational complexity to obtain a simplified skeleton super-network specifically includes: The candidate operation with the smallest importance parameter in the candidate operation set of the skeleton super network is selected for the first removal operation. The updated network is checked to see if it still meets the set accuracy threshold and computational complexity constraints. If removing the operation causes the computational complexity to exceed Cmax or the accuracy to fall below the threshold, the operation is restored. Otherwise, the removal operation is continued. After traversing all candidate operations with importance parameters less than θ, the streamlined skeleton super network is output. The first removal operation includes a candidate operation of removing a module and a candidate operation of removing a connection mode between blocks; When the candidate action for the removed module is the last candidate action for removing the module, perform the following operations: Disconnect the input and output connections of the module, delete the connections between subsequent modules and the module, and delete the module and its related computing nodes; When the candidate operation of the connection mode between the removed blocks is the last candidate operation, the following operations are performed: disconnect the connection, delete the subsequent modules and related nodes that depend on the connection, and delete the connection edge.

7. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 2 is characterized in that: The candidate operation set of the feature fusion super-network is constructed, and the feature fusion super-network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step B1: Constructing a candidate operation set of a feature fusion supernetwork, wherein the feature fusion supernetwork includes multiple fusion blocks, wherein nodes in each fusion block represent feature graphs, nodes are connected by edges, and a feature graph is connected to all its predecessor nodes. The candidate operation set of the feature fusion supernetwork includes an edge candidate operation set; the edge candidate operation set includes 1x1 convolution, 3x3 convolution, 5x5 convolution, and 3x3 dilated convolution; Step B2: Assign an importance parameter β to each edge e , represents the importance of the edge, and assigns an importance parameter β to each candidate operation in the edge candidate operation set on each edge o , represents the importance of the candidate operation, and the feature map calculation process in each fusion block is expressed as: ， Among them, z j The output feature map of the jth fusion block in the feature fusion hypernetwork, Represents the feature graph x of the i-th predecessor node i The output after applying the candidate operation o, where r o is the output channel expansion rate corresponding to the candidate operation o, represents the importance parameter of edge (i,j), The importance parameter of the candidate operation o to the edge (i, j); Step B3: Calculate the model error through the loss function L(A, W) according to the output feature map, fix the importance parameters, and update the network parameters W of each fusion block in the feature fusion super network through back propagation; Step B4: Fix the network parameters W and perform forward propagation to calculate the output of each candidate operation. Calculate the loss function L(A, W) based on the model output and the target output. Calculate the gradient of the loss function with respect to the importance parameter through backpropagation and update the importance parameter using gradient descent. Step B5: Repeat steps B3 and B4 until the network parameter W and importance parameter of the feature fusion hypernetwork converge; Step B6: After the training is completed, the feature fusion super-network is streamlined based on the values of the importance parameters under the constraints of model accuracy and computational complexity to obtain a streamlined feature fusion super-network.

8. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 7 is characterized in that: The method of simplifying the feature fusion super-network based on the value of the importance parameter by a greedy algorithm under the constraints of model accuracy and computational complexity to obtain a simplified feature fusion super-network specifically includes: According to the edge importance parameter β of each edge e Perform a preliminary pruning of the edges in the feature fusion hypernetwork and select the edge with the least importance to remove. If the removal operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, the operation is resumed; otherwise, the removal operation is continued. When all importance parameters β are traversed, the edge with the least importance is removed. e After the edges are smaller than θ, the candidate operations of the retained edges are concentrated with the importance parameter β o The largest candidate operation is used as the operation of this edge to obtain a streamlined feature fusion super network.

9. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 1, characterized in that: The Gumbel-Softmax is used to randomly sample the numerical accuracy of each layer of the neural network to obtain the importance parameter γ of the numerical accuracy candidate set of each neural network. The formula is: ， in, is the importance parameter of the i-th accuracy configuration in the t-th training, is the importance parameter of the i-th accuracy configuration in the t-1-th training, g i is the Gumbel noise, and τ is the temperature coefficient.

Images