Power transmission line hidden danger detection method in wind disaster scene based on model lightweight

Abstract

The invention provides a power transmission line hidden danger detection method in a wind disaster scene based on model lightweight, and the method comprises the steps: constructing a hidden danger detection super network comprising a skeleton super network and a feature fusion super network, and optimizing a super network structure through intelligent cutting based on important parameters and selective removal of unimportant candidate operations. Secondly, for the weight and the activation value of each layer, a numerical precision candidate set is defined, random sampling and dynamic adjustment of numerical precision are carried out through a Gumbel-Softmax mechanism, and it is ensured that precision configuration of each layer of neural network can be adaptively optimized in the training process; and simplifying the skeleton super network and the feature fusion super network through a greedy algorithm to obtain a lightweight hidden danger detection model meeting the requirements of precision and calculation complexity. According to the method, the calculation overhead can be effectively reduced, the reasoning efficiency of the hidden danger detection model is improved, and the method is suitable for embedded equipment with limited resources and edge calculation scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of power transmission line detection, and specifically relates to a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models. Background Art

[0002] With the rapid development of the power system, the safe operation of transmission lines has become an important part of ensuring power supply. Transmission line inspection is a key measure to ensure line safety. It can timely detect and deal with hidden dangers that may cause power outages, such as broken wires, foreign objects hanging on the wires, loose hardware, etc. If these hidden dangers are not detected and repaired in time, they may cause safety accidents, resulting in serious economic losses and social impacts.

[0003] Traditional power transmission line inspection methods mainly rely on manual inspections and ground vehicles. These methods are not only time-consuming and costly, but also greatly increase the safety risks of manual inspections under severe weather conditions such as windstorms and heavy rains, and the efficiency is extremely low. With the development of technology, drone inspection technology has gradually become an important means of power transmission line inspection due to its high efficiency and low cost. Drones can quickly reach inaccessible areas and conduct comprehensive inspections of power transmission lines from the air, greatly improving the efficiency and safety of inspections.

[0004] However, drone inspections face many challenges in extreme weather conditions such as windstorms. First, strong winds can cause the stability of drones to decrease, affecting operational safety and the quality of image acquisition. The violent shaking of wires caused by increased wind speeds, as well as the vibration of the drone itself, often cause the collected images to appear blurry and smearing, seriously affecting the clarity of the image and the subsequent image processing effect. In addition, severe weather conditions such as rain, wind and snow can also cause image blur, or even completely block important transmission line components, making it difficult to accurately identify hidden dangers.

[0005] In addition, communication facilities may be damaged during a storm, affecting real-time data transmission between drones and control centers. Even if the drone successfully collects image data, it may not be able to transmit it back to the analysis center in time due to communication obstruction, delaying the processing of hidden dangers. For the drone side, its computing resources are usually limited, making it difficult to process complex image analysis and hidden danger identification algorithms, which further increases the difficulty of implementing effective hidden danger detection in complex environments. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models.

[0007] The purpose of the present invention can be achieved by the following technical solutions: On the one hand, the present invention provides a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models, comprising the following steps: Construct a hidden danger detection super network, set model accuracy constraints and computational complexity constraints, construct a candidate operation set of the hidden danger detection super network, set importance parameters for the candidate operation set of the hidden danger detection super network, and intelligently trim the hidden danger detection super network based on the candidate operation set of the hidden danger detection super network and its importance parameters under the constraints of model accuracy and computational complexity; According to the weights and activation values ​​of each layer of the neural network in the pruned hidden danger detection super network, a candidate set of numerical precision is defined. According to the model precision constraint and computational complexity constraint, the numerical precision of the hidden danger detection super network is dynamically adjusted to obtain a lightweight hidden danger detection model. The lightweight hidden danger detection model is deployed on the drone side. The drone collects hidden danger image data of the transmission line, including broken strand images, images of foreign objects hanging on the wires, images of damaged poles and towers, and other hidden danger images. The hidden danger image data is input into the lightweight hidden danger detection model to realize the detection of hidden dangers of transmission lines in wind disaster scenarios.

[0008] Furthermore, the hidden danger detection super network includes a skeleton super network and a feature fusion super network; The method of constructing a candidate operation set of the hidden danger detection supernetwork, setting importance parameters for the candidate operation set of the hidden danger detection supernetwork, and intelligently trimming the hidden danger detection supernetwork based on the candidate operation set of the hidden danger detection supernetwork and its importance parameters under the constraints of model accuracy and computational complexity specifically includes: Construct a candidate operation set of the skeleton super network, and intelligently trim the skeleton super network based on the candidate operation set under the constraints of model accuracy and computational complexity; A candidate operation set of the feature fusion hypernetwork is constructed, and the feature fusion hypernetwork is intelligently trimmed based on the candidate operation set under the constraints of model accuracy and computational complexity.

[0009] Furthermore, the model accuracy constraint is L(A, W) ≤Lmax, where A represents the model architecture, W represents the network parameters trained under the given model architecture, L(A, W) is the error function of the model on the validation set, and Lmax is the preset maximum allowable error; The computational complexity constraint is C(A)≤Cmax, where C(A) includes the model's inference delay, floating-point operations per second, and parameter quantity, and Cmax is the preset maximum allowable computational complexity, which is determined based on the computing resources on the drone side.

[0010] Furthermore, the candidate operation set of the skeleton super network is constructed, and the skeleton super network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step A1: construct a candidate operation set of the skeleton hypernetwork; Step A2: Assign an importance parameter to each candidate operation in the candidate operation set of the skeleton hypernetwork α i , α i Indicates the contribution of the i-th candidate operation to network performance; Step A3: Initialize the network parameter W of the skeleton hypernetwork and initialize the importance parameter of each candidate operation to 1; Step A4: Use the current network structure and network parameters W to perform forward propagation and calculate the output of each node. For each candidate operation o i , the output is: , Among them, i (X; W) is the calculation result of the i-th candidate operation based on the shared network parameters W, α i is the importance parameter of the i-th candidate operation, and X is the input feature of the current node; Step A5: Weighted sum of the outputs of all candidate operations to get the final output of the current layer: , Where n is the number of candidate operations in the current layer; Step A6: Calculate the model error through the loss function L(A,W) according to the current output, fix the importance parameters, and update the network parameters W through back propagation; Step A7: Fix the network parameters W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) based on the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through back propagation, and update the importance parameter using gradient descent; Step A8: Repeat steps A6 and A7 until the network parameter W and importance parameter of the skeleton hypernetwork converge; Step A9: After the training is completed, the skeleton hypernetwork is simplified based on the value of the importance parameter through a greedy algorithm under the model accuracy constraint L(A, W) and the computational complexity constraint C(A) to obtain a simplified skeleton hypernetwork.

[0011] Further, the skeleton super network includes downsampling blocks and regular blocks, and the candidate operation set of the skeleton super network includes a candidate set of downsampling block operations, a candidate set of regular block operations, and a candidate set of connection modes between blocks; The candidate set of downsampling block operations includes 3x3 convolution, 5x5 convolution, 3x3 maximum pooling, 3x3 average pooling, and 3x3 dilated convolution; The conventional block operation candidate set includes 3x3 convolution, 1x1 convolution, depthwise separable convolution, and 3x3 maximum pooling; The candidate set of connection modes between the blocks includes a direct connection operation and a convolution operation.

[0012] Furthermore, the skeleton super-network is simplified by a greedy algorithm based on the value of the importance parameter under the constraints of model accuracy and computational complexity to obtain a simplified skeleton super-network, specifically including: Select the candidate operation with the smallest importance parameter in the candidate operation set of the skeleton super network for the first removal operation, check whether the updated network still meets the set accuracy threshold and computational complexity constraints, if removing the operation causes the computational complexity to exceed Cmax or the accuracy is lower than the threshold, then restore the operation; otherwise, continue to remove the operation, and after traversing all candidate operations with importance parameters less than θ, output the streamlined skeleton super network; The first removal operation includes a candidate operation of removing a module and a candidate operation of removing a connection mode between blocks; When the candidate action of the removed module is the last candidate action of the removed module, perform the following operations: Disconnect the input and output connections of the module, delete the connections between the subsequent modules and the module, and delete the module and its related computing nodes; When the candidate operation of the connection mode between the removed blocks is the last candidate operation, the following operations are performed: disconnect the connection, delete the subsequent modules and related nodes that depend on the connection, and delete the connection edge.

[0013] Furthermore, the candidate operation set of the feature fusion super network is constructed, and the feature fusion super network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step B1: constructing a candidate operation set of a feature fusion supernetwork, wherein the feature fusion supernetwork includes a plurality of fusion blocks, the nodes in each fusion block represent feature graphs, the nodes are connected to each other through edges, a feature graph is connected to all its predecessor nodes, and the candidate operation set of the feature fusion supernetwork includes an edge candidate operation set; the edge candidate operation set includes 1x1 convolution, 3x3 convolution, 5x5 convolution, and 3x3 dilated convolution; Step B2: Assign an importance parameter β to each edge e , represents the importance of the edge, and assigns an importance parameter β to each candidate operation in the edge candidate operation set on each edge o , represents the importance of the candidate operation, and the feature map calculation process in each fusion block is expressed as: , Among them, z jThe output feature map of the jth fusion block in the feature fusion hypernetwork, Represents the feature graph x of the i-th predecessor node i The output after applying the candidate operation o, where r o is the output channel expansion rate corresponding to the candidate operation o, represents the importance parameter of edge (i,j), Represents the importance parameter of candidate operation o to edge (i, j); Step B3: Calculate the model error through the loss function L(A,W) according to the output feature map, fix the importance parameters, and update the network parameters W of each fusion block in the feature fusion super network through back propagation; Step B4: Fix the network parameter W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) based on the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through back propagation, and update the importance parameter using gradient descent; Step B5: Repeat steps B3 and B4 until the network parameter W and the importance parameter of the feature fusion hypernetwork converge; Step B6: After the training is completed, the feature fusion super network is streamlined according to the values ​​of the importance parameters based on the model accuracy constraints and the computational complexity constraints to obtain a streamlined feature fusion super network.

[0014] Furthermore, the feature fusion super-network is simplified based on the value of the importance parameter by a greedy algorithm under the constraints of model accuracy and computational complexity to obtain a simplified feature fusion super-network, specifically including: According to the edge importance parameter β of each edge e Perform a preliminary pruning of the edges in the feature fusion hypernetwork and select the edge with the least importance to remove. If the removal of this operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, the operation is restored; otherwise, the removal operation is continued. When all importance parameters β are traversed, e After the edges are smaller than θ, the importance parameter β is set for the candidate operations of the retained edges o The largest candidate operation is used as the operation of this edge, and a streamlined feature fusion super network is obtained.

[0015] Furthermore, the method defines a candidate set of numerical precision for the weights and activation values ​​of each layer of the neural network in the pruned model, and dynamically adjusts the numerical precision of the model according to the model precision constraint and the computational complexity constraint, specifically including: For the weights and activation values ​​of each layer of the neural network in the pruned hidden danger detection model, define an optional numerical precision candidate set P = { p 1 , p2 、 …、 p n},in p i Indicates the i-th precision configuration, including FP32, FP16, INT8, and mixed precision; Introducing a learnable importance parameter γ for the candidate precisions in the numerical precision candidate set of each layer, wherein the dimension of the importance parameter γ is the same as the number of candidate precisions in the numerical precision candidate set; In the forward propagation of the training phase, Gumbel-Softmax is used to randomly sample the numerical precision of each layer of the neural network to obtain the importance parameter γ of the numerical precision candidate set of each neural network; Each forward propagation only activates the candidate accuracy with the largest importance parameter γ obtained by sampling as the accuracy configuration of the current neural network layer; According to the model accuracy constraints and computational complexity constraints, the current importance parameter γ is fixed, and the network parameter 𝑊 is updated through the conventional back-propagation process. After freezing the network parameter 𝑊, the importance parameter γ of the numerical accuracy candidate set is updated using the gradient estimation of Gumbel-Softmax. Repeat the above process until the network parameter 𝑊 and the importance parameter γ converge, and take the candidate accuracy with the largest importance parameter γ in the candidate set of numerical accuracy of each layer of the neural network as the accuracy configuration of the current neural network layer to obtain a lightweight hidden danger detection model.

[0016] Furthermore, the numerical precision of each layer of the neural network is randomly sampled using Gumbel-Softmax to obtain the importance parameter γ of the numerical precision candidate set of each neural network, and the formula is: , in, is the importance parameter of the i-th accuracy configuration in the t-th training, is the importance parameter of the i-th accuracy configuration in the t-1-th training, g i is the Gumbel noise, and τ is the temperature coefficient.

[0017] Compared with the prior art, the present invention has the following advantages: (1) The present invention adopts the technology of intelligent trimming of model structure and dynamic control of numerical accuracy under dual constraints. On the premise of ensuring the accuracy of hidden danger detection, it successfully realizes a lightweight model by optimizing the computational complexity. It can quickly and efficiently perform hidden danger detection on the UAV side, significantly reducing the consumption of computing resources and improving the working efficiency of the UAV.

[0018] (2) The present invention uses the Gumbel-Softmax method to dynamically adjust the numerical precision of each layer of the neural network. By calculating the importance parameters of the temperature coefficient and the Gumbel noise, the flexible configuration of the numerical precision is achieved, so that the model can adjust the precision according to actual needs, thereby balancing the computing resources and precision requirements, and further improving the efficiency and adaptability of the detection model.

[0019] (3) In the process of streamlining the skeleton hypernetwork and the feature fusion hypernetwork, the present invention adopts a greedy algorithm. Based on the constraints of model accuracy and computational complexity, it simplifies unnecessary candidate operations and reduces computational complexity. At the same time, it ensures the accuracy requirements of the model and optimizes the computational efficiency of the model. It is suitable for the end-side deployment of UAVs with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] Example 1.

[0023] like Figure 1 As shown, the present invention provides a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models, comprising the following steps: Construct a hidden danger detection super network, set model accuracy constraints and computational complexity constraints, construct a candidate operation set of the hidden danger detection super network, set importance parameters for the candidate operation set of the hidden danger detection super network, and intelligently trim the hidden danger detection super network based on the candidate operation set of the hidden danger detection super network and its importance parameters under the constraints of model accuracy and computational complexity; According to the weights and activation values ​​of each layer of the neural network in the pruned hidden danger detection super network, a candidate set of numerical precision is defined. According to the model precision constraint and computational complexity constraint, the numerical precision of the hidden danger detection super network is dynamically adjusted to obtain a lightweight hidden danger detection model. The lightweight hidden danger detection model is deployed on the drone side. The drone collects hidden danger image data of the transmission line, including broken strand images, images of foreign objects hanging on the wires, images of damaged poles and towers, and other hidden danger images. The hidden danger image data is input into the lightweight hidden danger detection model to realize the detection of hidden dangers of transmission lines in wind disaster scenarios.

[0024] Furthermore, the hidden danger detection super network includes a skeleton super network and a feature fusion super network; The method of constructing a candidate operation set of the hidden danger detection supernetwork, setting importance parameters for the candidate operation set of the hidden danger detection supernetwork, and intelligently trimming the hidden danger detection supernetwork based on the candidate operation set of the hidden danger detection supernetwork and its importance parameters under the constraints of model accuracy and computational complexity specifically includes: Construct a candidate operation set of the skeleton super network, and intelligently trim the skeleton super network based on the candidate operation set under the constraints of model accuracy and computational complexity; A candidate operation set of the feature fusion hypernetwork is constructed, and the feature fusion hypernetwork is intelligently trimmed based on the candidate operation set under the constraints of model accuracy and computational complexity.

[0025] The hidden danger detection hypernetwork is the basic network structure of the entire model, which includes two main parts: Skeleton hypernetwork: The skeleton hypernetwork is the basic framework of the network, which contains some core operations (such as convolution operations, pooling operations, etc.) for processing input data.

[0026] Feature fusion hypernetwork: The feature fusion hypernetwork is mainly used to extract information from different feature maps and combine this information to improve the representation ability of the model. It is usually used to enhance the feature learning and fusion effects of the model.

[0027] Set model accuracy constraints and computational complexity constraints; In actual deployment, it is necessary to balance the model's accuracy (i.e., the accuracy of the detection results) and computational complexity (i.e., the computing resources required for model inference). The purpose of this step is to ensure that the model does not exceed the computing resource limit while meeting the accuracy requirements. Specifically, two constraints are set: Accuracy constraint: Ensure that the error of the model does not exceed the predetermined maximum allowable error (Lmax), that is, the error of the model output cannot be too large to ensure the performance of the model.

[0028] Computational complexity constraints: Ensure that the computational complexity of the model (including inference latency, floating-point operations per second, number of parameters, etc.) does not exceed the preset limit (Cmax). This prevents the model from being too large to run on devices such as drones when it is actually deployed.

[0029] Construct a candidate operation set of the skeleton super network, and intelligently trim the skeleton super network based on the candidate operation set under the constraints of model accuracy and computational complexity; For each layer in the skeleton hypernetwork, there are multiple operations to choose from. For example, the convolution layer may have convolution kernels of different sizes, the pooling layer may have different pooling methods, etc. The candidate operation set is the set of all these optional operations. According to the accuracy and computational complexity constraints, the most suitable operations are selected from the candidate operation set to be retained, and the other operations are removed. The process of intelligent pruning automatically selects which operations have a greater impact on the model accuracy and which have a greater impact on the computational complexity, and finally optimizes the structure of the model.

[0030] Construct a candidate operation set of the feature fusion super-network, and intelligently trim the feature fusion super-network based on the candidate operation set under the constraints of model accuracy and computational complexity; Each layer of the feature fusion network also has multiple operations to choose from, usually different types of convolution operations (such as 1x1 convolution, 3x3 convolution, etc.). The choice of these operations will affect the effect of the model when fusing different feature maps. Similar to the intelligent trimming of the skeleton hypernetwork, the feature fusion network also performs intelligent trimming based on the constraints of accuracy and computational complexity, selects the optimal operation for fusion, and removes unnecessary operations.

[0031] According to the weights and activation values ​​of each layer of the neural network in the pruned hidden danger detection hypernetwork, a candidate set of numerical precision is defined. According to the model precision constraints and computational complexity constraints, the numerical precision of the hidden danger detection hypernetwork is dynamically adjusted to obtain a lightweight hidden danger detection model.

[0032] Different numerical precisions (such as FP32, FP16, INT8, mixed precision, etc.) will affect the computational efficiency and accuracy of the model. Different precision configurations can be selected for the weights and activation values ​​of each layer of the neural network. By introducing importance parameters (such as γ), the numerical precision of each layer of the network is adjusted. During the training process, the network automatically selects the most appropriate precision configuration by randomly sampling the precision of each layer (using the Gumbel-Softmax method). This process helps reduce the computational burden while ensuring accuracy, allowing the model to run more efficiently.

[0033] Furthermore, the model accuracy constraint is L(A, W) ≤Lmax, where A represents the model architecture, W represents the network parameters trained under the given model architecture, L(A, W) is the error function of the model on the validation set, and Lmax is the preset maximum allowable error; The computational complexity constraint is C(A)≤Cmax, where C(A) includes the inference delay, floating-point operations per second, and parameter quantity of the model, and Cmax is a preset value determined based on the computing resources on the drone side.

[0034] Furthermore, the candidate operation set of the skeleton super network is constructed, and the skeleton super network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step A1: construct a candidate operation set of the skeleton hypernetwork; Step A2: Assign an importance parameter to each candidate operation in the candidate operation set of the skeleton hypernetwork α i , α i Indicates the contribution of the i-th candidate operation to network performance; Step A3: Initialize the network parameter W of the skeleton hypernetwork and initialize the importance parameter of each candidate operation to 1; For each candidate operation, an importance parameter is assigned α i , which indicates the contribution of the operation to network performance. In simple terms, the importance parameter α i It measures the impact of the operation on the final model performance. The initial value of the importance parameter is 1, which means that it is assumed that all candidate operations contribute equally to network performance at the beginning.

[0035] Step A4: Use the current network structure and network parameters W to perform forward propagation and calculate the output of each node. For each candidate operation o i , the output is: , Among them, i (X; W) is the calculation result of the i-th candidate operation based on the shared network parameters W, α i is the importance parameter of the i-th candidate operation, and X is the input feature of the current node; Step A5: Weighted sum of the outputs of all candidate operations to get the final output of the current layer: , Where n is the number of candidate operations in the current layer; Step A6: Calculate the model error through the loss function L(A,W) according to the current output, fix the importance parameters, and update the network parameters W through back propagation; Step A7: Fix the network parameters W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) based on the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through back propagation, and update the importance parameter using gradient descent; Step A8: Repeat steps A6 and A7 until the network parameter W and importance parameter of the skeleton hypernetwork converge; Further, the skeleton super network includes downsampling blocks and regular blocks, and the candidate operation set of the skeleton super network includes a candidate set of downsampling block operations, a candidate set of regular block operations, and a candidate set of connection modes between blocks; The candidate set of downsampling block operations includes 3x3 convolution, 5x5 convolution, 3x3 maximum pooling, 3x3 average pooling, and 3x3 dilated convolution; The conventional block operation candidate set includes 3x3 convolution, 1x1 convolution, depthwise separable convolution, and 3x3 maximum pooling; The candidate set of connection modes between the blocks includes a direct connection operation and a convolution operation.

[0036] Furthermore, the skeleton super-network is simplified by a greedy algorithm based on the value of the importance parameter under the constraints of model accuracy and computational complexity to obtain a simplified skeleton super-network, specifically including: Select the candidate operation with the smallest importance parameter in the candidate operation set of the skeleton super network for removal, check whether the updated network still meets the set accuracy threshold and computational complexity constraints, if removing the operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, then restore the operation; otherwise, continue to remove the operation, and after traversing all candidate operations with importance parameters less than θ, output the streamlined skeleton super network; The candidate operations for removal include candidate operations for removing modules and the connection mode between removal blocks; When the candidate operation of the removed module is the last candidate operation, perform the following operations: Disconnect the input and output connections of the module, delete the connections between the subsequent modules and the module, and delete the module and its related computing nodes; When the candidate operation of the connection mode between the removed blocks is the last candidate operation, the following operations are performed: disconnect the connection, delete the subsequent modules and related nodes that depend on the connection, and delete the connection edge.

[0037] The importance parameter is the contribution of each candidate operation to the network performance (accuracy). A smaller value means that the operation has less impact on the network. Therefore, the greedy algorithm will first select those operations with smaller contributions and remove them first to reduce the amount of calculation. After each candidate operation is removed, the updated network performance needs to be checked; The core goal of this part is to use a greedy algorithm to gradually remove unimportant operations and connections according to the importance parameters of each candidate operation, streamline the network structure, and thus reduce computational complexity. The entire process follows the following principles: only remove operations that contribute less to accuracy to ensure that the network accuracy is not lower than the set threshold; after removal, check whether the constraints of computational complexity are met to ensure that the network still has sufficient computational efficiency; when removing operations, ensure the connectivity and structural consistency of the network to avoid incoherent network structures.

[0038] Through these steps, the final streamlined skeleton hypernetwork can reduce the amount of calculation as much as possible while meeting the constraints of accuracy and computational complexity, and is suitable for application in resource-constrained environments.

[0039] After the training is completed, the skeleton hypernetwork is simplified according to the value of the importance parameter through a greedy algorithm based on the model accuracy constraint L(A, W) and the computational complexity constraint C(A) to obtain the simplified skeleton hypernetwork.

[0040] Furthermore, the candidate operation set of the feature fusion super network is constructed, and the feature fusion super network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step B1: constructing a candidate operation set of a feature fusion supernetwork, wherein the feature fusion supernetwork includes a plurality of fusion blocks, the nodes in each fusion block represent feature graphs, the nodes are connected to each other through edges, a feature graph is connected to all its predecessor nodes, and the candidate operation set of the feature fusion supernetwork includes an edge candidate operation set; the edge candidate operation set includes 1x1 convolution, 3x3 convolution, 5x5 convolution, and 3x3 dilated convolution; Step B2: Assign an importance parameter β to each edge e , represents the importance of the edge, and assigns an importance parameter β to each candidate operation in the edge candidate operation set on each edge o , represents the importance of the candidate operation, and the feature map calculation process in each fusion block is expressed as: , Among them, z j The output feature map of the jth fusion block in the feature fusion hypernetwork, Represents the feature graph x of the i-th predecessor node i The output after applying the candidate operation o, where r o is the output channel expansion rate corresponding to the candidate operation o, represents the importance parameter of edge (i,j), Represents the importance parameter of candidate operation o to edge (i, j); Step B3: Calculate the model error through the loss function L(A,W) according to the output feature map, fix the importance parameters, and update the network parameters W of each fusion block in the feature fusion super network through back propagation; Step B4: Fix the network parameter W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) based on the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through back propagation, and update the importance parameter using gradient descent; Step B5: Repeat steps B3 and B4 until the network parameter W and the importance parameter of the feature fusion hypernetwork converge; Step B6: After the training is completed, the feature fusion super network is streamlined according to the values ​​of the importance parameters based on the model accuracy constraints and the computational complexity constraints to obtain a streamlined feature fusion super network.

[0041] Furthermore, the feature fusion super-network is simplified based on the value of the importance parameter by a greedy algorithm under the constraints of model accuracy and computational complexity to obtain a simplified feature fusion super-network, specifically including: According to the edge importance parameter β of each edge e Perform a preliminary pruning of the edges in the feature fusion hypernetwork and select the edge with the least importance to remove. If the removal of this operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, the operation is restored; otherwise, the removal operation is continued. When all importance parameters β are traversed, e After the edges are smaller than θ, the importance parameter β of the candidate operations of the retained edges is concentrated o The largest candidate operation is used as the operation of this edge, and a streamlined feature fusion super network is obtained.

[0042] According to the β of each edge e The edge with the smallest importance is selected for removal. The order of removal is from small to large according to the importance parameter of the edge. After removing each edge, check whether the network still meets the computational complexity constraint Cmax and the accuracy threshold requirements. If removing the edge causes the computational complexity to exceed Cmax or the accuracy to fall below the set threshold, restore the edge, that is, cancel the removal operation. If removing the edge does not affect the accuracy or computational complexity, keep the removal operation and continue to process the next edge. Repeat the above process until all importance parameters β e All edges smaller than the preset threshold θ are processed. For each retained edge, the importance parameter β in the edge candidate operation set is selected. o The largest operation is taken as the operation of the edge. Each operation in the candidate operation set has an importance parameter β o, indicating the contribution of the operation to the edge. Through the above steps, all unimportant edges are removed, and the remaining edges and the most important operations corresponding to each edge together form a simplified feature fusion super network. This simplified network can keep the accuracy and computational complexity of the model within an acceptable range while reducing redundant calculations.

[0043] Furthermore, the method defines a candidate set of numerical precision for the weights and activation values ​​of each layer of the neural network in the pruned model, and dynamically adjusts the numerical precision of the model according to the model precision constraint and the computational complexity constraint, specifically including: For the weights and activation values ​​of each layer of the neural network in the pruned hidden danger detection model, define an optional numerical precision candidate set P = {}, where represents the i-th precision configuration, including FP32, FP16, INT8, and mixed precision; Introducing a learnable importance parameter γ for the candidate precisions in the numerical precision candidate set of each layer, wherein the dimension of the importance parameter γ is the same as the number of candidate precisions in the numerical precision candidate set; In the forward propagation of the training phase, Gumbel-Softmax is used to randomly sample the numerical precision of each layer of the neural network to obtain the importance parameter γ of the numerical precision candidate set of each neural network; Each forward propagation only activates the candidate accuracy with the largest importance parameter γ obtained by sampling as the accuracy configuration of the current neural network layer; Gumbel-Softmax is a smooth approximation method that allows random sampling among discrete options and enables gradient estimation during backpropagation. In this way, the model can dynamically select the most appropriate numerical precision during training. Get the importance parameter γ: Each time sampling, Gumbel-Softmax randomly samples the γ parameters of each precision configuration in the numerical precision candidate set of each layer, generating a probability distribution between different precision configurations. Select the precision configuration corresponding to the maximum importance parameter: Each time forward propagation, only the precision configuration with the largest importance parameter γ is activated as the precision of the current neural network layer.

[0044] According to the model accuracy constraints and computational complexity constraints, the current importance parameter γ is fixed, and the network parameter 𝑊 is updated through the conventional back-propagation process. After freezing the network parameter 𝑊, the importance parameter γ of the numerical accuracy candidate set is updated using the gradient estimation of Gumbel-Softmax. Repeat the above process until the network parameter 𝑊 and the importance parameter γ converge, and take the candidate accuracy with the largest importance parameter γ in the candidate set of numerical accuracy of each layer of the neural network as the accuracy configuration of the current neural network layer to obtain a lightweight hidden danger detection model.

[0045] Furthermore, the numerical precision of each layer of the neural network is randomly sampled using Gumbel-Softmax to obtain the importance parameter γ of the numerical precision candidate set of each neural network, and the formula is: , in, is the importance parameter of the i-th accuracy configuration in the t-th training, is the importance parameter of the i-th accuracy configuration in the t-1-th training, g i is the Gumbel noise, and τ is the temperature coefficient.

[0046] Example 2.

[0047] This embodiment provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the program, a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models as described in any one of the above.

[0048] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models as described in any one of the above is implemented.

[0049] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0050] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models, characterized in that: The following steps are involved: Construct a hidden danger detection super network, set model accuracy constraints and computational complexity constraints, construct a candidate operation set of the hidden danger detection super network, set importance parameters for the candidate operation set of the hidden danger detection super network, and intelligently trim the hidden danger detection super network based on the candidate operation set of the hidden danger detection super network and its importance parameters under the constraints of model accuracy and computational complexity; According to the weights and activation values ​​of each layer of the neural network in the pruned hidden danger detection super network, a candidate set of numerical precision is defined. According to the model precision constraint and computational complexity constraint, the numerical precision of the hidden danger detection super network is dynamically adjusted to obtain a lightweight hidden danger detection model. The lightweight hidden danger detection model is deployed on the drone side. The drone collects hidden danger image data of the transmission line, including broken strand images, images of foreign objects hanging on the wires, images of damaged poles and towers, and other hidden danger images. The hidden danger image data is input into the lightweight hidden danger detection model to realize the detection of hidden dangers of transmission lines in wind disaster scenarios.

2. According to the method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight model according to claim 1, it is characterized in that: The hidden danger detection super network includes a skeleton super network and a feature fusion super network; The method of constructing a candidate operation set of the hidden danger detection supernetwork, setting importance parameters for the candidate operation set of the hidden danger detection supernetwork, and intelligently trimming the hidden danger detection supernetwork based on the candidate operation set of the hidden danger detection supernetwork and its importance parameters under the constraints of model accuracy and computational complexity specifically includes: Construct a candidate operation set of the skeleton super network, and intelligently trim the skeleton super network based on the candidate operation set under the constraints of model accuracy and computational complexity; A candidate operation set of the feature fusion hypernetwork is constructed, and the feature fusion hypernetwork is intelligently trimmed based on the candidate operation set under the constraints of model accuracy and computational complexity.

3. According to the method of detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight model in claim 1, it is characterized in that: The model accuracy constraint is L(A, W) ≤ Lmax, where A represents the model architecture, W represents the network parameters trained under the given model architecture, L(A, W) is the error function of the model on the validation set, and Lmax is the preset maximum allowable error; The computational complexity constraint is C(A)≤Cmax, where C(A) includes the model's inference delay, floating-point operations per second, and parameter quantity, and Cmax is the preset maximum allowable computational complexity, which is determined based on the computing resources on the drone side.

4. According to claim 2, a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models is characterized in that: The candidate operation set of the skeleton super network is constructed, and the skeleton super network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step A1: construct a candidate operation set of the skeleton hypernetwork; Step A2: Assign an importance parameter to each candidate operation in the candidate operation set of the skeleton hypernetwork α i , α i Indicates the contribution of the i-th candidate operation to network performance; Step A3: Initialize the network parameter W of the skeleton hypernetwork and initialize the importance parameter of each candidate operation to 1; Step A4: Use the current network structure and network parameters W to perform forward propagation and calculate the output of each node. For each candidate operation o i , the output is: ， Among them, i (X; W) is the calculation result of the i-th candidate operation based on the shared network parameters W, α i is the importance parameter of the i-th candidate operation, and X is the input feature of the current node; Step A5: Weighted sum of the outputs of all candidate operations to get the final output of the current layer: ， Where n is the number of candidate operations in the current layer; Step A6: Calculate the model error through the loss function L(A,W) according to the current output, fix the importance parameters, and update the network parameters W through back propagation; Step A7: Fix the network parameters W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) based on the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through back propagation, and update the importance parameter using gradient descent; Step A8: Repeat steps A6 and A7 until the network parameter W and importance parameter of the skeleton hypernetwork converge; Step A9: After the training is completed, the skeleton hypernetwork is simplified based on the model accuracy constraint L(A, W) and the computational complexity constraint C(A) through a greedy algorithm according to the value of the importance parameter to obtain a simplified skeleton hypernetwork.

5. According to claim 4, a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models is characterized in that: The skeleton super network includes downsampling blocks and regular blocks, and the candidate operation set of the skeleton super network includes a downsampling block operation candidate set, a regular block operation candidate set, and a connection mode candidate set between blocks; The candidate set of downsampling block operations includes 3x3 convolution, 5x5 convolution, 3x3 maximum pooling, 3x3 average pooling, and 3x3 dilated convolution; The conventional block operation candidate set includes 3x3 convolution, 1x1 convolution, depthwise separable convolution, and 3x3 maximum pooling; The candidate set of connection modes between the blocks includes a direct connection operation and a convolution operation.

6. According to claim 4, a method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models is characterized in that: The method of simplifying the skeleton super network according to the value of the importance parameter by a greedy algorithm based on the model accuracy constraint and the computational complexity constraint to obtain a simplified skeleton super network specifically includes: Select the candidate operation with the smallest importance parameter in the candidate operation set of the skeleton super network for the first removal operation, check whether the updated network still meets the set accuracy threshold and computational complexity constraints, if removing the operation causes the computational complexity to exceed Cmax or the accuracy is lower than the threshold, then restore the operation; otherwise, continue to remove the operation, and after traversing all candidate operations with importance parameters less than θ, output the streamlined skeleton super network; The first removal operation includes a candidate operation of removing a module and a candidate operation of removing a connection mode between blocks; When the candidate action of the removed module is the last candidate action of the removed module, perform the following operations: Disconnect the input and output connections of the module, delete the connections between the subsequent modules and the module, and delete the module and its related computing nodes; When the candidate operation of the connection mode between the removed blocks is the last candidate operation, the following operations are performed: disconnect the connection, delete the subsequent modules and related nodes that depend on the connection, and delete the connection edge.

7. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 2 is characterized in that: The candidate operation set of the feature fusion super network is constructed, and the feature fusion super network is intelligently tailored based on the candidate operation set under the constraints of model accuracy and computational complexity, specifically including: Step B1: constructing a candidate operation set of a feature fusion supernetwork, wherein the feature fusion supernetwork includes a plurality of fusion blocks, the nodes in each fusion block represent feature graphs, the nodes are connected to each other through edges, a feature graph is connected to all its predecessor nodes, and the candidate operation set of the feature fusion supernetwork includes an edge candidate operation set; the edge candidate operation set includes 1x1 convolution, 3x3 convolution, 5x5 convolution, and 3x3 dilated convolution; Step B2: Assign an importance parameter β to each edge e , represents the importance of the edge, and assigns an importance parameter β to each candidate operation in the edge candidate operation set on each edge o , represents the importance of the candidate operation, and the feature map calculation process in each fusion block is expressed as: ， Among them, z j The output feature map of the jth fusion block in the feature fusion hypernetwork, Represents the feature graph x of the i-th predecessor node i The output after applying the candidate operation o, where r o is the output channel expansion rate corresponding to the candidate operation o, represents the importance parameter of edge (i,j), Represents the importance parameter of candidate operation o to edge (i, j); Step B3: Calculate the model error through the loss function L(A,W) according to the output feature map, fix the importance parameters, and update the network parameters W of each fusion block in the feature fusion super network through back propagation; Step B4: Fix the network parameter W, perform forward propagation, calculate the output of each candidate operation, calculate the loss function L(A, W) based on the model output and the target output, calculate the gradient of the loss function with respect to the importance parameter through back propagation, and update the importance parameter using gradient descent; Step B5: Repeat steps B3 and B4 until the network parameter W and the importance parameter of the feature fusion hypernetwork converge; Step B6: After the training is completed, the feature fusion super network is streamlined according to the values ​​of the importance parameters based on the model accuracy constraints and the computational complexity constraints to obtain a streamlined feature fusion super network.

8. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 7 is characterized in that: The method of simplifying the feature fusion super network based on the value of the importance parameter by a greedy algorithm under the constraints of model accuracy and computational complexity to obtain a simplified feature fusion super network specifically includes: According to the edge importance parameter β of each edge e Perform a preliminary pruning of the edges in the feature fusion hypernetwork and select the edge with the least importance to remove. If the removal of this operation causes the computational complexity to exceed Cmax or the accuracy to be lower than the threshold, the operation is restored; otherwise, the removal operation is continued. When all importance parameters β are traversed, e After the edges are smaller than θ, the importance parameter β of the candidate operations of the retained edges is concentrated o The largest candidate operation is used as the operation of this edge, and a streamlined feature fusion super network is obtained.

9. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 1 is characterized in that: The method of defining a candidate set of numerical precision for the weights and activation values ​​of each layer of the neural network in the pruned model and dynamically adjusting the numerical precision of the model according to the model precision constraint and the computational complexity constraint specifically includes: For the weights and activation values ​​of each layer of the neural network in the pruned hidden danger detection model, define an optional numerical precision candidate set P = { p 1 , p 2 、 …、 p n },in p i Indicates the i-th precision configuration, including FP32, FP16, INT8, and mixed precision; Introducing a learnable importance parameter γ for the candidate precisions in the numerical precision candidate set of each layer, wherein the dimension of the importance parameter γ is the same as the number of candidate precisions in the numerical precision candidate set; In the forward propagation of the training phase, Gumbel-Softmax is used to randomly sample the numerical precision of each layer of the neural network to obtain the importance parameter γ of the numerical precision candidate set of each neural network; Each forward propagation only activates the candidate accuracy with the largest importance parameter γ obtained by sampling as the accuracy configuration of the current neural network layer; According to the model accuracy constraints and computational complexity constraints, the current importance parameter γ is fixed, and the network parameter 𝑊 is updated through the conventional back-propagation process. After freezing the network parameter 𝑊, the importance parameter γ of the numerical accuracy candidate set is updated using the gradient estimation of Gumbel-Softmax. Repeat the above process until the network parameter 𝑊 and the importance parameter γ converge, and take the candidate accuracy with the largest importance parameter γ in the candidate set of numerical accuracy of each layer of the neural network as the accuracy configuration of the current neural network layer to obtain a lightweight hidden danger detection model.

10. The method for detecting hidden dangers of power transmission lines in wind disaster scenarios based on lightweight models according to claim 9, characterized in that: The Gumbel-Softmax is used to randomly sample the numerical precision of each layer of the neural network to obtain the importance parameter γ of the numerical precision candidate set of each neural network, and the formula is: ， in, is the importance parameter of the i-th accuracy configuration in the t-th training, is the importance parameter of the i-th accuracy configuration in the t-1-th training, g i is the Gumbel noise, and τ is the temperature coefficient.

Images