Insulator detection method
By using multi-directional attention polymerization network and network lightweight pruning technology in the insulator detection algorithm, the problem of poor insulator string target recognition effect in complex environments is solved, and the insulator detection effect with high accuracy and low complexity is achieved.
Patent Information
- Application Number
- CN202411584621.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2025-03-14
AI Technical Summary
The existing insulator detection algorithm has poor recognition effect in complex environments, making it difficult to accurately identify insulator strings with large aspect ratios and insulator string targets with multiple poses.
The multi-directional attention aggregation network and network lightweight pruning technology are used to construct an insulator detection network model, so that it pays attention to the insulator target characteristics in a local-global-local way, reducing the complexity of the model.
Effectively detect multi-pose insulator string targets in complex environments, reduce errors in subsequent matching calculations, improve detection accuracy, and reduce model complexity for practical applications.
Smart Images
Figure CN119625056A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of insulator detection, and in particular to an insulator detection method. Background Art
[0002] The purpose of insulator detection is to identify the specific location of insulators in complex drone inspection images to provide a technical prerequisite for subsequent defect and fault detection. At present, the detection methods of insulators are mainly divided into traditional machine learning methods and deep learning methods. The two methods have different focuses. The traditional method focuses on the extraction of manual features for identification and positioning. Traditional methods have problems such as huge development costs, large computational and training costs. The deep learning-based method avoids the construction of a large number of artificial features, saves a lot of energy, has good recognition effect and high efficiency, and is also the current mainstream detection solution. Most of the existing deep learning-based insulator detection algorithms are extensions of conventional target detection, and there are still certain limitations for the detection of specific insulator targets.
[0003] The existing algorithms have the following deficiencies when completing the insulator detection task: (1) the recognition effect of insulator targets in complex environments is poor, and the recognition accuracy of insulator strings with large aspect ratios is low; (2) insulator strings with multiple postures cannot be effectively recognized and accurately locked in the anchor frame. Therefore, it is urgent to provide an insulator detection method to solve the problems that the existing target detection methods cannot accurately recognize insulator strings with multiple postures, have poor recognition effects on insulator targets in complex environments, and have low recognition accuracy for insulator strings with large aspect ratios when completing the insulator target detection task. Summary of the invention
[0004] The purpose of the present invention is to solve the technical problems in the above background and to provide an insulator detection method, comprising the following steps:
[0005] S1. Construct a multi-pose insulator string target detection dataset;
[0006] S2, constructing an insulator detection network model;
[0007] S3, select appropriate locations for network pruning to make the insulator detection network model lightweight;
[0008] S4, training the insulator detection network model;
[0009] S5. Detect the insulator string target to be detected, input the image containing the insulator string target into the trained insulator detection network model, and output the detection result of each circumscribed rectangular frame position where the insulator string target is located in the image.
[0010] In the preferred solution, the insulator detection network model uses the Drn-c42 backbone network to extract insulator string image features from drone inspection images.
[0011] In the preferred solution, a multi-directional attention aggregation network is used to encode image features, and the insulator detection network model is made to further focus on the feature information of the insulator string target in a local-global-local manner.
[0012] In a preferred solution, the insulator detection network model represents the insulator string target with a chain structure, and the rotating bounding box locates the insulator string.
[0013] In the preferred scheme, the insulator detection network model takes a color image as input, generates the two-dimensional position of the key points of the insulator string as output, and regards the insulator string as a chain, which is composed of several key points. The key points on the insulator string are defined by a bottom-up rule, the bottom key point is defined as the first, and then the remaining key points are arranged in sequence to the last one; in order to make the chain structure effectively represent the detected insulator string, the insulator detection network model is constructed as a double-stream structure, and finally a point chain structure is constructed according to the assembly algorithm in the human body posture estimation method to represent the detected insulator string.
[0014] In the preferred scheme, the insulator detection network model takes a w×h color image as input and generates the two-dimensional position of the key points of the insulator string as output; assuming that there are M insulator strings in the input image, the mth insulator string is regarded as a chain, which is connected by several key points, and the key points on the insulator string are defined by a bottom-up rule, the bottom key point is defined as the first, and then the remaining key points are arranged in sequence to the last; in order to make the chain structure effectively represent the detected insulator string, the insulator detection network model is constructed as a double-stream structure, and finally a point chain structure is constructed according to the assembly algorithm in the human body posture estimation method to represent the detected insulator string.
[0015] In a preferred solution, the dual-flow structure includes:
[0016] Keypoints Estimation stream, used to predict the two-dimensional confidence map K of the insulator key points;
[0017] Linkages Estimation stream, used to predict the two-dimensional vector field L of affinity between key points, encoding the degree of association between key points;
[0018] Set K = {K1, K2, K3...K J There are J confidence maps, K of which j ∈R w×h ,j∈{1...J}, set L={L1,L2,L3...LC}There are C vector fields, among which L c ∈R w×h×2 ,c∈{1...C},L C Each image position in encodes a 2D vector;
[0019] For each insulator string, the starting and ending positions of the string are defined as k1 and k j , the width of the string is w s , the length of the detection bounding box is determined by k1 and k j The distance between them is determined, and the width of the bounding box is determined by the vertical extension of the chain structure;
[0020] In order to reduce the difficulty of model convergence, a Gaussian function is used to define the true value key point feature K * :
[0021]
[0022] in represents the true coordinates of the jth key point, and the hyperparameter σ 2 represents the Gaussian variance; at the same time, the label of the chain Defined as:
[0023]
[0024] Among them, k i and k i+1 are the coordinates of two key points near point p, represents the true value of point p in the jth link, and P on the insulator string represents w s ≤2||pk i+1 ||2·sinθ≤w s , θ represents the vector pk i+1 and k i -k i+1 The angle between.
[0025] In the preferred solution, feature encoding is a multi-stage refinement process, divided into two parallel convolutional network branches, including:
[0026] Keypoints Estimation stream, used to encode key points on the insulator string;
[0027] Linkages Estimation stream, used to encode the chain structure between key points;
[0028] The encoding network consists of several 3×3 and 1×1 convolutional networks. In the first stage, the network estimates K 1 =ρ1 (F) and L 1 =φ 1 (F), where ρ 1 and φ 1 This is the convolutional neural network used for reasoning in the first stage. In each subsequent stage, the prediction results of the two branches in the previous stage are fused with the original image features to produce more accurate predictions:
[0029]
[0030] where ρ t and φ t is the convolutional neural network for inference at stage t;
[0031] In order to guide the network to iteratively predict the confidence map of the key points and affinity fields of each branch, the L2 loss function is applied at the end of each stage:
[0032]
[0033] Among them, p represents the position of the pixel, f t represents the loss at stage t, W is a binary mask where W(p) = 0 when an annotation is missing at image position p. The binary mask is used to avoid penalizing true predictions during training. The intermediate supervision at each stage solves the gradient vanishing problem by regularly replenishing the gradient. Therefore, the overall loss function is:
[0034]
[0035] Where T represents the total number of stages.
[0036] In the preferred solution, the multi-directional attention aggregation network is specifically:
[0037] Firstly, CA attention is used to generate a feature map with channel, direction and position awareness and then applied to the input feature map complementarily to enhance the representation of the insulator target;
[0038] Then the self-attention mechanism is calculated on a subset of the input tokens while preserving the underlying information exchange of the global-local context;
[0039] The self-attention mechanism is decomposed into consecutive modules and introduces local aggregation, global sparse attention and local propagation.
[0040] In the preferred scheme, local aggregation only integrates local aggregation from local approximate Token signals; for each Token, Depth-wise and Point-wise convolutions are used to aggregate information in a local window of size k×k;
[0041] Global sparse attention models the long-term relationship between a set of representative tokens, where each token is regarded as a representative of a local window; a sparse representative token set uniformly distributed in space is sampled, with one representative token for each r×r window, where r represents the sub-sampling rate, and then the self-attention mechanism is applied only to the selected tokens;
[0042] Local propagation diffuses the global context information learned by the delegate to non-representative tokens with the same window. The final multi-faceted attention information aggregation is expressed as:
[0043]
[0044] Where X in ∈R H×W×C represents the input tensor, CA represents Coordinate Attention, LocalAgg represents the local aggregation operator, FFN is a two-layer perceptron, GlobalSparseAttn is the global sparse self-attention, and Localprop is the local propagation operation.
[0045] In the preferred solution, the network pruning operation in step S3 is specifically: removing the last three keypointsEstimation modules and the last two LinkagesEstimation modules.
[0046] The beneficial effects of the present invention are:
[0047] (1) The multi-directional attention aggregation network and network lightweight pruning adopted by the method proposed in the present invention can make the network model pay more attention to the characteristic information of the insulator target from a local-global-local manner, suppress irrelevant information, and make the network model more lightweight, which is conducive to practical application and can effectively detect multi-posture insulator string targets in complex environments.
[0048] (2) The algorithm provided by the present invention can tightly encapsulate the multi-posture insulator string target in the target detection frame, reducing the error of subsequent matching calculation.
[0049] (3) The algorithm framework proposed in this invention can be more easily extended to string target detection tasks in other industrial fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 The overall framework diagram of the insulator detection network with attention and network pruning proposed in the present invention.
[0051] Figure 2 This is the algorithm flow chart.
[0052] Figure 3This is the structure diagram of the multi-directional attention aggregation network.
[0053] Figure 4 Diagram of the key steps for local-global-local information aggregation. DETAILED DESCRIPTION
[0054] The present invention is further described below in conjunction with the accompanying drawings.
[0055] An insulator detection method based on multi-directional attention and network pruning, comprising:
[0056] Step 1: Construct a multi-pose insulator string target detection dataset.
[0057] Step 2: Construct an insulator detection network model with multi-directional attention and network pruning;
[0058] like Figure 1 , 2 As shown, the insulator detection network model of azimuth attention and network pruning described in the present invention uses the Drn-c42 backbone network to extract the insulator string image features from the drone inspection image; then, a multi-azimuth attention aggregation network is used to refine the image features and make the network model pay more attention to the feature information of the insulator target in a local-global-local manner; in addition, in order to reduce the complexity of the model while taking into account the model performance, a suitable position is selected for network pruning to make the network model more lightweight and conducive to practical application. Finally, the network model can effectively characterize the insulator target with a chain structure, and the rotating bounding box can accurately locate the insulator.
[0059] The point chain representation and feature encoding network of the insulator are as follows:
[0060] Point chain representation of insulators: The proposed network model takes a w×h color image as input and generates the two-dimensional position of the key points of the insulator string as output. Assuming that there are M insulator strings in the input image, we regard the mth insulator string as a chain, which is linked by multiple key points. We define the key points on the insulator string according to the bottom-up rule. The bottom key point is defined as the first, and then the remaining key points are arranged in order to the last one. In order to make the chain structure effectively represent the detected insulator string, we construct the network model as a two-stream structure. The keypoints Estimation stream is used to predict the two-dimensional confidence map K of the insulator key points, and the Linkages Estimation stream is used to predict the two-dimensional vector field L of the affinity between key points, which encodes the degree of association between key points. Set K = {K1, K2, K3...K J There are J confidence maps, K of which j ∈R w×h,j∈{1...J}. The set L={L1,L2,L3...L C}There are C vector fields, among which L c ∈R w×h×2 ,c∈{1...C},L C Each image position in encodes a 2D vector. Finally, according to the assembly algorithm proposed by the human posture estimation method, a point chain structure is constructed to represent the detected insulator string. The specific process is as follows: Figure 2 For each insulator string, we define the starting and ending positions of the string as k1 and k j , the width of the string is w s , the length of the detection bounding box is determined by k1 and k j The width of the bounding box is determined by the vertical expansion of the chain structure. In addition, in order to reduce the difficulty of model convergence, we use a Gaussian function to define the true value key point feature K * :
[0061]
[0062] in represents the true coordinates of the jth key point, and the hyperparameter σ 2 represents the Gaussian variance. At the same time, the label of the chain Defined as
[0063]
[0064] Among them, k i and k i+1 are the coordinates of two key points near point p, represents the true value of point p in the jth link. P on the insulator string represents w s ≤2||pk i+1 ||2·sinθ≤w s , θ represents the vector pk i+1 and k i -k i+1 The angle between.
[0065] Feature encoding network: The feature encoding network is a multi-stage refinement process, which is divided into two parallel convolutional network branches: the keypoints Estimation stream is used to encode the key points on the insulator string, and the LinkagesEstimation stream is used to encode the link structure between the key points. The encoding network is mainly composed of multiple 3×3 and 1×1 convolutional networks. In the first stage, the network estimates K 1 =ρ 1 (F) and L 1 =φ1 (F), where ρ 1 and φ 1 It is a convolutional neural network used for reasoning in the first stage. In each subsequent stage, the prediction results of the two branches in the previous stage are fused with the original image features to produce more accurate predictions.
[0066]
[0067]
[0068] where ρ t and φ t Convolutional neural network for inference at stage t.
[0069] In order to guide the network to iteratively predict the confidence map of the key points and affinity fields of each branch, we apply the L2 loss function at the end of each stage as follows:
[0070]
[0071] Among them, p represents the position of the pixel, f t Denotes the loss at stage t, and W is a binary mask where W(p) = 0 when an annotation is missing at image position p. This mask is used to avoid penalizing true predictions during training. Intermediate supervision at each stage solves the gradient vanishing problem by periodically replenishing the gradient. Therefore, the overall loss function expression is shown in the following formula:
[0072]
[0073] Where T represents the total number of stages.
[0074] The multi-directional attention aggregation network is specifically:
[0075] like Figure 3 , 4 As shown in the figure, the multi-directional information aggregation network first captures the perceptual information of the three aspects of channel, direction, and position; then it processes the spatial tokens in different ranges in the image in a local-global-local manner and retains the underlying information exchange of the global and local contexts. Ultimately, the overall network model pays more attention to the feature information of the insulator target, suppresses irrelevant information, and improves the accuracy of insulator target detection. In addition, the newly created multi-directional attention information aggregation network in this paper is an efficient and lightweight feature processing module. The entire network module effectively solves the problem of balancing model accuracy and model complexity.
[0076] The multi-directional attention information aggregation network first uses CA attention to generate feature maps with channel, direction and position awareness and applies it complementary to the input feature map to enhance the representation of the insulator target. Then, the self-attention mechanism (SA) is calculated on a subset of the input token, while retaining the underlying information exchange of the global-local context. This paper decomposes the self-attention mechanism into continuous modules and introduces three effective steps: local aggregation, global sparse attention, and local propagation.
[0077] Local aggregation: Only integrate local aggregations from local approximate token signals; for each token, use depth-wise and point-wise convolutions to aggregate information in a local window of size k×k. Global sparse attention: Model long-term relationships between a set of representative tokens, where each token is regarded as a representative of a local window; sample a sparse set of representative tokens uniformly distributed in space, with one representative token for each r×r window. Here r represents the subsampling rate, and then the self-attention mechanism is applied only to these selected tokens. Local propagation: Spread the global contextual information learned by the delegate to non-representative tokens with the same window. Therefore, the final multi-faceted attention information aggregation can be expressed as:
[0078] X1=CA(X in )
[0079] X=LocalAgg(Norm(X1))+X1
[0080] Y=FFN(Norm(X))+X
[0081] Z=Localprop(GlobalSparseAttn(Norm(Y)))+Y
[0082] X out =FFN(Norm(Z))+Z
[0083] Where X in ∈R H×W×Crepresents the input tensor, CA represents Coordinate Attention, LocalAgg represents the local aggregation operator, FFN is a two-layer perceptron, GlobalSparseAttn is the global sparse self-attention, Localprop is the local propagation operation, all of which can be implemented by commonly used and highly optimized operations in standard deep learning platforms.
[0084] The specific network lightweight pruning is as follows:
[0085] Most of the existing insulator detection methods are applications and extensions of traditional target detection methods, and have achieved good results in insulator detection accuracy. However, the promotion and application of algorithm models is affected by factors such as the large number of model parameters. Inspired by the network pruning operation, this paper selects appropriate locations for network pruning to reduce the complexity and number of parameters of the model without reducing the performance of the model. The keypointsEstimation flow and Linkages Estimation flow of the original network model have 6 stages. This mode is mainly designed for multiple key points under multiple targets. It is not completely applicable to the mode of 5 key points of the insulator string target in this task scenario. Multiple stages undoubtedly increase the algorithm complexity and model parameter amount of the network model. This paper prunes and optimizes the stage steps of the keypointsEstimation flow and Linkages Estimation flow. It is found that when the last 3 keypoints Estimation modules and the last 2 Linkages Estimation modules are removed, the network model still has good detection accuracy, and the network complexity and parameter amount have been significantly reduced.
[0086] Step 3: Train the insulator detection network with multi-directional attention and network pruning
[0087] Step 4: Detect the insulator string target to be detected
[0088] Input the image containing the insulator string target into the trained network and output the detection results of each circumscribed rectangular box position where the insulator string target is located in the image.
[0089] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An insulator detection method, characterized in that: The method comprises the following steps: S1. Construct a multi-pose insulator string target detection dataset; S2, constructing an insulator detection network model; S3, select appropriate locations for network pruning to make the insulator detection network model lightweight; S4, training the insulator detection network model; S5. Detect the insulator string target to be detected, input the image containing the insulator string target into the trained insulator detection network model, and output the detection result of each circumscribed rectangular frame position where the insulator string target is located in the image.
2. An insulator detection method according to claim 1, characterized in that: The insulator detection network model uses the Drn-c42 backbone network to extract insulator string image features from UAV inspection images.
3. The insulator detection method according to claim 1, characterized in that: A multi-directional attention aggregation network is used to encode image features, and the insulator detection network model is made to further focus on the feature information of the insulator string target in a local-global-local manner.
4. The insulator detection method according to claim 1, characterized in that: The insulator detection network model represents the insulator string target with a chain structure and the rotating bounding box locates the insulator string.
5. The insulator detection method according to claim 1, characterized in that: The insulator detection network model takes a color image as input and generates the two-dimensional position of the key points of the insulator string as output. The insulator string is regarded as a chain composed of several key points. The key points on the insulator string are defined by a bottom-up rule. The bottom key point is defined as the first, and then the remaining key points are arranged in sequence to the last one. In order to make the chain structure effectively represent the detected insulator string, the insulator detection network model is constructed as a double-stream structure. Finally, according to the assembly algorithm in the human posture estimation method, a point chain structure is constructed to represent the detected insulator string.
6. An insulator detection method according to claim 5, characterized in that: The dual-stream structure includes: Keypoints Estimation stream, used to predict the two-dimensional confidence map K of the insulator key points; Linkages Estimation stream, used to predict the two-dimensional vector field L of affinity between key points, encoding the degree of association between key points; Set K = {K1, K2, K3...K J There are J confidence maps, K of which j ∈R w×h ,j∈{1...J}, set L={L1,L2,L3...L C }There are C vector fields, among which L c ∈R w×h×2 ,c∈{1...C},L C Each image position in encodes a 2D vector; For each insulator string, the starting and ending positions of the string are defined as k1 and k j , the width of the string is w s , the length of the detection bounding box is determined by k1 and k j The distance between them is determined, and the width of the bounding box is determined by the vertical extension of the chain structure; In order to reduce the difficulty of model convergence, a Gaussian function is used to define the true value key point feature K * : in represents the true coordinates of the jth key point, and the hyperparameter σ 2 represents the Gaussian variance; at the same time, the label of the chain Defined as: Among them, k i and k i+1 are the coordinates of two key points near point p, represents the true value of point p in the jth link, and P on the insulator string represents w s ≤2||pk i+1 ||2·sinθ≤w s , θ represents the vector pk i+1 and k i -k i+1 The angle between.
7. The insulator detection method according to claim 3, characterized in that: Feature encoding is a multi-stage refinement process, divided into two parallel convolutional network branches, including: Keypoints Estimation stream, used to encode key points on the insulator string; Linkages Estimation stream, used to encode the chain structure between key points; The encoding network consists of several 3×3 and 1×1 convolutional networks. In the first stage, the network estimates K 1 =ρ 1 (F) and L 1 =φ 1 (F), where ρ 1 and φ 1 This is the convolutional neural network used for reasoning in the first stage. In each subsequent stage, the prediction results of the two branches in the previous stage are fused with the original image features to produce more accurate predictions: K t =ρ t (F,K t-1 ,L t-1 ), L t =φt t (F,K t-1 ,L t-1 ), where ρ t and φ t is the convolutional neural network for inference at stage t; In order to guide the network to iteratively predict the confidence map of the key points and affinity fields of each branch, the L2 loss function is applied at the end of each stage: Among them, p represents the position of the pixel, f t Denotes the loss at stage t, W is a binary mask where W(p) = 0 when an annotation is missing at image position p. The binary mask is used to avoid penalizing true predictions during training. The intermediate supervision at each stage solves the gradient vanishing problem by periodically replenishing the gradient. Therefore, the overall loss function is: Where T represents the total number of stages.
8. The insulator detection method according to claim 3, characterized in that: The multi-directional attention aggregation network is specifically: Firstly, CA attention is used to generate a feature map with channel, direction and position awareness and then applied to the input feature map complementarily to enhance the representation of the insulator target; Then the self-attention mechanism is calculated on a subset of the input tokens while preserving the underlying information exchange of the global-local context; The self-attention mechanism is decomposed into consecutive modules and introduces local aggregation, global sparse attention and local propagation.
9. The insulator detection method according to claim 8, characterized in that: Local aggregation only integrates local aggregations from local approximate Token signals; for each Token, Depth-wise and Point-wise convolutions are used to aggregate information in a local window of size k×k; Global sparse attention models the long-term relationship between a set of representative tokens, where each token is regarded as a representative of a local window; Sampling a sparse set of representative tokens uniformly distributed in space, with one representative token for each r×r window, where r represents the subsampling rate, and then applying the self-attention mechanism only to the selected tokens; Local propagation diffuses the global context information learned by the delegate to non-representative tokens with the same window. The final multi-faceted attention information aggregation is expressed as: Where X in ∈R H×W×C represents the input tensor, CA represents Coordinate Attention, LocalAgg represents the local aggregation operator, FFN is a two-layer perceptron, GlobalSparseAttn is the global sparse self-attention, and Localprop is the local propagation operation.
10. The insulator detection method according to claim 1, characterized in that: The network pruning operation in step S3 is specifically: removing the last three keypoints Estimation modules and the last two Linkages Estimation modules.