Power line instance detection method based on cross-layer feature aggregation
Through the power line instance detection method based on cross-layer feature aggregation, the problem of low power line detection accuracy in the prior art in complex environments is solved, and accurate detection and regression of multi-form power line targets is achieved.
Patent Information
- Application Number
- CN202411882395.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-04-25
AI Technical Summary
The existing power line detection methods have low accuracy and poor detection effect in complex environments, making it difficult to effectively detect multi-form power line targets.
A power line instance detection method based on cross-layer feature aggregation is proposed. By constructing a multi-morphological power line detection data set, a cross-layer feature aggregation network and an OD_AttentionROI module are used, and combined with PLineIoU loss can achieve accurate detection of power lines.
It improves the accuracy and effect of power line detection, can accurately detect multi-form power line targets, and enhances the capture of global context information, improving regression accuracy.
Smart Images
Figure CN119888385A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of power line detection, and in particular to a power line instance detection method based on cross-layer feature aggregation. Background Art
[0002] As an important power transmission component, power lines need to be regularly inspected for safety. With the advancement of information technology and artificial intelligence technology, as well as the inherent advantages of drones such as being lightweight and convenient, power line detection based on drone inspection images has attracted much attention and has become an important means to improve the efficiency and safety of power inspections. At present, power line detection research can be roughly divided into two categories, one is based on traditional image processing technology, and the other is based on deep learning detection methods. It is difficult to achieve ideal detection results using traditional methods based on manual feature extraction, analysis and post-processing. Although deep learning-based methods have made certain progress in power line detection, most research methods detect power lines from the perspective of target detection and image segmentation. When applied to the field of power line detection in complex environments, the effect is poor and the accuracy is low.
[0003] Since most of the power lines photographed by drones in actual power inspection scenarios are in harsh outdoor environments, there are various complex environmental factors interfering in the inspection images. This leads to many deficiencies in existing algorithms when completing power line detection tasks: (1) low accuracy and poor detection effect; (2) inability to effectively detect multi-form power line targets and other issues. Therefore, it is urgent to propose a power line detection method with high detection accuracy and good effect. Summary of the invention
[0004] The purpose of the present invention is to solve the technical problems in the above background and propose a power line instance detection method based on cross-layer feature aggregation, which includes the following steps:
[0005] S1, construct a multi-modal power line detection dataset;
[0006] S2, building a power line instance detection network model based on a cross-layer feature aggregation network;
[0007] S3, training a power line instance detection network model;
[0008] S4. Detect the multi-form power line targets to be detected.
[0009] In a preferred solution, step S4 further comprises the following steps:
[0010] S41, using high-level features to detect and preliminarily locate power lines;
[0011] S42, further obtaining accurate positions and instance detection results of power lines by refining and aggregating graphic features;
[0012] S43, introduce OD_AttentionROI module to capture global context information;
[0013] S44. To solve the problem of similarity calculation of multi-modal power lines, the pixel point set of multi-modal power lines is regarded as a whole for regression through PLineIoU loss to improve the regression accuracy of multi-modal power lines.
[0014] In a preferred solution, a cross-layer feature aggregation network assigns power line objects to different levels of the FPN multi-level feature pyramid and detects power lines in sequence.
[0015] In the preferred solution, step S2 includes the following steps: constructing a cross-layer feature aggregation network, using high-level features to identify the position of the power line, and using low-level features to adjust the position of the power line; the image features extracted by the SwinTransformer_t backbone network generate three-layer features {F0, F1, F2} through the FPN multi-level feature pyramid; starting from the high-level F0, gradually cross-layer aggregation to F2; finally, {C0, C1, C2} is used to represent the image features obtained after the cross-layer feature aggregation network. The specific process is shown in the following formula:
[0016]
[0017] Where t=1,2,...,T, T is the total number of fusions, p t The power line prior parameters mainly include the horizontal and vertical coordinates x, y of the starting key point and the angle θ between the horizontal coordinate and the horizontal coordinate. For the first layer, F0 and P0 are evenly distributed on the image surface. In order to obtain the C t , with p t As input and the obtained power line features of the region of interest, the aggregation parameter p is then obtained through the FC layer t .
[0018] In the preferred solution, the OD_AttentionROI module in step S43 is specifically:
[0019] OD_AttentionROI power line image features and prior information are used as the input of the model. Each power line prior has N sampling points. For each power line prior, the ROI features of the power line prior are obtained according to the ROI Align method. Uniformly sample N from the power line prior p sampling points and uses bilinear interpolation to calculate the exact value of the feature at the sampling point location.
[0020] In a preferred solution, for multiple layers of ROI features, the ROI features of each layer are connected in series with the ROI features of the previous layer to enhance feature representation, and the extracted ROI features are subjected to multiple layers of convolution operations to obtain feature information near the target power line pixels.
[0021] In the preferred solution, in order to save memory space, a fully connected layer is used to further extract the prior features X of the power lines. p ∈R C×1 , and convert the acquired feature map size to X f ∈R C×H×W , and then flattened to X f ∈R C×HW .
[0022] In the preferred solution, in order to obtain the global context information of the power line prior feature, the ROI power line prior feature X is first calculated. p With the global feature map X f The attention matrix W is as follows:
[0023]
[0024] Where f is the normalization function softmax;
[0025] The aggregated features are expressed as:
[0026]
[0027] The output G reflects X f X p Reward, X p From X f Select from all positions;
[0028] Finally, add the output to the original input X p .
[0029] In the preferred solution, PLineIoU is specifically:
[0030] For each key point on the predicted power line, first Expand it into a line segment with radius e;
[0031] Then calculate the IoU between the extended line segment and the data label Ground Truth, which is specifically expressed as:
[0032]
[0033] in is the coordinate of the prediction point, is the coordinate of the data label point, e is the expansion radius, min() function means taking the minimum, max() means taking the maximum, is the intersection of the extended line segments, is the union of the extended line segments;
[0034] when When it is a negative value, the line segments can be optimized without overlapping;
[0035] Finally, PLineIoU, as a combination of infinite points and lines, is converted into a discrete form and simplified as follows:
[0036]
[0037] In the preferred solution, the PLineIoU loss is defined as:
[0038] L PLineIoU =1-PLineIoU
[0039] Among them, -1≤PLineIoU≤1, when the two lines completely overlap, PLineIoU=1, and when the two lines are far apart, PLineIoU converges to -1.
[0040] The beneficial effects of the present invention are:
[0041] (1) The cross-layer feature aggregation network proposed in the present invention can use high-level features to perform preliminary detection to roughly locate the position of the power line. Then, the network further refines and aggregates the features of the power line image to achieve accurate determination of the power line position.
[0042] (2) The OD_AttentionROI module adopted in the present invention can make the network model pay more attention to the global context information and enhance the power line representation to solve the problem of no visual clues. At the same time, the constructed PLineIoU Loss regards the multi-modal power line pixel point set as a whole for regression to improve the model's regression accuracy for the power line.
[0043] (3) The algorithm framework proposed in the present invention can accurately detect multi-form power line targets in actual power inspection scenarios.
[0044] (4) The algorithm framework proposed in the present invention can be easily extended to other multi-form linear and strip-shaped intelligent inspection tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 This is the overall framework diagram of the power line instance detection network based on cross-layer feature aggregation proposed in the present invention.
[0046] Figure 2This is the network structure diagram for cross-layer feature aggregation.
[0047] Figure 3 This is the structure diagram of the OD_AttentionROI module.
[0048] Figure 4 This is a diagram illustrating the PLineIoU method. DETAILED DESCRIPTION
[0049] The present invention is further described below in conjunction with the accompanying drawings.
[0050] A power line instance detection method based on cross-layer feature aggregation includes:
[0051] Step 1: Construct a multi-modal power line detection dataset;
[0052] Step 2: Construct a power line instance detection network model based on cross-layer feature aggregation;
[0053] like Figure 1 As shown, the power line instance detection network model based on cross-layer feature aggregation described in the present invention makes full use of low-level and high-level features to detect multi-modal power lines. Specifically, high-level features are first used for detection to roughly locate the power lines. Then, by refining and aggregating the graphic features, a more accurate position and instance detection result of the power lines is obtained. In order to solve the problem of no visual clues such as foreign object occlusion, the OD_AttentionROI module is introduced to enable the network model to capture more global context information and enhance the representation of multi-modal power lines. In addition, in order to solve the problem of similarity calculation of multi-modal power lines, the PLineIoU loss (Power Line Intersection over Union, PLineIoU for short) is proposed, and the multi-modal power line pixel point set is regarded as a whole for regression to improve the regression accuracy of multi-modal power lines.
[0054] The cross-layer feature aggregation network is specifically:
[0055] like Figure 2As shown in the figure, in the convolutional neural network, high-level features have more semantic responses to the target object, and low-level features contain more local context information. Effective use of high-level features of power lines can help develop more useful context information, such as effectively distinguishing power lines from other linear targets in the background. In addition, fine detail features can help detect power line targets that require high-precision positioning. Both high-level features and low-level features are important for power line detection. Power line objects are assigned to different FPN multi-level feature pyramid levels and power lines are detected in turn. Specifically, a cross-layer feature aggregation network (CLFA) is constructed. This network module mainly uses high-level features to identify the position of power lines, and low-level features to adjust the position of power lines. The image features extracted by the SwinTransformer_t backbone network are generated by the FPN multi-level feature pyramid to generate three-layer features {F0, F1, F2}. The cross-layer feature aggregation starts from the high-level F0 and gradually proceeds to F2. Finally, {C0, C1, C2} is used to represent the image features obtained after the cross-layer feature aggregation network. The specific process is shown in the following formula.
[0056]
[0057] Where t=1,2,...,T, T is the total number of fusions. t The power line prior parameters mainly include the horizontal and vertical coordinates x, y of the starting key point and the angle θ between the horizontal coordinate and the horizontal coordinate. For the first layer F0, P0 are evenly distributed on the image surface. t , with p t As input and the obtained power line features of the region of interest, the aggregation parameter p is then obtained through the FC layer t The above-mentioned stepwise aggregation and refinement of the extracted features and power line prior information are crucial for successful power line detection.
[0058] The OD_AttentionROI module is specifically:
[0059] like Figure 3 As shown in Figure 1, the OD_AttentionROI module is lightweight and efficient. It uses the power line image features and prior information as the input of the model. Each power line prior has N sampling points. For each power line prior, the ROI feature (X p ∈R C×Np ). The difference from the ROI Align method is that N is uniformly sampled from the power line prior. ppoints, and use bilinear interpolation to calculate the accurate feature values of these point positions. For multi-layer ROI features, they are connected in series with the ROI features of the previous layer to enhance feature representation. The extracted ROI features are subjected to multi-layer convolution operations to obtain feature information near the target power line pixels. In addition, in order to save memory space, a fully connected layer is used to further extract the prior feature information of the power line (X p ∈R C×1 ) and convert the acquired feature map size to X f ∈R C×H×W Then flatten it to X f ∈R C×HW .
[0060] In order to obtain the global context information of the power line prior feature, we first calculate the ROI power line prior feature X p With the global feature map X f The attention matrix W can be expressed as:
[0061]
[0062] Where f is the normalized function softmax. The aggregated features can be expressed as:
[0063]
[0064] The output G reflects X f X p Reward, X p From X f Finally, the output is added to the original input X p .
[0065] PLineIoU is specifically:
[0066] like Figure 4 As shown in , PLineIoU is derived from the target detection IoU, which is the ratio of intersection to union. Specifically, for each key point on the predicted power line, first Extend it into a line segment with radius e. Then calculate the IoU between the extended line segment and its ground truth, which can be expressed as:
[0067]
[0068] in yes The extension point, are the corresponding Ground Truth Points. In addition, Can be negative, which makes it possible to optimize line segments without overlapping.
[0069] Finally, PLineIoU can be regarded as a combination of infinite points and lines, and the function expression after conversion into discrete form and simplification is:
[0070]
[0071] PLineIoU loss is defined as:
[0072] L PLineIoU =1-PLineIoU
[0073] Among them, -1≤PLineIoU≤1, when the two lines completely overlap, PLineIoU=1, and when the two lines are far apart, PLineIoU converges to -1. The designed PLineIoU loss is simple and differentiable, easy to implement parallel calculation, and treating the power line as a whole for regression is conducive to improving the overall performance of the model.
[0074] Step 3: Train a power line instance detection network based on cross-layer feature aggregation;
[0075] Step 4: Detect the multi-form power line target to be detected;
[0076] Input the image containing multi-morphological power line targets into the trained network and output the segmentation and instance detection results of the multi-morphological power line targets in the image.
[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting power line instances based on cross-layer feature aggregation, characterized by: The following steps are involved: S1, construct a multi-modal power line detection dataset; S2, building a power line instance detection network model based on a cross-layer feature aggregation network; S3, training a power line instance detection network model; S4. Detect the multi-form power line target to be detected.
2. The method for detecting power line instances based on cross-layer feature aggregation according to claim 1, characterized in that: Step S4 also includes the following steps: S41, using high-level features to detect and preliminarily locate power lines; S42, further obtaining accurate positions and instance detection results of power lines by refining and aggregating graphic features; S43, introduce OD_AttentionROI module to capture global context information; S44. To solve the problem of similarity calculation of multi-modal power lines, the pixel point set of multi-modal power lines is regarded as a whole for regression through PLineIoU loss to improve the regression accuracy of multi-modal power lines.
3. The method for detecting power line instances based on cross-layer feature aggregation according to claim 1, characterized in that: The cross-layer feature aggregation network assigns power line objects to different levels of the FPN multi-level feature pyramid and detects power lines sequentially.
4. The method for detecting power line instances based on cross-layer feature aggregation according to claim 1, characterized in that: Step S2 includes the following steps: construct a cross-layer feature aggregation network, use high-level features to identify the position of the power line, and use low-level features to adjust the position of the power line; the image features extracted by the SwinTransformer_t backbone network are generated by the FPN multi-level feature pyramid to generate three-layer features {F0, F1, F2}; starting from the high-level F0, gradually cross-layer aggregation to F2; finally, {C0, C1, C2} is used to represent the image features obtained after the cross-layer feature aggregation network. The specific process is shown in the following formula: Where t=1,2,...,T, T is the total number of fusions, p t The power line prior parameters mainly include the horizontal and vertical coordinates x, y of the starting key point and the angle θ between the horizontal coordinate and the horizontal coordinate. For the first layer, F0 and P0 are evenly distributed on the image surface. In order to obtain the C t , with p t As input and the obtained power line features of the region of interest, the aggregation parameter p is then obtained through the FC layer t .
5. The method for detecting power line instances based on cross-layer feature aggregation according to claim 1, characterized in that: The OD_AttentionROI module in step S43 is specifically: OD_AttentionROI power line image features and prior information are used as the input of the model. Each power line prior has N sampling points. For each power line prior, the ROI features of the power line prior are obtained according to the ROI Align method. Uniformly sample N from the power line prior p sampling points and uses bilinear interpolation to calculate the exact value of the feature at the sampling point location.
6. The method for detecting power line instances based on cross-layer feature aggregation according to claim 5, characterized in that: For multi-layer ROI features, the ROI features of each layer are connected in series with the ROI features of the previous layer to enhance the feature representation. The extracted ROI features are subjected to multi-layer convolution operations to obtain the feature information near the target power line pixels.
7. The method for detecting power line instances based on cross-layer feature aggregation according to claim 5, characterized in that: In order to save memory space, a fully connected layer is used to further extract the prior features X of the power lines. p ∈R C×1 , and convert the acquired feature map size to X f ∈R C×H×W , and then flattened to X f ∈R C×HW .
8. The method for detecting power line instances based on cross-layer feature aggregation according to claim 2, characterized in that: In order to obtain the global context information of the power line prior feature, we first calculate the ROI power line prior feature X p With the global feature map X f The attention matrix W is as follows: Where f is the normalization function softmax; The aggregated features are expressed as: The output G reflects X f X p Reward, X p From X f Select from all positions; Finally, add the output to the original input X p .
9. The method for detecting power line instances based on cross-layer feature aggregation according to claim 2, characterized in that: PLineIoU is specifically: For each key point on the predicted power line, first Expand it into a line segment with radius e; Then calculate the IoU between the extended line segment and the data label Ground Truth, which is specifically expressed as: in is the coordinate of the prediction point, is the coordinate of the data label point, e is the expansion radius, min() function means taking the minimum, max() means taking the maximum, is the intersection of the extended line segments, is the union of the extended line segments; when When it is a negative value, the line segments can be optimized without overlapping; Finally, PLineIoU, as a combination of infinite points and lines, is converted into a discrete form and simplified as follows:
10. The method for detecting power line instances based on cross-layer feature aggregation according to claim 9, characterized in that: PLineIoU loss is defined as: L PLineIoU =1-PLineIoU; Among them, -1≤PLineIoU≤1, when the two lines completely overlap, PLineIoU=1, and when the two lines are far apart, PLineIoU converges to -1.