Power transmission line icing detection method based on structural prior constraint and terminal device

Abstract

This invention relates to a method and terminal device for detecting icing on transmission lines based on structural prior constraints. The method includes: acquiring an image of the transmission line containing conductors and icing areas to be identified; extracting multi-scale feature maps from the image; extracting conductor constraint features from the multi-scale feature maps; obtaining a conductor constraint feature map based on the multi-scale feature maps and the conductor constraint features; and outputting the icing detection result based on the conductor constraint feature map. By extracting conductor constraint features, the method focuses on the true position of the conductor based on the structural prior of the transmission line, suppresses background interference, and strengthens the feature representation of the conductor area, thereby improving the positioning accuracy and robustness of icing detection and achieving more accurate and stable icing identification of transmission lines.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of online monitoring of transmission lines and deep learning integration, and in particular to a method and terminal equipment for detecting icing on transmission lines based on structural prior constraints. Background Technology

[0002] In the field of transmission line icing disaster monitoring and prevention, the automatic detection and category identification of conductor icing status using image recognition technology has become an important technical direction to assist manual inspection and support de-icing decision-making. Existing technologies for automatic identification of transmission line icing status mostly employ single-stage target detection networks based on deep learning, especially detection frameworks represented by the YOLO series. For example, using networks such as YOLOv5 and YOLOv8 as the base model, multi-scale feature maps are extracted using convolutional or Transformer backbones, and then multi-scale fusion is performed through a feature pyramid structure. The detection head then performs bounding box regression and category prediction for icing targets on the transmission conductors. Some studies have further introduced attention mechanisms, lightweight convolutional modules, or improved multi-scale detection heads, and adjusted the loss function and small target detection layer to improve the detection rate and positioning accuracy of icing targets under complex backgrounds, different shooting angles, and various meteorological conditions.

[0003] While existing methods have achieved some success in detecting icing on transmission lines, these methods primarily focus on improving image recognition accuracy and fail to adequately optimize for the specific needs of de-icing robots in practical applications. In real-world power line inspections, the technology not only needs to accurately identify the location and type of icing targets but also ensure that the robot can efficiently and stably perform de-icing tasks in complex power environments. Therefore, current methods still suffer from some common shortcomings: First, there is a lack of explicit modeling of the structural prior that icing must be attached to the transmission line: Existing methods usually perform dense target search on the feature map of the whole image, which does not effectively distinguish the conductor area from the non-conductor background. It is easy to produce false detections on slender structures such as towers, tree branches, and cables. The ability to express slender targets such as slight icing and local icing attached to the line is limited, and the risk of missed detection is high. Second, there is insufficient modeling of the context along the line for slender icing targets: Icing on transmission lines is usually in the form of slender segments that are continuous along the line. However, the existing detection framework mainly relies on general convolution and multi-scale fusion, and lacks specific modeling of the continuous distribution of targets along the direction of the conductor. Under conditions of changing lighting, complex background or uneven icing distribution, local icing areas are easily treated as noise and smoothed out, resulting in insufficient stability of the detection results.

[0004] Third, the fine-grained recognition accuracy of ice types such as rime, frost, and white frost is limited: different ice types have significant physical differences in crystal morphology, surface roughness, and edge sharpness, but these differences are mainly reflected in subtle textures and high-frequency structures in visible light images. Existing methods mostly use deep features in the spatial domain to directly perform multi-class classification, lacking explicit modeling of frequency domain information and texture spectrum features. This makes it difficult to fully amplify the differences in high-frequency texture and edge roughness among different ice types, resulting in significant room for improvement in ice type discrimination ability and robustness under complex icing conditions.

[0005] In summary, in practical applications, de-icing robots need to perform efficient and stable de-icing tasks in complex power environments. This requires algorithms that can not only accurately identify and classify the types of ice on transmission lines, but also ensure that the robot can adapt in real time to different ice types, lighting changes, and complex background conditions in a dynamic environment, thereby ensuring the accurate execution of the de-icing task. How to improve upon traditional target detection frameworks to enable robots to more efficiently identify and handle complex transmission line icing problems is a pressing technical issue that needs to be addressed in this field. Summary of the Invention

[0006] To address at least one of the aforementioned technical problems, this invention provides a method for detecting icing on transmission lines based on structural prior constraints, comprising: Acquire images of the transmission line containing conductors and icing areas to be identified; Extract multi-scale feature maps from the image to be identified; Based on the multi-scale feature map, extract the conductor constraint features; Based on the multi-scale feature map and the traverse constraint features, the traverse constraint feature map is obtained; Based on the conductor constraint feature map, output the icing detection results.

[0007] Furthermore, the duct constraint features include: a duct probability mask and a duct orientation field; the step of obtaining a duct constraint feature map based on the multi-scale feature map and the duct constraint features includes: Based on the wire probability mask, spatial reweighting of each feature map in the multi-scale feature map is performed with wire constraints to obtain the local features of each feature map. Based on the directional field of the conductor, the local features of each feature map are aggregated along the direction of the conductor to obtain the directional aggregated features of each feature map. By fusing the local features and directional aggregation features of each feature map, the wire constraint feature map of each feature map is obtained.

[0008] Further, the steps for obtaining the local features of each feature map include: Based on the scale of each feature map in the multi-scale feature map, the wire probability mask is mapped to the resolution corresponding to the scale of each feature map to obtain the wire probability mask corresponding to each feature map. Apply a fixed-kernel-size average pooling layer to the wire probability mask corresponding to each feature map to obtain the soft mask corresponding to each feature map: Construct a spatial weight map based on the soft mask corresponding to each feature map; Based on the spatial weight map, each feature map is reweighted according to element-wise multiplication to obtain the local features of each feature map.

[0009] Further, the step of obtaining the directional aggregation features of each feature map includes: Based on the scale of each feature map in the multi-scale feature map, the conductor orientation field is mapped to the resolution corresponding to the scale of each feature map to obtain the conductor orientation field corresponding to each feature map. Based on the conductor direction field corresponding to each feature map, context features along the conductor direction are defined for each pixel position of the local features of each feature map through bilinear interpolation. Within the aggregation range, discrete sampling steps along the conductor direction are used with preset weights and normalization factors to perform one-dimensional directional aggregation of context features, resulting in directional aggregated features of each feature map.

[0010] Furthermore, the step of extracting duct constraint features based on the multi-scale feature map specifically involves extracting the duct probability mask and the duct orientation field, including: Based on the multi-scale feature maps, intermediate feature maps that form the prior of the conductor structure are extracted; Convolution is applied to the intermediate feature map, and then an activation function is used to predict the wire probability mask; Convolution is applied to the intermediate feature map to obtain a two-dimensional direction vector. Then, the two-dimensional direction vector is normalized pixel by pixel to estimate the direction field of the conductor.

[0011] Furthermore, the multi-scale feature map includes at least three sets of feature maps at different scales: a low-level feature map, a mid-level feature map, and a high-level feature map. The step of extracting intermediate feature maps to form the prior knowledge of the conductor structure based on the multi-scale feature map includes: Upsampling transformation is performed on the high-level feature map to obtain features with the same spatial resolution as the mid-level feature map, which is the transformed high-level feature map; The middle-layer feature map is channel-aligned with the transformed high-layer feature map to obtain the channel-aligned middle-layer feature map and the channel-aligned high-layer feature map. The intermediate feature maps after channel alignment and the high-level feature maps after channel alignment are summed, and nonlinear activation is applied to obtain the intermediate feature maps of the structural prior.

[0012] Furthermore, after the step of extracting the conductor constraint features based on the multi-scale feature map, the process also includes: Local frequency domain analysis was performed on the conductor constraint feature map to obtain its spectral characteristics; Several frequency bands are extracted based on spectral features, and each frequency band is reweighted based on enhancing the high-frequency texture and edge details related to icing to obtain the reweighted spectrum. Perform an inverse Fourier transform on the reweighted spectrum to obtain the frequency-domain enhanced texture features of the conductor region; Based on the texture features of the conductor region, spatial-frequency domain residual fusion is performed on the conductor constraint feature map to obtain the conductor region enhanced conductor constraint feature map.

[0013] Further, the step of performing local frequency domain analysis on the conductor constraint feature map to obtain the spectral features includes: The conductor constraint feature map is normalized channel by channel, and the region of interest is restricted to the vicinity of the conductor using a conductor probability mask, resulting in the transformed conductor constraint feature map. The transformed conductor constraint feature map is subjected to a two-dimensional discrete Fourier transform on a channel-by-channel basis to obtain the spectral features.

[0014] Furthermore, the steps to obtain the reweighted spectrum include: Based on the frequency radius, the spectrum is divided into low-frequency band, mid-frequency band, and high-frequency band, and a binary indicator function is constructed for each band; Based on the binary indicator function and the preset frequency band weighting coefficient, each frequency point and channel is reweighted to obtain the reweighted spectrum. The step of obtaining the enhanced conductor constraint feature map of the conductor region also includes: performing channel compression and recalibration on the texture features of the conductor region through convolution to obtain the texture residual; Based on texture residuals and wire constraint feature maps, an enhanced wire constraint feature map is constructed for the wire region.

[0015] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-mentioned transmission line icing detection methods.

[0016] This invention provides a method and terminal device for detecting icing on transmission lines based on structural prior constraints. By extracting conductor constraint features, the method focuses on the true position of the conductor based on the structural prior of the transmission line, suppresses background interference, and strengthens the feature representation of the conductor area, thereby improving the positioning accuracy and robustness of icing detection and achieving more accurate and stable icing identification of transmission lines. Attached Figure Description

[0017] Figure 1This is a flowchart of an embodiment of the transmission line icing detection method based on structural prior constraints of the present invention; Figure 2 This is a schematic diagram of the structure of one embodiment of the recognition model of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be noted that if the embodiments of this invention involve directional indications, such as up, down, left, right, front, back, etc., these directional indications are only used to explain the relative positional relationships and movement of the components in a specific posture. If the specific posture changes, the directional indications will also change accordingly. Furthermore, if the embodiments of this invention involve descriptions such as "first," "second," "S1," "S2," "step one," "step two," etc., these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance, or implicitly specifying the number of technical features indicated or the execution order of the method. Those skilled in the art will understand that anything that does not violate the inventive concept should be included within the scope of protection of this invention.

[0020] like Figure 1 As shown, this invention provides a method for detecting icing on transmission lines based on structural prior constraints, comprising: S1: Acquire an image of the transmission line containing the conductor and icing area to be identified; S2: Extract multi-scale feature maps from the image to be identified; S3: Extract conductor constraint features based on multi-scale feature maps; S4: Obtain the traverse constraint feature map based on the multi-scale feature map and the traverse constraint feature; S5: Output the icing detection results based on the conductor constraint feature map.

[0021] (a) Step S1: Preferably, in step S1, an airborne camera on a drone can be used to inspect the power transmission line, continuously acquiring RGB images containing the conductors and icing areas as the basis for environmental perception and task generation before the ground wire de-icing robot operates. The acquired raw images can be first uniformly scaled to a fixed resolution, and then scaled proportionally with edge padding to ensure that the conductors and icing areas are completely preserved within the field of view. Subsequently, the image pixels are normalized from 0-255 to [0,1], and the channel mean obtained from the training set is subtracted from the R, G, and B channels, and then divided by the channel standard deviation to obtain the standardized input tensor.

[0022] To enhance the model's adaptability to changes in pose and environment, during the training phase, data augmentation operations such as random horizontal flipping, random small-angle rotation, random contrast perturbation, and color jitter can be optionally and fixedly applied, while keeping the augmented image size unchanged. During the testing and deployment phases, only the above scaling, padding, and normalization steps are performed to ensure that the input distribution remains consistent with that during the training phase, providing stable input data for subsequent multi-scale feature map extraction and prior estimation of wire structures.

[0023] Steps S2-S5 can optionally be implemented using a pre-trained recognition model, such as... Figure 2 As shown, the recognition model includes a backbone network, a structural prior branch, a multi-scale feature weighted fusion module, and a detection head, which are used to execute steps S2-S5 respectively. In a preferred embodiment, a frequency domain enhancement step can be optionally included before step S5, in which case a frequency domain-texture enhancement head is added before the detection head. This recognition model, designed for the operational needs of de-icing robots, extracts conductor constraint features through the structural prior branch, focusing the detection area on the real conductor and reducing interference from non-operational targets. In the preferred embodiment, frequency domain texture enhancement is also performed within the conductor area to further improve the reliability of ice type discrimination, thereby outputting icing location, range, and ice type information that can be directly called by the robot control and operation planning modules. This supports the robot in completing subsequent task point generation, operation sequence arrangement, and de-icing strategy selection, achieving efficient detection and recognition of icing targets and their ice types on transmission lines.

[0024] (ii) Step S2: Step S2: Extract multi-scale feature maps from the image to be identified. An improved YOLO backbone network can be used as the basic backbone network to output features at different levels (corresponding to different scales). Further multi-scale downsampling / upsampling can be performed to obtain feature maps at different scales.

[0025] It is worth noting that step S2, as the front-end module of the robot's perception link, focuses on meeting three types of engineering constraints: real-time performance, resource controllability, and stability across operating conditions. On the one hand, it expands the receptive field through hierarchical downsampling to ensure that the robot can still stably grasp the overall direction of the guide wire under different line-of-sight distances and shooting postures. On the other hand, it retains high-resolution detailed features to support the robot's reliable detection of small targets such as those with slight icing or slender icing. Therefore, as long as multi-scale feature maps of different scales can be extracted and used as inputs for subsequent structural prior estimation and constraint feature fusion, a unified and directly callable feature foundation can be provided for the robot's subsequent target localization, work area generation, and strategy selection.

[0026] To adapt to different recognition environments and accuracy requirements, the number of groups, size, and specific extraction method of the multi-scale feature map can be arbitrarily set by those skilled in the art, and are not limited to the following preferred embodiments.

[0027] In a preferred embodiment, the multi-scale feature map extracted in step S2 includes at least three sets of feature maps at different scales; wherein: low-level feature maps Used to characterize conductor edges and small icing segments; mid-layer feature map Used to characterize the overall orientation of the conductor and medium-scale icing areas; high-level feature map It has the highest semantic level and is used to represent the global context information of the entire conductor and its surrounding environment, forming a multi-scale visual representation for the online operation of the ground wire de-icing robot.

[0028] In a preferred embodiment, such as Figure 2 As shown, the image acquisition and preprocessing module acquires the image to be recognized and preprocesses it to obtain a standardized input tensor. The input backbone network is used, and an improved YOLO backbone network is employed to normalize the input tensors. Hierarchical convolutional encoding is performed to extract three sets of feature maps at different scales; In a preferred embodiment, the backbone network consists of several sequentially connected convolutional modules; for example, the layer-by-layer computation process is represented as follows: ; in, Indicates the first Each output feature map; Indicates the first The corresponding parameters of each convolutional module; Indicates the first One convolutional module; Indicates the first Each output feature map Indicates the number of convolutional modules.

[0029] Specifically, in several specified layers, the convolution stride is set to 2, so that the height and width of the output feature map are about half of the input, which is used to achieve spatial downsampling and receptive field expansion. This hierarchical downsampling design can improve the feature consistency of the robot when switching between long-distance inspection and short-distance operation without significantly increasing the computing power overhead, and reduce the perception jitter caused by the change of viewpoint.

[0030] More specifically, each convolutional module processes the input features in the order of "convolution, batch normalization, activation"; after multiple layers of convolution and downsampling, the backbone network outputs at least three sets of feature maps at different scales. , represented as: ; The process of extracting this multi-scale feature map is denoted as:

[0031] in, This represents a standardized input tensor. If no preprocessing is required, the image to be recognized can be used as the raw input. The spatial dimension of the input; These are all the network parameters for the backbone network; Mapping to the backbone network; This is a low-level feature map used to characterize conductor edges and small icing fragments; an example of its spatial dimensions is... , This is a mid-level feature map used to characterize the overall orientation of the conductor and medium-scale icing areas. An example of its spatial dimensions is... ; This is a high-level feature map used to represent the global context information of the entire conductor and its surrounding environment. An example of its spatial dimensions is... Subsequently, Input a multi-scale feature map weighted fusion module for feature fusion; features An additional input structure prior branch is used to generate the wire probability mask. and conductor direction field .

[0032] (III) Step S3: Preferably, the conductor constraint features include a conductor probability mask and a conductor orientation field; based on these two constraint features, the multi-scale feature map is constrained in step S4 to obtain the conductor constraint feature map; It is worth noting that, S2: Based on the multi-scale feature map, the conductor probability mask and conductor orientation field are extracted. By upsampling + channel concatenation + convolution prediction on the feature maps at each scale, the conductor probability mask (binary confidence map) and conductor orientation field (angle / vector map) are output respectively. The conductor probability mask can be selected as a binary confidence map, and the conductor orientation field can be selected as an angle or / and a vector map, so as to realize the joint prediction of conductor region and orientation. The specific prediction extraction method and the specific structure of the structural prior branch can be arbitrarily set by those skilled in the art.

[0033] In a preferred embodiment, S3: the step of extracting conductor constraint features based on the multi-scale feature map includes: S31: Extract intermediate feature maps that form the prior of the conductor structure based on the multi-scale feature maps; Preferably, taking a multi-scale feature map, including a low-level feature map, a mid-level feature map, and a high-level feature map, as an example, step S31 includes: S311: Upsample the high-level feature map to obtain features with the same spatial resolution as the mid-level feature map; these are the transformed high-level features. Specifically, the high-level feature map... Perform an upsampling operation to adjust its spatial resolution to match that of the mid-level feature map. Consistent, resulting in the transformed high-level feature map:

[0034] in, High-level feature map Spatial dimension, (·) Bilinear interpolation with a scale factor of 2 is used. These are the high-level features after the transformation.

[0035] S312: Align the middle-layer feature map with the transformed high-layer feature map using channels to obtain the channel-aligned middle-layer feature map and the channel-aligned high-layer feature map; specifically: align the middle-layer feature map with the high-layer feature map using channels. With the transformed high-level feature map Perform a 1×1 convolution and then channel alignment to obtain:

[0036] in, This is the mid-layer feature map after channel alignment; This is a high-level feature map after channel alignment. Example: Both output channels have the same number of channels. .

[0037] S313: Summing the channel-aligned middle-layer feature map and the channel-aligned high-layer feature map, and applying nonlinear activation, yields the intermediate feature map of the structural prior; specifically: adding the two and applying nonlinear activation, extracting the intermediate feature map to form the conductor structure prior. ; in, The SiLU activation function is used. The spatial dimension of the intermediate feature map. This is an intermediate feature map, used for subsequent conductor probability mask prediction and conductor orientation field estimation.

[0038] S32: Apply convolution to the intermediate feature map, and then use an activation function to predict the wire probability mask; Specifically, in the intermediate feature map Above, a 3×3 convolution is applied, followed by Sigmoid activation, to predict the wire probability mask:

[0039]

[0040] in, For wire probability masks, an example of spatial dimensions is: , For the Sigmoid function, the output is... , which represents the probability that each pixel belongs to a wire.

[0041] S33: Apply convolution to the intermediate feature map to obtain a two-dimensional direction vector, then normalize the two-dimensional direction vector pixel by pixel to estimate the conductor direction field; specifically: Applying convolution to the intermediate feature map yields a two-dimensional direction vector:

[0042] Then, pixel-by-pixel L2 norm normalization is performed on the two-dimensional direction vector to obtain the unit conductor direction field:

[0043] in, For a two-dimensional direction vector, the spatial dimension is shown in the example. The first channel is the horizontal component. The second channel is the vertical component. ; For pixel index, To prevent division by zero constants, The two components are the unit tangential direction of the wire at that pixel. It is an L2 norm.

[0044] In a preferred embodiment, during the training phase of predicting the conductor probability mask and estimating the conductor orientation field, the weighted sum of the conductor probability mask loss and the conductor orientation field loss is used as the total loss of the structural prior branch; specifically: (1) Wire probability mask loss: During the training phase, based on the labeled wire binary mask , and the predicted wire probability mask Calculate the binary cross-entropy loss:

[0045] in, The binary mask for the labeled wires, with an example of spatial dimensions as follows: , representing the total number of pixels. Here, N represents the pixel index, and N represents the total number of pixels. For the predicted wire probability mask, This is the binary cross-entropy loss.

[0046] (2) Conductor directional field loss: During the training phase, the unit tangential direction field generated based on the traverse markings... As a monitoring signal, it is related to the predicted conductor direction field. The direction prediction results are constrained by cosine similarity loss:

[0047] in, For the set of pixels in the conductor region, To predict the cosine similarity between the direction and the true direction, The loss is the field loss in the conductor direction, which drives the predicted direction to be consistent with the actual conductor direction.

[0048] (3) Total loss of structural prior branches: The total loss of the structural prior branches is determined by a weighted sum of the conductor probability mask loss and the conductor direction field loss:

[0049] in, The total loss of the structural prior branches, , These are the weighting coefficients for the conductor probability mask loss and the conductor orientation field loss, respectively.

[0050] This embodiment provides a preferred embodiment for step S3, which involves extracting conductor constraint features from a multi-scale feature map, but is not limited thereto; in the preferred embodiment, after obtaining the multi-scale feature map... Next, we will first utilize the characteristics of middle and high-level personnel. Construct structural prior branches to predict the probability mask for the wires. Estimate the direction field of the conductor This serves as the structural prior input for subsequent wire constraint feature fusion and frequency domain-texture enhancement.

[0051] (iv) Step S4: In a preferred embodiment, step S4: obtaining the traverse constraint feature map based on the multi-scale feature map and the traverse constraint features, includes: S41: Based on the wire probability mask, spatial reweighting of each feature map in the multi-scale feature map is performed with wire constraints to obtain the local features of each feature map. Specifically: Step S41 uses the wire probability mask as the basis for spatial weighting, and assigns “high weight to wire regions and low weight to non-wire regions” to each pixel of the multi-scale feature map, thereby realizing feature space reweighting, highlighting the local features related to the wire, suppressing background interference, and finally obtaining the local features of each feature map under the constraint of the wire.

[0052] S42: Based on the conductor direction field, perform context aggregation of the local features of each feature map along the conductor direction to obtain the direction aggregation features of each feature map; Specifically, in step S42, using the conductor direction field as the basis for spatial aggregation, one-dimensional directional aggregation is performed on each local feature in step S41 along the conductor direction, defining contextual features along the conductor direction, and making explicit predictions on the conductor direction, ultimately obtaining the directional aggregation features of each feature map.

[0053] S43: Fuse the local features and directional aggregation features of each feature map to obtain the wire constraint feature map of each feature map.

[0054] Specifically, in step S43, local features and directional aggregation features can be weighted and fused according to a certain ratio to obtain the wire constraint feature map of each feature map.

[0055] In this embodiment, a preferred embodiment of step S4 is given. Based on the multi-scale feature map obtained in step S2 and the structural prior (conductor probability mask and conductor orientation field) obtained in step S3, this invention introduces step S4 (adding a structural prior branch). On the one hand, based on the conductor probability mask, it can spatially reweight the conductor constraints of each feature map at each scale, perform conductor region enhancement and background suppression, and display the spatial location of the predicted transmission conductor. On the other hand, based on the conductor orientation field, it can perform context aggregation along the conductor direction to display the predicted direction of the transmission conductor. The two are combined to obtain the conductor constraint feature map (examples include...). , respectively corresponding The de-icing robot can focus more precisely on the conductor area, improving its ability to perform de-icing tasks efficiently in complex power environments.

[0056] In a preferred embodiment, step S41 includes: S411: Based on the scale of each feature map in the multi-scale feature map, map the wire probability mask to the resolution corresponding to the scale of each feature map to obtain the wire probability mask corresponding to each feature map. For example, in step S2, the multi-scale feature map includes For example, the original wire probability mask By using bilinear interpolation downsampling or upsampling, the probability masks of the conductors corresponding to the features at each scale are obtained, i.e., the feature maps. Corresponding resolution: ; in, This is the probability mask for the wires corresponding to each feature map. , , ,

[0057] S412: Apply a fixed-kernel-size average pooling layer to the wire probability mask corresponding to each feature map to obtain the soft mask corresponding to each feature map:

[0058] in, This is the probability mask for the wires corresponding to each feature map. The soft mask corresponding to each feature map is given by k, where k is the preset window size. The above conductor probability mask is then subjected to average pooling to obtain a soft mask covering the conductor and its neighborhood. This can further expand the receptive field in the vicinity of the conductor and improve the accuracy of subsequent detection.

[0059] S413: Construct a spatial weight map based on the soft mask corresponding to each feature map: ; in, Spatial weighted graph; >0 represents the conductor region enhancement factor.

[0060] S414: Based on the spatial weight map, reweight each feature map using element-wise multiplication to obtain the local features of each feature map:

[0061] in, These are the local features of each feature map. For pixel index, For feature map indexing.

[0062] In this embodiment, a preferred embodiment of step S41 is given. First, the scale is unified to match the resolution and improve the recognition accuracy. Then, a soft mask is constructed to expand the receptive field of the area near the conductor and avoid missed detections. Finally, the spatial weight map is used as an index to reweight each feature map element by element to obtain local features for conductor region enhancement and background suppression. These three-step improvements can further improve the detection accuracy. It is worth noting that steps S411-S414 are preferred embodiments of step S41, but are not limited thereto. For example, step S41 can be omitted, scale alignment is not performed, and mask reweighting is performed directly. Step S412 can also be omitted, the receptive field of the area near the conductor is not expanded, and step S413 is performed directly using the conductor probability mask to construct the spatial weight map. Of course, it is also possible not to construct the spatial weight map, but to directly use the conductor probability mask as the region weight index to enhance the weight of the conductor region and obtain the local features of each feature map.

[0063] More preferably, step S42 includes: S421: Based on the scale of each feature map in the multi-scale feature map, map the conductor orientation field to the resolution corresponding to the scale of each feature map to obtain the conductor orientation field corresponding to each feature map. For example, in step S2, the multi-scale feature map includes For example, the original conductor direction field Bilinear interpolation downsampling or upsampling is used to obtain the conductor direction field corresponding to each scale feature, i.e., each feature map. Corresponding resolution:

[0064] in, For each feature map, the conductor direction field is... , , ,

[0065] S422: Based on the conductor orientation field corresponding to each feature map, context features along the conductor orientation are defined for each pixel position of the local features of each feature map through bilinear interpolation. ; S423: Within the aggregation range, the discrete sampling step size along the direction of the conductor is used to perform one-dimensional directional aggregation of the context features with preset weights and normalization factors to obtain the directional aggregation features of each feature map.

[0066] Specifically, the directional aggregation feature is represented as:

[0067] in, Contextual features are obtained through bilinear interpolation; t is the pixel index; t is the discrete sampling step size along the conductor direction. The preset weights are T, where T is the aggregation range. As the normalization factor, This is a directional aggregation feature.

[0068] More preferably, step S43 includes: proportionally fusing the local features and orientation aggregation features of each feature map to obtain the wire constraint feature map of each feature map.

[0069] in, The fusion weights for directional aggregation features,

[0070] (v) Step S5: Preferably, the output icing detection result can be the icing target location; the icing target location can be represented by an icing bounding box; more preferably, the icing detection result can also include target confidence, ice type, etc.

[0071] Preferably, taking a multi-scale feature map including three sets of feature maps as an example, icing target detection and icing category prediction are performed at these three scales. The bounding box position, presence confidence, and ice type category (rime, rime, wet snow, uniced) of the icing target are output. The above results are output in a structured form that can be directly called by the robot control module for subsequent task target generation, priority ranking, and de-icing strategy selection (such as adaptive configuration of task intensity, speed, and de-icing method). At the same time, the network is trained end-to-end through a joint loss consisting of bounding box regression, target confidence, and ice type classification to ensure that the model outputs stable, reliable, and usable perception results for closed-loop engineering execution under complex working conditions, meeting the online deployment and real-time decision-making requirements of the ground wire de-icing robot.

[0072] Specifically, the detection head adopts a three-branch structure: Taking a multi-scale feature map consisting of three sets of feature maps as an example, Let the input features be:

[0073] At each scale, a decoupled detection head is used to construct: a bounding box regression branch, a target confidence branch, and an ice-type classification branch.

[0074] First, intermediate features are obtained through several 3×3 convolutions:

[0075] in, The SiLU activation function is used. These are intermediate features at each scale. Based on this, we obtain: Bounding box regression output:

[0076] Target confidence output:

[0077] Ice type classification output: ; Among them, K=4 corresponds to the four categories of "rime, rime, wet snow, and unice". This is the Sigmoid function.

[0078] Anchor-free representation in grid form, for scale Each grid cell The bounding box parameters are represented by the center coordinates ( ) and width and height Through the By applying an appropriate transformation, we obtain:

[0079] ; in, , , , for The four components, , For the scale normalization factor, ( () represents the center coordinates of the bounding box. This represents the width and height of the bounding box.

[0080] Denote the corresponding true bounding box. for For bounding box regression, the loss function is constructed using CIoU loss:

[0081] in, For bounding box regression loss, The number of positive samples. For the positive sample set, for Positive sample pixels on the layer feature map To predict the bounding box, For the true bounding box, The intersection-union ratio (IUGR) is a metric used to measure the degree of overlap between predicted bounding boxes and ground truth bounding boxes.

[0082] Target confidence represents positive sample pixels The probability of an icy target exists is determined by the target confidence task, and a loss function is constructed using binary cross-entropy loss.

[0083] in, For target confidence loss, For all pixels involved in the calculation, including positive and negative samples, Layer feature map, location The target confidence level is predicted. The true label for the target confidence level.

[0084] For each positive sample pixel The category probability vector output at the corresponding position is:

[0085] in, For the model in The output class probability vector, The probabilities of the four ice types are respectively; For the ice type classification task, a multi-class cross-entropy loss function is used to construct the loss function:

[0086] in, For ice type classification loss, The true ice shape is labeled, denoted as a one-hot vector. For the first Ice-like type.

[0087] During the training sample construction phase, only the probability mask of the wire is applied. Target boxes with high overlap are assigned positive samples to ensure that the ice classification branch focuses on the conductor and icy areas.

[0088] The total loss for object detection and ice pattern recognition is a weighted sum of the bounding box regression loss, object confidence loss, and ice pattern classification loss: ; During overall training, the loss is minimized as follows: ; in, For the total loss, These are the bounding box regression loss, the target confidence loss, and the ice type classification loss, respectively. These are the weight coefficients for the bounding box regression loss, the target confidence loss, and the ice type classification loss, respectively. For mask loss; These are the weighting coefficients for the mask loss; To minimize the loss.

[0089] After completing target detection and ice type identification, this method can optionally map the prediction results at each scale back to the original image coordinate system, outputting structured results including target bounding boxes, target confidence scores, and ice type categories (rime, rime, wet snow, uniced). This structured output is then sent in real-time to / provided to the de-icing robot control module for direct invocation, according to the data format required by the robot control interface (such as target coordinates, confidence scores, ice type labels, etc.). The control module can then use this data to confirm the location of iced targets, prioritize tasks, select de-icing strategies, and drive the robot to perform subsequent path planning and end-effector tasks (such as approaching, attaching, and de-icing / clearing). Simultaneously, the system visually overlays different colored borders and category labels on the image, facilitating manual verification, algorithm parameter tuning, and providing traceable evidence for robot task playback, operation log recording, and acceptance report generation.

[0090] (vi) Frequency domain enhancement: In a preferred embodiment, to further enhance the frequency domain texture in the conductor area and improve the reliability of ice pattern discrimination, step S5 may also include step S4': frequency domain enhancement step, in which case a frequency domain-texture enhancement head is added in front of the detection head.

[0091] Specifically, the wire probability mask is obtained in step S3. Step S4 yields the wire constraint feature map. For example, to meet the engineering requirements of "operational availability, strategy selectability, and controllable execution" for ground wire de-icing robots, step S4' can be optionally added. This involves performing local frequency domain analysis and frequency band reweighting within the conductor and its neighborhood, transforming ice morphology differences into texture discrimination features that the robot can stably perceive. By explicitly amplifying the differences in high-frequency texture, edge roughness, and crystal structure details among different ice types, a more discriminative enhanced texture feature map is generated, thereby improving the robot's reliability in identifying ice types under complex lighting, fog, vibration, and background interference conditions. Subsequently, the conductor constraint features are enhanced in the spatial domain in the form of residuals, preserving both the conductor structure and positioning information while injecting high-frequency details sensitive to ice types. This enhances the output features, which, as input to the output detection head in step S5, can be directly utilized by the ice type prediction branch, further providing the robot control layer with more reliable ice type criteria. This supports subsequent decisions regarding de-icing method selection, operation force / speed parameter setting, and operation priority ranking.

[0092] In a preferred embodiment, step S4' includes: S41': Perform local frequency domain analysis on the conductor constraint feature map to obtain the spectral characteristics; specifically, step S41' includes: S411': Channel-by-channel normalization is performed on the conductor constraint feature map, and the region of interest is restricted to the vicinity of the conductor using a conductor probability mask, resulting in the transformed conductor constraint feature map; represented as:

[0093] ; in, These are the wire-constrained feature maps for feature maps at various scales. The normalized wire constraint feature map is the feature map after each scale. , The mean and variance are given. This is the balance coefficient; These are the wire constraint feature maps after transformation of feature maps at various scales.

[0094] S412': Perform a two-dimensional discrete Fourier transform on the transformed conductor constraint feature map, channel by channel, to obtain the spectral features:

[0095] in For two-dimensional FFT, It is a spectral feature.

[0096] S42': Extract several frequency bands based on spectral features, and reweight each frequency band based on enhancing high-frequency textures and edge details related to icing to obtain a reweighted spectrum. Specifically, step S42' includes: S421': Based on frequency radius The spectrum is divided into low-frequency, mid-frequency, and high-frequency bands, and a binary indicator function is constructed for each band; it can be represented as: ; The selection of low-frequency, mid-frequency, and high-frequency bands is to ensure mutual exclusion and full frequency domain coverage. Different frequency bands are used to characterize the overall brightness pattern, mesoscale structure, and fine-grained texture. Without compromising the geometry of the conductors and the stability of spatial positioning, the focus is on enhancing high-frequency textures and edge details related to icing, enabling the robot to more sensitively capture subtle differences in different icing patterns during inspection / operation. At the same time, the necessary structural information such as conductor orientation and icing contours carried by low and mid-frequency bands is preserved, ensuring that the output features can both identify ice types and be used for robot stable positioning and subsequent operation decisions.

[0097] S422': Based on the binary indicator function and preset frequency band weighting coefficients, each frequency point and channel is reweighted to obtain the reweighted spectrum; expressed as:

[0098] in, , , For preset frequency band weighting coefficients, ≥ This is used to highlight high-frequency textures and edge details; This is a reweighted spectrum.

[0099] S43': Perform an inverse Fourier transform on the reweighted spectrum to obtain the enhanced conductor region texture features in the frequency domain, represented as: ; in, This refers to the texture features of the conductor region.

[0100] S44': Based on the texture features of the conductor region, spatial-frequency domain residual fusion is performed on the conductor constraint feature map to obtain the enhanced conductor constraint feature map. Specifically, to inject high-frequency texture differences while preserving the spatial location information of the conductor constraints, this step uses residual fusion to integrate the texture features of the conductor region. Fusion loop constraint feature map The enhanced conductor constraint feature map of the conductor region is obtained. Specifically, step S44' includes: S441': Texture features of the wire region are processed by 1×1 convolution. (Frequency domain enhanced texture features) Channel compression and recalibration are performed to obtain texture residuals. :

[0101] S442': Based on texture residuals and wire constraint feature map Constructing a conductor constraint feature map after enhancing the conductor region. :

[0102] Through the above processing, taking a multi-scale feature map consisting of three sets of feature maps as an example, we obtain the following at each of the three scales: They inherit both the characteristic map of the wire constraint and the characteristic map of the wire constraint. The geometric structure and location information, and simultaneously through frequency-domain reweighted texture residuals. The high-frequency texture differences related to icing type were explicitly encoded to obtain the conductor constraint feature map after conductor region enhancement.

[0103] Subsequently, the enhanced conductor constraint feature map of the conductor region extracted in step S4' is used. The wire constraint feature map extracted in step S4 is used instead. Input the detection head from step S5 and output the icing detection result to further improve the ice type recognition result.

[0104] The beneficial effects of the transmission line icing detection method based on structural prior constraints of this invention are as follows: (1) Step S3 introduces conductor constraint features (structural prior constraints), and step S4 obtains conductor constraint feature map based on conductor constraint features; step S5 outputs icing detection results based on conductor constraint feature map: introducing structural prior branches into the traditional recognition framework makes the icing detection process more accurately focused on the conductor area, improving the ability of the de-icing robot to efficiently perform de-icing tasks in complex power environments; in the preferred scheme: conductor constraint features, including conductor probability mask and conductor direction field, explicitly predict the spatial position and direction of the transmission conductor, and directly apply this prior to multi-scale feature map reweighting and target confidence calculation, further ensuring that the robot only searches for icing targets in the conductor area when performing de-icing tasks, and aggregates context information along the conductor direction, effectively reducing false detections caused by non-conductor structures such as tower materials, branches, and cables in complex environments, while significantly improving the accuracy and stability of the robot in identifying targets with slight icing, slender icing and local icing.

[0105] Compared to existing recognition frameworks that mainly focus on general attention, FPN structure adjustment, or optimization of small target detection layers, but fail to abstract the prior knowledge that icing is necessarily attached to the conductor into an independent network branch and explicitly constrain the detection area, this application introduces a design with structural prior constraints, which closely depends on the specific scenario of power transmission icing, and has outstanding substantive features and significant progress.

[0106] (2) In the further optimized scheme, step S4' is introduced: frequency domain enhancement step: Based on the lightweight frequency domain-texture enhancement detection head of the conductor region, the conductor region is locally decomposed in the frequency domain and the frequency band is reweighted, and then fused with the spatial domain features for ice type classification, thereby explicitly amplifying the subtle differences in high-frequency texture, surface roughness and edge sharpness of ice types such as rime, frost and white frost, which can further improve the ice recognition capability of the de-icing robot and significantly improve the fine-grained recognition accuracy and robustness of multiple types of ice under complex lighting and noise backgrounds. This design limits the frequency domain module to a specific structural region and combines it with the differences in the physical morphology of ice types. It further tightly couples the region selected by the prior screening of the conductor structure with the frequency domain-texture enhancement, forming a synergistic effect of structural prior and texture prior. This cross-level prior combination is an innovative design for the problem of power transmission icing. The structural prior branch and frequency domain-texture enhancement detection head design of this invention can be embedded in real-time detection frameworks such as the YOLO series and RT-DETR, further improving the application performance of de-icing robots in dynamic power line inspection.

[0107] On the other hand, the present invention also provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned transmission line icing detection methods.

[0108] On the other hand, the present invention also provides a terminal device, optionally an ice removal robot, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-mentioned transmission line icing detection methods.

[0109] For example, the program code can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the program code in the terminal device.

[0110] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the terminal device may also include input / output devices, network access devices, buses, etc.

[0111] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0112] The memory can be an internal storage unit of the terminal device, such as a hard drive or RAM. The memory can also be an external storage device of the terminal device, such as a plug-in hard drive, SmartMediaCard (SMC), Secure Digital (SD) card, or FlashCard. Furthermore, the memory can include both internal and external storage units of the terminal device. The memory is used to store the program code and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output.

[0113] The aforementioned computer storage medium and terminal equipment are created based on the aforementioned transmission line icing detection method. Their technical functions and beneficial effects will not be elaborated here. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for detecting icing on transmission lines based on structural prior constraints, characterized in that, include: Acquire images of the transmission line containing conductors and icing areas to be identified; Extract multi-scale feature maps from the image to be identified; Based on the multi-scale feature map, extract the conductor constraint features; Based on the multi-scale feature map and the traverse constraint features, the traverse constraint feature map is obtained; Based on the conductor constraint feature map, output the icing detection results.

2. The method for detecting icing on transmission lines according to claim 1, characterized in that, The traverse constraint features include: traverse probability mask and traverse orientation field; the steps to obtain the traverse constraint feature map based on the multi-scale feature map and the traverse constraint features include: Based on the wire probability mask, spatial reweighting of each feature map in the multi-scale feature map is performed with wire constraints to obtain the local features of each feature map. Based on the directional field of the conductor, the local features of each feature map are aggregated along the direction of the conductor to obtain the directional aggregated features of each feature map. By fusing the local features and directional aggregation features of each feature map, the wire constraint feature map of each feature map is obtained.

3. The method for detecting icing on transmission lines according to claim 2, characterized in that, The steps to obtain the local features of each feature map include: Based on the scale of each feature map in the multi-scale feature map, the wire probability mask is mapped to the resolution corresponding to the scale of each feature map to obtain the wire probability mask corresponding to each feature map. Apply a fixed-kernel-size average pooling layer to the wire probability mask corresponding to each feature map to obtain the soft mask corresponding to each feature map: Construct a spatial weight map based on the soft mask corresponding to each feature map; Based on the spatial weight map, each feature map is reweighted according to element-wise multiplication to obtain the local features of each feature map.

4. The method for detecting icing on transmission lines according to claim 2, characterized in that, The steps to obtain the directional aggregated features of each feature map include: Based on the scale of each feature map in the multi-scale feature map, the conductor orientation field is mapped to the resolution corresponding to the scale of each feature map to obtain the conductor orientation field corresponding to each feature map. Based on the conductor direction field corresponding to each feature map, context features along the conductor direction are defined for each pixel position of the local features of each feature map through bilinear interpolation. Within the aggregation range, discrete sampling steps along the conductor direction are used with preset weights and normalization factors to perform one-dimensional directional aggregation of context features, resulting in directional aggregated features of each feature map.

5. The method for detecting icing on transmission lines according to claim 1, characterized in that, The steps for extracting duct constraint features based on multi-scale feature maps specifically involve extracting the duct probability mask and duct orientation field, including: Based on the multi-scale feature maps, intermediate feature maps that form the prior of the conductor structure are extracted; Convolution is applied to the intermediate feature map, and then an activation function is used to predict the wire probability mask; Convolution is applied to the intermediate feature map to obtain a two-dimensional direction vector. Then, the two-dimensional direction vector is normalized pixel by pixel to estimate the direction field of the conductor.

6. The method for detecting icing on transmission lines according to claim 5, characterized in that, Multi-scale feature maps include at least three sets of feature maps at different scales: low-level feature maps, mid-level feature maps, and high-level feature maps. The steps for extracting intermediate feature maps that form the prior knowledge of the conductor structure based on multi-scale feature maps include: Upsampling transformation is performed on the high-level feature map to obtain features with the same spatial resolution as the mid-level feature map, which is the transformed high-level feature map; The middle-layer feature map is channel-aligned with the transformed high-layer feature map to obtain the channel-aligned middle-layer feature map and the channel-aligned high-layer feature map. The intermediate feature maps after channel alignment and the high-level feature maps after channel alignment are summed, and nonlinear activation is applied to obtain the intermediate feature maps of the structural prior.

7. The method for detecting icing on transmission lines according to any one of claims 1 to 6, characterized in that, After extracting the conductor constraint features based on the multi-scale feature map, the process also includes: Local frequency domain analysis was performed on the conductor constraint feature map to obtain its spectral characteristics. Several frequency bands are extracted based on spectral features, and each frequency band is reweighted based on enhancing the high-frequency texture and edge details related to icing to obtain the reweighted spectrum. Perform an inverse Fourier transform on the reweighted spectrum to obtain the frequency-domain enhanced texture features of the conductor region; Based on the texture features of the conductor region, spatial-frequency domain residual fusion is performed on the conductor constraint feature map to obtain the conductor region enhanced conductor constraint feature map.

8. The method for detecting icing on transmission lines according to claim 7, characterized in that, The steps for performing local frequency domain analysis on the conductor constraint feature map to obtain the spectral features include: The duct constraint feature map is normalized channel by channel, and the region of interest is restricted to the vicinity of the duct using a duct probability mask, resulting in the transformed duct constraint feature map. The transformed conductor constraint feature map is subjected to a two-dimensional discrete Fourier transform on a channel-by-channel basis to obtain the spectral features.

9. The method for detecting icing on transmission lines according to claim 8, characterized in that, The steps to obtain the reweighted spectrum include: Based on the frequency radius, the spectrum is divided into low-frequency band, mid-frequency band, and high-frequency band, and a binary indicator function is constructed for each band. Based on the binary indicator function and the preset frequency band weighting coefficient, each frequency point and channel is reweighted to obtain the reweighted spectrum. The step of obtaining the enhanced conductor constraint feature map of the conductor region also includes: performing channel compression and recalibration on the texture features of the conductor region through convolution to obtain the texture residual; Based on texture residuals and wire constraint feature maps, an enhanced wire constraint feature map is constructed for the wire region.

10. A terminal device, characterized in that, It includes a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute the transmission line icing detection method according to any one of claims 1 to 9.

Images