A transmission line fitting detection method based on geometric and gray feature fusion

Abstract

The embodiment discloses a transmission line fitting detection method based on geometric and gray feature fusion, follows the principle of "first positioning, then extraction, then fusion, and finally detection", and aims at the problems that the existing transmission line fitting detection algorithm seriously misjudges under complex background and shielding interference and physical characteristics are not fully utilized. The fitting region of interest is accurately extracted through the pre-training lightweight model, and the global background interference is eliminated. Based on the inherent physical prior of fitting "regular geometric topology + strong metal reflection", a feature extraction network with parallel geometric contour feature branch and gray mutation feature branch is designed to realize independent learning of two types of complementary features. A topological constraint adaptive feature fusion mechanism and a multi-dimensional physical constraint joint loss function are introduced to strengthen the feature expression conforming to the physical form of the fitting. Combined with the fitting size prior, an exclusive candidate box is generated, and the detection efficiency is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of image target detection technology, and in particular to a method for detecting power transmission line fittings based on the fusion of geometric and grayscale features. Background Technology

[0002] With the continued advancement of dual-carbon goals and the accelerated construction of new power systems, the safe and stable operation of transmission lines, as the core carriers of energy transmission, is of paramount importance. Transmission line fittings are key components ensuring the mechanical connection, conductor fixation, and electrical safety of the lines. Long-term exposure to the complex outdoor environment makes them prone to defects such as loosening, missing parts, deformation, and corrosion. Statistics show that line faults caused by fitting defects account for more than 30% of all transmission line faults, seriously threatening the safe and stable operation of the power grid.

[0003] Currently, drone inspection has become the mainstream method for power line inspection, but manual interpretation has problems such as low efficiency, high missed detection rate, and strong subjectivity. While general-purpose deep learning-based object detection algorithms have been applied to power line inspection, they still face several core technical bottlenecks in hardware inspection: First, insufficient detection capability for small targets. At aerial photography distances of 30-50 meters, hardware such as cotter pins and bolts have pixel sizes of only 8-20 pixels, resulting in weak feature information and a false negative rate exceeding 30% for general networks, leading to low detection accuracy. Second, insufficient utilization of physical characteristics. Existing algorithms do not incorporate the inherent physical characteristics of hardware geometric structures and significant grayscale variations in metal materials, resulting in poor feature extraction targeting. Third, weak background noise suppression. Aerial images in the field contain a large number of invalid background pixels, including weeds, towers, and the sky, resulting in high background noise amplitude. This can easily cause redundant backgrounds to cover the hardware's features, interfering with network feature extraction. Fourth, a single feature fusion method. General stitching fusion cannot distinguish the contribution of geometric structural features from metal reflection features, causing effective features to be submerged by background noise. Fifth, poor robustness in occluded scenes. Under conditions of shadows, dust, or debris occlusion, detection accuracy drops by more than 50%, failing to meet practical engineering needs. Summary of the Invention

[0004] This invention discloses a method for detecting transmission line fittings based on the fusion of geometric and grayscale features, in order to overcome the above-mentioned technical problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A method for detecting transmission line fittings based on the fusion of geometric and grayscale features includes the following steps: S1: Acquire panoramic images of the transmission line and preprocess them. Based on the preprocessed panoramic images, use a pre-trained lightweight YOLOv8s model to obtain the region of interest images of the fittings. S2: Establish a dual-branch physical feature extraction network and train it using a multi-dimensional physical constraint joint loss function. The dual-branch physical feature extraction network includes a geometric contour feature branch, a gray-scale abrupt change feature branch, and a topological constraint adaptive feature fusion module. The geometric contour feature branch is used to extract features from the region of interest image of the hardware, and three parallel edge detection sub-branches are used to obtain three edge features and then stitch them together to obtain a multi-scale refined edge feature map. The Hough circle detection is used to obtain a set of candidate circle parameters, and then the geometric contour feature map of the hardware is obtained. The gray-scale mutation feature branch is used to obtain a gray-scale mutation binary map based on the region of interest image of the hardware, to obtain a shallow gray-scale feature map, and then to obtain a gray-scale mutation feature map of the hardware. The topology constraint adaptive feature fusion module is used to obtain the comprehensive feature map of the hardware after topology constraint fusion based on the hardware geometric contour feature map and the hardware grayscale abrupt change feature map; S3: Obtain a panoramic image of the transmission line to be detected, and obtain the comprehensive feature map of the hardware of the panoramic image of the transmission line to be detected based on the trained dual-branch physical feature extraction network. Then, obtain the metal-specific candidate boxes of the hardware size, and output the final detection result of the hardware spatial position.

[0006] Furthermore, the geometric contour feature branch includes: an input layer, a first convolutional layer, a second convolutional layer, a first edge detection sub-branch, a second edge detection sub-branch, a third edge detection sub-branch, a feature concatenation layer, a non-maximum suppression layer, a channel mapping convolutional layer, a Hough circle detection layer, a geometric mask generation layer, and a pixel-wise multiplication layer; The input of the first convolutional layer is connected to the input layer to receive the region of interest image of the hardware through the input layer and extract the low-level visual features of the region of interest image of the hardware to obtain a basic texture feature map; the input of the second convolutional layer is connected to the output of the first convolutional layer to filter high-frequency noise while preserving spatial information based on the basic texture feature map to obtain a shallow structure feature map. The inputs of the first, second, and third edge detection sub-branches are all connected to the output of the second convolutional layer to extract the first, second, and third edge features using Sobel operators at three scales (3×3, 5×5, and 7×7) based on the shallow structure feature map. The input of the feature concatenation layer is connected to the outputs of the first, second, and third edge detection sub-branches to obtain a multi-scale edge fusion feature map based on the first, second, and third edge features. The input of the non-maximum suppression layer is connected to the output of the feature concatenation layer to perform pixel-wise non-maximum suppression based on the multi-scale edge fusion feature map, eliminating redundant edge responses and compressing the edge width to a single pixel to obtain a refined edge feature map. The input of the channel mapping convolutional layer is connected to the output of the non-maximum suppression layer to obtain a multi-scale refined edge feature map based on the refined edge feature map. The input of the Hough circle detection layer is connected to the output of the channel mapping convolutional layer to perform Hough circle detection on the multi-scale refined edge feature map and obtain a set of candidate circle parameters. The input of the geometric mask generation layer is connected to the output of the Hough circle detection layer and is used to obtain a geometric constraint mask with the same size as the multi-scale refined edge feature map based on the set of candidate circle parameters. The input of the pixel-wise multiplication layer is connected to the output of the geometric mask generation layer and the output of the channel mapping convolutional layer and is used to obtain the geometric contour features of the fitting based on the geometric constraint mask and the multi-scale refined edge feature map, so as to output the geometric contour feature map of the fitting through the output layer.

[0007] Furthermore, the gray-scale abrupt change feature branch includes: a gray-scale conversion layer, a neighborhood difference calculation layer, a threshold segmentation layer, a morphological dilation layer, a third convolutional layer, and a fourth convolutional layer; The grayscale conversion layer is used to convert the region of interest (ROI) image of the RGB image into a single-channel grayscale image based on the ROI image of the fittings, thereby obtaining a grayscale base image; the input end of the neighborhood difference calculation layer is connected to the output end of the grayscale conversion layer to enhance the grayscale jump characteristics of the metal reflective area based on the grayscale base image, thereby obtaining a grayscale difference image. The input of the threshold segmentation layer is connected to the output of the neighborhood difference calculation layer, and is used to obtain a gray-level abrupt change binary map based on the gray-level difference map; the input of the morphological dilation layer is connected to the output of the threshold segmentation layer, so as to connect discrete abrupt change points into continuous regions based on the gray-level abrupt change binary map, fill in the breaks in the reflective region, and obtain a continuous abrupt change region binary map. The input of the third convolutional layer is connected to the output of the morphological dilation layer to extract the semantic features of the bottom gray-level mutation based on the binary map of the continuous mutation region and obtain a shallow gray-level feature map; the input of the fourth convolutional layer is connected to the output of the third convolutional layer to align the output dimension of the geometric branch based on the shallow gray-level feature map and obtain a gray-level mutation feature map of the hardware.

[0008] Furthermore, the topology constraint adaptive feature fusion module includes: a spatial topology weight generation module, a spatial feature enhancement module, a channel adaptive weighting module, and a channel-by-channel feature fusion module; The spatial topology weight generation module is used to obtain the topological confidence of the candidate circle based on the candidate circle parameter set, so as to obtain the geometric constraint weight matrix based on the hardware geometric contour feature map. The spatial feature enhancement module is used to obtain a spatially enhanced geometric feature map based on the geometric constraint weight matrix; The channel adaptive weighting module is used to obtain the original geometric channel weight vector by using the first channel weight learning layer according to the spatial enhanced geometric feature map, and to obtain the original grayscale channel weight vector by using the second channel weight learning layer according to the hardware grayscale change feature map. The channel-by-channel feature fusion module is used to obtain the fused feature map, i.e., the comprehensive feature map of the fittings after topological constraint fusion, based on the original geometric channel weight vector and the original grayscale channel weight vector.

[0009] Furthermore, the formula used to obtain the topological confidence score of the candidate circle is as follows:

[0010] In the formula, The candidate circle number is... For the first Topological confidence of each candidate circle; For the first The cumulative vote count of the centers of each candidate circle; The threshold is accumulated for the center of the Hough circle detection; The formula used to obtain the geometric constraint weight matrix is ​​as follows:

[0011] In the formula: This is the geometric constraint weight matrix; All are pixel coordinates on the geometric contour feature map of the hardware; For the first Topological confidence of each candidate circle; For the first The pixel coverage area of ​​each candidate circle The background region is the basic weight.

[0012] Furthermore, the formula used to obtain the spatially enhanced geometric feature map is as follows:

[0013] In the formula, The first geometric profile feature diagram of the hardware c Channels, pixel coordinates Eigenvalues ​​at; The geometric constraint weight matrix in pixel coordinates The weight value at the location; For the spatially enhanced geometric feature map, the first c Channels, pixel coordinates Eigenvalues ​​at; This is the channel index for the geometric profile feature map of the hardware.

[0014] Furthermore, the formula used to obtain the fused feature map is as follows:

[0015]

[0016] In the formula, To fuse feature maps c Channels, pixel coordinates Eigenvalues ​​at; For the spatially enhanced geometric feature map, the first c Channels, pixel coordinates Eigenvalues ​​at; The grayscale mutation feature map c Channels, pixel coordinates Eigenvalues ​​at; , The first c Normalized geometric channel weights and normalized grayscale channel weights; ' is the original geometric channel weight vector; This is the original grayscale channel weight vector; It is a very small constant.

[0017] Furthermore, the joint loss function for the multi-dimensional physical constraints is expressed as follows:

[0018] In the formula: The total loss value is the joint loss function for multi-dimensional physical constraints. Weighted classification loss for difficult samples; For multi-scale geometric perception regression loss; The loss is due to multi-level topology consistency constraints; For bi-branch feature consistency loss; , , These are all the balancing weight coefficients for the loss term; in,

[0019] In the formula: is the total number of hardware samples in the training batch; i is the index of the hardware sample in the training batch; Let be the loss weight for the i-th sample; The categories are numbered sequentially, and include five categories: normal, missing, deformed, corroded, and obscured. For the i-th sample The true label of the class; Let i be the probability that the i-th sample belongs to the j-th class as predicted by the network.

[0020] In the formula, The number of detection scales; For testing scale serial number; Let be the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box at the m-th detection scale; It is a Euclidean distance metric function; The coordinates of the center point of the prediction box. The coordinates of the center point of the annotation box; For the first Under the detection scale, the diagonal length of the smallest bounding rectangle that simultaneously contains the predicted bounding box and the labeled bounding box; This parameter ensures consistency in aspect ratio between the predicted bounding box and the labeled bounding box. This is the balancing weighting coefficient for aspect ratio loss; The weighting coefficients for the geometric constraints of the circumcircle are used to balance the contribution ratios of position regression and shape constraints. Let be the intersection-union ratio of the minimum outer circle of the predicted bounding box and the minimum outer circle of the labeled bounding box at the m-th detection scale;

[0021]

[0022] In the formula, , These are the width and height of the prediction box, respectively. These are the width and height of the annotation box, respectively.

[0023] In the formula, The set of pixels representing the actual area of ​​the labeled hardware; For geometric contour feature maps in pixels The response intensity at that location; For grayscale abrupt change feature map at pixel The response intensity at that location.

[0024] Beneficial Effects: This invention provides a transmission line fitting detection method based on geometric and grayscale feature fusion. Following the principle of "first localization, then extraction, then fusion, and finally detection," it addresses the problems of severe misjudgment and insufficient utilization of physical characteristics in existing transmission line fitting detection algorithms under complex backgrounds and occlusion interference. It accurately extracts the region of interest (ROI) of the fitting through a pre-trained lightweight model, eliminating global background interference. Based on the inherent physical prior of the fitting's "regular geometric topology + strong metallic reflectivity," a feature extraction network with parallel geometric contour feature branches and grayscale abrupt change feature branches is designed to achieve independent learning of these two complementary features. A topological constraint adaptive feature fusion mechanism and a multi-dimensional physical constraint joint loss function are introduced to strengthen the feature expression that conforms to the fitting's physical morphology. Combined with the fitting's size prior, a dedicated candidate box is generated, significantly improving detection efficiency. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the transmission line fitting inspection method based on geometric and grayscale feature fusion of the present invention; Figure 2 This is a schematic diagram of the transmission line fitting testing method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the overall structure of the dual-branch physical feature extraction network according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the topology constraint adaptive feature fusion module structure according to an embodiment of the present invention; Figure 5 These are schematic diagrams of various hardware fittings according to embodiments of the present invention; Figure 6 This is an experimental diagram illustrating the anti-shading performance of an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This embodiment introduces a method for detecting transmission line fittings based on the fusion of geometric and grayscale features, including the following steps: Figure 1 and Figure 2 As shown: S1: Acquire panoramic images of the transmission line and preprocess them. Based on the preprocessed panoramic images, use a lightweight YOLOv8s model to locate the regions of interest (ROI) of the hardware in the preprocessed panoramic images and obtain the ROI images of the hardware. In this embodiment, image acquisition includes three sources: first, low-altitude aerial photography along the transmission line tower route using an inspection drone equipped with a visible light industrial camera to collect panoramic inspection images of the transmission line in the field; second, close-up images of the hardware on the line taken by maintenance personnel using a handheld portable high-definition industrial camera; and third, historical inspection photos archived in the power grid maintenance database to supplement and expand the sample dataset. After image acquisition is completed, median filtering is used to remove noise from the aerial footage, and all images are uniformly normalized and scaled to 640×640 pixels.

[0029] Specifically, based on the preprocessed panoramic image, a lightweight YOLOv8s model is used to locate the region of interest (ROI) of the fittings. The original image is then cropped according to the location bounding box to obtain a uniform 256×256 ROI image of the fittings. Images from the geometric contour branch and the grayscale abrupt change branch are the same ROI image output by S2.

[0030] Specifically, transmission line fittings generally possess two types of strong a priori physical characteristics: First, geometric structure a priori characteristics. The vast majority of fittings (such as U-rings, bolts, and cotter pins) have regular circular, annular, or rectangular topological structures with clear edges and stable outlines; deformation or loss directly leads to topological disruption. Second, material reflection a priori characteristics. Fittings are made of metal with smooth surfaces, producing strong specular reflections under illumination, forming localized areas of abrupt grayscale changes that differ significantly from the grayscale distribution of the non-metallic background. Based on these characteristics, this embodiment transforms the fitting detection problem into a combined problem of "geometric structure matching + metal reflective area detection."

[0031] Define the observed feature vector of the hardware region as: It can be broken down into: (1) In the formula: This represents the observed feature vector of the hardware region; These are the geometric structural characteristic components of the hardware. This refers to the characteristic component of abrupt changes in metallic grayscale; This represents the background noise component.

[0032] In complex scenarios, the amplitude of background noise is usually much larger than the amplitude of the fitting feature, satisfying: (2) In the formula: It is the L2 norm, used to calculate the magnitude of a vector; The threshold for determining noise amplitude; This represents the background noise component. However, the two types of features have complementary distribution characteristics. Geometric features exhibit topological sparsity, with a significant response only in the hardware outline region and a weak response in the background region. Gray-scale features exhibit local clustering: significant gray-scale abrupt changes only exist in the reflective metal region, while the gray-scale distribution in the background region is smooth. This complementary prior provides a theoretical basis for the dual-branch decoupled feature extraction.

[0033] S2: Establish a dual-branch physical feature extraction network, which includes a geometric contour feature branch, a gray-level abrupt change feature branch, and a topological constraint adaptive feature fusion module; The geometric contour feature branch is used to obtain the geometric contour features of the fittings based on the region of interest image of the fittings; The grayscale mutation feature branch is used to obtain the grayscale mutation features of the fittings based on the region of interest image of the fittings. The topology constraint adaptive feature fusion module is used to obtain the comprehensive features of the fitting after topology constraint fusion based on the fitting's geometric contour features and grayscale abrupt change features.

[0034] In this embodiment, as Figure 3 As shown, a dual-branch parallel backbone network, i.e., a dual-branch physical feature extraction network, is constructed. The input size is 256×256 pixels. The two branches process the same hardware ROI input completely in parallel and independently, and finally output feature maps of the same size (256×256×64), providing a unified feature dimension for subsequent topological constraint adaptive fusion. The core idea of ​​this network design is to decouple the two types of inherent physical characteristics of hardware into independent feature learning tasks, avoiding mutual interference between different types of features in general convolutional networks, thereby significantly improving the recognition accuracy of small target features.

[0035] Preferably, the geometric contour feature branch includes: an input layer, a first convolutional layer, a second convolutional layer, a first edge detection sub-branch, a second edge detection sub-branch, a third edge detection sub-branch, a feature concatenation layer, a non-maximum suppression layer, a channel mapping convolutional layer, a Hough circle detection layer, a geometric mask generation layer, and a pixel-wise multiplication layer; The input of the first convolutional layer is connected to the input layer to receive the region of interest image of the hardware through the input layer, and extract the underlying visual features based on the region of interest image of the hardware to obtain a basic texture feature map; the input of the second convolutional layer is connected to the output of the first convolutional layer, and is used to obtain a shallow structure feature map based on the basic texture feature map while preserving spatial information and filtering high-frequency noise. The input ends of the first edge detection sub-branch, the second edge detection sub-branch, and the third edge detection sub-branch are all connected to the output end of the second convolutional layer, and are used to extract the first edge feature, the second edge feature, and the third edge feature respectively based on the shallow structure feature map; wherein, the first edge feature, the second edge feature, and the third edge feature are edge features in three directions: horizontal, vertical, and 45° diagonal, and different pixel fittings are used.

[0036] The input of the feature splicing layer is connected to the output of the first edge detection sub-branch, the second edge detection sub-branch, and the third edge detection sub-branch, respectively, and is used to obtain a multi-scale edge fusion feature map based on the first edge feature, the second edge feature, and the third edge feature. The input of the non-maximum suppression layer is connected to the output of the feature splicing layer. It is used to perform pixel-wise non-maximum suppression based on the multi-scale edge fusion feature map, eliminate redundant edge responses, compress the edge width to a single pixel, and obtain a refined edge feature map. The input of the channel mapping convolutional layer is connected to the output of the non-maximum suppression layer, which is used to obtain a multi-scale refined edge feature map based on the refined edge feature map. The input of the Hough circle detection layer is connected to the output of the channel mapping convolutional layer. It is used to detect circular candidate regions that conform to the ring topology of the fittings in the feature map based on the multi-scale refined edge feature map, and obtain a set of candidate circle parameters. The input of the geometric mask generation layer is connected to the output of the Hough circle detection layer. It is used to obtain a geometric constraint mask with the same size as the multi-scale refined edge feature map based on the set of candidate circle parameters. The input of the pixel-wise multiplication layer is connected to the output of the geometric mask generation layer and the output of the channel mapping convolutional layer. It is used to obtain the geometric contour features of the fittings based on the geometric constraint mask and the multi-scale refined edge feature map, so as to output the geometric contour feature map of the fittings through the output layer.

[0037] Specifically, the geometric contour feature branch is designed specifically for the physical structure of the hardware's "regular geometric topology." It adopts a hybrid architecture of modular serial and multi-branch parallel processing, without any downsampling operations, fully preserving spatial resolution to ensure the positioning accuracy of extremely small targets. The basic convolutional encoding stage takes a 256×256×3 region of interest image of the hardware as input, and extracts basic visual features through two 3×3 convolutional layers. Both convolutional layers use a stride of 1 and padding=1, without pooling operations or activation functions. While preserving spatial information, it initially filters high-frequency noise, and finally outputs a 256×256×64 basic feature map.

[0038] Specifically, the execution process of the geometric contour feature branch includes the following three stages: (1) Basic convolutional encoding stage: This stage consists of a first convolutional layer and a second convolutional layer cascaded in series: The input end of the first convolutional layer is connected to the input layer, and the input is an RGB image of the region of interest of the hardware of 256×256×3. It adopts a 3×3 convolutional kernel, stride 1, padding 1 setting, without pooling layer and activation function, and is used to extract the low-level visual features and output a basic texture feature map of 256×256×32; The input end of the second convolutional layer is connected to the output end of the first convolutional layer, and adopts a 3×3 convolutional kernel, stride 1, padding 1 setting, without pooling layer and activation function, and is used to filter high-frequency noise while preserving spatial information and output a shallow structure feature map of 256×256×64.

[0039] (2) Multi-scale edge extraction stage: To address the issue of large differences in fitting size, a three-branch parallel edge detection structure is adopted. Sobel operators of three scales, namely 3×3, 5×5, and 7×7, are used to extract edges in the horizontal, vertical, and diagonal directions, corresponding to fittings of different sizes from 8 to 20 pixels. This stage includes a first edge detection sub-branch, a second edge detection sub-branch, a third edge detection sub-branch, a feature concatenation layer, a non-maximum suppression layer, and a channel mapping convolutional layer. The inputs of the first, second, and third edge detection sub-branches are all connected to the output of the second convolutional layer, and the three processes are performed in parallel. The first edge detection sub-branch uses a 3×3 Sobel operator to extract edge features in the horizontal, vertical, and 45° diagonal directions, suitable for ultra-small fittings of 8-10 pixels. The second edge detection sub-branch uses a 5×5 Sobel operator to extract edge features in the horizontal, vertical, and 45° diagonal directions, suitable for medium-sized fittings of 10-15 pixels. The third edge detection sub-branch uses a 7×7 Sobel operator to extract edge features in the horizontal, vertical, and 45° diagonal directions, suitable for large fittings of 15-20 pixels.

[0040] Specifically, the input of the feature concatenation layer is connected to the output of the three edge detection sub-branches, respectively, to concatenate the three edge features along the channel dimension. After concatenating the edge features of the three branches, redundant edge responses are removed by pixel-wise non-maximum suppression, retaining a continuous and clear contour of one pixel width, to obtain a multi-scale edge fusion feature map. The input of the non-maximum suppression layer is connected to the output of the feature concatenation layer, to perform pixel-wise non-maximum suppression, eliminate redundant edge responses, compress the edge width to one pixel, and output a refined edge feature map. The input of the channel mapping convolutional layer is connected to the output of the non-maximum suppression layer, using a 1×1 convolutional kernel to map the number of channels back to 64, outputting a 256×256×64 multi-scale refined edge feature map.

[0041] (3) Hough Topology Filtering Stage: This stage utilizes the prior knowledge that most hardware has a ring / circular topology structure. It filters background clutter using the ring topology prior and consists of a Hough circle detection layer, a geometric mask generation layer, and a pixel-wise multiplication layer. The input of the Hough circle detection layer is connected to the output of the channel mapping convolutional layer. It performs Hough circle detection on the multi-scale refined edge feature map, with a preset minimum center spacing of 10px, a detection radius range of 4~12px, and a center accumulation threshold of 15. This is used to detect circular candidate regions in the feature map that conform to the ring topology of the hardware, and outputs a set of candidate circle parameters. The input of the geometric mask generation layer is connected to the output of the Hough circle detection layer. It generates a geometric constraint mask with the same size as the multi-scale refined edge feature map based on the candidate circle parameters. The weight of regions conforming to the hardware topology is set to 1.2, and the weight of background regions is set to 0.8, outputting a 256×256×1 geometric constraint mask image. The input of the pixel-wise multiplication layer is connected to the output of the channel mapping convolutional layer and the output of the geometric mask generation layer, respectively. The multi-scale refined edge feature map and the geometric mask are subjected to pixel-wise Hadamard product operation to enhance the regional features that conform to the hardware topology and suppress the response of irregular background contours. The output is a 256×256×64 geometric contour feature map, which is the final output of this branch.

[0042] Preferably, the grayscale abrupt change feature branch includes: a grayscale conversion layer, a neighborhood difference calculation layer, a threshold segmentation layer, a morphological dilation layer, a third convolutional layer, and a fourth convolutional layer; The grayscale conversion layer is used to convert the RGB image into a single-channel grayscale image based on the region of interest image of the hardware using a standard grayscale conversion formula, thereby obtaining a grayscale base image. The input of the neighborhood difference calculation layer is connected to the output of the grayscale conversion layer, and is used to enhance the grayscale jump characteristics of the metal reflection area based on the grayscale base image, thereby obtaining a grayscale difference image. The input of the threshold segmentation layer is connected to the output of the neighborhood difference calculation layer, and is used to obtain a grayscale abrupt change binary image based on the grayscale difference image. The input of the morphological dilation layer is connected to the output of the threshold segmentation layer. The output connection is used to connect discrete mutation points into continuous regions based on the gray-scale mutation binary map, fill in the breaks in the reflective area, and obtain a continuous mutation region binary map. The input of the third convolutional layer is connected to the output of the morphological dilation layer, and is used to extract the underlying gray-scale mutation semantic features based on the continuous mutation region binary map to obtain a shallow gray-scale feature map. The input of the fourth convolutional layer is connected to the output of the third convolutional layer, and is used to enhance the nonlinear expression capability of the features and align the output dimension of the geometric branch based on the shallow gray-scale feature map to obtain a hardware gray-scale mutation feature map. The output of the fourth convolutional layer is the final output of the gray-scale mutation feature branch, outputting a 256×256×64 gray-scale mutation feature map.

[0043] Specifically, the grayscale abrupt change feature branch is designed specifically for the physical characteristic of "strong metallic reflectivity" in hardware. It processes the image directly in the grayscale space to maximize the preservation of the original metallic reflective features. The execution process of this branch includes three stages: (1) Neighborhood gray-level difference calculation stage: This stage consists of a gray-level conversion layer and a neighborhood difference calculation layer in sequence: The input end of the gray-level conversion layer is connected to the input layer. The input is a 256×256×3 metal ROI RGB image. The RGB image is converted into a single-channel gray-level image using the standard gray-level conversion formula. Then, through the neighborhood difference calculation layer, the maximum gray-level difference between each pixel and its 8 neighboring pixels in the 3×3 neighborhood is calculated. Using the maximum difference instead of the average difference can more sensitively capture the local drastic gray-level changes, realize the initial separation of metal and non-metal backgrounds, strengthen the gray-level jump characteristics of the metal reflective area, and output a 256×256×1 gray-level difference image.

[0044] (2) Threshold dilation enhancement stage: This stage includes a threshold segmentation layer and a morphological dilation layer. Taking the gray-level difference image as input, a fixed threshold T=32 optimized by statistical analysis of 6000 samples is used for segmentation. Pixels with a gray-level difference greater than 32 are marked as gray-level abrupt change points. Then, the morphological dilation layer is used to perform a morphological dilation operation on the binary image using a 3×3 full-1 convolution kernel, connecting the discrete abrupt change points into continuous regions, filling the gaps caused by uneven illumination in the reflective areas, and finally outputting a 256×256×1 gray-level abrupt change region binary image.

[0045] (3) Gray-scale feature encoding stage: This stage includes a third convolutional layer and a fourth convolutional layer. To address the lack of high-level semantic features in binary images, feature encoding is performed using two 3×3 convolutional layers. Both convolutional layers use a stride of 1 and padding of 1. The third and fourth convolutional layers use the ReLU activation function to enhance the non-linear expressive power of the features. The third convolutional layer maps the number of channels from 1 to 32 to extract the semantic features of the underlying gray-scale abrupt changes, outputting a shallow gray-scale feature map of 256×256×32. The fourth convolutional layer maps the number of channels from 32 to 64 to enhance the non-linear expressive power of the features, aligns the output dimension of the geometric branch with the output dimension of the geometric contour feature branch, provides a unified input for subsequent feature fusion, and finally outputs a gray-scale abrupt change feature map of 256×256×64.

[0046] Preferably, the topology constraint adaptive feature fusion module includes: a spatial topology weight generation module, a spatial feature enhancement module, a channel adaptive weighting module, and a channel-by-channel feature fusion module; Specifically, such as Figure 4 As shown, the topology constraint adaptive feature fusion module includes an input layer, a spatial topology weight generation module, a spatial feature enhancement module, a channel adaptive weighting module, a channel-by-channel feature fusion module, and an output layer. The module as a whole follows a serial progressive logic of "spatial topology enhancement → channel adaptive weighting → element-by-element fusion". The input end is connected to the output end of the dual-branch physical feature extraction network, and the output end is connected to the subsequent hardware-specific small-size candidate box generation module and the detection output head.

[0047] The input layer includes a first input terminal, a second input terminal, and a third input terminal; The first input terminal is connected to the output terminal of the geometric contour feature branch, and is used to obtain a geometric contour feature map with a size of 256×256×64 based on the metal fitting geometric contour feature map output by the geometric contour feature branch; The second input terminal is connected to the output terminal of the gray-scale abrupt change feature branch, and is used to obtain a gray-scale abrupt change feature map with a size of 256×256×64 based on the gray-scale abrupt change feature map of the hardware output by the gray-scale abrupt change feature branch; the spatial resolution and channel number of the two feature maps are completely consistent, satisfying the pixel-level spatial alignment requirements; The third input terminal is connected to the output terminal of the Hough topology filtering layer of the geometric contour feature branch, and is used to obtain a set of candidate circle parameters based on the detection results output by the Hough topology filtering layer. The candidate circle parameters include the center coordinates, radius, and cumulative vote count of each candidate circle, eliminating the need to repeatedly perform Hough circle detection and avoiding computational redundancy.

[0048] Preferably, the spatial topology weight generation module is used to obtain the topological confidence of the candidate circle based on the candidate circle parameter set, so as to obtain the geometric constraint weight matrix based on the fitting geometric contour feature map; Specifically, the spatial topology weight generation module includes a topology confidence calculation layer and a topology weight generation layer connected in series. The two layers are cascaded in series along the data flow direction, and the output of the previous layer is the input of the next layer. It is used to generate a geometric constraint weight matrix of spatial dimension based on the hardware ring topology prior, and to perform differentiated weight allocation between the hardware region and the background region.

[0049] The input terminal of the topology confidence calculation layer is connected to the third input terminal, and is used to calculate the topology confidence of each candidate circle based on the candidate circle parameter set, thereby obtaining the single-circle confidence value. The topology confidence calculation formula is: (3) In the formula, The candidate circle number is... For the first Topological confidence of each candidate circle; For the first The center of each candidate circle is summed up with the number of votes. The higher the number of votes, the better the edge continuity and the clearer the topological structure of the circular region. The threshold for accumulating the center of the Hough circle detection is set to 15. The calculated confidence score is constrained within the interval [1, 2]. The more clearly the topological candidate circle is identified, the higher the confidence score, and the greater the subsequent feature enhancement.

[0050] The input of the topology weight generation layer is connected to the output of the topology confidence calculation layer. It is used to generate a geometric constraint weight matrix with the same spatial size as the input feature map, based on the position parameters of the candidate circles and their corresponding topology confidence, thus obtaining a geometric constraint weight matrix with a size of 256×256×1. The pixel-by-pixel generation rule for the weight matrix is ​​as follows: for any pixel point on the geometric contour feature map of the hardware... If a pixel falls within the coverage area of ​​any candidate circle, the highest confidence value among all candidate circles covering the pixel is taken as the weight of that pixel; if the pixel does not fall within the coverage area of ​​any candidate circle, a fixed base weight of 0.8 is taken as the weight of that pixel. The pixel-by-pixel calculation formula is expanded as follows: (4) In the formula: This is the geometric constraint weight matrix; All are pixel coordinates on the geometric contour feature map of the hardware; For the first Topological confidence of each candidate circle; For the first The pixel coverage area of ​​each candidate circle As the basic weight for the background region, =0.8.

[0051] Preferably, the spatial feature enhancement module is used to obtain a spatially enhanced geometric feature map based on the geometric constraint weight matrix.

[0052] Specifically, the spatial feature enhancement module is a pixel-wise Hadamard product operation layer, whose input is connected to the first input and the output of the topology weight generation layer, respectively. It is used to enhance the feature amplitude of regions that conform to the topology of the fittings and suppress the feature amplitude of cluttered edge regions in the background, based on the fitting geometric contour feature map and the geometric constraint weight matrix, to obtain a spatially enhanced geometric feature map with a size of 256×256×64.

[0053] The specific calculation process employs a channel broadcasting mechanism: the geometric constraint weight matrix of a single channel is copied 64 times along the channel dimension, ensuring that the geometric constraint weight matrix perfectly matches the dimension of the hardware geometric contour feature map. Then, element-wise multiplication is performed on the pixel values ​​at corresponding positions. The calculation formulas for each channel and each pixel are as follows: (5) In the formula, The c-th channel and pixel coordinates of the geometric contour feature map of the hardware Eigenvalues ​​at; The geometric constraint weight matrix in pixel coordinates The weight value at the location; The c-th channel and pixel coordinates of the spatially enhanced geometric feature map The eigenvalues ​​at the location are calculated. Through this operation, the characteristic amplitude of the hardware topology region is amplified, and the stray features of the background region are compressed, achieving precise noise reduction guided by physical priors.

[0054] Preferably, the channel adaptive weighting module includes a first channel weight learning layer, a second channel weight learning layer, and a channel normalization layer; wherein the first channel weight learning layer and the second channel weight learning layer are completely parallel independent network structures, respectively corresponding to the channel importance learning of the two features; the channel normalization layer receives the output of the two weights and performs normalization constraints to ensure the stability of the fused numerical values.

[0055] The input of the first channel weight learning layer is connected to the output of the spatial feature enhancement module, and is used to automatically learn the importance weights of each channel of the geometric features based on the spatially enhanced geometric feature map, thereby obtaining an original geometric channel weight vector with a dimension of 64. '.

[0056] Specifically, the first channel weight learning layer adopts a three-level sequential channel attention architecture consisting of a global average pooling layer, a first fully connected layer, and a second fully connected layer. The structure and function of each layer are as follows: The global average pooling layer receives a spatially enhanced geometric feature map with dimensions of 256×256×64 at its input. It performs an average value calculation on all spatial pixels of each feature channel, compressing the two-dimensional spatial features into a single-valued global channel representation, thus achieving channel-level global information aggregation. The calculation formula expands to: (6) In the formula, These represent the height and width of the feature map, respectively. =256、 =256; The c-th channel and pixel coordinates of the spatially enhanced geometric feature map Eigenvalues ​​at; For the first c The global average pooling output value of each channel. This layer outputs channel feature vectors of size 1×1×64.

[0057] The first fully connected layer (dimensionality reduction layer) receives a 1×1×64-channel feature vector from the global average pooling layer as input. It has 16 neurons and uses the ReLU activation function. This layer reduces the dimensionality of the high-dimensional channel features, decreasing the number of network parameters and computational cost, while simultaneously learning the non-linear relationships between channels. The output of this layer is a 1×1×16 dimensionality-reduced feature vector.

[0058] The second fully connected layer (dimensionality-upgrading layer) receives the 1×1×16 dimensionality-reduced feature vector output from the first fully connected layer as input. It has 64 neurons and uses the sigmoid function to map the output values ​​to the [0, 1] interval. This layer restores the feature dimension to the original number of channels and outputs the importance weight coefficient for each channel. The final output of this layer is a 64-dimensional original geometric channel weight vector. in The weights are the original geometric features of the c-th channel.

[0059] The input of the second channel weight learning layer is connected to the second input, and is used to automatically learn the importance weights of each channel of the grayscale feature based on the grayscale abrupt change feature map of the hardware, to obtain an original grayscale channel weight vector with a dimension of 64. The network structure, parameter settings, and hierarchical connection relationships of the second channel weight learning layer are completely consistent with those of the first channel weight learning layer. It also contains a three-level structure of a global average pooling layer, a first fully connected layer, and a second fully connected layer, which will not be described in detail here.

[0060] The input of the channel normalization layer is connected to the output of the first channel weight learning layer and the output of the second channel weight learning layer, respectively. Referring to the weighted normalization mechanism of bidirectional cross-scale feature fusion, the weights of the two original channels are normalized channel by channel to obtain the normalized geometric channel weight vector. With normalized grayscale channel weight vector .

[0061] Specifically, introduce a minimal constant. =0.0001 To avoid numerical instability caused by a denominator of 0, the channel-by-channel normalization calculation formula is as follows: (7) Preferably, the channel-by-channel feature fusion module has its input terminals connected to the output terminals of the spatial feature enhancement module, the second input terminal, and the channel normalization layer, respectively, for using the spatial enhanced geometric feature map, the hardware grayscale abrupt change feature map, and the normalized geometric channel weight vector. With normalized grayscale channel weight vector Perform a channel-by-channel weighted summation operation to obtain a fused feature map with a size of 256×256×64, which is the comprehensive feature of the hardware after topological constraint fusion.

[0062] The specific calculation process employs a spatial broadcasting mechanism: the single-value channel weights are copied along the spatial dimension to a size of 256×256, ensuring a one-to-one correspondence between the weights and all spatial pixels of the corresponding channel. These weights are then multiplied by the two feature maps and summed pixel-by-pixel. The pixel-by-pixel, channel-by-channel fusion calculation formula is as follows: (8) In the formula, To fuse the c-th channel of the feature map and pixel coordinates Eigenvalues ​​at; The c-th channel and pixel coordinates of the spatially enhanced geometric feature map Eigenvalues ​​at; The c-th channel and pixel coordinates of the grayscale abrupt change feature map Eigenvalues ​​at; , These are the normalized geometric channel weights and normalized grayscale channel weights for the c-th channel, respectively.

[0063] Substituting equation (5) into equation (8), we obtain the complete expansion of the fusion formula: (9) The input end of the output layer is connected to the output end of the channel-by-channel feature fusion module, and is used to output the fused feature map to the subsequent hardware-specific small-size candidate box generation module and detection output head for final hardware classification and position regression.

[0064] S3: Obtain a panoramic image of the transmission line to be detected. Based on the trained dual-branch physical feature extraction network, obtain the comprehensive feature map of the hardware in the panoramic image of the transmission line to be detected. Then, use a multi-scale small target anchor box clustering algorithm to obtain metal-specific candidate boxes for the size of the hardware, and output the final detection result of the spatial position of the hardware.

[0065] Specifically, based on the comprehensive feature map of the fittings fused with the aforementioned topological constraints, a multi-scale small target anchor box clustering algorithm is used to generate fitting-specific candidate boxes adapted to the size of miniature fittings. Further, the fitting-specific candidate boxes are predicted and corrected using CIoU bounding box regression, and then redundant and overlapping candidate boxes are removed using non-maximum suppression (NMS), finally outputting the detection result of the fitting's spatial location.

[0066] This embodiment is based on the fused feature map output by the topological constraint adaptive feature fusion module. It uses a two-stage method of offline clustering design + online feature map deployment to generate small candidate boxes specific to hardware. The specific process is as follows: (1) Offline anchor frame clustering stage: For the scenario prior where the pixel size of transmission line fittings is concentrated in 8~20 pixels, the width and height of all fittings annotation boxes in the training set are statistically analyzed, and the k-means clustering algorithm is used to cluster the anchor frame reference size; the number of cluster centers is set to 3, and the intersection-over-union ratio (IoU) is used as the distance metric. Through iterative optimization, three types of exclusive anchor frame reference sizes adapted to micro fittings are obtained: 8×8, 12×12, and 16×16, which correspond to the size range of 8~10 pixel ultra-small fittings, 10~15 pixel medium fittings, and 15~20 pixel large fittings, respectively; this exclusive anchor frame replaces the large-size anchor frame of the general detection algorithm, which can significantly reduce the number of invalid candidate boxes and improve the matching accuracy of small target detection.

[0067] (2) Online candidate box deployment stage: At each pixel position of the fusion feature map of 256×256×64, the above three types of pre-set small anchor boxes for hardware are deployed in sequence as initial detection candidate boxes; since the fusion feature map is not downsampled throughout the process and retains the complete spatial resolution, each candidate box can accurately correspond to the small-sized hardware area of ​​the original image, ensuring the localization granularity of extremely small targets.

[0068] Specifically, based on the hardware-specific small candidate box, the final detection result is obtained through four steps: detection head prediction, confidence screening, non-maximum suppression, and coordinate mapping. The specific process is as follows: (1) Parallel prediction by the detection head: The fused feature map with dedicated small candidate boxes is sent to the detection output head. The detection output head contains two parallel processing branches: a classification branch and a regression branch. The classification branch predicts the category of each candidate box and outputs the probability score of each candidate box for five categories: normal, missing, deformed, corroded, and occluded. During the training phase, the weighted cross-entropy loss function for difficult samples is used for supervision. The loss weight is increased for extremely small-sized and occluded hardware samples to guide the network to focus on optimizing the classification accuracy of difficult scenarios.

[0069] The regression branch performs offset regression on the bounding box position of each candidate box, predicting the center coordinate offset and width-height scaling factor of the candidate box relative to the actual hardware position. During the training phase, the CIoU loss function with circumcircle geometric constraints is used for supervision. In addition to the three constraints of intersection-union ratio, center distance, and aspect ratio, a new constraint of circular topology of the hardware is added, so that the position regression process takes into account both the positioning accuracy of the rectangular box and the fit of the actual hardware contour.

[0070] (2) Initial screening of confidence: Set the classification confidence threshold to 0.5, filter out low confidence candidate boxes with category scores below the threshold, retain high confidence effective candidate boxes, and reduce subsequent calculations.

[0071] (3) Non-maximum suppression (NMS): Non-maximum suppression is performed on the retained high-confidence candidate boxes. The boxes are sorted from high to low classification confidence and the intersection-union ratio (IoU) between the current candidate box and the remaining candidate boxes is calculated in turn. When the IoU between candidate boxes is greater than the preset threshold of 0.45, the candidate boxes with higher confidence are retained and redundant overlapping candidate boxes are removed to avoid the same hardware being detected repeatedly.

[0072] (4) Coordinate mapping: The coordinates of the candidate boxes that are finally retained are mapped from the scale of the fused feature map back to the scale of the 256×256 hardware ROI image. Then, combined with the positioning coordinates of the hardware ROI in the original panoramic image, the coordinates are mapped back to the coordinate system of the original transmission line panoramic image. Finally, the complete detection result containing hardware category, confidence score and spatial location box is output.

[0073] Preferably, the joint loss function of the multi-dimensional physical constraints is expressed as follows: Specifically, to further enhance the network's learning of the physical characteristics of hardware and improve the detection accuracy of extremely small targets and robustness in occlusion scenarios, this embodiment constructs a multi-dimensional physical constraint joint loss function for end-to-end supervised training of the network. The total loss function consists of four weighted components: hard sample weighted classification loss, multi-scale geometric perception regression loss, multi-level topological consistency constraint loss, and bi-branch feature consistency loss. The total loss formula is as follows: (10) In the formula: The total loss value is the joint loss function for multi-dimensional physical constraints. A weighted classification loss for difficult samples is used to monitor the classification accuracy of hardware categories and defect types. Multi-scale geometric perception regression loss is used to monitor the position regression accuracy of the hardware inspection frame; This is a multi-level topology consistency constraint loss, used to supervise the geometric topological integrity of the detection results; This is a bi-branch feature consistency loss, used to monitor the response consistency of bi-branch features in the hardware region; , , These are all balancing weight coefficients for the loss term, and the optimal values ​​have been verified through ablation experiments. =1.0、 =0.3、 =0.2.

[0074] Specifically, based on the standard cross-entropy classification loss, an adaptive weighted mechanism for difficult samples is introduced. This increases the loss weight for samples with high detection difficulty, such as extremely small-sized hardware and occluded hardware, guiding the network to focus on optimizing classification accuracy for industry pain points. The difficult sample weighted classification loss is as follows: (11) In the formula: is the total number of hardware samples in the training batch; i is the index of the hardware sample in the training batch; Let be the loss weight for the i-th sample. If the target pixel size of the sample is less than 8 pixels or the occlusion ratio is greater than 20%, then... =2.0, other ordinary samples =1.0. The categories are numbered sequentially, and include five categories: normal, missing, deformed, corroded, and obscured. For the i-th sample The true label for a class, with a value of 1 for positive samples and a value of 0 for negative samples; The probability that the i-th sample predicted by the network belongs to the j-th class is given; corresponding to the small-sized candidate boxes for the three types of hardware (8×8, 12×12, and 16×16), regression supervision is performed simultaneously on the three detection scales; based on the standard CIoU loss, the geometric constraints of the circumscribed circle are incorporated to ensure that the position regression process takes into account both the positioning accuracy of the rectangular box and the matching degree of the hardware geometry.

[0075] Specifically, the multi-scale geometric perception regression loss formula is expanded as follows: (12) In the formula, The number of detection scales; This is the detection scale number. Let be the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box at the m-th detection scale; It is a Euclidean distance metric function; The coordinates of the center point of the prediction box. The coordinates of the center point of the annotation box; For the first Under the detection scale, the diagonal length of the smallest bounding rectangle that simultaneously contains the predicted bounding box and the labeled bounding box; This parameter ensures consistency in aspect ratio between the predicted bounding box and the labeled bounding box. This is the balancing weighting coefficient for aspect ratio loss; These are the weighting coefficients for the circumcircle geometric constraints, used to balance the contribution ratios of position regression and shape constraints. =0.3; Let be the intersection-union ratio of the minimum outer circle of the predicted bounding box and the minimum outer circle of the labeled bounding box at the m-th detection scale; (13) (14) In the formula, , These are the width and height of the prediction box, respectively. These represent the width and height of the annotation box, respectively.

[0076] Specifically, the physical prior supervision is moved from the detection output layer to the feature extraction layer, constraining the consistency of geometric and grayscale feature responses within the real area of ​​the hardware. This guides the dual-branch network to focus on the same hardware target, suppressing stray feature responses from the background region and further reducing the background misclassification rate. The loss is constructed based on the cosine distance between the two types of feature responses within the real area of ​​the hardware. The formula for calculating the dual-branch feature consistency loss is as follows: (15) In the formula, The set of pixels representing the actual area of ​​the labeled hardware; For geometric contour feature maps in pixels The response intensity at that pixel location is the maximum value of all channel feature values ​​at that pixel location. For grayscale abrupt change feature map at pixel The response intensity at a given pixel is the maximum value of all channel feature values ​​at that pixel location. A smaller loss value indicates a higher consistency in the response distribution of the two types of features in the hardware region, and better feature focus of the dual-branch network; conversely, a larger loss value indicates misalignment of the two types of feature responses, and insufficient targeting of the network's feature learning.

[0077] The detection performance comparison experiment of this embodiment is as follows: This embodiment's dataset is constructed based on publicly available power line inspection datasets and aerial images of actual transmission lines. A total of 6000 images containing various types of hardware were selected, including: 2400 normal hardware samples, 1200 missing samples, 1000 deformed samples, 800 corroded samples, and 600 partially occluded samples. The LabelImg annotation tool was used to accurately annotate the location and defect type of the hardware, with the annotation box size controlled between 8-25 pixels. Schematic diagrams of various hardware types are shown below. Figure 5 As shown.

[0078] The input aerial images were normalized to 640×640 pixels, and median filtering was used to remove salt-and-pepper noise. A lightweight YOLOv8s model was used as the fitting localization module. During training, only the insulator fitting area was labeled, not the specific fittings. The training batch size was 16, the initial learning rate was 0.001, and the AdamW optimizer was used for 100 iterations. The final fitting localization accuracy reached 98.9%.

[0079] The hardware positioning module generates a 256×256 pixel ROI for hardware, eliminates irrelevant background interference across the entire area, and restricts subsequent detection tasks to the ROI, which greatly reduces the amount of computation and improves detection accuracy.

[0080] Furthermore, the hardware detection method based on a dual-branch physical feature extraction network proposed in this embodiment is compared with the performance of four representative mainstream target detection algorithms, including Faster R-CNN, RetinaNet, YOLOv5s, and YOLOv8s. The first two are two-stage detection algorithms, while the latter three are single-stage lightweight detection algorithms.

[0081] Five standard metrics were used to evaluate detection performance: mean accuracy (mAP@0.5) measures the overall detection accuracy of the model; mAP@0.75 measures the model's high-precision detection capability; overall false negative rate is the proportion of undetected hardware items to the total number of hardware items; occlusion false positive rate is the proportion of background areas misidentified as hardware items to the total detection results; and single-image inference time is the average processing time from input to output for a single image. Lower values ​​for the first four metrics indicate better detection performance, while a lower value for the last metric indicates a more lightweight model.

[0082] On a self-built test set, the comprehensive performance of the method in this embodiment is compared with that of the current mainstream small target detection algorithms, as shown in Table 1.

[0083] Table 1. Comparison of overall performance of different algorithms

[0084] Experimental results show that the method in this embodiment significantly outperforms mainstream algorithms in all performance metrics. Specifically, mAP@0.5 is 7.8 percentage points higher than the current best YOLOv8s, the occlusion false positive rate is reduced by 30.0 percentage points, and the inference time is shortened by 49%. It achieves a qualitative breakthrough in small target detection and anti-occlusion capabilities, and fully meets the real-time requirements of UAV inspection.

[0085] Table 2 Comparison of testing performance of hardware of different sizes

[0086] Considering the extremely small size of hardware fittings, the test set was divided into three intervals based on target pixel size: <8 pixels, 8-12 pixels, and 12-20 pixels. The detection performance of different algorithms in each size range was compared, and the results are shown in Table 2. Experimental results show that the method in this embodiment has the most significant advantage in detecting ultra-small hardware fittings <8 pixels, with mAP@0.5 improved by 22.0 percentage points compared to YOLOv8s. This proves that the dual-branch design based on physical features can effectively extract the weak features of extremely small targets, solving the industry pain point of general algorithms' serious missed detection of ultra-small targets.

[0087] To verify the independent contribution of each innovative module, this embodiment designed a univariate ablation experiment, as shown in Table 3. Using YOLOv8s as the baseline model, improvements were gradually added starting from the baseline model. Prior region guidance shortened the inference time by 17% and improved mAP by 1.6 percentage points, demonstrating the importance of removing background interference for small target detection. Dual-branch feature extraction performed better than using either branch alone, indicating that the geometric contour and gray-scale abrupt change features in this embodiment are complementary. Topological constraint fusion improved mAP by 1.1 percentage points and reduced the occlusion false positive rate by 7.5 percentage points compared to ordinary splicing fusion, proving that physical prior-guided feature fusion is more effective. The optimal λ value for geometric constraint loss was 0.3, at which point the model achieved the best balance between detection accuracy and anti-occlusion capability. Small prior candidate boxes further shortened the inference time by 47% without sacrificing accuracy, significantly improving detection efficiency.

[0088] Table 3 Ablation Experiment Results

[0089] The detection performance of the algorithm was tested at five different occlusion levels: 0%, 10%, 20%, 30%, and 40%. Figure 6 As shown, the experimental results indicate that the method of the present invention can still maintain 77.8% mAP under 30% occlusion and control the false detection rate within 11.5%. Even under 40% heavy occlusion, it can still detect most defects, demonstrating strong anti-occlusion capabilities and meeting the actual inspection needs in complex outdoor environments.

[0090] This embodiment addresses the industry challenges of high false negative rates for extremely small targets, severe misjudgments under complex backgrounds and occlusion interference, and insufficient utilization of physical characteristics in transmission line fitting inspection. It constructs a two-stage deep learning framework of "localization first, detection later," accurately extracting the region of interest (ROI) of the fitting through a pre-trained lightweight model, effectively eliminating over 90% of globally irrelevant background interference. Based on the inherent physical prior of the fitting's "regular geometric topology + strong metallic reflectivity," a decoupled feature extraction network is designed with parallel geometric contour feature branches and gray-scale abrupt change feature branches, enabling independent learning of two complementary features. Simultaneously, a topological constraint adaptive feature fusion mechanism and a geometric topological constraint loss function are introduced to strengthen the feature representation that conforms to the fitting's physical morphology, solving the problems of insufficient feature specificity and poor robustness in occluded scenarios in general algorithms. Combining the prior of the fitting's actual pixel size, three types of dedicated small-sized candidate boxes (8×8, 12×12, and 16×16) are generated, significantly reducing invalid computations without sacrificing detection accuracy, thus significantly improving detection efficiency and enabling the model to meet the real-time requirements of UAV inspection.

[0091] Experimental results show that the method in this embodiment significantly outperforms mainstream target detection algorithms such as Faster R-CNN and YOLOv8s in terms of overall detection accuracy, detection capability of extremely small targets, anti-occlusion performance, and inference speed. It can be directly deployed on UAV edge computing terminals. In the future, we will further expand the types of defect detection, integrate infrared and ultraviolet multimodal data to improve the detection accuracy in complex working conditions, and continuously optimize the lightweight nature of the model to adapt to edge devices with lower computing power.

[0092] This embodiment fully leverages the physical characteristics of hardware, constructs a two-stage framework of "first positioning, then detection," and extracts geometric contours and gray-scale abrupt change features through a dual-branch decoupled network. Combined with a topological constraint fusion mechanism and a geometric constraint loss function (a multi-dimensional physical constraint joint loss function), it effectively solves the problems of missed detection of extremely small targets and misjudgment in complex scenarios.

[0093] In summary, the main contributions of this embodiment are as follows: (1) A two-stage “localization-detection” cascade architecture is proposed. The pre-trained lightweight model achieves 98.9% accuracy in localizing the hardware region, accurately extracts the region of interest (ROI) of the hardware and removes more than 90% of the global irrelevant background interference. Ablation experiments show that this prior region guidance mechanism can improve the overall detection mAP by 1.6% and shorten the inference time of a single image by 17%.

[0094] (2) Design a physical prior-driven dual-branch decoupled feature extraction network, and construct independent feature learning branches for the regular geometric topology of hardware and the strong reflective properties of metal respectively, so as to realize the independent learning of two complementary features of geometric structure and metal reflectivity. Compared with using a single feature branch alone, the dual-branch fusion design can improve the average detection accuracy (mAP) by 1.9~3.2 percentage points, effectively solving the problem of insufficient feature targeting of general network.

[0095] (3) Introducing a topological constraint adaptive feature fusion mechanism and a geometric topological constraint loss function (multi-dimensional physical constraint joint loss function) to enhance the feature expression that conforms to the physical form of the fittings, and to achieve accurate differentiation between shadow occlusion and real defects. The overall detection mAP@0.5 (the suffix @0.5 represents the IoU (intersection over union ratio of the predicted box and the real labeled box) judgment threshold, which is the passing line for judging whether the fittings are successfully detected) can reach 86.3%. For ultra-small fittings below 8×8 pixels, the mAP reaches 73.6%. The misjudgment rate in the 30% occlusion scene is reduced to 11.5%, and the inference time is shortened to 42ms. (4) Based on the prior of the actual pixel size of the hardware, three types of exclusive small-sized candidate boxes of 8×8, 12×12 and 16×16 are generated. This mechanism can further shorten the inference time of a single image by 47% without losing detection accuracy.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting transmission line fittings based on the fusion of geometric and grayscale features, characterized in that, Includes the following steps: S1: Acquire panoramic images of the transmission line and preprocess them. Based on the preprocessed panoramic images, use a pre-trained lightweight YOLOv8s model to obtain the region of interest images of the fittings. S2: Establish a dual-branch physical feature extraction network and train it using a multi-dimensional physical constraint joint loss function. The dual-branch physical feature extraction network includes a geometric contour feature branch, a gray-scale abrupt change feature branch, and a topological constraint adaptive feature fusion module. The geometric contour feature branch is used to extract features from the region of interest image of the hardware, and three parallel edge detection sub-branches are used to obtain three edge features and then stitch them together to obtain a multi-scale refined edge feature map. The Hough circle detection is used to obtain a set of candidate circle parameters, and then the geometric contour feature map of the hardware is obtained. The gray-scale mutation feature branch is used to obtain a gray-scale mutation binary map based on the region of interest image of the hardware, to obtain a shallow gray-scale feature map, and then to obtain a gray-scale mutation feature map of the hardware. The topology constraint adaptive feature fusion module is used to obtain the comprehensive feature map of the hardware after topology constraint fusion based on the hardware geometric contour feature map and the hardware grayscale abrupt change feature map; S3: Obtain a panoramic image of the transmission line to be detected, and obtain the comprehensive feature map of the hardware of the panoramic image of the transmission line to be detected based on the trained dual-branch physical feature extraction network. Then, obtain the metal-specific candidate boxes of the hardware size, and output the final detection result of the hardware spatial position.

2. The method for detecting transmission line fittings based on geometric and grayscale feature fusion according to claim 1, characterized in that, The geometric contour feature branch includes: an input layer, a first convolutional layer, a second convolutional layer, a first edge detection sub-branch, a second edge detection sub-branch, a third edge detection sub-branch, a feature concatenation layer, a non-maximum suppression layer, a channel mapping convolutional layer, a Hough circle detection layer, a geometric mask generation layer, and a pixel-wise multiplication layer. The input of the first convolutional layer is connected to the input layer to receive the region of interest image of the hardware through the input layer and extract the low-level visual features of the region of interest image of the hardware to obtain a basic texture feature map; the input of the second convolutional layer is connected to the output of the first convolutional layer to filter high-frequency noise while preserving spatial information based on the basic texture feature map to obtain a shallow structure feature map. The inputs of the first, second, and third edge detection sub-branches are all connected to the output of the second convolutional layer to extract the first, second, and third edge features using Sobel operators at three scales (3×3, 5×5, and 7×7) based on the shallow structure feature map. The input of the feature concatenation layer is connected to the outputs of the first, second, and third edge detection sub-branches to obtain a multi-scale edge fusion feature map based on the first, second, and third edge features. The input of the non-maximum suppression layer is connected to the output of the feature concatenation layer to perform pixel-wise non-maximum suppression based on the multi-scale edge fusion feature map, eliminating redundant edge responses and compressing the edge width to a single pixel to obtain a refined edge feature map. The input of the channel mapping convolutional layer is connected to the output of the non-maximum suppression layer to obtain a multi-scale refined edge feature map based on the refined edge feature map. The input of the Hough circle detection layer is connected to the output of the channel mapping convolutional layer to perform Hough circle detection on the multi-scale refined edge feature map and obtain a set of candidate circle parameters. The input of the geometric mask generation layer is connected to the output of the Hough circle detection layer and is used to obtain a geometric constraint mask with the same size as the multi-scale refined edge feature map based on the set of candidate circle parameters. The input of the pixel-wise multiplication layer is connected to the output of the geometric mask generation layer and the output of the channel mapping convolutional layer and is used to obtain the geometric contour features of the fitting based on the geometric constraint mask and the multi-scale refined edge feature map, so as to output the geometric contour feature map of the fitting through the output layer.

3. The method for detecting transmission line fittings based on geometric and grayscale feature fusion according to claim 2, characterized in that, The gray-level abrupt change feature branch includes: gray-level conversion layer, neighborhood difference calculation layer, threshold segmentation layer, morphological dilation layer, third convolutional layer, and fourth convolutional layer; The grayscale conversion layer is used to convert the region of interest (ROI) image of the RGB image into a single-channel grayscale image based on the ROI image of the fittings, thereby obtaining a grayscale base image; the input end of the neighborhood difference calculation layer is connected to the output end of the grayscale conversion layer to enhance the grayscale jump characteristics of the metal reflective area based on the grayscale base image, thereby obtaining a grayscale difference image. The input of the threshold segmentation layer is connected to the output of the neighborhood difference calculation layer, and is used to obtain a gray-level abrupt change binary map based on the gray-level difference map; the input of the morphological dilation layer is connected to the output of the threshold segmentation layer, so as to connect discrete abrupt change points into continuous regions based on the gray-level abrupt change binary map, fill in the breaks in the reflective region, and obtain a continuous abrupt change region binary map. The input of the third convolutional layer is connected to the output of the morphological dilation layer to extract the semantic features of the bottom gray-level mutation based on the binary map of the continuous mutation region and obtain a shallow gray-level feature map; the input of the fourth convolutional layer is connected to the output of the third convolutional layer to align the output dimension of the geometric branch based on the shallow gray-level feature map and obtain a gray-level mutation feature map of the hardware.

4. The method for detecting transmission line fittings based on geometric and grayscale feature fusion according to claim 3, characterized in that, The topology constraint adaptive feature fusion module includes: a spatial topology weight generation module, a spatial feature enhancement module, a channel adaptive weighting module, and a channel-by-channel feature fusion module; The spatial topology weight generation module is used to obtain the topological confidence of the candidate circle based on the candidate circle parameter set, so as to obtain the geometric constraint weight matrix based on the hardware geometric contour feature map. The spatial feature enhancement module is used to obtain a spatially enhanced geometric feature map based on the geometric constraint weight matrix; The channel adaptive weighting module is used to obtain the original geometric channel weight vector by using the first channel weight learning layer according to the spatial enhanced geometric feature map, and to obtain the original grayscale channel weight vector by using the second channel weight learning layer according to the hardware grayscale change feature map. The channel-by-channel feature fusion module is used to obtain the fused feature map, i.e., the comprehensive feature map of the fittings after topological constraint fusion, based on the original geometric channel weight vector and the original grayscale channel weight vector.

5. The method for detecting transmission line fittings based on geometric and grayscale feature fusion according to claim 4, characterized in that, The formula used to obtain the topological confidence of candidate circles is as follows: In the formula, The candidate circle number is... For the first Topological confidence of each candidate circle; For the first The cumulative vote count of the centers of each candidate circle; The threshold is accumulated for the center of the Hough circle detection; The formula used to obtain the geometric constraint weight matrix is ​​as follows: In the formula: This is the geometric constraint weight matrix; All are pixel coordinates on the geometric contour feature map of the hardware; For the first Topological confidence of each candidate circle; For the first The pixel coverage area of ​​each candidate circle The background region is the basic weight.

6. The method for detecting transmission line fittings based on geometric and grayscale feature fusion according to claim 5, characterized in that, The formula used to obtain the spatially augmented geometric feature map is as follows: In the formula, The first geometric profile feature diagram of the hardware c Channels, pixel coordinates Eigenvalues ​​at; The geometric constraint weight matrix in pixel coordinates The weight value at the location; For the spatially enhanced geometric feature map, the first c Channels, pixel coordinates Eigenvalues ​​at; This is the channel index for the geometric profile feature map of the hardware.

7. The method for detecting transmission line fittings based on geometric and grayscale feature fusion according to claim 6, characterized in that, The formula used to obtain the fused feature map is as follows: In the formula, To fuse feature maps c Channels, pixel coordinates Eigenvalues ​​at; For the spatially enhanced geometric feature map, the first c Channels, pixel coordinates Eigenvalues ​​at; The grayscale mutation feature map c Channels, pixel coordinates Eigenvalues ​​at; , The first c Normalized geometric channel weights and normalized grayscale channel weights; ' is the original geometric channel weight vector; This is the original grayscale channel weight vector; It is a very small constant.

8. The method for detecting transmission line fittings based on geometric and grayscale feature fusion according to claim 1, characterized in that, The joint loss function for the multi-dimensional physical constraints is expressed as follows: In the formula: The total loss value is the joint loss function for multi-dimensional physical constraints. Weighted classification loss for difficult samples; For multi-scale geometric perception regression loss; The loss is due to multi-level topology consistency constraints; For bi-branch feature consistency loss; , , These are all the balancing weight coefficients for the loss term; in, In the formula: is the total number of hardware samples in the training batch; i is the index of the hardware sample in the training batch; Let be the loss weight for the i-th sample; The categories are numbered sequentially, and include five categories: normal, missing, deformed, corroded, and obscured. For the i-th sample The true label of the class; Let i be the probability that the i-th sample belongs to the j-th class as predicted by the network. In the formula, The number of detection scales; For testing scale serial number; Let be the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box at the m-th detection scale; It is a Euclidean distance metric function; The coordinates of the center point of the prediction box. The coordinates of the center point of the annotation box; For the first Under the detection scale, the diagonal length of the smallest bounding rectangle that simultaneously contains the predicted bounding box and the labeled bounding box; This parameter ensures consistency in aspect ratio between the predicted bounding box and the labeled bounding box. This is the balancing weighting coefficient for aspect ratio loss; The weighting coefficients for the geometric constraints of the circumcircle are used to balance the contribution ratios of position regression and shape constraints. Let be the intersection-union ratio of the minimum outer circle of the predicted bounding box and the minimum outer circle of the labeled bounding box at the m-th detection scale; In the formula, , These are the width and height of the prediction box, respectively. These are the width and height of the annotation box, respectively. In the formula, The set of pixels representing the actual area of ​​the labeled hardware; For geometric contour feature maps in pixels The response intensity at that location; For grayscale abrupt change feature map at pixel The response intensity at that location.

Images