Power transmission line detection method and device and storage medium

Abstract

This application discloses a method, apparatus, and storage medium for detecting transmission lines. The method includes: extracting image feature maps and text feature vectors from aerial images of transmission lines and natural language detection commands; aligning the text feature vectors and image feature maps in phase space based on a cyclic cross-correlation operator, and incorporating geomagnetic topological prior bias to correct spatial topological errors, thereby obtaining a coherent field matrix and candidate regions; determining the semantic directional derivatives of the multi-scale visual features of the candidate regions; determining the singularity values ​​of the candidate regions based on the semantic directional derivatives and structural integrity constraint tensors; determining an adaptive detection threshold based on the feature distribution differences between the candidate regions and background regions and the background noise covariance matrix; and when the singularity value is greater than the adaptive detection threshold, using the coherent field matrix as the energy density field, simulating the gravitational collapse process through closed-loop integration to obtain the sub-pixel-level center coordinates of the target defect.

Description

Technical Field

[0001] This application relates to the field of non-destructive testing technology for power transmission lines, and in particular to methods, devices and storage media for power transmission line testing. Background Technology

[0002] With the widespread adoption of drone-based power line inspection technology, the use of deep learning models for automatic defect identification in images of power transmission lines has become mainstream in the industry. Currently, image detection of power transmission lines mainly relies on traditional detection techniques based on supervised learning (such as the YOLO series and Faster R-CNN), recognition techniques based on attention mechanisms and Transformers, and basic cross-modal detection techniques. However, these techniques have significant shortcomings when dealing with hidden defects such as missing cotter pins, cracked fittings, and loose bolts. On the one hand, hidden defects account for a very small percentage of pixels in aerial images and are often accompanied by complex vegetation backgrounds and strong light and shadow interference. Traditional convolutional neural networks or simple attention mechanisms struggle to separate extremely weak defect feature signals from high-noise backgrounds, resulting in weak and easily submerged feature responses and severely insufficient weak signal recognition capabilities. On the other hand, current image detection models are mostly closed-set recognition models, lacking semantic understanding capabilities. They cannot understand the specific instructions given by inspectors through natural language, leading to insufficient detection flexibility, poor generalization ability, and poor interactivity. Furthermore, the vibrations, lighting changes, and motion blur of drones during flight in the wild cause traditional bounding box regression algorithms to deviate significantly when locating small targets, making it difficult to meet the requirements of centimeter-level precision operation and maintenance, and the false alarm rate is extremely high in complex backgrounds. Summary of the Invention

[0003] This invention provides a method, apparatus, and storage medium for detecting transmission lines, aiming to at least solve the problems of low accuracy in identifying hidden defects in complex environments, poor positioning accuracy, and insufficient interactivity in related technologies. The technical solution of this invention is as follows:

[0004] According to a first aspect of the present invention, a method for detecting transmission lines is provided. The method includes: extracting features from aerial images of the transmission line and natural language detection commands input by a user, respectively, to obtain an image feature map and a text feature vector; performing phase space alignment between the text feature vector and the image feature map based on a cyclic cross-correlation operator, and incorporating a geomagnetic topological prior bias during the phase space alignment process to correct spatial topological errors caused by changes in aerial attitude, thereby obtaining a coherent field matrix; the magnitude of each coordinate element in the coherent field matrix characterizes the semantic matching degree between the image region at that coordinate element and the text command; determining candidate regions from the image feature map based on the coherent field matrix; extracting multi-scale visual features of the candidate regions, and determining the semantic direction derivative of the multi-scale visual features along the semantic direction on a normal equipment feature manifold; the normal equipment feature manifold is constructed from a large number of normal samples; the semantic direction is the direction indicated by the text feature vector; the semantic direction derivative characterizes... The variation of multi-scale visual feature distribution on the feature manifold of normal equipment is analyzed. Singularity values ​​of candidate regions are determined based on semantic directional derivatives and structural integrity constraint tensors. The structural integrity constraint tensor is constructed based on the inherent topological logic of the physical structure of transmission line components, which includes physical constraint rules for hardware morphology and component connection relationships. Singularity values ​​characterize the possibility of hidden defects in candidate regions. An adaptive detection threshold for candidate regions is determined based on the first feature distribution of the candidate regions, the second feature distribution of the background region in the aerial image, and the background noise covariance matrix. The background noise covariance matrix characterizes the degree of fluctuation of visual noise in the background region across each feature dimension and the degree of correlation between dimensions. When the singularity value of a candidate region is determined to be greater than the adaptive detection threshold, the coherence field matrix is ​​used as the energy density field. In the candidate region, a closed-loop integral is used to simulate the gravitational collapse process, and the coordinate point that maximizes the integral is solved to obtain the sub-pixel-level center coordinates of the target defect.

[0005] Multi-scale visual features include semantic and detail features extracted from different layers of the convolutional neural network, which are used to balance global semantic understanding and local detail perception.

[0006] The structural integrity constraint tensor is used to penalize spurious changes that do not conform to the laws of physical connectivity.

[0007] The main diagonal element Σnoise(ii) of the background noise covariance matrix: the variance of the i-th feature dimension, which represents the degree of fluctuation of the visual noise of the background region in this single dimension. The larger the value, the more drastic the fluctuation of the background noise feature in this dimension (such as the large variance of the gray-scale feature dimension of vegetation texture).

[0008] The off-diagonal element Σnoise(i,j) of the background noise covariance matrix represents the covariance between the i-th and j-th feature dimensions, indicating the degree of correlation between the noise changes of the two feature dimensions. A positive number indicates a positive correlation (if the noise in one dimension increases, the noise in the other dimension also increases), a negative number indicates a negative correlation, and 0 indicates no correlation.

[0009] In the above implementation, the spatial alignment inaccuracy caused by UAV jitter and attitude changes is solved by introducing a cyclic cross-correlation operator and a geomagnetic topological prior bias; by calculating the semantic directional derivative and singularity value, weak defect perturbation signals can be extracted from normal equipment manifolds, improving the recognition rate of hidden defects; and robust detection and sub-pixel-level accurate positioning in complex backgrounds are achieved through adaptive detection thresholds and a gravitational collapse model.

[0010] As one implementation method, based on the cyclic cross-correlation operator, the text feature vector and the image feature map are phase-space aligned. During the phase-space alignment process, a geomagnetic topological prior bias is incorporated. Specifically, this includes: using the text feature vector as a one-dimensional kernel and performing cyclic cross-correlation with the image feature map in the spatial dimension to generate a preliminary response map; obtaining the geomagnetic azimuth and UAV attitude angles corresponding to the aerial image; constructing a geomagnetic topological bias matrix based on the geomagnetic azimuth and UAV attitude angles, where the weights in the geomagnetic topological bias matrix aligned with the standard geomagnetic topology of the transmission line are enhanced, while the weights in the geomagnetic topological bias matrix not aligned with the standard geomagnetic topology of the transmission line are suppressed; and performing element-wise weighted fusion of the preliminary response map and the geomagnetic topological bias matrix to obtain the coherent field matrix after spatial topological error correction.

[0011] In the above implementation, by constructing and fusing a geomagnetic topology bias matrix, the spatial topology error caused by the UAV attitude change is effectively corrected, the response in the target direction is enhanced, and interference in the non-target direction is suppressed.

[0012] As one implementation method, candidate regions are determined from the image feature map based on the coherence field matrix. Specifically, this includes: extracting response regions from the image feature map whose coordinate elements of the coherence field matrix are larger than a preset threshold; performing connected component analysis on the extracted response regions and aggregating adjacent pixels into at least one connected region; and generating corresponding bounding boxes for the connected regions as candidate regions.

[0013] Threshold segmentation is performed on the coherent field matrix to extract response regions that exceed a preset threshold; connected component analysis is performed on the extracted response regions to aggregate adjacent pixels into at least one connected region; corresponding bounding boxes are generated based on the connected regions as candidate regions.

[0014] In the above implementation, by aggregating pixels through connected component analysis, potential defect areas can be more accurately defined, reducing fragmented responses.

[0015] As one implementation method, the semantic directional derivative of multi-scale visual features on the normal device feature manifold along the semantic direction is determined. Specifically, this includes: using an autoencoder or manifold learning model trained on normal samples to project multi-scale visual features onto the normal device feature manifold; determining the projection point on the normal device feature manifold and the tangent vector on the manifold along the direction indicated by the text feature vector; and determining the semantic directional derivative based on the magnitude and direction coefficient of the tangent vector.

[0016] In the above implementation method, by using the manifold learning model and tangent vector calculation, the changing trend of features on the manifold can be accurately captured, thereby more sensitively detecting the small perturbations caused by hidden defects.

[0017] As one implementation method, the singularity value of the candidate region is determined based on the semantic directional derivative and the structural integrity constraint tensor. Specifically, this includes: projecting the semantic directional derivative onto a physically reasonable change subspace; the physically reasonable change subspace is composed of the structural integrity constraint tensor; determining the vector magnitude after projecting the physically reasonable change subspace as the singularity value; the vector magnitude after projecting the physically reasonable change subspace does not include components perpendicular to the physically reasonable change subspace, so that components perpendicular to the physically reasonable change subspace are discarded in the calculation. The discarded components constitute a penalty for the pseudo-change, thereby achieving the purpose of penalizing pseudo-changes that do not conform to the physical connection rules.

[0018] In the above implementation, by introducing a physically reasonable subspace for projection, pseudo-singularities that do not conform to physical logic are eliminated, the false alarm rate is reduced, and the accuracy of singularity values ​​is improved.

[0019] As one implementation method, an adaptive detection threshold for the candidate region is determined based on the first feature distribution of the candidate region, the second feature distribution of the background region of the aerial image, and the background noise covariance matrix. This includes: determining the KL divergence between the first and second feature distributions, and determining the trace of the background noise covariance matrix. The KL divergence characterizes the feature difference between the candidate region and the background region. The adaptive detection threshold is obtained by weighting the KL divergence with the reciprocal of the trace of the background noise covariance matrix.

[0020] In the above implementation, the threshold is dynamically calculated using the KL divergence and the trace of the background noise covariance matrix, so that the detection threshold can be adaptively adjusted according to the degree of environmental clutter and feature differences, thus balancing detection sensitivity and false alarm rate.

[0021] As one implementation method, the gravitational collapse process is simulated through closed-region loop integration, and the coordinate points that maximize the integral are solved. Specifically, this includes: defining a variable closed boundary curve associated with preset parameters, centered on the centroid of the candidate region; the preset parameters include parameter vectors characterizing the shape and position of the variable closed boundary curve; using the norm of the coherent field matrix as the energy density, the total energy within the region enclosed by the variable closed boundary curve is determined as the region integral; with the goal of maximizing the region integral and with the perimeter of the variable closed boundary curve within a preset perimeter range as a constraint, the preset parameters are increased sequentially to obtain the adjusted variable closed boundary curve and the adjusted region integral, until the adjusted region integral reaches the convergence condition; the geometric center of the variable closed boundary curve corresponding to the region integral that reaches the convergence condition is determined as the sub-pixel-level center coordinates.

[0022] In the above implementation method, by simulating the gravitational collapse process and using loop integration to find the energy center, sub-pixel-level positioning accuracy is achieved, solving the drift problem of traditional rectangular box regression in the positioning of small targets.

[0023] As one implementation method, the method also includes: fitting the contour and range of the target defect based on the distribution of the coherent field matrix in the neighborhood of the sub-pixel level center coordinates; and determining the defect type of the target defect based on the contour and range of the target defect.

[0024] In the above implementation method, by fitting the contour and range, the defect features can be further refined, providing a more accurate basis for subsequent defect type determination.

[0025] According to a second aspect of the present invention, a transmission line detection apparatus is provided, which is configured to perform a transmission line detection method as described in the first aspect and any possible implementation thereof.

[0026] According to a third aspect of the present invention, a transmission line detection system is provided, which is configured to perform a transmission line detection method as described in the first aspect and any possible implementation thereof.

[0027] According to a fourth aspect of the present invention, an electronic device is provided, comprising: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement a transmission line detection method as described in the first aspect and any possible implementation thereof.

[0028] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform a transmission line detection method as described in the first aspect and any possible implementation thereof.

[0029] According to a sixth aspect of the present invention, a computer program product is provided, the computer program product including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the transmission line detection method of the first aspect and any possible implementation thereof.

[0030] The technical solution provided by this invention brings at least the following beneficial effects: Based on natural language guidance, this application achieves the alignment of visual features and textual semantics in phase space by introducing a cyclic cross-correlation operator and a geomagnetic topological prior bias, effectively correcting spatial topological errors caused by UAV attitude changes and solving the spatial offset problem in complex environments. By calculating the semantic directional derivatives of multi-scale visual features on normal equipment feature manifolds and combining them with the structural integrity constraint tensor to determine singularity values, weak perturbation signals generated by hidden defects can be extracted from normal structural features, significantly improving the ability to identify hidden defects. By determining an adaptive detection threshold based on the feature distribution differences between candidate and background regions and the background noise covariance matrix, the system can adaptively balance detection sensitivity and false alarm rate according to the clutter of the inspection environment, improving robustness in complex backgrounds. By using the coherent field matrix as the energy density field and simulating the gravitational collapse process through closed-loop integration, sub-pixel-level ultra-precise positioning of the target defect center was achieved, meeting the requirements of centimeter-level precise operation and maintenance. This effectively solves the problems of difficult extraction of hidden defect signals, poor environmental adaptability, low positioning accuracy, and insufficient interactivity in related technologies.

[0031] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0033] Figure 1 This is a flowchart illustrating a transmission line testing method according to an exemplary embodiment;

[0034] Figure 2 This is a block diagram illustrating a power transmission line detection device according to an exemplary embodiment;

[0035] Figure 3 This is a schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0037] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0038] For ease of understanding, the transmission line testing method provided in this application will be described in detail below with reference to the accompanying drawings.

[0039] like Figure 1 As shown, this embodiment provides a method for detecting power transmission lines. This method aims to address the problems of weak characteristic signals, significant environmental interference, and insufficient positioning accuracy encountered when using UAV aerial images for hidden defect detection in complex field environments. The method of this embodiment specifically includes the following steps.

[0040] Step S1: Extract features from the aerial images of the transmission line and the natural language detection commands input by the user to obtain image feature maps and text feature vectors.

[0041] The aerial images were taken by a drone.

[0042] Specifically, during inspections, drones collect high-resolution aerial images of power transmission lines. Simultaneously, inspection personnel input natural language detection commands via terminals, such as "check for cracks in the insulator strings" or "locate the missing cotter pin." Upon receiving the image data, the system uses a pre-trained visual encoder (such as a deep neural network like ResNet, VisionTransformer, or SwinTransformer) to extract features from the image, mapping high-dimensional pixel data into semantically meaningful image feature maps. Simultaneously, a text encoder (such as the text branch of BERT or CLIP) encodes the natural language detection commands, extracting text feature vectors that characterize the semantic core of the commands. This step achieves vectorized representation of unstructured image and text data, laying the foundation for subsequent cross-modal interaction. It should be understood that the choice of feature extraction network is not limited to the above example; any network structure capable of extracting discriminative features is within the scope of this invention.

[0043] Step S2: Based on the cyclic cross-correlation operator, the text feature vector and the image feature map are phase-space aligned. During the phase-space alignment process, the geomagnetic topological prior bias is incorporated to correct the spatial topological error caused by the change in aerial photography attitude, and the coherent field matrix is ​​obtained.

[0044] The coherence field matrix is ​​also called the semantic coherence field matrix.

[0045] In this context, the magnitude of each element at a coordinate point in the coherence field matrix represents the semantic matching degree between the image region at that coordinate point and the text command. Specifically, due to wind vibrations during drone flight and the continuous changes in shooting attitude angles (pitch, roll, and yaw), the target position in the image is offset and distorted from the actual geographic space. Directly using simple dot product operations for image-text matching is insufficient to overcome this spatial offset problem. This implementation introduces a cyclic cross-correlation operator, using the text feature vector as a one-dimensional kernel to perform a sliding window cross-correlation operation on the image feature map, thereby searching for the optimal matching position in the phase space and effectively capturing the semantic invariance under local spatial displacement.

[0046] Furthermore, considering that transmission lines have a defined geomagnetic orientation and spatial topology, alignment based solely on visual features may still be affected by background clutter. Therefore, this step also incorporates a geomagnetic topology prior bias. This bias utilizes the geomagnetic azimuth and attitude angle information recorded by the UAV to construct a bias matrix, which is used to correct for line topology deformation caused by the tilt of the shooting angle, making the alignment result more consistent with the actual physical layout of the transmission line. The final generated coherence field matrix is ​​a two-dimensional response map, and the high-response regions in this two-dimensional response map are the potential target regions that are highly correlated with user commands.

[0047] In this step, the coherence field matrix can be calculated using the following formula.

[0048] (1).

[0049] The semantic coherence field matrix represents the resonant distribution of image pixel space and text instructions in the feature coherence domain, and its value determines the salience of the target object. The cyclic cross-correlation operator, compared to the dot product, can capture the semantic invariance under local spatial displacement (solving the deviation caused by drone jitter). : Nonlinear phase mapping function, used to adjust the phase consistency between visual and text features; : Environmental scattering coefficient, with correlation score dynamically adjusted based on aerial light intensity; : Geomagnetic topological prior bias, combined with the spatial topological position of the UAV attitude angle correction device; Scaling factor. Typically the square root of the feature dimension, used to prevent excessively large inner products from causing gradient vanishing. , These are the visual feature matrix of the image feature map and the text feature matrix represented by the text feature vector: representing the pixel-level feature set extracted from the aerial image and the semantic encoding vector of the defect description corpus, respectively. These are linear projection weight matrices for visual and textual features, respectively, used to map visual and textual features to a unified metric space for inner product operations.

[0050] Step S3: Based on the coherence field matrix, candidate regions are determined from the image feature map.

[0051] Specifically, after obtaining the coherent field matrix representing the semantic matching degree, the system needs to filter and locate high-response regions. This step extracts regions with significantly higher response values ​​than the surrounding background by analyzing the distribution characteristics of the coherent field matrix. This can be achieved by setting a threshold or using an adaptive segmentation algorithm, as the extracted high-response regions often correspond to specific components on the transmission line (such as insulators, fittings, etc.).

[0052] To remove isolated noise and form meaningful detection targets, this step also aggregates the extracted regions to generate candidate regions (such as bounding boxes) containing potential targets. This process narrows the large image search space down to a few high-probability local regions, significantly reducing the computational cost of subsequent detailed analysis.

[0053] Step S4: Extract multi-scale visual features of candidate regions and determine the semantic direction derivative of the multi-scale visual features along the semantic direction on the normal device feature manifold.

[0054] Among them, the normal device feature manifold is constructed from a large number of normal samples, the semantic direction is the direction indicated by the text feature vector, and the semantic direction derivative represents the change of the feature distribution of multi-scale visual features on the normal device feature manifold.

[0055] Specifically, hidden defects (such as minute cracks or missing pins) differ extremely little from normal equipment in visual characteristics, making direct calculation of feature distance in Euclidean space often indistinguishable. This implementation introduces the concept of differential geometry, constructing a "normal equipment feature manifold" using a large number of normal samples. This manifold describes the feature distribution of normal equipment under various viewing angles and lighting conditions. For each candidate region, its multi-scale visual features are extracted and projected onto this manifold. Subsequently, the tangent vector of the projected point along the direction indicated by the text feature vector (i.e., the semantic direction) is calculated, i.e., the semantic direction derivative. The physical meaning of this derivative is that if the candidate region is normal equipment, its feature changes should smoothly transition along the tangent plane of the manifold; however, if hidden defects exist, feature points often deviate from the manifold surface, causing abnormal transitions or abrupt changes in the directional derivative. In this way, the system can sensitively perceive weak defect signals, achieving amplified detection of hidden defects.

[0056] Step S5: Determine the singularity value of the candidate region based on the semantic directional derivative and the structural integrity constraint tensor.

[0057] The structural integrity constraint tensor is constructed based on the inherent topological logic of the physical structure of transmission line components. This inherent topological logic includes physical constraint rules for hardware morphology and component connection relationships. Singularity values ​​characterize the probability of hidden defects in candidate regions. Specifically, although the semantic directional derivative can capture feature anomalies, interference factors such as changes in light and shadow, and foliage obstruction in complex environments can also trigger similar feature mutations, leading to false alarms. To eliminate these false singularities, this step introduces the structural integrity constraint tensor. This tensor encodes the physical prior knowledge of transmission line components, such as "bolts should match nuts" and "insulator strings should form a continuous chain structure." By projecting the semantic directional derivative onto a physically reasonable variation subspace constructed from this tensor, the system can determine whether the feature mutation conforms to physical logic. If the mutation direction is perpendicular to the physically reasonable variation subspace (i.e., does not conform to physical constraints), it is determined to be environmental interference and suppressed; if the mutation direction has a large projection modulus within the subspace, it indicates that the mutation is highly likely caused by a real structural defect. The final calculated singularity value combines the degree of feature anomaly with physical plausibility, and can more accurately quantify the probability that a candidate region has hidden defects.

[0058] Step S6: Determine the adaptive detection threshold of the candidate region based on the first feature distribution of the candidate region, the second feature distribution of the background region of the aerial image, and the background noise covariance matrix.

[0059] The background noise covariance matrix characterizes the degree of fluctuation of visual noise in the background region across each feature dimension and the degree of correlation between dimensions. Specifically, traditional fixed thresholds are difficult to adapt to changing inspection environments. In areas with dense vegetation or strong backlighting, the background noise is high, and a fixed low threshold will lead to a large number of false alarms; in areas with clean backgrounds, a fixed high threshold is prone to missed detections.

[0060] This implementation dynamically generates an adaptive detection threshold by calculating the feature distribution differences (such as KL divergence) between the candidate region and the background region, as well as the trace of the background noise covariance matrix. When the environment is cluttered and the noise covariance is large, the threshold automatically increases, adopting a robust strategy; when the environment is clear and the background differences are significant, the threshold automatically decreases, adopting a sensitive strategy. This mechanism effectively balances the detection sensitivity and false alarm rate, ensuring the robustness of the algorithm in different scenarios.

[0061] Step S7: When the singularity value of the candidate region is determined to be greater than the adaptive detection threshold, the coherent field matrix is ​​used as the energy density field. In the candidate region, the gravitational collapse process is simulated by the closed-loop integral, and the coordinate point that maximizes the integral is solved to obtain the sub-pixel level center coordinates of the target defect.

[0062] Specifically, once a candidate region is determined to have a defect, the system needs to output a high-precision localization result. Traditional bounding box regression often only provides a rough, pixel-level location and is easily affected by irregularities in the target's shape.

[0063] This implementation treats the coherence field matrix as an energy density field, simulating the gravitational collapse process in astrophysics. By performing closed-loop integration within the candidate region, the degree of energy accumulation at different locations is calculated. As the integration path shrinks, the energy gradually collapses towards the center point where the response is most intense, eventually converging to a single point. The coordinates of this point represent the geometric center of the defect. Due to the analytical continuity of the integration operation, these coordinates can achieve sub-pixel accuracy, thus meeting the requirements of power transmission line operation and maintenance for precise positioning of small targets.

[0064] As one implementation method, the above implementation method is further refined, and the specific implementation process of phase space alignment in step S2 is explained in detail below.

[0065] Specifically, based on the cyclic cross-correlation operator, the text feature vector and the image feature map are phase space aligned, and geomagnetic topological prior bias is incorporated into the phase space alignment process, which includes the following steps.

[0066] Step S201: The text feature vector is used as a one-dimensional kernel and subjected to cyclic cross-correlation operation with the image feature map in the spatial dimension to generate a preliminary response map.

[0067] In actual drone inspection scenarios, the drone fuselage often experiences high-frequency vibrations due to wind interference or flight control errors, causing unexpected spatial offsets in the acquired aerial images at the pixel level. Traditional dot product operations for image-text feature matching are extremely demanding in terms of feature alignment; even minute pixel shifts can lead to a sharp drop in matching scores, resulting in missed target detections. This implementation introduces a cyclic cross-correlation operator, treating the text feature vector as a sliding "semantic probe" and performing traversal cross-correlation operations on the image feature map. Essentially, this process constructs a "translation search" mechanism within the phase space, allowing the system to capture semantic invariance under local spatial displacements within a certain range. Even if the target object shifts position in the image, the cyclic cross-correlation operation can still lock the optimal matching position through phase compensation, thereby generating a preliminary response map with higher spatial resolution and stronger anti-shake capability.

[0068] Step S202: Obtain the geomagnetic azimuth and UAV attitude angle corresponding to the aerial image.

[0069] At the moment of taking a picture, the drone's flight control system records the current attitude data, including pitch angle, roll angle, yaw angle, and geomagnetic azimuth angle. This data reflects the camera's true orientation in three-dimensional space and is a key basis for subsequent correction of spatial topology errors.

[0070] It should be understood that attitude angles can be obtained not only through flight control recording, but also directly through inertial measurement units (IMUs) or electronic compasses.

[0071] Step S203: Construct a geomagnetic topological bias matrix based on the geomagnetic azimuth angle and the UAV attitude angle.

[0072] Transmission lines typically exhibit well-defined geomagnetic characteristics, such as an east-west or north-south orientation, and their components, including fittings and insulators, possess relatively fixed spatial topologies. When a drone captures images from a non-perpendicular perspective (e.g., a steeply tilted side view), the line topology in the image undergoes geometrical distortion, leading to a mismatch between visual features and standard topology templates. This step utilizes the acquired attitude angle and geomagnetic azimuth angle to construct a geomagnetic topology bias matrix. This matrix is ​​essentially a spatial weight mask, constructed according to the physical principle of "enhancing consistent directions and suppressing inconsistent directions."

[0073] Specifically, for directions in the image that align with the standard geomagnetic topology of the transmission line (e.g., along the conductor's extension direction), the matrix assigns higher weight values ​​to enhance the semantic response of that direction; for directions that deviate from the standard topology (e.g., background clutter perpendicular to the conductor), the matrix assigns lower weight values ​​to suppress irrelevant interference. In this way, the geomagnetic topology bias matrix transforms prior knowledge of the physical space into constraints in the feature space.

[0074] Step S204: The preliminary response map and the geomagnetic topological bias matrix are fused element-wise to obtain the coherent field matrix after spatial topological error correction.

[0075] In the specific calculations, the weighted fusion process can be understood as modulating the preliminary response map. Let the response value of a certain coordinate point in the preliminary response map be R, and the weight coefficient of the corresponding position in the geomagnetic topological bias matrix be W. Then, the element values ​​of the fused coherent field matrix can be expressed as the product or weighted sum of R and W. This fusion operation ensures that the coherent field matrix not only reflects the degree of semantic matching between the image and text but also incorporates the rational constraints of the spatial physical structure. The final coherent field matrix can effectively eliminate false responses caused by the tilt of the UAV attitude, significantly improving the geometric accuracy of locating transmission line components.

[0076] Through the above steps, this implementation not only solves the airspace offset problem caused by UAV jitter, but also uses geomagnetic topology priors to correct the spatial geometric errors caused by attitude changes, providing a more accurate semantic heatmap for subsequent candidate region determination.

[0077] As one implementation method, the above implementation method is further refined, and the specific logic of candidate region generation in step S3 is explained in detail.

[0078] Specifically, candidate regions are determined from the image feature map based on the coherence field matrix, which includes the following steps:

[0079] Step S301: Extract the response region from the image feature map where the coordinate element size of the coherent field matrix is ​​higher than a preset threshold.

[0080] After obtaining the coherence field matrix, the value of each point in the matrix represents the semantic relevance of that location to the user's input natural language command. To filter out potential target regions, a preset threshold is set for initial filtering. The specific value of this preset threshold can be adjusted according to the actual application scenario. For example, in one implementation, the preset threshold can be dynamically calculated based on the global statistical characteristics of the coherence field matrix (such as a linear combination of the mean and standard deviation) to adapt to differences in response distribution under different lighting and background complexities; in another implementation, the preset threshold can also be an empirical constant trained on a large number of historical samples. All coordinate points with values ​​greater than the preset threshold are marked as potential response points, and the set of these points constitutes the initial response region. This step effectively eliminates a large number of low-relevance background regions, significantly reducing the data size for subsequent processing.

[0081] Step S302: Perform connected component analysis on the extracted response region and aggregate adjacent pixels into at least one connected region.

[0082] Because hidden defects (such as minute cracks or missing pins) often appear as continuous local areas in images, while spurious responses caused by environmental noise (such as swaying leaves or dappled light) typically appear as isolated discrete points, this step introduces a connected component analysis algorithm to further remove noise interference and aggregate effective signals. Specifically, the system scans the initial response region, searching for pixels that are spatially adjacent to each other (e.g., 4-neighborhood or 8-neighborhood connected), and aggregates these interconnected pixels into a whole, forming at least one connected region. It should be understood that "adjacent" here includes not only direct adjacency in physical pixel coordinates, but also, in some high-precision implementations, approximate adjacency relationships with a feature space distance less than a certain threshold. Through connected component analysis, the originally fragmented response points are organized into geometric clusters with spatial structural significance, effectively suppressing isolated responses caused by random noise and improving the signal-to-noise ratio.

[0083] Step S303: Generate corresponding bounding boxes for connected regions as candidate regions.

[0084] After obtaining the connected regions, the system needs to generate a geometric description for each connected region to facilitate subsequent feature extraction and defect identification. This step uses a bounding box generation algorithm to calculate the circumscribed rectangle of each connected region, i.e., the bounding box. This bounding box not only contains the spatial coordinates of the connected region (such as the coordinates of the center point and the top-left corner), but also the length and width dimensions of the region. The generated bounding boxes are then determined as the final candidate regions. These candidate regions will serve as the input range for subsequent differential manifold analysis and singularity calculation. Through the above steps, this implementation achieves the transformation from sparse semantic response points to compact candidate target regions, ensuring the integrity of the potential defect signal while avoiding the interference of fragmented responses on subsequent detection accuracy, providing high-quality input data for subsequent fine feature analysis.

[0085] As one implementation method, the above implementation method is further refined, and the specific calculation method of the semantic direction derivative in step S4 is explained in detail.

[0086] Specifically, determining the semantic directional derivative of multi-scale visual features along the semantic direction on the normal device feature manifold includes the following steps.

[0087] Step S401: Using an autoencoder or manifold learning model trained on normal samples, project multi-scale visual features onto the normal device feature manifold.

[0088] During the long-term operation of transmission lines, the vast majority of equipment is in a normal state, and its visual features are often distributed near a low-dimensional manifold structure in high-dimensional space. To construct this "normal equipment feature manifold," this implementation method collects a large number of normal equipment samples from historical inspection data and trains them using an autoencoder or manifold learning model. After training, the model learns the inherent patterns of the normal equipment feature distribution. When a multi-scale visual feature of a candidate region to be detected is input, the model maps (projects) it onto the manifold space, finding the optimal corresponding position of the feature point on the manifold, i.e., the projection point.

[0089] It should be understood that, apart from autoencoders, other algorithms that can achieve dimensionality reduction and manifold embedding of high-dimensional data, such as isometric mapping or local linear embedding, can also be applied to this step, and the present invention does not limit them.

[0090] Step S402: Determine the projection point on the normal device feature manifold, and the tangent vector on the manifold along the direction indicated by the text feature vector.

[0091] In differential geometry, every point on a manifold has a tangent plane, and the vectors within that tangent plane describe the manifold's tendency to change at that point. In this step, the text feature vector indicates the semantic direction of interest to the user, such as "missing cotter pin." The system calculates the tangent vector along this semantic direction of the manifold at the projection point. This tangent vector represents the theoretical trajectory of the features that should change on the manifold if the device experiences a small "missing cotter pin" change. By calculating the relationship between the actual feature point and this tangent vector, it is possible to determine whether an anomaly exists in that region.

[0092] Step S403: Determine the semantic directional derivative based on the magnitude and direction coefficient of the tangent vector.

[0093] Specifically, the semantic directional derivative depends not only on the length (magnitude) of the tangent vector but also on the consistency of its direction with the semantic direction (directional coefficient). If the tangent vector has a large magnitude and its direction is highly consistent with the text feature vector, it indicates that the feature changes in that region are highly correlated with the defect type queried by the user, and the semantic directional derivative value is large. Conversely, if the tangent vector has a small magnitude or its direction is inconsistent, it indicates that the feature changes in that region are gradual or irrelevant to the query, and the semantic directional derivative value is small.

[0094] In this way, the semantic directional derivative can keenly capture minute perturbations on the manifold surface. For hidden defects, although their feature distance in Euclidean space may be small, they often appear as "singularities" deviating from the normal trajectory on the manifold structure, leading to significant transitions in the directional derivative. This mechanism effectively amplifies the signal intensity of hidden defects, making weak features that were originally submerged in background noise detectable, thereby significantly improving detection sensitivity.

[0095] As one implementation method, the above implementation method is further refined, and the physical constraint logic of the singularity value in step S5 is explained in detail.

[0096] Specifically, the singularity values ​​of candidate regions are determined based on the semantic direction derivative and the structural integrity constraint tensor, which includes the following steps:

[0097] Step S501: Project the semantic direction derivative onto the physically reasonable change subspace; the physically reasonable change subspace is composed of the structural integrity constraint tensor.

[0098] In the actual operating environment of power transmission lines, abnormal changes in visual features are not entirely equivalent to defects. For example, dappled sunlight filtering through leaves, partial shading caused by birds perched on branches, or reflections from raindrops can all trigger significant abrupt changes at the image feature level, leading to overestimation of the semantic directional derivative. However, these changes do not conform to the physical structural logic of power transmission line components.

[0099] To distinguish between "physically real deformation" and "visually pseudo-singular interference," this implementation introduces a structural integrity constraint tensor. This tensor is constructed based on the inherent topological logic of the physical structure of transmission line components. For example, the form of hardware should maintain continuity, bolts and nuts should have a matching connection, and insulator strings should exhibit a specific chain structure. These physical constraint rules are encoded as basis vectors in the tensor space, collectively constructing a "physically reasonable variation subspace." Characteristic changes within this subspace are considered to be physically consistent and reasonable deformations or displacements; while changes perpendicular to this subspace often represent interference signals that violate physical logic. The projection operation decomposes the high-dimensional semantic direction derivative vector into components within this subspace and components perpendicular to this subspace.

[0100] Step S502: The vector magnitude after the physical reasonable change of the subspace projection is determined as the singularity value.

[0101] The vector magnitude after projection of the physically rationally changed subspace does not include the component perpendicular to the physically rationally changed subspace.

[0102] When calculating singularity values, this implementation retains only the projection modulus of the semantic directional derivative within the physically reasonable variation subspace, while explicitly discarding the vertical component. The physical significance of this design lies in the fact that genuine hidden defects (such as structural fractures caused by hardware cracks or connection gaps caused by missing pins) typically disrupt the original physical topology of components, and their characteristic change directions must lie within or be highly correlated with the physically reasonable variation subspace; while pseudo-singularities such as light and shadow interference and background clutter often have random and disordered characteristic change directions, highly probably perpendicular to this subspace. By discarding the vertical component, the system effectively performs a "filtering" operation based on physical priors.

[0103] For example, when the movement of light and shadow causes a sudden change in brightness in a certain area, although the semantic directional derivative may be large, because the direction of change does not conform to the rigid constraints of the hardware structure, its modulus after projection onto the physically reasonable change subspace is extremely small. Therefore, the calculated singularity value will be significantly reduced, thus avoiding false alarms. Conversely, if a pin is missing, its characteristic change breaks the integrity of the component connection, generating a large projection component in the physically reasonable change subspace, thus being accurately identified as a high singularity area. This projection mechanism based on physical constraints effectively reduces the false alarm rate in complex field environments and significantly improves the reliability of detection results.

[0104] In this step, the singularity value can be calculated using the following formula.

[0105] (2).

[0106] The manifold singularity score represents the degree of deviation of the current hardware feature from the standard semantic manifold. The larger the value, the stronger the defect concealment but the higher the certainty. Semantic directional derivative: the rate of change of visual features in the direction of the normal semantic space; Characteristic manifold mapping operator; Tensor product operator, used to fuse singularity detection results with structural physical constraints in a multidimensional way; The structural integrity constraint tensor represents the inherent topological logic of electrical equipment (such as the pairing of bolted connections). It is used to filter out false singularities caused by light and shadow interference, such as ensuring the basic physical morphological logic of hardware (e.g., bolts will not disappear out of thin air, they can only loosen). : Feature vector of the region to be tested, a high-dimensional visual descriptor extracted from the region of interest (RoI) of the image; Infinitesimal perturbation operator. Mathematically used to define a limiting process, in algorithm implementation it represents a tiny sampling step size in the feature space; Standard state baseline vector, the semantic center of the device's intact and undamaged state as defined by the expert system; : Feature manifold mapping function, used to map complex visual observation data to a low-dimensional and continuous physical manifold.

[0107] As one implementation method, the above implementation method is further refined, and the method for determining the adaptive detection threshold in step S6 is described in detail.

[0108] Specifically, the adaptive detection threshold for the candidate region is determined based on the first feature distribution of the candidate region, the second feature distribution of the background region of the aerial image, and the background noise covariance matrix. This process includes the following steps:

[0109] Step S601: Determine the KL divergence between the first feature distribution and the second feature distribution, and determine the trace of the background noise covariance matrix.

[0110] In this context, KL divergence characterizes the feature differences between candidate regions and background regions. In UAV inspection images, the background region typically occupies most of the image area, and its feature distribution represents the general visual characteristics of the current scene (such as vegetation texture, sky color, etc.). Candidate regions are the potential defect locations initially identified by the system. KL divergence, as an asymmetric similarity metric, can effectively quantify the degree of difference between two probability distributions. When the feature distribution of the candidate region differs significantly from that of the background region (e.g., the candidate region is a metal fitting, while the background is green vegetation), the KL divergence value is large, indicating that the region has significant visual saliency and is highly likely to be a target component or defect; conversely, if the candidate region is only a part of the background (such as a leaf), its feature distribution is highly similar to the overall background, and the KL divergence value is small. Therefore, in this embodiment, KL divergence acts as an indicator of "signal strength".

[0111] Meanwhile, the trace of the background noise covariance matrix is ​​used to characterize the degree of fluctuation of visual noise in the background region across each feature dimension and the degree of correlation between dimensions. The trace is the sum of the diagonal elements of the matrix, and in the context of the covariance matrix, it represents the total variance of each feature dimension. In the scenario of power transmission line inspection, the environment is complex and variable. When the background is in a stable state (such as a blue sky on a clear day or a stationary rock), the fluctuation of each feature dimension is small, and the trace of the covariance matrix is ​​small; while when the background is in a cluttered state (such as dense forests swaying in the wind, shimmering water, or mountains with strong light and shadow), the fluctuation of pixel features across each dimension is drastic, and the trace of the covariance matrix will increase significantly. Therefore, the size of the trace directly reflects the "clutter" or "interference level" of the environment.

[0112] Step S602: The KL divergence is weighted and calculated with the inverse of the trace of the background noise covariance matrix to obtain the adaptive detection threshold.

[0113] This step establishes a dynamic balancing mechanism that integrates the two key indicators mentioned above into a unified judgment threshold. Specifically, the adaptive detection threshold is positively correlated with the KL divergence and negatively correlated with the trace of the background noise covariance matrix (i.e., positively correlated with the reciprocal of the trace). The physical logic behind this design is that when the environment is cluttered (large trace), the reciprocal of the background noise decreases, and the weighted threshold automatically increases. This means that the system's criteria for singularity judgment become more stringent; only areas with extremely high singularity values ​​will be identified as defects, effectively avoiding a large number of false alarms caused by random noise interference in complex backgrounds, reflecting a "robust" detection strategy. Conversely, when the environment is clear (small trace), the reciprocal of the background noise increases, and the weighted threshold automatically decreases. At this time, the system is more sensitive to weak anomalous signals, and even hidden defects with inconspicuous feature differences can be detected, reflecting a "sensitive" detection strategy. Through this weighted calculation, this implementation method achieves adaptive adjustment of the detection threshold according to the environmental signal-to-noise ratio, solves the problem of the traditional fixed threshold being inconsistent in different environments, and significantly improves the robustness of the algorithm in variable field environments.

[0114] In this step, the adaptive detection threshold can be calculated using the following formula.

[0115] (3).

[0116] Γ: Dynamic detection energy level threshold; KL divergence (relative entropy) measures the probability distribution of the region under test. Distribution with background Differences; The trace of the background noise covariance matrix is ​​used to quantitatively describe the degree of visual clutter in the environment. Temporal decay factor; Δt: Exposure time difference compensation, used to correct motion blur gain caused by high-speed movement of the UAV; This represents the static baseline threshold, which is the primary sensitivity threshold preset by the algorithm.

[0117] As one implementation method, the above implementation method is further refined, and the specific implementation process of sub-pixel level positioning in step S7 is explained in detail.

[0118] Specifically, the gravitational collapse process is simulated through closed-loop integration, and the coordinates that maximize the integral are determined. This involves the following steps:

[0119] Step S701: Define a variable closed boundary curve associated with preset parameters, centered on the centroid of the candidate region.

[0120] The preset parameters include parameter vectors that characterize the shape and position of the variable closed boundary curve. After identifying candidate regions with hidden defects, traditional target detection algorithms typically output a rectangular bounding box, whose center point is often just pixel-level integer coordinates and is easily shifted by irregular target shapes or background clutter.

[0121] To achieve higher-precision positioning, this embodiment introduces the concept of gravitational field collapse from physics. First, the preliminary geometric centroid of the candidate region is calculated, serving as the initial anchor point for the integration operation. Then, a variable closed boundary curve is defined, the shape and position of which are controlled by a set of parameter vectors. It should be understood that this curve can be circular, elliptical, or other irregular closed curves, its specific form depending on the geometric characteristics of the energy distribution in the coherent field matrix; this invention does not impose any limitations on this.

[0122] Step S702: Using the norm of the coherent field matrix as the energy density, the total energy within the region enclosed by the variable closed boundary curve is determined as the regional integral.

[0123] In this step, the coherence field matrix is ​​no longer merely a semantic matching score map, but is redefined as an energy density field in space. The numerical value of each point in the matrix represents the "semantic quality" or "energy intensity" inherent at that location. Integration is performed over the region enclosed by the variable closed boundary curve to calculate the total energy within that region. This process simulates the calculation of mass distribution in a gravitational field; the more concentrated the energy in a region, the stronger its semantic relevance to the user's input natural language command, such as "missing cotter pin," and the higher the probability that the defect actually exists.

[0124] Step S703: With the goal of maximizing the regional integral and the constraint that the perimeter of the variable closed boundary curve is within a preset perimeter range, the adjusted variable closed boundary curve and the adjusted regional integral are obtained by sequentially increasing the preset parameters, until the adjusted regional integral reaches the convergence condition.

[0125] This is the core evolutionary step in the gravitational collapse process. Through iterative optimization algorithms, preset parameters are continuously adjusted, thereby changing the shape and position of the variable closed boundary curve. Each adjustment is accompanied by a recalculation of the region integral. The optimization goal is to find a closed region that maximizes the energy integral, which is equivalent to finding a "potential well" or "gravitational center" in the energy field.

[0126] Meanwhile, to prevent the curve from shrinking infinitely to a single point or expanding to the entire background, this implementation introduces a perimeter constraint, requiring the curve perimeter to remain within a preset range. This constraint mechanism ensures that the localization result is focused on the core defect area while also having a certain spatial coverage, preventing localization divergence caused by noise interference. As iterations proceed, the regional integral value gradually stabilizes, reaching the convergence condition. At this point, it can be considered that the energy has collapsed to the core region.

[0127] In this step, the sub-pixel-level center coordinates can be calculated using the following formula.

[0128] Formula (4).

[0129] The coordinates of the collapsed singularity are the final output sub-pixel fine-tuning center. Closed-loop integration, performing energy field integration at the edge of the candidate box; : Semantic energy density function, the original response intensity of the pixel at coordinates (u,v) to the defect semantic instruction; Attenuation index: controls the focusing intensity during center-of-gravity collapse. The larger the pixel, the closer the positioning is to the pixel center. : Optimal parameter solution operator; finding the spatial coordinate point that maximizes the loop integral value; The initial coarse positioning point is the center coordinate of the rectangle initially generated by the detector; Euclidean distance: the geometric distance between pixel (u,v) and the initial anchor point.

[0130] Step S704: Determine the geometric center of the variable closed boundary curve corresponding to the region integral that has reached the convergence condition as the sub-pixel level center coordinates.

[0131] After the gravitational collapse process ends, the region enclosed by the final determined closed boundary curve is the optimal coverage area of ​​the defect. The system calculates the geometric center of this closed region and uses it as the final location coordinates of the target defect. Since both the integral operation and the geometric center calculation are based on continuous mathematical field theory derivation, rather than discrete pixel grids, the obtained coordinate values ​​can achieve sub-pixel accuracy.

[0132] For example, when detecting a missing cotter pin with a diameter of only a few pixels, this implementation can output sub-pixel coordinates such as (125.34, 340.78), instead of the traditional coordinates (125, 341). This improvement in accuracy is crucial for guiding UAVs in close-range re-shooting or for the precise operation of automated repair robotic arms. Compared to traditional bounding box regression methods, the gravitational collapse model in this implementation utilizes global energy distribution information, exhibiting stronger robustness to local noise and edge blurring, thereby achieving ultra-precise localization of minute, hidden defects.

[0133] Optionally, the logic for contour fitting and type determination after defect localization is explained in detail below.

[0134] After obtaining the sub-pixel-level center coordinates of the target defect through the gravitational collapse model, the system acquires highly accurate location information. However, in actual operation and maintenance, a single center point is often insufficient to support a complete defect assessment. For example, maintenance personnel need to know the crack extension length and the specific shape range of the missing cotter pin in order to determine the severity level of the defect and formulate corresponding maintenance strategies. Therefore, this implementation introduces a contour fitting and type determination mechanism based on the coherent field matrix.

[0135] Specifically, based on the distribution of the coherent field matrix within the neighborhood of the sub-pixel-level center coordinates, the contour and range of the target defect are fitted. The specific implementation process is as follows: First, a local neighborhood is defined within the coherent field matrix, with the sub-pixel-level center coordinates as the origin. The size of this neighborhood is not fixed but adaptively determined. In one implementation, the system can calculate the rate at which the energy density of the coherent field matrix decays from the center to the periphery. When the energy decays to a preset proportion (such as 10% or 20%) of the central peak, this distance is defined as the neighborhood radius. This neighborhood definition method based on energy decay ensures that the fitted range tightly encompasses the core response region of the defect, avoiding the introduction of surrounding background noise.

[0136] Within a defined neighborhood, the distribution of the coherence field matrix is ​​analyzed using an image segmentation algorithm. Since the coherence field matrix is ​​essentially a continuous energy field, its numerical gradient reflects the spatial rate of change of the semantic matching degree.

[0137] An active contour model or level set method can be used, starting with the sub-pixel center as the initial seed point, and evolving towards the edge under the guidance of the energy gradient, eventually converging at the position where the energy change is most drastic, i.e., the physical boundary of the defect. In this way, not only is the center point of the defect output, but also a closed contour curve describing the shape of the defect and the pixel area enclosed by the contour (i.e., the defect range) is generated.

[0138] It should be understood that the contour fitting algorithm is not limited to the above example, and the edge detection algorithm based on threshold segmentation is also applicable to this implementation.

[0139] Furthermore, based on the outline and extent of the target defect, the defect type is determined, specifically including the following logic: different types of hidden defects have significant differences in visual geometry. The system pre-stores geometric feature templates for various typical defects. After fitting the defect outline, the system extracts the geometric features of the outline, such as aspect ratio, roundness, convexity, and edge curvature. For example, if the fitted outline is long and thin, and the aspect ratio exceeds a preset threshold (e.g., aspect ratio greater than 5:1), and the edge curvature is high, the system classifies it as a "crack" defect, because cracks usually exhibit long linear features. Conversely, if the fitted outline presents an approximately circular or elliptical cavity at a connection point, and a pin should exist at that location, the system classifies it as a "missing cotter pin." Furthermore, if the outline covers the surface of the hardware and presents a large area of ​​irregular patches, combined with abnormal texture features, the system may classify it as "hardware corrosion."

[0140] By introducing contour fitting and type determination steps, this implementation method upgrades simple "location" to "qualitative location," providing richer and more comprehensive defect information for transmission line operation and maintenance. It not only informs inspection personnel where the defect is, but also what the defect looks like and what its approximate type is, thereby effectively improving the practical value of inspection results and the efficiency of operation and maintenance decisions.

[0141] To achieve the above functions, the transmission line detection device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art will readily recognize that, based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] This disclosure also provides an embodiment such as Figure 2 The transmission line detection device shown includes: a feature extraction module 21, a phase space alignment module 22, a candidate region determination module 23, a semantic direction derivative calculation module 24, a singularity value determination module 25, an adaptive threshold determination module 26, and a sub-pixel positioning module 27.

[0143] The feature extraction module 21 is used to extract features from aerial images of power transmission lines and natural language detection commands input by users, respectively, to obtain image feature maps and text feature vectors.

[0144] The phase space alignment module 22 is used to perform phase space alignment between text feature vectors and image feature maps based on cyclic cross-correlation operators. During the phase space alignment process, geomagnetic topological prior bias is incorporated to correct spatial topological errors caused by changes in aerial photography attitude, resulting in a coherent field matrix. The size of each coordinate point element in the coherent field matrix represents the semantic matching degree between the image region at that coordinate point and the text command.

[0145] Candidate region determination module 23 is used to determine candidate regions from the image feature map based on the coherence field matrix.

[0146] The semantic direction derivative calculation module 24 is used to extract multi-scale visual features of candidate regions and determine the semantic direction derivative of multi-scale visual features along the semantic direction on the normal device feature manifold; the normal device feature manifold is constructed from a large number of normal samples; the semantic direction is the direction indicated by the text feature vector; the semantic direction derivative represents the change of the feature distribution of multi-scale visual features on the normal device feature manifold.

[0147] The singularity value determination module 25 is used to determine the singularity value of the candidate region based on the semantic directional derivative and the structural integrity constraint tensor. The structural integrity constraint tensor is constructed based on the inherent topological logic of the physical structure of the transmission line components. The inherent topological logic includes physical constraint rules for the form of hardware and the connection relationship of components. The singularity value characterizes the possibility of hidden defects in the candidate region.

[0148] The adaptive threshold determination module 26 is used to determine the adaptive detection threshold of the candidate region based on the first feature distribution of the candidate region, the second feature distribution of the background region of the aerial image, and the background noise covariance matrix; the background noise covariance matrix characterizes the degree of fluctuation of the visual noise of the background region in each feature dimension and the degree of correlation between dimensions.

[0149] The subpixel localization module 27 is used to obtain the subpixel-level center coordinates of the target defect when the singularity value of the candidate region is greater than the adaptive detection threshold. It uses the coherent field matrix as the energy density field and simulates the gravitational collapse process in the candidate region through closed-loop integration to solve for the coordinate point that maximizes the integral.

[0150] Furthermore, the feature extraction module 21, phase space alignment module 22, candidate region determination module 23, semantic direction derivative calculation module 24, singularity value determination module 25, adaptive threshold determination module 26, and sub-pixel localization module 27 are respectively configured to implement the corresponding steps in the above embodiments. To avoid repetition, the functional implementation steps of the corresponding modules will not be described again here.

[0151] Regarding the apparatus in the above embodiments, the specific manner in which each unit module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0152] Figure 3 This is a schematic diagram of an electronic device provided in this application. (For example...) Figure 3 The electronic device 30 may include at least one processor 301 and a memory 303 for storing processor-executable instructions. The processor 301 is configured to execute the instructions in the memory 303 to implement the power line detection method in the following embodiments.

[0153] In addition, the electronic device 30 may also include a communication bus 302, at least one communication interface 304, an input device 306, and an output device 305.

[0154] Processor 301 may be a processor (central processing unit, CPU), microprocessor unit, ASIC, or one or more integrated circuits for controlling the execution of programs according to the present application.

[0155] The communication bus 302 may include a path for transmitting information between the aforementioned components.

[0156] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0157] Input device 306 is used to receive input signals and output device 305 is used to output signals.

[0158] Memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0159] The memory 303 stores instructions for executing the scheme of this application, and the processor 301 controls the execution. The processor 301 executes the instructions stored in the memory 303 to realize the functions of the method of this application.

[0160] In a specific implementation, as one example, processor 301 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 in the CPU.

[0161] In a specific implementation, as one example, the electronic device 30 may include multiple processors, such as... Figure 3 Processors 301 and 307 are described in the text. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0162] The electronic device is as follows Figure 3 The diagram includes a processor 301 and a memory 303 for storing executable instructions of the processor 301; wherein the processor 301 is configured to execute executable instructions to implement a transmission line detection method as described in any of the possible implementations above. And it can achieve the same technical effect, so to avoid repetition, it will not be described again here.

[0163] This application also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of a power transmission line detection device or electronic device, the power transmission line detection device or electronic device is able to perform a power transmission line detection method as described in any of the possible implementations above. And it can achieve the same technical effect; to avoid repetition, it will not be described again here.

[0164] This application also provides a computer program product, including a computer program or instructions, which are executed by a processor as a transmission line detection method according to any of the possible implementations described above. It achieves the same technical effect, and to avoid repetition, will not be described again here.

[0165] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0166] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A power line detection method characterized by comprising: The method includes: Feature extraction was performed on aerial images of power transmission lines and natural language detection commands input by users to obtain image feature maps and text feature vectors. Based on the cyclic cross-correlation operator, the text feature vector and the image feature map are phase-space aligned. During the phase-space alignment process, a geomagnetic topological prior bias is incorporated to correct the spatial topological error caused by the change in aerial photography attitude, resulting in a coherent field matrix. The size of each coordinate point element in the coherent field matrix represents the semantic matching degree between the image region at that coordinate point and the text command. Candidate regions are determined from the image feature map based on the coherence field matrix; Multi-scale visual features of the candidate region are extracted, and the semantic direction derivative of the multi-scale visual features along the semantic direction on the normal device feature manifold is determined; the normal device feature manifold is constructed from a large number of normal samples; the semantic direction is the direction indicated by the text feature vector; the semantic direction derivative characterizes the change of the feature distribution of the multi-scale visual features on the normal device feature manifold. The singularity value of the candidate region is determined based on the semantic directional derivative and the structural integrity constraint tensor. The structural integrity constraint tensor is constructed based on the inherent topological logic of the physical structure of the transmission line components. The inherent topological logic includes physical constraint rules for hardware morphology and component connection relationships. The singularity value characterizes the possibility of hidden defects in the candidate region. An adaptive detection threshold for the candidate region is determined based on the first feature distribution of the candidate region, the second feature distribution of the background region of the aerial image, and the background noise covariance matrix; the background noise covariance matrix characterizes the degree of fluctuation of visual noise in the background region across each feature dimension and the degree of correlation between dimensions. When the singularity value of the candidate region is determined to be greater than the adaptive detection threshold, the coherent field matrix is ​​used as the energy density field. In the candidate region, the gravitational collapse process is simulated by closed-loop integration, and the coordinate point that maximizes the integration is solved to obtain the sub-pixel-level center coordinates of the target defect. The step of performing phase space alignment between the text feature vector and the image feature map based on the cyclic cross-correlation operator, and incorporating geomagnetic topological prior bias during the phase space alignment process, specifically includes: The text feature vector is used as a one-dimensional kernel, and a cyclic cross-correlation operation is performed with the image feature map in the spatial domain to generate a preliminary response map. Obtain the geomagnetic azimuth and UAV attitude angle corresponding to the aerial image; Based on the geomagnetic azimuth and UAV attitude angle, a geomagnetic topology bias matrix is ​​constructed, wherein the weights in the geomagnetic topology bias matrix that are consistent with the standard geomagnetic topology of the transmission line are enhanced, and the weights in the geomagnetic topology bias matrix that are inconsistent with the standard geomagnetic topology of the transmission line are suppressed. The preliminary response map is fused with the geomagnetic topological bias matrix element by element to obtain the coherent field matrix after spatial topological error correction.

2. The method of claim 1, wherein, The step of determining candidate regions from the image feature map based on the coherence field matrix specifically includes: From the image feature map, extract the response regions where the coordinate elements of the coherent field matrix are larger than a preset threshold; Connectivity analysis is performed on the extracted response region, and adjacent pixels are aggregated into at least one connected region; The connected regions are used to generate corresponding bounding boxes, which are then used as candidate regions.

3. The method of claim 1, wherein, Determining the semantic direction derivative of the multi-scale visual features along the semantic direction on the normal device feature manifold specifically includes: The multi-scale visual features are projected onto the normal device feature manifold using an autoencoder or manifold learning model trained on normal samples. Determine the projection point on the normal device feature manifold, and the tangent vector on the manifold along the direction indicated by the text feature vector; The semantic directional derivative is determined based on the magnitude and direction coefficient of the tangent vector.

4. The method of claim 1, wherein, The step of determining the singularity value of the candidate region based on the semantic direction derivative and the structural integrity constraint tensor specifically includes: The semantic directional derivative is projected onto the physically reasonable change subspace; the physically reasonable change subspace is composed of the structural integrity constraint tensor. The vector magnitude after projection of the physically rationally changed subspace is determined as the singularity value; the vector magnitude after projection of the physically rationally changed subspace does not include the component perpendicular to the physically rationally changed subspace.

5. The method of claim 1, wherein, The step of determining the adaptive detection threshold of the candidate region based on the first feature distribution of the candidate region, the second feature distribution of the background region of the aerial image, and the background noise covariance matrix includes: Determine the KL divergence between the first feature distribution and the second feature distribution, and determine the trace of the background noise covariance matrix; the KL divergence characterizes the feature differences between the candidate region and the background region. The adaptive detection threshold is obtained by weighting the KL divergence with the reciprocal of the trace of the background noise covariance matrix.

6. The method of claim 1, wherein, The method of simulating the gravitational collapse process through closed-loop integration and finding the coordinate point that maximizes the integral specifically includes: A variable closed boundary curve is defined with the centroid of the candidate region as the center and associated with preset parameters; the preset parameters include a parameter vector characterizing the shape and position of the variable closed boundary curve. Using the norm of the coherent field matrix as the energy density, the total energy within the region enclosed by the variable closed boundary curve is determined as the regional integral. With the goal of maximizing the regional integral and the constraint that the perimeter of the variable closed boundary curve is within a preset perimeter range, the preset parameters are increased sequentially to obtain the adjusted variable closed boundary curve and the adjusted regional integral, until the adjusted regional integral reaches the convergence condition. The geometric center of the variable closed boundary curve corresponding to the region integral that meets the convergence condition is determined as the sub-pixel-level center coordinates.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Based on the distribution of the coherent field matrix in the neighborhood of the sub-pixel-level center coordinates, the contour and range of the target defect are fitted. Based on the outline and extent of the target defect, the defect type of the target defect is determined.

8. A power line detection device for performing the power line detection method according to any one of claims 1 to 7, characterized by The device includes: The feature extraction module is used to extract features from aerial images of power transmission lines and natural language detection commands input by users, respectively, to obtain image feature maps and text feature vectors. The phase space alignment module is used to perform phase space alignment between the text feature vector and the image feature map based on the cyclic cross-correlation operator. During the phase space alignment process, a geomagnetic topological prior bias is incorporated to correct the spatial topological error caused by changes in aerial photography attitude, resulting in a coherent field matrix. The size of each coordinate point element in the coherent field matrix represents the semantic matching degree between the image region at that coordinate point element and the text command. The candidate region determination module is used to determine candidate regions from the image feature map based on the coherence field matrix. The semantic direction derivative calculation module is used to extract multi-scale visual features of the candidate region and determine the semantic direction derivative of the multi-scale visual features along the semantic direction on the normal device feature manifold; the normal device feature manifold is constructed from a large number of normal samples; the semantic direction is the direction indicated by the text feature vector; the semantic direction derivative characterizes the change of the feature distribution of the multi-scale visual features on the normal device feature manifold; The singularity value determination module is used to determine the singularity value of the candidate region based on the semantic direction derivative and the structural integrity constraint tensor; the structural integrity constraint tensor is constructed based on the inherent topological logic of the physical structure of the transmission line components, and the inherent topological logic includes physical constraint rules for hardware shape and component connection relationship; the singularity value characterizes the possibility of hidden defects in the candidate region. An adaptive threshold determination module is used to determine an adaptive detection threshold for the candidate region based on the first feature distribution of the candidate region, the second feature distribution of the background region of the aerial image, and the background noise covariance matrix; the background noise covariance matrix characterizes the degree of fluctuation of visual noise in the background region in each feature dimension and the degree of correlation between dimensions; The subpixel localization module is used to, when the singularity value of the candidate region is determined to be greater than the adaptive detection threshold, use the coherent field matrix as the energy density field, and in the candidate region, simulate the gravitational collapse process through closed-loop integration to solve for the coordinate point that maximizes the integral, thereby obtaining the subpixel-level center coordinates of the target defect. The phase space alignment module is specifically used to take the text feature vector as a one-dimensional kernel and perform cyclic cross-correlation operation with the image feature map in the spatial dimension to generate a preliminary response map; obtain the geomagnetic azimuth angle and UAV attitude angle corresponding to the aerial image; construct a geomagnetic topology bias matrix based on the geomagnetic azimuth angle and UAV attitude angle, wherein the weights in the geomagnetic topology bias matrix that are consistent with the standard geomagnetic topology of the transmission line are enhanced, and the weights in the geomagnetic topology bias matrix that are inconsistent with the standard geomagnetic topology of the transmission line are suppressed; and perform element-wise weighted fusion of the preliminary response map and the geomagnetic topology bias matrix to obtain the coherent field matrix after spatial topology error correction.

9. A computer-readable storage medium having stored thereon instructions, the computer-readable storage medium comprising: When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the transmission line detection method as described in any one of claims 1-7.

Images