Power line real-time identification and spatial positioning method based on depth vision assistance

Abstract

The invention discloses a power line real-time identification and spatial positioning method based on depth vision assistance, and relates to the technical field of power line real-time identification, and the method comprises the steps: constructing a lead identification and positioning frame with a strong generalization capability through deep fusion of a physical law and a depth vision technology; a vibration propagation model based on a wave equation can accurately capture signal interruption characteristics at a cross point, and a frequency domain fusion mechanism significantly improves the linear structure recovery capability of a backlight scene; dynamic vibration phases are coded into a continuous radiation density function by adopting an implicit neural radiation field technology, and the dependence of traditional point cloud reconstruction on dense sampling is broken through; and a dynamic deformable convolution kernel is introduced to realize self-adaptive tracking of the wire form through a physical deformation model, so that extreme working conditions such as icing can be effectively handled.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time recognition of power lines, and in particular to a method for real-time recognition and spatial positioning of power lines based on depth vision assistance. Background Art

[0002] With the deep integration of drones and computer vision technology, intelligent power line inspection has achieved core capabilities such as real-time analysis of aerial images and dynamic calibration of three-dimensional spatial coordinates. The current mainstream solution adopts a collaborative framework of multispectral imaging, laser point cloud and deep learning, extracts conductor pixel features through an improved U-Net segmentation network, and combines SLAM technology to achieve sub-meter positioning. However, in the scenario of multi-circuit transmission lines on the same tower at 500kV and above, the conductor crossing area causes point cloud scattering due to spatial projection overlap and electromagnetic interference, which leads to feature confusion.

[0003] Some solutions use three-dimensional point cloud dynamic reconstruction algorithms, such as NeRF++, which implicitly models the spatial distribution of wires through neural radiation fields to alleviate the misjudgment of static intersections; some solutions introduce graph neural networks (GNNs) to process wire topological relationships and use node embedding to characterize the connection characteristics of intersection areas; some solutions use spatiotemporal attention mechanisms to track the movement trajectories of wires in video streams to distinguish real intersections from temporary obstructions; however, these solutions rely on dense sampling for point cloud reconstruction, and holes are prone to appear in intersection areas due to wire occlusion; graph node features are difficult to stably represent topological changes caused by high-frequency vibrations in dynamic scenarios; and the attention mechanism is insufficient to model the long-range dependencies of low-texture metal wires.

[0004] Recently, some studies have introduced physical prior constraints to optimize the feature extraction process; the vibration model based on the conductor wave equation is encoded into the Transformer architecture, and the intersection motion pattern is predicted through differential homeomorphism mapping; some schemes design multi-scale frequency domain fusion modules to separate the conductor vibration signal and environmental noise in the Hilbert-Huang transform domain; there are also methods to transform intersection recognition into an optimal transmission problem, and use the Sinkhorn algorithm to calculate the matching cost of conductor pixels; but these improvements are still limited by problems such as sensor noise amplification and a surge in computational complexity, and have not yet achieved stable generalization under complex working conditions; current technical routes generally face the difficult problem of balancing the physical interpretability of the algorithm with engineering practicality, and there is an urgent need to develop a new feature decoupling mechanism to break through the bottleneck of intersection misidentification. Summary of the Invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] The present invention provides a method for real-time identification and spatial positioning of power lines based on deep vision assistance to solve the problems of existing solutions relying on dense point cloud reconstruction and static topology modeling, which are difficult to deal with problems such as wire crossing vibration, strong light interference and dynamic deformation, and the imbalance between the algorithm's physical interpretability and engineering practicality.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] The embodiment of the present invention provides a method for real-time recognition and spatial positioning of power lines based on depth vision assistance, which includes:

[0009] Step S1, acquiring real-time video stream data of the power transmission line through an airborne visual sensor, and performing multimodal data alignment processing on the video stream;

[0010] Step S2: constructing a multi-scale frequency domain fusion module to extract frequency domain features of the wire area in the video stream, and extracting spatial frequency distribution features including wire vibration characteristics;

[0011] Step S3: Based on the spatiotemporal topological relationship modeling module, a motion transmission model of the wire pixels between adjacent frames is established to generate a wire vibration signal propagation path;

[0012] Step S4, using implicit neural radiation field technology to construct a three-dimensional motion trajectory constraint field of the wire according to the vibration signal propagation path;

[0013] Step S5, adjusting the wire shape detection template in real time by combining the environmental meteorological parameters through the dynamic deformable convolution kernel matching module;

[0014] Step S6: output the location and three-dimensional coordinates of the wire intersection, and correct the misidentified area based on the topological connectivity rule.

[0015] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance described in the present invention, the multimodal data alignment process specifically includes:

[0016] The polarization reflection feature map of the metal wire is generated by fusing the polarization degree map of the polarization depth camera with the RGB image.

[0017] Perform sparse processing on the laser point cloud data to extract the main direction vector of the axial distribution of the wire;

[0018] The spatiotemporal registration of multi-source data is achieved through the Lie group SE(3) transformation matrix.

[0019] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance described in the present invention, wherein: in step S1, in the process of realizing the spatiotemporal registration of multi-source data by using the Lie group SE (3) transformation matrix, an initial external participation motion model of each sensor is established, and then the time offset is compensated, the steps include:

[0020] Define the initial extrinsic parameter matrix of sensor No. m in the reference coordinate system:

[0021]

[0022] Among them, T m,0 Represents the initial external parameter transformation matrix of the mth sensor, R m,0 represents the corresponding 3×3 rotation matrix, t m,0 Indicates the corresponding three-dimensional translation vector, 0 indicates the three-dimensional zero vector, and the subscript represents transpose, scalar 1 is the homogeneous coordinate unit;

[0023] Represent the sensor motion as Lie algebra elements:

[0024]

[0025] Among them, ξ m is the Lie algebra vector, ω m represents the angular velocity vector, v m represents the translation velocity vector, [ω m ] × Indicates that ω m The mapping is an antisymmetric matrix;

[0026] Calculate the time offset using the formula:

[0027] Δt=t ref -t 0,m ,

[0028] Where Δt represents the difference between the target alignment time and the calibration time of the mth sensor, t ref represents the selected reference alignment moment, t 0,m Indicates the calibration time of sensor No. m;

[0029] Compensating motion via exponential mapping:

[0030]

[0031] Among them, T m (t ref ) represents the external parameter matrix compensated to the reference time, exp represents the matrix exponential mapping, is the Lie algebra generator, Δt is the time offset, T m,0 is the initial external parameter;

[0032] The matrix exponential is:

[0033]

[0034] Among them, i4 represents the 4×4 identity matrix, Represents matrix multiplication;

[0035] A source data point p m Mapping to world coordinates:

[0036]

[0037] Among them, p m represents the homogeneous coordinate point in the sensor coordinate system, represents the inverse of the reference sensor extrinsic matrix, p w Indicates the unified world coordinate point.

[0038] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on deep vision assistance described in the present invention, the multi-scale frequency domain fusion module is composed of a cascaded Gabor filter bank, and the filter parameters satisfy:

[0039] Spatial frequency range: 0.5-1.5 lines / pixel;

[0040] Direction selection: 6 discrete angles of ±15° along the conductor axis;

[0041] Scale division: bandwidth is set based on 1 / 2, 1, and 2 times the wire diameter;

[0042] The multi-scale frequency domain fusion module includes:

[0043] Short-time Fourier transform layer, used to extract time-frequency features of video frame sequences;

[0044] Frequency domain attention selection layer, dynamically enhancing the 0.5-1.5 lines / pixel frequency response corresponding to the wire features;

[0045] The phase consistency reconstruction layer restores the edge features of linear structures that are submerged by strong light.

[0046] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance described in the present invention, the spatiotemporal topological relationship modeling module is specifically implemented as follows:

[0047] The micro-vibration displacement field of wire pixels between adjacent frames is extracted based on the improved RAFT optical flow algorithm;

[0048] Construct the conductor wave differential equation and identify the intersection interruption location through the vibration signal attenuation characteristics;

[0049] A graph neural network is used to perform topological embedding representation of the connection relationship of wires at intersections;

[0050] The improved RAFT optical flow algorithm adds a vibration perception branch after the original feature extraction network, which includes:

[0051] Temporal difference convolution layer: calculates the gradient of pixel intensity changes of three consecutive frames;

[0052] Frequency domain attention gate: suppresses ambient jitter noise with a frequency below 0.5Hz;

[0053] Displacement correction unit: constrains the optical flow vector amplitude according to the Young's modulus of the wire.

[0054] As a preferred solution of the method for real-time identification and spatial positioning of power lines based on depth vision assistance described in the present invention, wherein: in step S3, the transmission line is regarded as a continuous string under tension and containing damping, and a damped wave equation is established based on mechanical equilibrium;

[0055] Establish the partial differential equation for the lateral displacement w(x,t):

[0056] Where ρ represents the wire density, A represents the cross-sectional area, represents the second-order derivative with respect to time, c d represents the total damping coefficient, represents the first derivative with respect to time, T represents the line tension, represents the second-order derivative along the length, and the scalar 0 is the equilibrium state;

[0057] Define the equivalent elastic modulus:

[0058]

[0059] Where k represents the equivalent elastic coefficient, Δx represents the length unit of discrete modeling;

[0060] The damping coefficient is:

[0061]

[0062] Here, ζ represents the damping ratio.

[0063] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance described in the present invention, the implicit neural radiation field technology uses a position encoding function to map three-dimensional coordinates to a high-dimensional feature space, and uses a multi-layer perceptron to model the implicit relationship between the radiation density and vibration phase of the conductor, wherein the vibration phase parameter is dynamically injected by the motion transfer model output in step S3;

[0064] The construction process of the implicit neural radiation field technology includes:

[0065] Encode the conductor vibration signal propagation path as a position-direction related radiation density function;

[0066] Generate continuous implicit representation of wire motion trajectory through differentiable rendering;

[0067] The reflection characteristics of the wire material are introduced as physical constraints for constructing the radiation field.

[0068] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance described in the present invention, in step S4, the propagation path of the wire vibration signal is encoded as a position-direction related radiation density function, and in the obtained vibration signal propagation path set {(x i ,d i ,φ i )}, high-dimensional mapping is performed on the spatial position x and direction d respectively:

[0069]

[0070] Among them, γ p (x) represents the position encoding vector, Represents three-dimensional coordinates, L p Represents the number of position encoding layers, π represents pi, and the function calculates the sine and cosine of each coordinate component one by one;

[0071] Similarly, the direction encoding is:

[0072]

[0073] Among them, γ d (d) represents the direction encoding vector, Represents the unit direction vector, L d Indicates the number of direction encoding layers;

[0074] The encoded vector and the vibration phase φ(x) are combined to form the input:

[0075] z=[γ p (x),γ d (d),φ(x)],

[0076] Wherein, φ(x) represents the vibration phase at the position output in step S3, and φ(x) is obtained by calculating the discrete phase {φ i}At the spatial position {x i} is obtained by interpolation or fitting;

[0077] Use multi-layer perceptron to map to radiation density σ(x,d):

[0078] h 0 =z,

[0079] h l =ReLU(W l h l-1 +b l )(l=1,…,L-1),

[0080] σ(x,d)=W σ h L-1 +b σ ,

[0081] Among them, h l represents the hidden state of the lth layer, are the weight matrix and bias vector of the lth layer, N l is the number of hidden units in the lth layer of the network, N0=dim(z) represents the input dimension, ReLU(·) represents the linear rectification function, and L represents the total number of network layers. b σ ∈R are the output layer parameters, and σ(x, d) is the spatial-directional correlation radiation density function after encoding the vibration signal propagation path, which is used for subsequent differentiable rendering.

[0082] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance described in the present invention, the dynamic deformable convolution kernel matching module specifically includes:

[0083] Establish the physical deformation mapping relationship between conductor diameter change and temperature and wind speed;

[0084] Generate parameterized deformable convolution kernels whose shapes are dynamically adjusted with meteorological parameters;

[0085] A meta-learning strategy is used to optimize the online update process of the convolution kernel deformation parameters.

[0086] As a preferred solution of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance described in the present invention, the topological connectivity rule is implemented as follows:

[0087] Generate a priori knowledge base of conductor connection relationship based on the topological structure of transmission lines;

[0088] Perform connected domain analysis on the misidentified area and calculate its curvature continuity with the adjacent wires;

[0089] The spatial coordinate offset at the intersection point is corrected by the optimal transmission algorithm.

[0090] The beneficial effects of the present invention are as follows: the present invention constructs a wire identification and positioning framework with strong generalization ability through the deep integration of physical laws and deep vision technology; the vibration propagation model based on the wave equation can accurately capture the signal interruption characteristics at the intersection, and the frequency domain fusion mechanism significantly improves the linear structure recovery ability of the backlight scene; the implicit neural radiation field technology is used to encode the dynamic vibration phase into a continuous radiation density function, breaking through the traditional point cloud reconstruction's dependence on dense sampling; the dynamic deformable convolution kernel is introduced to realize the adaptive tracking of the wire morphology through the physical deformation model, which effectively copes with extreme working conditions such as icing; the topological connectivity rules are combined with the prior knowledge base to systematically reduce the risk of misidentification. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0092] Figure 1 This is a flow chart of the method for real-time recognition and spatial positioning of power lines based on depth vision assistance in Example 1. DETAILED DESCRIPTION

[0093] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0094] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0095] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0096] Example 1, with reference to Figure 1 This embodiment provides a method for real-time recognition and spatial positioning of power lines based on depth vision assistance, comprising the following steps:

[0097] Step S1: acquiring real-time video stream data of the power transmission line through an airborne visual sensor and performing multimodal data alignment processing on the video stream;

[0098] Multimodal data alignment processing specifically includes:

[0099] The polarization reflection feature map of the metal wire is generated by fusing the polarization degree map of the polarization depth camera with the RGB image.

[0100] Perform sparse processing on the laser point cloud data to extract the main direction vector of the axial distribution of the wire;

[0101] The spatiotemporal registration of multi-source data is achieved through the Lie group SE(3) transformation matrix;

[0102] In step S1, in the process of realizing the spatiotemporal registration of multi-source data by using the Lie group SE (3) transformation matrix, an initial external motion model of each sensor is established, and then the time offset is compensated. The steps include:

[0103] Define the initial extrinsic parameter matrix of sensor No. m in the reference coordinate system:

[0104]

[0105] Among them, T m,0 Represents the initial external parameter transformation matrix of the mth sensor, R m,0 represents the corresponding 3×3 rotation matrix, t m,0 Indicates the corresponding three-dimensional translation vector, 0 indicates the three-dimensional zero vector, and the subscript represents transpose, scalar 1 is the homogeneous coordinate unit;

[0106] Represent the sensor motion as Lie algebra elements:

[0107]

[0108] Among them, ξ m is the Lie algebra vector, ω m represents the angular velocity vector, v m represents the translation velocity vector, [ω m ] × Indicates that ω m The mapping is an antisymmetric matrix;

[0109] Calculate the time offset using the formula:

[0110] Δt=t ref -t 0,m ,

[0111] Where Δt represents the difference between the target alignment time and the calibration time of the mth sensor, t ref represents the selected reference alignment moment, t 0,m Indicates the calibration time of sensor No. m;

[0112] Compensating motion via exponential mapping:

[0113]

[0114] Among them, T m (t ref ) represents the external parameter matrix compensated to the reference time, exp represents the matrix exponential mapping, is the Lie algebra generator, Δt is the time offset, T m,0 is the initial external parameter;

[0115] The matrix exponential is:

[0116]

[0117] Where I4 represents the 4×4 identity matrix, Represents matrix multiplication;

[0118] A source data point p m Mapping to world coordinates:

[0119]

[0120] Among them, p m represents the homogeneous coordinate point in the sensor coordinate system, represents the inverse of the reference sensor extrinsic matrix, p w Indicates the unified world coordinate point;

[0121] Specifically, this method achieves dual alignment of multi-source sensors in space and time by establishing an extrinsic parameter matrix and a Lie algebra motion model. In space, the rotation and translation differences of different sensors are unified in the form of SE(3) transformation to ensure geometric consistency of each mode. In time, by interpolating and compensating the exponential mapping of the sensor motion generator, any delayed data can be synchronized at the reference time, thereby eliminating the geometric misalignment caused by timing deviation. The matrix exponent takes into account both linear and second-order motion effects, meeting real-time requirements while maintaining sub-pixel alignment accuracy. The framework has good modularity and can be flexibly expanded to more sensors.

[0122] Step S2: construct a multi-scale frequency domain fusion module to extract frequency domain features of the wire area in the video stream, and extract the spatial frequency distribution features including the wire vibration characteristics;

[0123] The multi-scale frequency domain fusion module is composed of a cascaded Gabor filter bank, and the filter parameters satisfy:

[0124] Spatial frequency range: 0.5-1.5 lines / pixel;

[0125] Direction selection: 6 discrete angles of ±15° along the conductor axis;

[0126] Scale division: bandwidth is set based on 1 / 2, 1, and 2 times the wire diameter;

[0127] The multi-scale frequency domain fusion module includes:

[0128] Short-time Fourier transform layer, used to extract time-frequency features of video frame sequences;

[0129] Frequency domain attention selection layer, dynamically enhancing the 0.5-1.5 lines / pixel frequency response corresponding to the wire features;

[0130] Phase consistency reconstruction layer, which restores the edge features of linear structures submerged by strong light;

[0131] Step S3: Based on the spatiotemporal topological relationship modeling module, a motion transmission model of the wire pixels between adjacent frames is established to generate a wire vibration signal propagation path;

[0132] The specific implementation of the spatiotemporal topological relationship modeling module is as follows:

[0133] The micro-vibration displacement field of wire pixels between adjacent frames is extracted based on the improved RAFT optical flow algorithm;

[0134] Construct the conductor wave differential equation and identify the intersection interruption location through the vibration signal attenuation characteristics;

[0135] A graph neural network is used to perform topological embedding representation of the connection relationship of wires at intersections;

[0136] The improved RAFT optical flow algorithm adds a vibration perception branch after the original feature extraction network. This branch includes:

[0137] Temporal difference convolution layer: calculates the gradient of pixel intensity changes of three consecutive frames;

[0138] Frequency domain attention gate: suppresses ambient jitter noise with a frequency below 0.5Hz;

[0139] Displacement correction unit: constrains the optical flow vector amplitude according to the Young's modulus of the wire;

[0140] In step S3, the transmission line is regarded as a continuous string under tension and containing damping, and a damped wave equation is established based on mechanical equilibrium;

[0141] Establish the partial differential equation for the lateral displacement w(x,t):

[0142] Where ρ represents the wire density, A represents the cross-sectional area, represents the second-order derivative with respect to time, c d represents the total damping coefficient, represents the first derivative with respect to time, T represents the line tension, represents the second-order derivative along the length, and the scalar 0 is the equilibrium state;

[0143] Define the equivalent elastic modulus:

[0144]

[0145] Where k represents the equivalent elastic coefficient, Δx represents the length unit of discrete modeling;

[0146] The damping coefficient is:

[0147]

[0148] Where ζ represents the damping ratio;

[0149] Specifically, this differential equation integrates the inertia, elasticity, and damping effects of the conductor to reflect the propagation and attenuation of lateral vibrations in the space-time domain. The inertia term ensures that the wave propagates at the physical speed of the wave, the tension term controls the wavelength and frequency characteristics, and the damping term describes the energy dissipation process through the damping ratio, which helps to filter low-frequency environmental jitter.

[0150] Step S4, using implicit neural radiation field technology to construct a three-dimensional motion trajectory constraint field of the wire according to the vibration signal propagation path;

[0151] The implicit neural radiation field technology uses a position encoding function to map three-dimensional coordinates to a high-dimensional feature space, and uses a multi-layer perceptron to model the implicit relationship between the wire radiation density and the vibration phase, where the vibration phase parameter is dynamically injected by the motion transfer model output in step S3;

[0152] The construction process of implicit neural radiation field technology includes:

[0153] Encode the conductor vibration signal propagation path as a position-direction related radiation density function;

[0154] Generate continuous implicit representation of wire motion trajectory through differentiable rendering;

[0155] The reflective characteristics of the conductor material are introduced as physical constraints for constructing the radiation field;

[0156] In step S4, the wire vibration signal propagation path is encoded as a position-direction related radiation density function. i ,d i ,φ i )}, high-dimensional mapping is performed on the spatial position x and direction d respectively:

[0157]

[0158] Among them, γ p (x) represents the position encoding vector, Represents three-dimensional coordinates, L p Represents the number of position encoding layers, π represents pi, and the function calculates the sine and cosine of each coordinate component one by one;

[0159] Similarly, the direction encoding is:

[0160]

[0161] Among them, γ d (d) represents the direction encoding vector, Represents the unit direction vector, L d Indicates the number of direction encoding layers;

[0162] The encoded vector and the vibration phase φ(x) are combined to form the input:

[0163] z=[γ p (x),γ d (d),φ(x)],

[0164] Wherein, φ(x) represents the vibration phase at the position output in step S3, and φ(x) is obtained by calculating the discrete phase {φ i}At the spatial position {x i} is obtained by interpolation or fitting;

[0165] Use multi-layer perceptron to map to radiation density σ(x,d):

[0166] h 0 =z,

[0167] h l =ReLU(W l h l-1 +b l )(l=1,…,L-1),

[0168] σ(x,d)=W σ h L-1 +b σ ,

[0169] Among them, h l represents the hidden state of the lth layer, are the weight matrix and bias vector of the lth layer, N l is the number of hidden units in the lth layer of the network, N0=dim(z) represents the input dimension, ReLU(·) represents the linear rectification function, and L represents the total number of network layers. b σ ∈R are the output layer parameters respectively, and σ(x, d) is the spatial-directional correlation radiation density function after encoding the vibration signal propagation path, which is used for subsequent differentiable rendering;

[0170] Specifically, through high-frequency position and direction encoding, the model can distinguish subtle vibration differences in space and the characteristics of different propagation directions, thereby forming a continuous implicit representation of the wire micro-vibration path. The vibration phase output from step S3 is introduced as a dynamic input, so that the radiation density function can perceive real-time vibration information rather than static geometric structure. The multi-layer perceptron structure and its weight parameterization design have sufficient expression capacity to fit the complex vibration-radiation relationship, while maintaining the differentiable characteristics of the network, meeting the gradient requirements of the differentiable renderer. In addition, this method has good scalability: the coding frequency band L can be increased or decreased. p ,L d Or adjust the hidden layer width N l , to adapt to different resolutions and computing resource limitations;

[0171] Step S5, adjusting the wire shape detection template in real time by combining the environmental meteorological parameters through the dynamic deformable convolution kernel matching module;

[0172] The dynamic deformable convolution kernel matching module specifically includes:

[0173] Establish the physical deformation mapping relationship between conductor diameter change and temperature and wind speed;

[0174] Generate parameterized deformable convolution kernels whose shapes are dynamically adjusted with meteorological parameters;

[0175] A meta-learning strategy is used to optimize the online update process of the convolution kernel deformation parameters;

[0176] In step S5, the physical deformation mapping relationship between the wire diameter change and the temperature and wind speed is established as follows:

[0177] A physical mapping model is constructed based on the influence of temperature and wind speed on the conductor diameter. The radial strain caused by thermal expansion and wind load is combined to determine the influence of the conductor diameter d on the temperature T and wind speed v. w The change relationship can be expressed as:

[0178]

[0179] Among them, d(T,v w ) indicates that the ambient temperature is T and the wind speed is v w The conductor diameter at the time of calibration is d0, which is the initial diameter at the reference temperature T0 and no wind. α is the linear thermal expansion coefficient. ΔT = T-T0 is the temperature deviation. T0 is the reference ambient temperature during initial calibration. ν is the Poisson's ratio. ρ a Indicates the air density, C d represents the aerodynamic drag coefficient, E represents Young's modulus, v w represents the wind speed; the formula consists of two parts: the thermal expansion term αΔT is used to estimate the diameter increase caused by temperature change, and the wind load term Based on aerodynamic drag and material Poisson effect, the secondary effect of wind speed on radial contraction is characterized;

[0180] Specifically, the mapping incorporates the coupled effects of temperature and wind speed into the conductor diameter prediction: the thermal expansion term dominates the diameter change under low and medium wind speed conditions, the wind load term interacts with the Poisson's ratio through aerodynamic drag, and significantly reflects the radial compression effect under high wind speed scenarios. d The proposed method can be obtained through experimental calibration. It has low computational complexity and simple structure, making it very suitable for embedding in the dynamic deformable convolution kernel matching module to achieve real-time online update of the detection template size and shape, thereby enhancing the tracking and recognition accuracy of the wire contour under complex meteorological conditions.

[0181] Step S6: outputting the location and three-dimensional coordinates of the wire intersection, and correcting the misidentified area based on the topological connectivity rule;

[0182] Topological connectivity rules are implemented as follows:

[0183] Generate a priori knowledge base of conductor connection relationship based on the topological structure of transmission lines;

[0184] Perform connected domain analysis on the misidentified area and calculate its curvature continuity with the adjacent wires;

[0185] The spatial coordinate offset at the intersection point is corrected by the optimal transmission algorithm.

[0186] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for real-time recognition and spatial positioning of power lines based on deep vision assistance, characterized in that: include, Step S1, acquiring real-time video stream data of the power transmission line through an airborne visual sensor, and performing multimodal data alignment processing on the video stream; Step S2: constructing a multi-scale frequency domain fusion module to extract frequency domain features of the wire area in the video stream, and extracting spatial frequency distribution features including wire vibration characteristics; Step S3: Based on the spatiotemporal topological relationship modeling module, a motion transmission model of the wire pixels between adjacent frames is established to generate a wire vibration signal propagation path; Step S4, using implicit neural radiation field technology to construct a three-dimensional motion trajectory constraint field of the wire according to the vibration signal propagation path; Step S5, adjusting the wire shape detection template in real time by combining the environmental meteorological parameters through the dynamic deformable convolution kernel matching module; Step S6: output the location and three-dimensional coordinates of the wire intersection, and correct the misidentified area based on the topological connectivity rule.

2. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 1, characterized in that: The multimodal data alignment process specifically includes: The polarization reflection feature map of the metal wire is generated by fusing the polarization degree map of the polarization depth camera with the RGB image. Perform sparse processing on the laser point cloud data to extract the main direction vector of the axial distribution of the wire; The spatiotemporal registration of multi-source data is achieved through the Lie group SE(3) transformation matrix.

3. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 2, characterized in that: In step S1, in the process of realizing the spatiotemporal registration of multi-source data by the Lie group SE (3) transformation matrix, an initial external motion model of each sensor is established, and then the time offset is compensated. The steps include: Define the initial extrinsic parameter matrix of sensor No. m in the reference coordinate system: Among them, T m,0 Represents the initial external parameter transformation matrix of the mth sensor, R m,0 represents the corresponding 3×3 rotation matrix, t m,0 Indicates the corresponding three-dimensional translation vector, 0 indicates the three-dimensional zero vector, and the subscript represents transpose, scalar 1 is the homogeneous coordinate unit; Represent the sensor motion as Lie algebra elements: Among them, ξ m is the Lie algebra vector, ω m represents the angular velocity vector, v m represents the translation velocity vector, [ω m ] × Indicates that ω m The mapping is an antisymmetric matrix; Calculate the time offset using the formula: Δt=t ref -t 0,m , Where Δt represents the difference between the target alignment time and the calibration time of the mth sensor, t ref represents the selected reference alignment moment, t 0,m Indicates the calibration time of sensor No. Compensating motion via exponential mapping: Among them, T m (t ref ) represents the external parameter matrix compensated to the reference time, exp represents the matrix exponential mapping, is the Lie algebra generator, Δt is the time offset, T m,0 is the initial external parameter; The matrix exponential is: Where I4 represents the 4×4 identity matrix, Represents matrix multiplication; A source data point p m Mapping to world coordinates: Among them, p m represents the homogeneous coordinate point in the sensor coordinate system, represents the inverse of the reference sensor extrinsic matrix, p w Indicates the unified world coordinate point.

4. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 1, characterized in that: The multi-scale frequency domain fusion module is composed of a cascaded Gabor filter bank, and the filter parameters satisfy: Spatial frequency range: 0.5-1.5 lines / pixel; Direction selection: 6 discrete angles of ±15° along the conductor axis; Scale division: bandwidth is set based on 1 / 2, 1, and 2 times the wire diameter; The multi-scale frequency domain fusion module includes: Short-time Fourier transform layer, used to extract time-frequency features of video frame sequences; Frequency domain attention selection layer, dynamically enhancing the 0.5-1.5 lines / pixel frequency response corresponding to the wire features; The phase consistency reconstruction layer restores the edge features of linear structures that are submerged by strong light.

5. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 1, characterized in that: The spatiotemporal topological relationship modeling module is specifically implemented as follows: The micro-vibration displacement field of wire pixels between adjacent frames is extracted based on the improved RAFT optical flow algorithm; Construct the conductor wave differential equation and identify the intersection interruption location through the vibration signal attenuation characteristics; A graph neural network is used to perform topological embedding representation of the connection relationship of wires at intersections; The improved RAFT optical flow algorithm adds a vibration perception branch after the original feature extraction network, which includes: Temporal difference convolution layer: calculates the gradient of pixel intensity changes of three consecutive frames; Frequency domain attention gate: suppresses ambient jitter noise with a frequency below 0.5Hz; Displacement correction unit: constrains the optical flow vector amplitude according to the Young's modulus of the wire.

6. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 5, characterized in that: In step S3, the transmission line is regarded as a continuous string under tension and containing damping, and a damped wave equation is established based on mechanical equilibrium; Establish the partial differential equation for the lateral displacement w(x,t): Where ρ represents the wire density, a represents the cross-sectional area, represents the second-order derivative with respect to time, c d represents the total damping coefficient, represents the first derivative with respect to time, T represents the line tension, represents the second-order derivative along the length, and the scalar 0 is the equilibrium state; Define the equivalent elastic modulus: Where k represents the equivalent elastic coefficient, Δx represents the length unit of discrete modeling; The damping coefficient is: Here, ζ represents the damping ratio.

7. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 1, characterized in that: The implicit neural radiation field technology uses a position encoding function to map three-dimensional coordinates to a high-dimensional feature space, and uses a multi-layer perceptron to model the implicit relationship between the wire radiation density and the vibration phase, where the vibration phase parameter is dynamically injected by the motion transfer model output in step S3; The construction process of the implicit neural radiation field technology includes: Encode the conductor vibration signal propagation path as a position-direction related radiation density function; Generate continuous implicit representation of wire motion trajectory through differentiable rendering; The reflection characteristics of the wire material are introduced as physical constraints for constructing the radiation field.

8. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 7, characterized in that: In step S4, the wire vibration signal propagation path is encoded as a position-direction related radiation density function. i ,d i ,φ i )}, high-dimensional mapping is performed on the spatial position x and direction d respectively: Among them, γ p (x) represents the position encoding vector, Represents three-dimensional coordinates, L p Represents the number of position encoding layers, π represents pi, and the function calculates the sine and cosine of each coordinate component one by one; Similarly, the direction encoding is: Among them, γ d (d) represents the direction encoding vector, Represents the unit direction vector, L d Indicates the number of direction encoding layers; The encoded vector and the vibration phase φ(x) are combined to form the input: z=[γ p (x),c d (d),φ(x)], Wherein, φ(x) represents the vibration phase at the position output in step S3, and φ(x) is obtained by calculating the discrete phase {φ i }At the spatial position {x i } is obtained by interpolation or fitting; Use multi-layer perceptron to map to radiation density σ(x,d): h 0 =z, h l =ReLU(W l h l-1 +b l )(l=1,…,L-1), σ(x,d)=W σ h L-1 +b σ , Among them, h l represents the hidden state of the lth layer, are the weight matrix and bias vector of the lth layer, N l is the number of hidden units in the lth layer of the network, N0=dim(z) represents the input dimension, ReLU(·) represents the linear rectification function, and L represents the total number of network layers. b σ ∈R are the output layer parameters, and σ(x, d) is the spatial-directional correlation radiation density function after encoding the vibration signal propagation path, which is used for subsequent differentiable rendering.

9. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 1, characterized in that: The dynamic deformable convolution kernel matching module specifically includes: Establish the physical deformation mapping relationship between conductor diameter change and temperature and wind speed; Generate parameterized deformable convolution kernels whose shapes are dynamically adjusted with meteorological parameters; A meta-learning strategy is used to optimize the online update process of the convolution kernel deformation parameters.

10. The method for real-time recognition and spatial positioning of power lines based on depth vision assistance according to claim 1, characterized in that: The topology connectivity rule is implemented as follows: Generate a priori knowledge base of conductor connection relationship based on the topological structure of transmission lines; Perform connected domain analysis on the misidentified area and calculate its curvature continuity with the adjacent wires; The spatial coordinate offset at the intersection point is corrected by the optimal transmission algorithm.

Images