Monocular Camera-Based Adaptive Obstacle Detection Method and System for UAV Inspection Routes

Abstract

The present invention discloses a method and system for adaptive obstacle detection in an unmanned aerial vehicle (UAV) inspection route based on a monocular camera. The method includes obtaining initial flight parameters, optimizing the flight parameters based on scene complexity assessment, collecting an original image sequence and performing quality assessment and exposure compensation; constructing a multi-resolution feature pyramid, extracting feature descriptors, performing feature matching and verification; performing multi-constraint pose optimization and depth estimation based on the feature matching results, and reconstructing a three-dimensional point cloud; calculating the direction enhanced response to extract the power line candidate region, performing multi-view line segment matching and three-dimensional reconstruction, and analyzing the dynamic characteristics of the power line; performing obstacle segmentation based on the power line three-dimensional model and the depth map, calculating the distance of the obstacle, evaluating the risk level, and generating early warning information and a monitoring plan. The present invention can adaptively process different scenarios, accurately identify and evaluate the obstacle risk, and improve the inspection efficiency and reliability.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and in particular, to a method and system for detecting obstacles in a wire assembly based on a monocular camera. Background Art

[0002] In the maintenance and monitoring of power facilities, the safe operation of power lines is directly related to the reliability and stability of power supply. The traditional manual inspection method is not only time-consuming and laborious, but also inefficient in complex terrains or under adverse weather conditions, and there are safety hazards. With the rapid development of computer vision algorithms, the automatic inspection system based on a monocular camera has gradually become an important means for intelligent monitoring of power lines. This method can not only improve the inspection efficiency, but also obtain more comprehensive and objective monitoring data, providing important support for the preventive maintenance of power lines.

[0003] At present, the domestic and foreign research mainly focuses on two directions: object detection based on deep learning and feature matching based on traditional image processing. In terms of object detection, improved algorithms such as YOLO or Faster R-CNN are mainly used for obstacle recognition, but these methods often require a large amount of labeled data for training and are difficult to adapt to different scenarios and lighting conditions. In terms of feature matching, algorithms such as SIFT or SURF are commonly used to extract image features, and methods such as RANSAC are used for matching and screening, but these methods have high computational complexity and insufficient real-time performance. In terms of three-dimensional reconstruction, traditional methods mainly rely on SfM (Structure from Motion) technology to restore the three-dimensional structure of the scene through feature point matching and Bundle Adjustment optimization. However, when dealing with slender objects such as power lines, problems such as incomplete reconstruction or insufficient accuracy often occur.

[0004] However, the existing technical solutions still have the following key problems: First, the image acquisition process of power lines lacks scene adaptability, and fixed flight parameters are difficult to cope with scenes of different complexities, resulting in unstable quality of the acquired data; Second, in the feature extraction and matching links, the existing methods fail to fully utilize the directional and continuous features of power lines, resulting in a significant decrease in matching accuracy when the feature distribution is uneven or there is local occlusion; Third, in the three-dimensional reconstruction process, there is a lack of modeling of the unique deformation and dynamic characteristics of power lines, making the reconstruction results unable to accurately reflect the actual state of power lines; Fourth, the existing obstacle risk assessment methods are mainly based on static distance measurement, and do not consider the growth trend of obstacles and the dynamic influence of environmental factors, making it difficult to achieve the foresight of early warning; Fifth, in terms of image quality assessment, the existing methods often use a single clarity or brightness index, lacking a comprehensive assessment of scene integrity and feature reliability, which affects the accuracy of subsequent processing. These technical problems seriously restrict the actual application effect of the monocular camera inspection system. Summary of the Invention

[0005] Objective of the invention: to propose a method and system for detecting obstacles of wire components based on a monocular camera, in order to solve at least one technical problem existing in the prior art.

[0006] Technical solution: A method for detecting obstacles of wire components based on a monocular camera includes the following steps:

[0007] S1. Obtain initial flight parameters, and calculate optimized flight parameters through a scene complexity evaluation function; based on the optimized flight parameters, collect an original image sequence, calculate an image quality index, and obtain an image quality score; based on the image quality score, perform exposure compensation processing on the original image sequence to obtain a compensated image sequence; based on the compensated image sequence, calculate the spatial similarity and motion continuity between images, generate an image association matrix; based on the image association matrix, screen to obtain an optimal image combination; where the initial flight parameters include the initial flight altitude, the initial flight speed, and the initial sampling interval;

[0008] S2. Based on the optimal image combination and the image quality score, construct a multi-resolution feature pyramid to obtain a multi-scale feature tensor set; calculate orientation descriptors for the multi-scale feature tensor set to generate a set of feature descriptors; based on the set of feature descriptors and the image association matrix, calculate feature matching scores, and output an initial set of feature matching pairs; perform multi-level verification on the initial set of feature matching pairs to obtain a reliable matching set; analyze the feature distribution of the reliable matching set, generate supplementary features, and obtain an enhanced set of feature matching pairs;

[0009] S3. Based on the enhanced set of feature matching pairs, as well as the pre-stored camera parameter matrix and GPS raw data, calculate the initial pose to obtain a pose sequence; perform multi-view geometric optimization on the pose sequence, and output the optimized camera pose; based on the optimized camera pose, calculate depth estimation, and generate a set of multi-scale depth maps; perform consistency optimization on the set of multi-scale depth maps to obtain an optimized unified depth map; combine the optimized unified depth map and the optimized camera pose, and perform reconstruction quality evaluation, and output a quality distribution map;

[0010] S4. Based on the compensated image sequence and the quality distribution map, calculate the direction enhancement response to obtain a power line candidate region map; perform topological constraint segmentation on the power line candidate region map, and output a set of segmented line segments; based on the set of segmented line segments and the optimized camera pose, perform multi-view matching to generate a set of line segment matching pairs; based on the set of line segment matching pairs and the optimized unified depth map, perform 3D reconstruction to obtain a 3D power line model; analyze the temporal characteristics of the 3D power line model, and output a power line dynamic characteristic report;

[0011] S5. Based on the compensated image sequence, the optimized unified depth map, and the three-dimensional power line model, perform scene segmentation to obtain obstacle feature data; calculate the distance between the power line and the obstacle based on the obstacle feature data and the power line dynamic characteristic report, and output the obstacle distance data; combine the obstacle feature data, the obstacle distance data, and the three-dimensional power line model to perform obstacle risk assessment, generate an obstacle risk level table; generate obstacle warning information based on the obstacle risk level table and the power line dynamic characteristic report, and output an obstacle warning report; optimize the inspection strategy based on the obstacle warning report and the obstacle risk level table, and output an obstacle monitoring plan.

[0012] A conductor assembly obstacle detection system based on a monocular camera, comprising:

[0013] At least one processor; and,

[0014] A memory communicatively connected to at least one of the processors; wherein,

[0015] The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned conductor assembly obstacle detection method based on a monocular camera.

[0016] Beneficial effects: The present invention ensures the quality of data acquisition, realizes the accurate identification and feature expression of power lines; guarantees the accuracy and reliability of the reconstruction result, and realizes the comprehensive monitoring of the operating state of power lines and the risks of the surrounding environment; improves the automation level and accuracy of power line inspection, and realizes the early warning of potential risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the method of the present invention.

[0018] Figure 2 It is a flowchart of step S1 of the present invention.

[0019] Figure 3 It is a flowchart of step S2 of the present invention.

[0020] Figure 4 It is a flowchart of step S3 of the present invention.

[0021] Figure 5 It is a flowchart of step S4 of the present invention.

[0022] Figure 6 It is a flowchart of step S5 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0023] As Figure 1 shown, the present application proposes a conductor assembly obstacle detection method based on a monocular camera, comprising the following steps:

[0024] S1. Obtain initial flight parameters, and calculate optimized flight parameters through a scene complexity evaluation function; based on the optimized flight parameters, collect the original image sequence, calculate the image quality index, and obtain the image quality score; based on the image quality score, perform exposure compensation processing on the original image sequence to obtain the compensated image sequence; based on the compensated image sequence, calculate the spatial similarity and motion continuity between images, and generate an image association matrix; based on the image association matrix, screen to obtain the optimal image combination; where the initial flight parameters include the initial flight altitude, the initial flight speed, and the initial sampling interval;

[0025] S2. Based on the optimal image combination and the image quality score, construct a multi-resolution feature pyramid to obtain a multi-scale feature tensor set; calculate the orientation descriptor for the multi-scale feature tensor set to generate a set of feature descriptors; based on the set of feature descriptors and the image association matrix, calculate the feature matching score, and output the initial set of feature matching pairs; perform multi-level verification on the initial set of feature matching pairs to obtain a reliable matching set; analyze the feature distribution of the reliable matching set to generate supplementary features, and obtain an enhanced set of feature matching pairs;

[0026] S3. Based on the enhanced set of feature matching pairs, as well as the pre-stored camera parameter matrix and GPS raw data, calculate the initial pose to obtain a pose sequence; perform multi-view geometric optimization on the pose sequence, and output the optimized camera pose; based on the optimized camera pose, calculate the depth estimation, and generate a multi-scale depth map set; perform consistency optimization on the multi-scale depth map set to obtain an optimized unified depth map; combine the optimized unified depth map and the optimized camera pose to perform reconstruction quality evaluation, and output the quality distribution map;

[0027] S4. Based on the compensated image sequence and the quality distribution map, calculate the direction enhancement response to obtain a power line candidate region map; perform topological constraint segmentation on the power line candidate region map, and output a set of segmented line segments; based on the set of segmented line segments and the optimized camera pose, perform multi-view matching to generate a set of line segment matching pairs; based on the set of line segment matching pairs and the optimized unified depth map, perform 3D reconstruction to obtain a 3D power line model; analyze the temporal characteristics of the 3D power line model, and output a power line dynamic characteristic report;

[0028] S5. Based on the compensated image sequence, the optimized unified depth map, and the three-dimensional power line model, perform scene segmentation to obtain obstacle feature data; calculate the distance between the power line and the obstacle based on the obstacle feature data and the power line dynamic characteristic report, and output the obstacle distance data; combine the obstacle feature data, the obstacle distance data, and the three-dimensional power line model to perform obstacle risk assessment, generate an obstacle risk level table; generate obstacle warning information based on the obstacle risk level table and the power line dynamic characteristic report, and output an obstacle warning report; optimize the inspection strategy based on the obstacle warning report and the obstacle risk level table, and output an obstacle monitoring plan.

[0029] As Figure 2 shown, according to one aspect of the present application, step S1 is further as follows:

[0030] S11. Obtain the initial flight parameters, calculate the edge density value, local variance value, and depth change rate of the image, and substitute them into the scene complexity evaluation function for calculation to obtain the scene complexity score; based on the scene complexity score, perform optimized calculation on the initial flight parameters, and output the optimized flight parameters;

[0031] S12. Based on the optimized flight parameters, collect the original image sequence; obtain the pre-configured camera parameter matrix, perform wavelet transform calculation on each image in the original image sequence to obtain the clarity index; calculate the brightness distribution of each region in the original image sequence to obtain the illumination uniformity index; analyze the blur degree of the original image sequence to obtain the motion blur index; evaluate the scene coverage in the original image sequence to obtain the scene integrity index; combine the clarity index, illumination uniformity index, motion blur index, and scene integrity index to form an image quality scoring matrix;

[0032] S13. Based on the image quality scoring matrix, divide the original image sequence into a predetermined number of regions, calculate the local brightness mapping value of each region, and generate a region weight mask; multiply and superimpose the local brightness mapping value with the region weight mask to obtain the exposure compensation value; based on the exposure compensation value, perform brightness adjustment on the original image sequence, and output the compensated image sequence;

[0033] S14. Based on the compensated image sequence and the optimized flight parameters, calculate the spatial similarity value between adjacent images; analyze the motion continuity of the compensated image sequence to obtain the continuity score; combine the spatial similarity value and the continuity score to generate a spatio-temporal correlation matrix; based on the spatio-temporal correlation matrix, screen the optimal image combination.

[0034] In an embodiment of the present application, the calculation of the edge density value includes the following steps: calculate the image gradient ED = ∑∑sqrt[(ΨI / Ψx) 2 + (ΨI / Ψy) 2 / (M·N); where ED is the edge density value; I is the image grayscale value; M and N are the height and width of the image; ΨI / Ψx is the gradient in the x direction; ΨI / Ψy is the gradient in the y direction; (ΨI / Ψx) 2 is the square of the gradient in the x direction; (ΨI / Ψy) 2 is the square of the gradient in the y direction, and Ψ is the partial derivative.

[0035] The local variance calculation includes the following steps: Calculate the local region variance LV = ∑∑(I(x, y) - μ(x, y)) 2 / k 2 ; where LV is the local variance value; I(x, y) is the pixel value at position (x, y); μ(x, y) is the mean within a k×k window centered on (x, y); k is the window size; dividing by k 2 for normalization.

[0036] The depth change rate calculation includes the following steps: Calculate the depth gradient DR = ∑∑|D(x + 1, y) - D(x, y)| + |D(x, y + 1) - D(x, y)| / (M·N); where DR is the depth change rate; D(x, y) is the depth value at position (x, y); M and N are the dimensions of the depth map; |D(x + 1, y) - D(x, y)| is the depth difference in the horizontal direction; |D(x, y + 1) - D(x, y)| is the depth difference in the vertical direction.

[0037] The scene complexity score calculation includes the following steps: Calculate the comprehensive score SC = w1·ED + w2·LV + w3·DR + w4·∑(EDi·LVi·DRi) / n; where SC is the scene complexity score; ED is the edge density value; LV is the local variance value; DR is the depth change rate; EDi, LVi, DRi are the corresponding index values of the i-th sub-region; n is the number of sub-regions; w1, w2, w3, w4 are the weight coefficients, satisfying ∑wi = 1.

[0038] The sharpness index calculation includes the following steps: Calculate the wavelet transform coefficient SI = ∑|Wψ(s, p)| 2 / N; where SI is the sharpness index; Wψ(s, p) is the wavelet coefficient at scale s and position p; N is the total number of coefficients; |Wψ(s, p)| 2 is the energy of the wavelet coefficient; ψ is the wavelet basis function.

[0039] The illumination uniformity index calculation includes the following steps: Calculate the luminance distribution LI = 1 - ∑|B(x, y) - Bm| / (255·M·N); where LI is the illumination uniformity index; B(x, y) is the luminance value at position (x, y); Bm is the average image luminance; M and N are the image dimensions; 255 is the maximum luminance value.

[0040] The calculation of the motion blur index includes the following steps: Calculate the frequency domain energy ratio MI = ∑|F(u, v)| 2 _high / ∑|F(u, v)| 2 _total; where MI is the motion blur index; F(u, v) is the Fourier transform coefficient of the image; |F(u, v)| 2 _high is the energy of the high-frequency component; |F(u, v)| 2 _total is the total energy; u, v are the frequency domain coordinates.

[0041] The calculation of the scene integrity index includes the following steps: Calculate the effective coverage rate CI = ∑V(x, y) / (M·N)·(1 - O(x, y) / A); where CI is the scene integrity index; V(x, y) is the effective pixel marker at position (x, y); O(x, y) is the occlusion area marker; A is the total area; M, N are the image sizes.

[0042] The generation of the quality scoring matrix includes the following steps: Calculate the comprehensive quality score QM = α1·SI + α2·LI + α3·MI + α4·CI - β·∑|grad 2 I(x, y)| / N; where QM is the quality score; SI is the sharpness index; LI is the illumination uniformity index; MI is the motion blur index; CI is the scene integrity index; grad 2 I(x, y) is the response of the Laplacian operator; α1, α2, α3, α4, β are weight coefficients, satisfying ∑αi = 1.

[0043] The calculation of the local brightness mapping value includes the following steps: Calculate the brightness mapping function L(x, y) = k·log(1 + σ·I(x, y)) / (μ + σ·I(x, y)); where L(x, y) is the mapped brightness value; I(x, y) is the original brightness value; k is the gain coefficient; σ is the contrast parameter; μ is the local mean.

[0044] The generation of the regional weight mask includes the following steps: Calculate the weight value W(x, y) = exp(-((x - xc) 2 + (y - yc) 2 ) / (2σ 2 ))·(1 + γ·|grad I(x, y)|); where W(x, y) is the weight value at position (x, y); xc, yc are the regional center coordinates; σ is the Gaussian kernel parameter; |grad I(x, y)| is the gradient amplitude; γ is the edge enhancement coefficient.

[0045] Building a scene complexity evaluation model includes the following steps: Calculate the scene complexity score SC = w1·ED + w2·LV + w3·DR; where SC is the comprehensive scene complexity score; ED is the edge density value, ED = ∑|grad I(x, y)| / N, grad I(x, y) is the gradient value at the position (x, y) of the image, and N is the total number of image pixels; LV is the local variance value, LV = ∑(I(x, y) - μ) 2 / N, I(x, y) is the grayscale value at the position (x, y) of the image, and μ is the average grayscale value of the local area; DR is the depth change rate, DR = ∑|D(x + 1, y) - D(x, y)| / N, D(x, y) is the depth value at the position (x, y); w1, w2, and w3 are adaptive weight coefficients, dynamically adjusted by the neural network according to the scene characteristics, satisfying w1 + w2 + w3 = 1; The weight update adopts the gradient descent method, w_k(t + 1) = w_k(t) - η·ΨL / Ψw_k, where η is the learning rate, L is the loss function, w_k(t + 1) is the weight coefficient value at time t + 1, w_k(t) is the weight coefficient value at time t, and w_k is the general symbol of the weight coefficient, used to represent the weight value at any time.

[0046] In this embodiment, by building a multi-dimensional scene evaluation system and an adaptive flight parameter optimization mechanism, precise control of the inspection task is achieved. Specifically, first, by calculating three key indicators of the edge density value, local variance value, and depth change rate of the image, the scene complexity is comprehensively evaluated, avoiding the one-sidedness that may be brought by a single indicator; then, based on the scene complexity score, an adaptive optimization algorithm is used to dynamically adjust the flight parameters, so that the flight altitude, speed, and sampling interval can be adjusted in real time according to the scene characteristics. The quality of the collected images is evaluated and screened through the image quality scoring mechanism, ensuring the data quality for subsequent processing; finally, through exposure compensation processing and spatial similarity calculation, the continuity and quality consistency of the image sequence are ensured. This embodiment improves the adaptability and reliability of power line image acquisition, laying a solid foundation for subsequent feature extraction and matching. In practical applications, it can effectively cope with power line inspection scenarios under different weather conditions and different terrain features, improving the inspection efficiency and data quality.

[0047] According to one aspect of the present application, step S11 is further:

[0048] S111. Obtain the initial flight altitude, initial flight speed, and initial sampling interval. Calculate the edge intensity map by using the Sobel operator to compute the image gradient magnitude. Obtain the edge binary map by performing adaptive threshold processing on the edge intensity map. Calculate the edge density value by computing the proportion of non-zero pixels in the edge binary map. Perform block processing on the image. Calculate the gray variance for each image block to obtain the local variance matrix, and perform weighted averaging on the local variance matrix to obtain the local variance value. Calculate the depth map of adjacent image frames based on disparity estimation to obtain a depth sequence, calculate the spatial gradient of the depth sequence to obtain the depth gradient map, and statistically calculate the change rate of the depth gradient map to obtain the depth change rate.

[0049] S112. According to the edge density value, local variance value, and depth change rate, calculate the structural complexity weight to obtain the structural weight vector, calculate the texture complexity weight to obtain the texture weight vector, and calculate the depth complexity weight to obtain the depth weight vector. Substitute the structural weight vector, texture weight vector, and depth weight vector into a multi-layer perceptron for non-linear mapping to obtain the complexity mapping value. Perform normalization processing on the complexity mapping value and output the scene complexity score.

[0050] S113. Input the scene complexity score, initial flight altitude, initial flight speed, and initial sampling interval. Construct a parameter mapping matrix based on the scene complexity to obtain the parameter mapping matrix. Multiply the scene complexity score by the parameter mapping matrix to obtain the parameter adjustment vector. Perform weighted fusion of the initial flight altitude, initial flight speed, and initial sampling interval with the parameter adjustment vector to obtain the optimized flight altitude, optimized flight speed, and optimized sampling interval.

[0051] In this embodiment, by constructing an adaptive scene complexity evaluation system and a multi-level parameter optimization mechanism, precise regulation of flight parameters is achieved. Using multi-scale decomposition and the Sobel operator to calculate the edge density value can accurately capture the structural features in the scene. At the same time, through local variance calculation and depth estimation, the texture complexity and spatial variation characteristics of the scene are comprehensively evaluated. In the parameter optimization link, a non-linear mapping method based on a multi-layer perceptron is used to convert complex scene features into specific flight parameter adjustment values, considering the mutual relationship and influence between scene features. By constructing a parameter mapping matrix, precise conversion from scene complexity to specific flight parameters is realized, ensuring the rationality and reliability of the adjustment. This embodiment not only improves the setting accuracy of flight parameters but also enhances the system's adaptability to different scenes. In actual power line inspection, it can automatically adjust flight parameters according to the characteristics of different line segments, effectively avoiding the limitations of traditional fixed parameter schemes in complex scenes and improving the quality and efficiency of data collection.

[0052] According to one aspect of the present application, step S13 is further as follows:

[0053] S131. Based on the image quality scoring matrix, for each image in the original image sequence, use the adaptive grid partitioning algorithm to obtain an image region set; based on the image region set, calculate the brightness mean of each region to obtain a region brightness matrix; analyze the distribution characteristics of the region brightness matrix to obtain a brightness distribution map; apply Gaussian smoothing processing to the brightness distribution map to obtain a smoothed brightness map; calculate the difference between the smoothed brightness map and the preset standard brightness to obtain a brightness deviation map; based on the brightness deviation map and the image quality scoring matrix, generate a local mapping matrix.

[0054] S132. Based on the local mapping matrix and the image region set, calculate the edge intensity of each region to obtain an edge intensity map; based on the edge intensity map, extract the region texture features to obtain a texture feature map; fuse the edge intensity map and the texture feature map to obtain a region importance map; perform normalization processing on the region importance map to obtain a weight mask matrix; perform element-wise multiplication on the local mapping matrix and the weight mask matrix to obtain a weighted mapping matrix; perform spatial domain filtering on the weighted mapping matrix to obtain a smoothed mapping matrix.

[0055] S133. Based on the smoothed mapping matrix, perform brightness mapping on each image in the original image sequence to obtain a mapped image sequence; calculate the local contrast of the mapped image sequence to obtain a contrast map; based on the contrast map, analyze the degree of image detail preservation to obtain a detail preservation map; fuse the contrast map and the detail preservation map to obtain a quality assessment map; based on the quality assessment map, perform local fine-tuning on the mapped image sequence and output a compensated image sequence.

[0056] In this embodiment, by constructing an adaptive grid partitioning and multi-level exposure compensation mechanism, the problem of uneven exposure commonly existing in power line inspection images is effectively solved. The image is segmented into regions through the adaptive grid partitioning algorithm, and the segmentation granularity is automatically adjusted according to the complexity of the image content, avoiding the boundary effect that may occur in fixed grid partitioning; by calculating local brightness mapping and weight masks, differential processing of different regions is realized. Especially in the process of generating the weight mask, by fusing edge intensity and texture features, the details of key target regions such as power lines are ensured to be preserved, while avoiding image distortion caused by overcompensation. In the final exposure compensation stage, through local contrast and detail preservation evaluation, the adaptive adjustment of the compensation intensity is realized, ensuring the overall visual quality of the image. This embodiment not only effectively balances the light and dark contrast of the image, but also preserves the detail features of the power line and its surrounding environment, improving the accuracy of subsequent feature extraction and matching. In practical applications, it can effectively process images under complex lighting conditions such as backlight and strong light, providing high-quality image data support for power line detection.

[0057] As Figure 3 shown, according to one aspect of the present application, step S2 is further as follows:

[0058] S21. Calculate the scale parameter for each level based on the optimal image combination and image quality score; perform tensor transformation and weight kernel function calculation on the images in the optimal image combination based on the scale parameter for each level to generate a single-layer feature tensor; combine the single-layer feature tensors of all levels to generate a multi-scale feature tensor set.

[0059] S22. Calculate the anisotropy intensity value of each feature point based on the multi-scale feature tensor set to obtain an anisotropy intensity vector; generate a histogram of oriented gradients based on the anisotropy intensity vector to obtain oriented gradient information; calculate the context correlation based on the oriented gradient information to obtain context encoding data; combine the anisotropy intensity vector, oriented gradient information, and context encoding data to generate a set of feature descriptors.

[0060] S23. Calculate the neighborhood constraint of feature points based on the set of feature descriptors and the image association matrix to obtain neighborhood constraint data; calculate the matching probability between feature points based on the neighborhood constraint data; generate an initial set of feature matching pairs based on the matching probability and a preset threshold.

[0061] S24. Calculate the local structure consistency score based on the initial set of feature matching pairs and the image association matrix, evaluate the global topological consistency, analyze the temporal continuity, and generate a multi-level verification score; screen the initial set of feature matching pairs based on the multi-level verification score and output a reliable matching set.

[0062] S25. Analyze the feature space distribution based on the reliable matching set and the set of feature descriptors, identify the feature sparse regions, generate supplementary feature points in the feature sparse regions; calculate descriptors for the supplementary feature points and combine the descriptors with the reliable matching set to output an enhanced set of feature matches.

[0063] In an embodiment of the present application, the calculation of the anisotropy intensity value includes the following steps: calculate the eigenvalue ratio of the feature point covariance matrix AI = (λ1 - λ2) / (λ1 + λ2); where AI is the anisotropy intensity value; λ1, λ2 are the eigenvalues of the covariance matrix, and λ1 ≥ λ2; the covariance matrix C = [∑Ix 2 , ∑IxIy; ∑IxIy, ∑Iy 2 ; Ix, Iy are the image gradients.

[0064] The generation of the histogram of oriented gradients includes the following steps: calculate the histogram of oriented gradients HOG = ∑w(x, y)·δ(θ(x, y) - θbin); where HOG is the histogram of oriented gradients; w(x, y) is the gradient magnitude at the position (x, y); θ(x, y) is the gradient direction; θbin is the central angle of the direction bin; δ is the Dirac function.

[0065] Context relevance calculation includes the following steps: Calculate the attention weight matrix A(i, j) = softmax(Q·K T / sqrt(d))·V; where A(i, j) is the relevance between feature points i and j; Q is the query matrix; K is the key matrix; V is the value matrix; d is the feature dimension; softmax is the normalization function.

[0066] Feature point neighborhood constraint calculation includes the following steps: Calculate the neighborhood consistency score NC = ∑exp(-||di, j - dk, l|| 2 / (2σ 2 ))·δ(||pi - pk|| < r); where NC is the neighborhood consistency score; di, j is the descriptor distance between feature points i and j; dk, l is the descriptor distance between corresponding matching points k and l; pi, pk are the spatial positions of the feature points; r is the neighborhood radius; σ is the Gaussian kernel parameter; δ is the indicator function.

[0067] The matching probability calculation model includes the following steps: Calculate the matching probability P(mi, j) = exp(-λ1·d(fi, fj) 2 -λ2·||xi - xj|| 2 ) / (Z·(1 + exp(-γ·NC))); where P(mi, j) is the matching probability of the feature point pair (i, j); d(fi, fj) is the descriptor distance; xi, xj are the feature point positions; NC is the neighborhood consistency score; λ1, λ2, γ are the weight coefficients; Z is the normalization factor.

[0068] Multi-level verification score calculation includes the following steps: Calculate the verification score VS = w1·LS + w2·GS + w3·TS; where VS is the multi-level verification score; LS is the local structure consistency score, LS = ∑||Ri, j - Rk, l||F / n; GS is the global topological consistency score, GS = exp(-||Fi - Fj|| 2 / σ 2 )); TS is the temporal continuity score, TS = 1 - |vi, t - vi, t-1| / vmax; Ri, j is the local relative transformation matrix; Fi, Fj are the global feature descriptors; vi, t is the motion speed at time t; w1, w2, w3 are the weight coefficients, vmax is the maximum motion speed.

[0069] Feature space distribution analysis includes the following steps: Calculate the feature distribution density D(x, y) = ∑K(||pi - (x, y)|| / h) / (n·h 2)); where D(x, y) is the density value at position (x, y); K is the kernel function, K(u) = (1 - u 2 )·I(|u|≤ 1); pi is the position of the feature point; h is the bandwidth parameter; n is the total number of feature points; I is the indicator function.

[0070] In another embodiment of the present application, the timing constraint TC = ∑||M(xt)-x(t + 1)|| 2 ·exp(-||ft - f(t + 1)|| 2 / σ 2 ); where TC is the timing constraint value; M is the motion prediction function; xt is the state at time t; ft is the feature at time t; σ is the feature similarity parameter.

[0071] In this embodiment, by constructing a multi - level feature extraction and matching optimization framework, the accurate recognition and reliable matching of power line features are realized. First, multi - scale features are extracted through a multi - resolution feature pyramid, which can capture the feature performance of power lines at different scales; in the feature description link, a multi - head attention network and a context - aware mechanism are adopted to effectively extract key features such as the directionality and continuity of power lines. Especially in the feature matching process, by constructing a multi - level verification mechanism and a spatial distribution analysis framework, high - precision screening of feature matching is realized, and at the same time, feature points in sparse areas are supplemented through a feature enhancement network, ensuring the uniformity and integrity of feature distribution. This embodiment not only improves the accuracy of power line feature extraction but also enhances the robustness of feature matching. In actual inspection tasks, it can effectively handle the difficulties in feature extraction caused by factors such as light changes and partial occlusion, providing high - quality feature matching data support for subsequent 3D reconstruction.

[0072] According to one aspect of the present application, step S22 is further as follows:

[0073] S221. Based on the multi - scale feature tensor set, construct a covariance matrix for each scale layer to obtain a covariance tensor set; calculate the eigenvalues of the covariance tensor set to obtain an eigenvalue sequence; based on the eigenvalue sequence, calculate the anisotropy metric to obtain an anisotropy intensity vector; based on the anisotropy intensity vector, use the self - attention mechanism to calculate the correlation degree between feature points to obtain an attention weight map; fuse the anisotropy intensity vector and the attention weight map to obtain an enhanced feature vector;

[0074] S222. Based on the enhanced feature vector, construct a local coordinate system to obtain a local coordinate map; calculate the direction gradient in the local coordinate map to obtain a gradient magnitude map and a gradient direction map; perform direction quantization on the gradient direction map to obtain a direction histogram; calculate the statistical features of the gradient magnitude map to obtain a magnitude feature vector; combine the direction histogram and the magnitude feature vector to obtain direction gradient information;

[0075] S223. Based on the directional gradient information and the enhanced feature vector, construct a multi-head attention network to obtain an attention feature map; based on the attention feature map, calculate the semantic correlation of the neighborhood of the feature points to obtain a semantic association map; fuse the attention feature map and the semantic association map to obtain context-encoded data; perform feature concatenation on the anisotropic intensity vector, the directional gradient information, and the context-encoded data, and perform dimensionality reduction through a multi-layer perceptron to output a set of feature descriptors.

[0076] In this embodiment, by designing a multi-head attention feature extraction network and a context-aware feature description mechanism, accurate description and robust expression of power line features are achieved. First, by calculating the anisotropic intensity of the feature points, the directional features of the power lines are accurately captured; during the feature extraction process, the self-attention mechanism is used to calculate the correlation between feature points, which can effectively suppress background interference and highlight the main features of the power lines. In the process of extracting directional gradient information, by constructing a local adaptive coordinate system, the rotation invariance of feature description is improved, enabling the system to accurately process power line images at different angles. Especially in the context encoding link, by using a multi-head attention network and semantic correlation analysis, multi-scale expression of features and effective fusion of context information are achieved, enhancing the discriminability and robustness of feature description. This embodiment not only improves the extraction accuracy of power line features but also enhances the system's resistance to interference factors such as occlusion and illumination changes. In actual power line detection tasks, it can accurately extract and describe the key features of power lines, providing reliable feature data support for subsequent matching and 3D reconstruction.

[0077] According to one aspect of the present application, step S25 is further as follows:

[0078] S251. Obtain a reliable matching set and a set of feature descriptors, calculate the spatial distribution density of the feature points to obtain a density distribution map, use an adaptive threshold to segment the density distribution map to obtain a region segmentation map, and extract the low-density regions in the region segmentation map to obtain a sparse region marking map; perform morphological processing on the sparse region marking map to obtain an optimized region map, and calculate the geometric features of each region in the optimized region map to obtain a region feature vector.

[0079] S252. Based on the region feature vector and the sparse region marking map, construct a multi-layer attention feature generation network to obtain an attention feature map, generate candidate feature points under the guidance of the attention feature map to obtain a set of candidate points; calculate the local response of each point in the set of candidate points to obtain a response intensity map, and screen the feature points according to the response intensity map to obtain a set of supplementary feature points; extract the local image patches of the set of supplementary feature points to obtain a set of feature image patches.

[0080] S253. Apply a multi-scale feature extraction network to the set of feature image patches to obtain multi-scale feature maps, calculate local descriptors of the multi-scale feature maps to obtain a local descriptor subset; process the local descriptor subset using a context attention module to obtain a context feature map, fuse the local descriptor subset and the context feature map to obtain a supplementary descriptor subset; perform feature fusion on the supplementary descriptor subset and the reliable matching set, and output an enhanced feature matching set.

[0081] In one embodiment of the present application, the generation of multi-layer attention features includes the following steps: Calculate the attention feature A = ∑Heads(Q, K, V)·W; where A is the attention feature; Heads is the multi-head attention function, Heads = Concat(head1,..., headh); headi = Attention(QWi Q , KWi K , VWi V ); Attention(Q, K, V) = softmax(QK T / sqrt(dk))V; Wi is a learnable parameter matrix, Q is the query matrix, K is the key matrix, V is the value matrix, W is a learnable parameter matrix, Concat is the operation of concatenating the outputs of multiple heads, Attention is the core calculation formula of the attention mechanism, and dk is the dimension of the key matrix K.

[0082] The process of performing feature fusion is specifically as follows: Calculate the fused feature F = ∑αk·Φk(fk)·Wk(x, y); where F is the fused feature; Φk is the k-th layer feature transformation function; fk is the k-th layer feature map; Wk(x, y) is the spatial attention weight, Wk = softmax(θ(fk)); αk is the scale weight, which is obtained through network learning.

[0083] In this embodiment, by constructing a feature space distribution analysis and multi-layer attention feature generation network, the problem of uneven distribution of power line features is effectively solved. First, through calculating the spatial distribution density of feature points and adaptive threshold segmentation, the feature sparse regions are accurately identified; during the feature supplementation process, a multi-layer attention network is used to generate candidate feature points, which can adaptively generate supplementary points that conform to the power line feature distribution law according to the context information. Especially in the feature verification stage, through the multi-scale feature extraction network and the context attention module, the quality and reliability of the supplementary features are ensured. The system realizes the effective screening of the supplementary features by analyzing the local response intensity and the matching of feature descriptors, and avoids introducing noise feature points. This embodiment not only fills the blank regions of the feature distribution, but also maintains the continuity and consistency of the features, improving the integrity and accuracy of subsequent feature matching.

[0084] Such asFigure 4 As shown, according to one aspect of the present application, step S3 is further as follows:

[0085] S31. Based on the enhanced feature matching set and the pre-stored camera parameter matrix, calculate the projection error value, generate a temporal smoothing constraint, including a velocity constraint term and an acceleration constraint term; based on the pre-stored GPS raw data, construct a GPS constraint term; based on the temporal smoothing constraint and the GPS constraint term, calculate the camera position and attitude, perform motion prediction, and output an initial pose sequence;

[0086] Among them, calculating the projection error value includes calculating the projected coordinates of feature points based on the reprojection model, calculating the Euclidean distance from the actual coordinates, and generating a projection error matrix; generating the temporal smoothing constraint includes analyzing the motion continuity between adjacent frames, constructing a velocity constraint term, and generating an acceleration constraint term; constructing the GPS constraint term includes analyzing the GPS signal quality, generating a position weight matrix, and constructing a GPS error term;

[0087] S32. Based on the initial pose sequence and the enhanced feature matching set, calculate the reprojection error to obtain reprojection error data; based on the reprojection error data, evaluate the structural consistency to obtain structural constraint data; based on the structural constraint data, analyze the scene continuity to obtain continuity constraint data; based on the reprojection error data, structural constraint data, and continuity constraint data, perform optimization calculations and output the optimized camera pose;

[0088] S33. Based on the optimized camera pose and the compensated image sequence, divide the local image blocks, calculate the local depth probability, and generate a depth estimation probability distribution; based on the depth estimation probability distribution, perform depth estimation calculations for each scale level, combine the multi-scale results, and output a multi-scale depth map set;

[0089] S34. Based on the multi-scale depth map set and the optimized camera pose, calculate the depth data item to obtain depth consistency data; based on the depth consistency data, analyze the depth gradient to obtain gradient constraint data; based on the gradient constraint data, evaluate the surface smoothness to obtain smoothness constraint data; combine the depth consistency data, gradient constraint data, and smoothness constraint data for optimization and output the optimized unified depth map;

[0090] S35. Based on the optimized unified depth map and the optimized camera pose, calculate the reconstruction integrity score, evaluate the geometric consistency and scale accuracy, and generate a reconstruction quality score; based on the reconstruction quality score, calculate the spatial distribution to obtain a quality distribution map.

[0091] In an embodiment of the present application, the reprojection error calculation includes the following steps: calculating the reprojection error RE = ∑||π(K[R|t]X) - x|| 2·w(x); where RE is the reprojection error; π is the projection function; K is the camera intrinsic matrix; [R|t] is the rotation and translation matrix; X is the 3D point coordinate; x is the corresponding image coordinate; w(x) is the feature point reliability weight.

[0092] The structural consistency evaluation includes the following steps: Calculate the structural similarity SS = ∑||Si - Sj|| 2 ·exp(-di,j / σ) / (∑exp(-di,j / σ)); where SS is the structural similarity; Si, Sj are local structure descriptors; di,j is the distance between feature points; σ is the distance attenuation coefficient.

[0093] Calculate the depth probability P(d|I) = softmax(Φ(fθ(I), d))·∏P(d|Ii); where P(d|I) is the probability distribution of depth d given the image I; Φ is the depth network feature extraction function; fθ is a neural network with parameter θ; P(d|Ii) is the depth probability predicted for each view; ∏ represents the multiplication of probabilities.

[0094] Calculate the consistency energy function E(D) = ∑(D - Di) 2 ·wi + λ·∑|grad D|·exp(-β·|grad I|); where E(D) is the energy function of the depth map D; Di is the initial depth estimate; wi is the depth confidence weight; gradD is the depth gradient; grad I is the image gradient; λ, β are smoothing parameters.

[0095] The process of performing the optimization calculation is specifically as follows: Construct the total error energy function E = λ1·Er + λ2·Es + λ3·Ec; where Er is the reprojection error, Er = ∑||π(KTP) - p|| 2 , π is the projection function, K is the camera intrinsic matrix, T is the pose transformation matrix, P is the 3D point coordinate, p is the corresponding 2D image coordinate; Es is the structural consistency error, Es = ∑||S(Pi) - S(Pj)|| 2 , S(P) is the local structure descriptor of point P; Ec is the continuity constraint error, Ec = ∑||T(t) - T(t - 1)|| 2 , T(t) is the pose matrix at time t; λ1, λ2, λ3 are weight coefficients, which are dynamically adjusted according to the scene features through an adaptive algorithm; The optimization solution uses the LM algorithm, and the pose parameters are iteratively updated until convergence.

[0096] The smoothness constraint function includes the following steps: Calculate the smoothness constraint SC = ∑∑wij·ρ(di - dj); where SC is the smoothness constraint value; wij is the weight between pixels i and j, wij = exp(-α·||Ii - Ij|| 2 ); di and dj are the depth values of pixels i and j; ρ is the Huber loss function; Ii and Ij are the color values of pixels i and j; α is the color similarity parameter.

[0097] In this embodiment, by designing a multi-constraint joint optimization framework and a depth estimation network, high-precision reconstruction of the three-dimensional structure of the power line is achieved. First, through the fusion optimization of GPS data and visual features, an accurate initial pose estimation is established; in the multi-view geometric optimization process, triple constraints of reprojection error, structural consistency, and scene continuity are adopted to improve the accuracy of camera pose estimation. Especially in the depth estimation link, through multi-scale depth map generation and consistency optimization, the problems of depth discontinuity and occlusion in the power line scene are effectively solved. By establishing a complete reconstruction quality evaluation system, the reliability verification of the reconstruction result is realized. This embodiment not only improves the accuracy and integrity of the reconstruction, but also enhances the adaptability of the system to complex scenes. In practical applications, it can accurately reconstruct the three-dimensional structure of the power line in various complex environments, laying a solid foundation for subsequent condition monitoring and risk assessment.

[0098] According to one aspect of the present application, step S32 is further as follows:

[0099] S321. Based on the initial pose sequence and the enhanced feature matching set, construct a projection model to obtain a set of projection matrices; based on the set of projection matrices, calculate the reprojection coordinates of the feature points to obtain a set of projection coordinates; calculate the Euclidean distance between the set of projection coordinates and the actual coordinates to obtain a distance error map; analyze the statistical characteristics of the distance error map to obtain reprojection error data; based on the reprojection error data, construct an error weight matrix to obtain an error weight map;

[0100] S322. Based on the error weight map and the enhanced feature matching set, construct a scene structure map to obtain a set of structure maps; analyze the local rigidity of the set of structure maps to obtain a rigidity constraint map; based on the rigidity constraint map, calculate the geometric consistency between the feature points to obtain a geometric constraint map; fuse the rigidity constraint map and the geometric constraint map to obtain structure constraint data; based on the structure constraint data, calculate the constraint weight to obtain a structure weight matrix;

[0101] S323. Analyze the temporal variation of the initial pose sequence to obtain a motion trajectory map; calculate the motion continuity between adjacent frames based on the motion trajectory map and the structure weight matrix to obtain a continuity score map; evaluate the smoothness of the scene change based on the continuity score map to obtain a smoothness map; fuse the continuity score map and the smoothness map to obtain continuity constraint data; substitute the reprojection error data, the structure constraint data, and the continuity constraint data into an optimization solver to output the optimized camera pose and 3D point cloud data.

[0102] In this embodiment, a multi-constraint joint optimization mechanism and a scene structure consistency analysis framework are constructed to achieve high-precision optimization of the camera pose and 3D point cloud. First, a reprojection error evaluation system is established by calculating through a projection model and Euclidean distance. By analyzing the deviation between the projected coordinates and the actual coordinates, the accuracy of pose estimation is accurately quantified. During the construction of the structure constraint, local rigidity analysis and geometric consistency evaluation are adopted, which can effectively maintain the structural integrity of the scene and avoid structural deformation caused by pose optimization. Especially in terms of continuity constraint, by analyzing the temporal characteristics of the camera motion trajectory and the smoothness of the scene change, a complete motion continuity evaluation mechanism is established to ensure the temporal consistency of the pose estimation result. This embodiment not only improves the accuracy of pose estimation but also ensures the quality of 3D point cloud reconstruction, showing excellent robustness when dealing with complex power line scenes. In actual inspection tasks, it can accurately reconstruct the 3D structure of the power line, providing reliable spatial information support for subsequent defect detection and risk assessment.

[0103] According to one aspect of the present application, step S34 is further as follows:

[0104] S341. Input the multi-scale depth map set and the optimized camera pose, calculate the depth consistency between adjacent views to obtain a consistency map, analyze the complementarity of the multi-scale depth data to obtain a scale complementarity map, and fuse the consistency map and the scale complementarity map to obtain a consistency weight matrix; perform weighted fusion on the depth data based on the consistency weight matrix to obtain depth consistency data. Figure 1 Fuse the consistency map and the scale complementarity map to obtain a consistency weight matrix; perform weighted fusion on the depth data based on the consistency weight matrix to obtain depth consistency data. Figure 1 S342. Calculate the spatial gradient of the multi-scale depth map set to obtain a gradient magnitude map, analyze the gradient direction distribution to obtain a gradient direction map, calculate the local continuity of the gradient to obtain a continuity score map; construct a gradient constraint model by combining the gradient magnitude map, the gradient direction map, and the continuity score map, and output gradient constraint data; generate an edge-preserving weight based on the gradient constraint data to obtain an edge weight map.

[0105]

[0106] S343. Calculate the local surface features based on the depth consistency data to obtain a surface feature map, analyze the surface normal distribution to obtain a normal distribution map, and evaluate the local smoothness to obtain a local smoothness map; fuse the surface feature map, the normal distribution map, and the local smoothness map to obtain smoothness constraint data; input the depth consistency data, the gradient constraint data, and the smoothness constraint data into an optimization solver to generate an optimized unified depth map.

[0107] In this embodiment, a multi-scale depth consistency optimization framework and an edge-preserving smoothness constraint mechanism are constructed to achieve high-quality unified depth map reconstruction. First, by analyzing the depth consistency between adjacent views and the complementarity of multi-scale data, a complete depth fusion evaluation system is established; during the construction of the gradient constraint, spatial gradient analysis and local continuity evaluation are adopted, which can effectively preserve the edge features of the depth map and avoid over-smoothing during the depth optimization process. Especially in terms of surface smoothness constraint, by analyzing the local surface features and normal distribution, an adaptive smoothness constraint model is constructed to ensure that the depth map has good continuity while maintaining details. This embodiment not only improves the accuracy of depth estimation but also enhances the robustness of the system to occlusion and noise. In the actual three-dimensional reconstruction task of power lines, it can accurately reconstruct the spatial position and morphological features of power lines, providing accurate depth information support for subsequent line monitoring and risk assessment.

[0108] As Figure 5 shown, according to one aspect of the present application, step S4 is further as follows:

[0109] S41. Based on the compensated image sequence and the quality distribution map, construct a multi-scale directional filter bank, and calculate the point cloud projection consistency score; based on the point cloud projection consistency score, use an anisotropic Gaussian filter to calculate the local directionality score and generate a direction response tensor; based on the direction response tensor, perform edge enhancement processing to obtain an enhanced response map; perform directionality analysis on the enhanced response map, and output a power line candidate region map and a direction distribution map;

[0110] S42. Based on the power line candidate region map and the direction distribution map, calculate the local direction consistency, evaluate the line segment extensibility, and generate a linear feature weight; based on the linear feature weight, calculate the curvature change rate, evaluate the segment smoothness, and generate a continuity score; based on the continuity score, analyze the line segment intersection state, calculate the parallelism index, and generate a topological relationship matrix; combine the linear feature weight, the continuity score, and the topological relationship matrix to perform segmentation optimization, and output a segmented line segment set;

[0111] S43. Calculate the direction similarity based on the segmented line segment set and the optimized camera pose, evaluate the consistency of the length ratio, and generate a geometric similarity score; calculate the relative position relationship based on the geometric similarity score, evaluate the spatial distribution pattern, and generate a topological constraint matrix; analyze the peripheral feature distribution based on the topological constraint matrix, calculate the region descriptor, and generate a context matching score; screen the matching pairs based on the geometric similarity score, the topological constraint matrix, and the context matching score, and output the set of line segment matching pairs.

[0112] S44. Evaluate the shape features based on the set of line segment matching pairs, the direction distribution map, and the optimized unified depth map, and generate the basic model parameters; calculate the local deformation features based on the basic model parameters, evaluate the elastic deformation, and generate the deformation function; analyze the measurement error based on the deformation function, calculate the uncertainty, and generate the noise model; perform optimized reconstruction based on the basic model parameters, the deformation function, and the noise model, and output the three-dimensional power line model.

[0113] S45. Calculate the sag change rate based on the three-dimensional power line model and the optimized camera pose to obtain the sag data; measure the lateral offset based on the sag data to obtain the offset data; analyze the vibration frequency based on the offset data to obtain the vibration data; estimate the tension value based on the vibration data to obtain the tension data; perform time series filtering and weighting processing on the sag data, the offset data, the vibration data, and the tension data, and output the power line dynamic characteristic report.

[0114] In an embodiment of the present application, the direction enhancement response calculation includes the following steps: calculate the direction response intensity DR = ∑G(θk)·F(I, θk) / (1 + λ·|▽ 2 I|); where DR is the direction response intensity; G(θk) is the Gaussian weight of the direction θk; F(I, θk) is the filtering response of the direction θk; ▽ 2 I is the Laplacian operator response; λ is the smoothing coefficient; I is the input image.

[0115] The topological constraint segmentation model includes the following steps: calculate the segmentation energy E(S) = Ed(S) + αEc(S) + βEt(S); where E(S) is the total energy of the segmentation result S; Ed is the data term, Ed = ∑|I(x) - μk| 2 ; Ec is the continuity term, Ec = ∑|sk – s(k + 1)|; Et is the topological term, Et = ∑exp(-||di - dj|| 2 / σ 2 )·δ(li ≠ lj); μk is the mean of the region k; sk is the line segment k; di, dj are the feature descriptors; li, lj are the labels; α, β are the weight coefficients.

[0116] The calculation of the geometric similarity of line segments includes the following steps: Calculate the geometric similarity GS = exp(-w1·θ 2 / σ1 2 -w2·(l1 / l2 - 1) 2 / σ2 2 - w3·d 2 / σ3 2 ); where GS is the geometric similarity; θ is the line segment direction difference; l1 and l2 are the line segment lengths; d is the distance between the midpoints of the line segments; w1, w2, and w3 are the weight coefficients; σ1, σ2, and σ3 are the scale parameters.

[0117] The calculation of the sag change rate includes the following steps: Calculate the sag change rate CS = (h(t) - h(t-1)) / h(t-1); where CS is the sag change rate; h(t) is the sag value at time t, h(t) = h0 +αL 2 / (8T(t)); h0 is the initial sag; α is the temperature coefficient; L is the span; T(t) is the tension function.

[0118] The vibration frequency analysis model includes the following steps: Calculate the vibration characteristic VF = FFT(y(t))·W(ω); where VF is the vibration frequency characteristic; y(t) is the displacement time series; FFT is the fast Fourier transform; W(ω) is the frequency weight function, W(ω) = exp(-|ω-ω0| 2 / σ 2 ); ω0 is the fundamental frequency; σ is the frequency band width parameter.

[0119] The tension estimation function includes the following steps: Calculate the tension value T = mgL 2 / (8h)·(1 + kv 2 / g 2 ); where T is the tension estimated value; m is the mass per unit length; g is the acceleration due to gravity; L is the span; h is the sag value; v is the transverse vibration velocity; k is the correction coefficient, k = exp(-βT / T0); β is the material property parameter; T0 is the reference tension.

[0120] In this embodiment, by constructing a direction-enhanced power line detection network and a dynamic characteristic analysis framework, the accurate identification and status evaluation of power lines are achieved. First, through direction-enhanced response and topological constraint segmentation, the spatial structure features of power lines are accurately extracted; in the multi-view matching process, a reliable line segment matching mechanism is established by using geometric similarity and topological relationship analysis. Especially in the 3D reconstruction and dynamic characteristic analysis links, through the multi-head attention network and spatio-temporal feature analysis, the accurate description of the shape and motion state of power lines is realized. This embodiment not only improves the accuracy of power line detection, but also can effectively identify abnormal states of the line. In actual inspection tasks, it can accurately monitor the sag changes, vibration characteristics and tension states of power lines, providing comprehensive technical support for line safety operation and maintenance.

[0121] In another embodiment of the present application, a power line dynamic characteristic description DM = {C(t), O(t), V(t), T(t)} is constructed; where C(t) is the sag function, C(t) = h0 + αL 2 / (8T(t)), h0 is the initial sag height, α is the temperature coefficient, L is the span, and T(t) is the tension; O(t) is the offset function, O(t) = A·sin(ωt + φ), A is the amplitude, ω is the angular frequency, and φ is the phase; V(t) is the vibration characteristic, V(t) = FFT(O(t)), and the spectral characteristics are obtained through fast Fourier transform; T(t) is the tension function, T(t) = mg·L 2 / (8h), m is the mass per unit length, g is the acceleration due to gravity, and h is the real-time sag value; each parameter is obtained by least squares fitting.

[0122] According to one aspect of the present application, step S44 is further as follows:

[0123] S441. Input the set of line segment matching pairs and the direction distribution map, construct a multi-head attention network to process the line segment features to obtain an attention feature map, calculate the geometric attributes of the line segments to obtain a geometric feature map, and analyze the topological relationship between the line segments to obtain a topological relationship map; fuse the attention feature map, geometric feature map and topological relationship map, extract the basic shape parameters of the power line, and output the basic shape data.

[0124] S442. Based on the basic shape data and the optimized unified depth map, calculate the elastic deformation characteristics of the line segments to obtain an elastic deformation map, analyze the influence of environmental factors to obtain an environmental impact map, and evaluate the load distribution to obtain a load distribution map; establish a deformation model by combining the elastic deformation map, environmental impact map and load distribution map, and output the deformation function data.

[0125] S443. Analyze the uncertainty of the measurement data to obtain an uncertainty map, evaluate the system error distribution to obtain an error distribution map, and calculate the random noise characteristics to obtain a noise characteristic map; fuse the uncertainty map, the error distribution map, and the noise characteristic map to obtain the noise parameters; substitute the basic shape data, the deformation function data, and the noise parameters into the optimizer to output the three-dimensional power line model.

[0126] In this embodiment, by constructing a multi-head attention feature fusion network and a deformation parameter adaptive estimation framework, the accurate reconstruction of the three-dimensional power line model is realized. First, the multi-head attention network analyzes the line segment features and topological relationships, and can accurately capture the spatial structure features of the power line; in the process of estimating the deformation parameters, elastic deformation analysis and environmental factor influence evaluation are adopted to establish a complete deformation model, effectively considering the influence of external factors such as temperature change and wind load on the shape of the power line. Especially in terms of noise processing, by analyzing the uncertainty of the measurement data and the system error distribution, a reliable noise model is constructed to ensure the accuracy and reliability of the reconstruction result. This embodiment not only realizes the accurate reconstruction of the geometric shape of the power line, but also can reflect its dynamic change characteristics under different environmental conditions. In practical applications, it can accurately reconstruct power lines of different types and states, providing a reliable three-dimensional model support for line operation state monitoring and fault diagnosis.

[0127] According to one aspect of the present application, step S45 is further as follows:

[0128] S451. Obtain the three-dimensional power line model, construct a multi-scale tensor network to extract shape features to obtain a shape feature map, calculate the sag values of each segment to obtain a sag sequence, and analyze the sag change trend to obtain a change trend map; establish a deformation mapping relationship based on the shape feature map and the sag sequence to obtain a deformation mapping matrix; combine the change trend map and the deformation mapping matrix to calculate the dynamic characteristics and output the sag data.

[0129] S452. According to the three-dimensional power line model and the optimized camera pose, construct a spatio-temporal attention network to analyze the motion features to obtain a motion feature map, calculate the horizontal displacement to obtain a displacement sequence, and evaluate the vertical deviation to obtain a deviation sequence; fuse the motion feature map, the displacement sequence, and the deviation sequence to construct a motion model to obtain a motion state map; analyze the temporal characteristics of the motion state map and output the offset data and vibration data.

[0130] S453. Input the sag data, the offset data, and the vibration data, construct a multi-constraint state estimator to obtain a state estimation map, calculate the material stress distribution to obtain a stress distribution map, and evaluate the external load to obtain a load distribution map; fuse the state estimation map, the stress distribution map, and the load distribution map for mechanical analysis to obtain the tension data; perform temporal filtering on the sag data, the offset data, the vibration data, and the tension data and output the power line dynamic characteristic report.

[0131] In this embodiment, by constructing a multi-scale tensor feature extraction network and a spatio-temporal attention motion analysis framework, an accurate evaluation of the dynamic characteristics of the power line is achieved. First, the shape features and sag change features are extracted through the multi-scale tensor network, which can comprehensively capture the morphological changes of the power line under different load conditions. In the process of motion characteristic analysis, the spatio-temporal attention network is used to analyze the displacement and vibration features, and a complete motion state evaluation system is established to effectively identify the abnormal swing and vibration of the power line. Especially in the tension estimation link, through the multi-constraint state estimator and the mechanical analysis model, an accurate evaluation of the stress state of the power line is achieved, comprehensively considering the influence of multiple factors such as material properties, environmental loads, and installation states, and improving the accuracy of tension estimation. This embodiment can not only monitor the operating state of the power line in real time but also predict potential safety hazards. In the actual line inspection task, it can timely detect safety risks caused by abnormal tension, excessive vibration, etc., providing reliable data support for line maintenance decision-making.

[0132] As Figure 6 shown, according to one aspect of the present application, step S5 is further as follows:

[0133] S51. Based on the compensated image sequence, the optimized unified depth map, and the 3D power line model, evaluate the height stratification to generate spatial obstacle features; based on the spatial obstacle features, calculate the motion features, evaluate the change pattern, and generate dynamic obstacle features; based on the dynamic obstacle features, analyze the geometric shape, calculate the texture features, and generate obstacle morphology features; combine the spatial obstacle features, dynamic obstacle features, and obstacle morphology features for obstacle recognition and segmentation, and output obstacle feature data;

[0134] S52. Based on the obstacle feature data and the power line dynamic characteristic report, calculate the shortest distance between the obstacle and the power line to obtain the vertical distance data; based on the vertical distance data, analyze the motion trend of the obstacle to obtain the dynamic approach data; based on the dynamic approach data, evaluate the environmental impact to obtain the environmental impact data; based on the vertical distance data, dynamic approach data, and environmental impact data, construct an obstacle distance evaluation model and output obstacle distance data;

[0135] S53. Based on the obstacle feature data, obstacle distance data, and 3D power line model, calculate the static distance risk to obtain the static threat level; based on the static threat level, evaluate the dynamic collision risk to obtain the dynamic threat level; based on the dynamic threat level, analyze the growth trend of the obstacle to obtain the development threat level; based on the development threat level, measure the environmental impact to obtain the environmental threat level; perform weighted combination on the static threat level, dynamic threat level, development threat level, and environmental threat level, and output the obstacle risk rating table;

[0136] S54. Calculate the obstacle type weights based on the obstacle risk level table, obstacle feature data, and power line dynamic characteristic report to obtain an obstacle weight matrix; evaluate the weather impact based on the obstacle weight matrix to obtain a weather impact factor; perform obstacle warning classification based on the obstacle risk level table, obstacle weight matrix, and weather impact factor, and output an obstacle warning report.

[0137] S55. Calculate the obstacle monitoring coverage based on the obstacle warning report, obstacle risk level table, and pre-stored historical inspection data to obtain monitoring coverage data; evaluate key areas based on the monitoring coverage data to obtain area priorities; analyze the monitoring frequency based on the area priorities to obtain frequency optimization data; perform optimization calculations based on the monitoring coverage data, area priorities, and frequency optimization data to generate an obstacle monitoring plan including the key monitoring path of obstacles, sampling frequency, priority processing area, and monitoring period.

[0138] In an embodiment of the present application, the calculation of the static threat degree includes the following steps: Calculate the static threat degree ST = (d0 / d) α ·exp(-v·t / τ); where ST is the static threat degree; d0 is the safety threshold distance; d is the actual distance; α is the distance attenuation exponent; v is the relative speed; t is the observation time; τ is the time constant.

[0139] The calculation of the dynamic threat degree includes the following steps: Calculate the dynamic threat degree DT = ∑wi·[vi(t) / v0i]·exp(-|θi(t) - θp(t)| / σθ); where DT is the dynamic threat degree; vi(t) is the moving speed of obstacle i at time t; v0i is the corresponding speed threshold; θi(t) is the moving direction; θp(t) is the predicted direction; wi is the obstacle type weight; σθ is the direction similarity parameter.

[0140] The calculation of the environmental threat degree includes the following steps: Calculate the environmental threat degree ET = w1·R(t) + w2·V(t) + w3·G(t); where ET is the environmental threat degree; R(t) is the rainfall impact factor, R(t) = (r(t) / r0) α ; V(t) is the visibility impact factor, V(t) = exp(-v0 / v(t)); G(t) is the terrain impact factor, G(t) = 1 + kg·∑|▽h(x, y)| / A; r(t) is the rainfall; v(t) is the visibility; h(x, y) is the terrain elevation; w1, w2, w3 are weight coefficients; α, kg are adjustment parameters.

[0141] The comprehensive risk level assessment model includes the following steps: calculating the comprehensive risk level R = f(ST, DT, ET)·(1 + γ·dR / dt); where R is the comprehensive risk level; f(ST, DT, ET) is the risk fusion function, f = (w1·ST p + w2·DT p + w3·ET p ); 1 / p dR / dt is the risk change rate; γ is the time series influence coefficient; p is the non-linear fusion parameter; w1, w2, w3 are adaptive weights, dynamically adjusted by the scenario type and environmental conditions.

[0142] In this embodiment, by establishing a multi-dimensional obstacle analysis and risk early warning system, the intelligent monitoring and risk management of the environment around the power line are realized. First, through scene segmentation and feature extraction, various potential obstacles are accurately identified; in the risk assessment process, a multi-layer position encoding network and a time series attention mechanism are adopted to achieve the accurate quantification of static and dynamic threats. Especially in the link of formulating the early warning strategy, by comprehensively analyzing the environmental impact and development trend, a complete risk level assessment system and a monitoring plan optimization mechanism are established. This embodiment can not only timely discover the current potential safety hazards, but also predict the potential risk development trend. In practical applications, it can accurately assess the risks of different types of obstacles and automatically generate targeted monitoring plans, improving the efficiency and reliability of line safety management.

[0143] In another embodiment of the present application, the comprehensive risk level R is calculated as R = w1·Rs + w2·Rd + w3·Rg + w4·Re; where Rs is the static threat degree, Rs = d0 / d(t), d0 is the safety threshold distance, d(t) is the real-time measured distance; Rd is the dynamic threat degree, Rd = v(t) / v0, v(t) is the relative movement speed of the obstacle, v0 is the speed threshold; Rg is the development threat degree, Rg = g(t) / g0, g(t) is the growth rate of the obstacle (such as a tree), g0 is the growth rate threshold; Re is the environmental threat degree, Re = e(t) / e0, e(t) is the comprehensive influence degree of environmental factors, e0 is the environmental impact threshold; w1, w2, w3, w4 are dynamic weight coefficients, adaptively adjusted according to factors such as season and weather, and satisfy ∑wi = 1.

[0144] According to one aspect of the present application, step S53 is further:

[0145] S531. Obtain obstacle feature data and obstacle distance data, construct a multi-layer position encoding network to analyze the spatial relationship to obtain a spatial relationship map, calculate the minimum safety distance to obtain a safety distance map, and evaluate the spatial interference degree to obtain an interference degree map; fuse the spatial relationship map, safety distance map and interference degree map to establish a risk assessment model, and output the static threat degree; calculate the relative motion characteristics based on the three-dimensional power line model to obtain motion characteristic data.

[0146] S532. According to the motion characteristic data and obstacle feature data, construct a temporal attention network to analyze the dynamic characteristics to obtain a dynamic characteristic map, calculate the collision probability to obtain a collision probability map, and evaluate the motion trend to obtain a trend prediction map; combine the dynamic characteristic map, collision probability map and trend prediction map to construct a dynamic risk model, and output the dynamic threat degree and development threat degree.

[0147] S533. Input the obstacle feature data and the three-dimensional power line model, analyze the influence of meteorological conditions to obtain a meteorological influence map, calculate the terrain constraint to obtain a terrain constraint map, and evaluate the seasonal change to obtain a seasonal change map; fuse the meteorological influence map, terrain constraint map and seasonal change map to construct an environmental influence model, and output the environmental threat degree; use an adaptive weight network to perform weighted fusion on the static threat degree, dynamic threat degree, development threat degree and environmental threat degree, and output an obstacle risk level table.

[0148] In this embodiment, by constructing a multi-layer position encoding network and a temporal attention risk assessment framework, the accurate quantification of the threat degree of obstacles is realized. First, analyze the spatial relationship between the obstacle and the power line through the multi-layer position encoding network, and establish a complete static threat assessment system; in the process of dynamic risk analysis, use the temporal attention network to predict the motion trend and potential collision risk of the obstacle, and can accurately evaluate the potential threats brought by dynamic obstacles such as tree growth and building construction. Especially in the aspect of environmental impact assessment, by analyzing meteorological conditions, terrain constraints and seasonal changes, a complete environmental impact model is constructed, and various environmental factors affecting the obstacle risk are comprehensively considered. This embodiment can not only accurately identify the current safety hazards, but also predict the potential risk development trend. In practical applications, it can accurately classify the risk levels of different types of obstacles, providing a scientific decision-making basis for line safety management and preventive maintenance.

[0149] According to one aspect of the present application, a method for detecting obstacles of a conductor assembly based on a monocular camera includes the following steps:

[0150] S1. Take multiple images taken at different positions and angles, and combine GPS and camera internal parameter data to provide high-quality original data for subsequent feature matching and three-dimensional reconstruction.

[0151] Fly along the direction of the vertical wire and use a monocular camera to take an image at regular intervals. Collect multiple images (at least 3) of the same scene as the processing data. Each image needs to contain the GPS data of the current pose of the camera and the internal parameter matrix information of the current camera.

[0152] S2. Apply advanced feature detection algorithms such as SIFT to stably detect feature points from the images, find the corresponding points between different images through the matching algorithm, and calculate the pose change of the camera using the principles of multi-view geometry to lay the foundation for 3D reconstruction.

[0153] Perform feature detection and matching on multiple images collected by the same camera. The means of feature detection include but are not limited to using feature detectors such as the Scale-Invariant Feature Transform (SIFT) operator. In this embodiment, taking the SIFT detection operator as an example, perform SIFT feature detection on each image to generate feature points and feature descriptors. Use the Euclidean distance as a measure of the similarity of feature points to calculate the Euclidean distance of the key point descriptors in two images.

[0154] After feature extraction and matching, obtain a series of matching feature point pairs of two images. According to a series of matching point pairs detected in two images, the rotation matrix R and translation matrix t of the pose transformation between the two images can be solved. The basic principle is as follows: A point P C =(x C , y C , z C ) projected onto the pixel point p = (u, v) can be expressed as:

[0155] ;

[0156] where u and v are pixel coordinates respectively;

[0157] Then, the pixel point on the second image can be expressed as follows:

[0158] ;

[0159] After substituting and converting and normalizing, it is expressed as: K -1 p2 = RK -1 p1 + t;

[0160] After substituting a series of pixel coordinates, the rotation matrix and translation matrix can be solved, and further optimize the camera pose through algorithms such as Bundle Adjustment.

[0161] S3. Combine the GPS data for scale correction to ensure the accuracy and reliability of 3D reconstruction.

[0162] Since the camera system only retains 2D information, the depth obtained at this time is a normalized data. Obstacle analysis requires 3D coordinate judgment in the actual scene, and the scale correction of the posture is required. The camera posture can be calibrated using the GPS data in the image EXIF ​​to restore the actual position information of the point cloud.

[0163] Assume that the camera pose obtained in the previous step is T cal , the camera position obtained by GPS is T GPS There is a rotation R, a translation matrix t and a scale transformation s between the two postures. R, t and the scale factor s can be solved by minimizing f. f represents the difference between the scale-transformed posture and the actual GPS posture, expressed as: min||sRT cal +tT GPS || F 2 ;

[0164] After taking the derivative of the translation matrix t and setting it equal to 0, we can solve for t:

[0165] ;

[0166] in represents the center of N poses, that is, the average of N poses. Substituting the analytical result of the translation matrix t into the difference f, the difference f can be converted:

[0167] ;

[0168] Furthermore, we differentiate s and set it equal to 0 to solve for the scale factor s:

[0169] ;

[0170] At this time, the minimum transformation of the difference f is f R Find the maximum: .

[0171] Pair Matrix Perform SVD decomposition to solve the rotation matrix R: M = UΣV T , R=VU T .

[0172] By using the above formula, the scale factor is calculated for N poses to obtain accurate 3D reconstruction points.

[0173] S4. Find the matching points of the wire through wire segmentation and epipolar geometry constraints.

[0174] Due to the characteristics of the wire, it is difficult to find feature matching points using traditional feature detection algorithms, resulting in the fact that general 3D reconstruction can only reconstruct the scene but cannot match wire information. Therefore, the wire data is processed separately.

[0175] First, use some image segmentation algorithms to segment the wires in the images, obtain the wire segmentation results in two images, and obtain the matching relationship of the wires. For the reference image, a series of points on the wires are extracted according to the segmentation map, and the matching points of these points on the other image can be obtained through the epipolar geometry relationship and the wire matching relationship.

[0176] The internal parameter of the camera is K, and the coordinate of a wire point P in the world coordinate system is X W , and the projection coordinates of this point at the two images are p0 and p1 respectively. The transformation matrices of the camera poses at the two places are R0, t0 and R1, t1 respectively. According to the expressions in step S2, it can be deduced that: z0p0 = K(R0X W +t0); z1p1 = K(R1X W +t1), where z0 and z1 respectively represent the normalization constants of P projected in the two camera coordinate systems. When P is at different depths, such as at P1, P2, and P3, its image formed on the second image must be located on the epipolar line. In the extreme case, when z0 = 0: X W =-R0 T t0, at this time, the image of P z0=0 formed at camera C1 is: e0 = -K(R1R0 T t0 + t1); when z0 = ∞: X W =R0 T K -1 p0, at this time, the image of P z0=∞ formed at camera C1 is: e1 = -K(R1R0 T K -1 p0); the epipolar line can be expressed as the straight line connecting point e0 and point e1. Further, the expression of the epipolar line is obtained as e0×e1, and through the skew-symmetric matrix, it can be converted to: epipolar line: e0×e1 = skew(e0)∙e1, where epipolar line is the epipolar line and skew is the skew-symmetric matrix. After obtaining the points of the wire on the first image, the position of p1 is solved through the epipolar line and the wire segmentation mask.

[0177] S5. Triangulate the wire matching points and perform dense reconstruction on the wire to obtain 3D information.

[0178] Due to the existence of errors, the position X W cannot be accurately solved, that is, x = PX W and x' = P'X WCannot be satisfied simultaneously. The triangulation problem of the wire points is transformed into an optimization problem, that is: min XW |x - PX W | + |x' - P'X W |.

[0179] The projection of the wire points on Camera 1 can be expressed as:

[0180] ;

[0181] It can be split into:

[0182] ;

[0183] After transforming the above formula, two constraints can be obtained:

[0184] ;

[0185] After introducing the projection points of the second camera, an overdetermined equation can be obtained:

[0186] ;

[0187] Solve for X through SVD W . Dense reconstruction results of the wire can be obtained by reconstructing a series of matching points on the wire.

[0188] S6. Through dense three-dimensional reconstruction technology, recover the three-dimensional model of the scene from multiple images, determine the distance by the point clouds of the wire and the background, determine obstacles, and provide accurate spatial information for the maintenance of power facilities.

[0189] Perform three-dimensional reconstruction on the scene points, including but not limited to using a network such as MVSNet to perform dense three-dimensional reconstruction on the scene. Determine obstacles by measuring the distance of the scene points near the wire points.

[0190] This embodiment uses image feature matching to accurately detect obstacles near the wire, calculate the closest distance between the obstacle and the wire, and provide accurate data for the safe operation of power facilities; combined with GPS data and camera pose information, calibrate the camera pose through an optimization algorithm to ensure the accuracy of the three-dimensional reconstruction model and enhance the robustness of the system; generate a three-dimensional model of the wire and its environment through multi-view stereo vision technology and dense three-dimensional reconstruction algorithm, which helps to evaluate the impact of obstacles on the line.

[0191] According to one aspect of the present application, a wire component obstacle detection system based on a monocular camera includes:

[0192] At least one processor; and,

[0193] A memory communicatively connected to at least one of the processors; wherein,

[0194] The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the method for detecting obstacles of the wire assembly based on a monocular camera described in any one of the above embodiments.

[0195] The present invention realizes the full-process intelligent processing from data acquisition to risk warning by constructing a complete adaptive power line inspection and risk assessment system. First, the quality of data acquisition is ensured through scene adaptive parameter optimization, and precise identification and feature expression of power lines are achieved by using multi-level image processing and feature extraction technologies; in the 3D reconstruction link, the accuracy and reliability of the reconstruction results are guaranteed through multi-constraint optimization and deep learning methods; especially in the aspect of dynamic characteristic analysis and risk assessment, a spatio-temporal attention network and a multi-dimensional risk assessment framework are adopted to realize the comprehensive monitoring of the operating state of power lines and the risks of the surrounding environment. The present invention not only improves the automation level and accuracy of power line inspection, but also realizes the early warning of potential risks. In practical applications, it can effectively cope with the challenges brought by complex meteorological conditions, diverse terrain features and various obstacles, provides all-round technical support for the safe operation and maintenance of power lines, and improves the operation reliability and maintenance efficiency of the power system.

[0196] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. The method of adaptive obstacle detection on the inspection route of UAV based on monocular camera is characterized by: The steps include: S1. Obtain initial flight parameters, and calculate optimized flight parameters through scene complexity evaluation function; based on the optimized flight parameters, collect original image sequences, calculate image quality indicators, and obtain image quality scores; Based on the image quality score, the original image sequence is subjected to exposure compensation to obtain a compensated image sequence; Based on the compensated image sequence, the spatial similarity and motion continuity between images are calculated to generate an image correlation matrix; based on the image correlation matrix, the optimal image combination is screened; the initial flight parameters include the initial flight altitude, the initial flight speed and the initial sampling interval; S2. Based on the optimal image combination and image quality score, a multi-resolution feature pyramid is constructed to obtain a multi-scale feature tensor set; a direction descriptor is calculated for the multi-scale feature tensor set to generate a feature descriptor set; Based on the feature descriptor set and the image association matrix, the feature matching score is calculated and the initial feature matching pair set is output; the initial feature matching pair set is verified at multiple levels to obtain a reliable matching set; the feature distribution of the reliable matching set is analyzed, and supplementary features are generated to obtain an enhanced feature matching set; S3, based on the enhanced feature matching set, the pre-stored camera parameter matrix and the GPS raw data, calculate the initialization pose to obtain a pose sequence; Perform multi-view geometry optimization on the pose sequence and output the optimized camera pose; calculate the depth estimate based on the optimized camera pose and generate a multi-scale depth map set; perform consistency optimization on the multi-scale depth map set to obtain an optimized unified depth map; Combine the optimized unified depth map and the optimized camera pose to evaluate the reconstruction quality and output the quality distribution map. S4. Based on the compensated image sequence and the quality distribution map, the directional enhancement response is calculated to obtain a power line candidate area map; the power line candidate area map is segmented by topological constraints to output a segmented line segment set; based on the segmented line segment set and the optimized camera pose, multi-view matching is performed to generate a segment matching pair set; based on the segment matching pair set and the optimized unified depth map, three-dimensional reconstruction is performed to obtain a three-dimensional model of the power line; Analyze the timing characteristics of the three-dimensional model of the power line and output the power line dynamic characteristics report; S5. Based on the compensated image sequence, the optimized unified depth map and the power line 3D model, scene segmentation is performed to obtain obstacle feature data; Based on obstacle feature data and power line dynamic characteristics report, calculate the distance between power lines and obstacles and output obstacle distance data; Combine obstacle feature data, obstacle distance data and power line 3D model to conduct obstacle risk assessment and generate obstacle risk level table; Generate obstacle warning information and output obstacle warning report based on obstacle risk level table and power line dynamic characteristics report; Based on the obstacle warning report and obstacle risk level table, the inspection strategy is optimized and the obstacle monitoring plan is output.

2. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 1 is characterized in that: Step S1 is further as follows: S11, obtaining initial flight parameters, calculating the edge density value, local variance value and depth change rate of the image, and substituting them into the scene complexity evaluation function for calculation to obtain the scene complexity score; Based on the scene complexity score, the initial flight parameters are optimized and calculated, and the optimized flight parameters are output; S12, based on the optimized flight parameters, collecting the original image sequence; obtaining the pre-configured camera parameter matrix, performing wavelet transform calculation on each image in the original image sequence, and obtaining a clarity index; Calculate the brightness distribution of each area in the original image sequence to obtain the illumination uniformity index; analyze the blur degree of the original image sequence to obtain the motion blur index; evaluate the scene coverage in the original image sequence to obtain the scene integrity index; combine the clarity index, illumination uniformity index, motion blur index and scene integrity index to form an image quality scoring matrix; S13, based on the image quality score matrix, dividing the original image sequence into a predetermined number of regions, calculating the local brightness mapping value of each region, and generating a regional weight mask; Multiply the local brightness map value with the regional weight mask and superimpose them to obtain the exposure compensation value; Based on the exposure compensation value, the brightness of the original image sequence is adjusted, and the compensated image sequence is output; S14, calculating the spatial similarity value between adjacent images based on the compensated image sequence and the optimized flight parameters; Analyze the motion continuity of the compensated image sequence to obtain a continuity score; The spatial similarity values ​​and continuity scores are combined to generate a spatiotemporal association matrix; Based on the spatiotemporal correlation matrix, the optimal image combination is screened.

3. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 2 is characterized in that: Step S2 is further as follows: S21. Based on the optimal image combination and the image quality score, the scale parameter of each level is calculated; based on the scale parameter of each level, the image in the optimal image combination is subjected to tensor transformation and weight kernel function calculation to generate a single-layer feature tensor; the single-layer feature tensors of all levels are combined to generate a multi-scale feature tensor set; S22. Based on the multi-scale feature tensor set, the anisotropic intensity value of each feature point is calculated to obtain an anisotropic intensity vector; based on the anisotropic intensity vector, a directional gradient histogram is generated to obtain directional gradient information; Based on the directional gradient information, the context correlation is calculated to obtain context coding data; the anisotropic intensity vector, the directional gradient information and the context coding data are combined to generate a feature descriptor set; S23, based on the feature descriptor set and the image association matrix, calculate the neighborhood constraints of the feature points to obtain neighborhood constraint data; based on the neighborhood constraint data, calculate the matching probability between the feature points; based on the matching probability and a preset threshold, generate an initial feature matching pair set; S24, based on the initial feature matching pair set and the image association matrix, calculate the local structure consistency score, evaluate the global topological consistency, analyze the temporal continuity, and generate a multi-level verification score; Based on the multi-level verification scores, the initial feature matching pair set is screened and a reliable matching set is output; S25. Based on the reliable matching set and the feature descriptor set, analyze the feature space distribution, identify the feature sparse area, and generate supplementary feature points in the feature sparse area; calculate the descriptor for the supplementary feature point, merge the descriptor with the reliable matching set, and output the enhanced feature matching set.

4. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 3 is characterized in that: Step S3 is further as follows: S31. Based on the enhanced feature matching set and the pre-stored camera parameter matrix, the projection error value is calculated, and the time series smoothing constraint, including the speed constraint term and the acceleration constraint term, is generated; based on the pre-stored GPS raw data, the GPS constraint term is constructed; based on the time series smoothing constraint and the GPS constraint term, the camera position and attitude are calculated, motion prediction is performed, and the initialization posture sequence is output; S32, based on the initialization pose sequence and the enhanced feature matching set, calculating the reprojection error to obtain reprojection error data; based on the reprojection error data, evaluating the structural consistency to obtain the structural constraint data; Based on the structural constraint data, the scene continuity is analyzed to obtain the continuity constraint data; Based on the reprojection error data, structural constraint data and continuity constraint data, optimization calculation is performed to output the optimized camera pose; S33, based on the optimized camera pose and compensated image sequence, divide the local image blocks, calculate the local depth probability, and generate a depth estimation probability distribution; based on the depth estimation probability distribution, perform depth estimation calculation on each scale level, combine multi-scale results, and output a multi-scale depth map set; S34, based on the multi-scale depth map set and the optimized camera pose, calculate the depth data item to obtain depth consistency data; based on the depth consistency data, analyze the depth gradient to obtain gradient constraint data; based on the gradient constraint data, evaluate the surface smoothness to obtain smoothness constraint data; combine the depth consistency data, the gradient constraint data and the smoothness constraint data for optimization, and output the optimized unified depth map; S35. Based on the optimized unified depth map and the optimized camera pose, calculate the reconstruction completeness score, evaluate the geometric consistency and scale accuracy, and generate a reconstruction quality score; Based on the reconstruction quality score, the spatial distribution is calculated to obtain a quality distribution map.

5. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 4 is characterized in that: Step S4 is further as follows: S41. Based on the compensated image sequence and the quality distribution map, a multi-scale directional filter group is constructed to calculate the point cloud projection consistency score; based on the point cloud projection consistency score, an anisotropic Gaussian filter is used to calculate the local directionality score and generate a directional response tensor; Based on the directional response tensor, edge enhancement processing is performed to obtain an enhanced response map; the enhanced response map is subjected to directionality analysis to output a power line candidate area map and a directional distribution map; S42, based on the power line candidate area map and the direction distribution map, calculate the local direction consistency, evaluate the line segment ductility, and generate the linear feature weight; Based on the linear feature weights, the curvature change rate is calculated, the piecewise smoothness is evaluated, and a continuity score is generated; Based on the continuity score, the intersection status of the line segments is analyzed, the parallelism index is calculated, and the topological relationship matrix is ​​generated; combined with Linear feature weights, continuity scores and topological relationship matrices are used to optimize segmentation and output a set of segmented line segments; S43, based on the segmented line segment set and the optimized camera pose, calculating the direction similarity, evaluating the length ratio consistency, and generating a geometric similarity score; Based on the geometric similarity score, the relative position relationship is calculated, the spatial distribution pattern is evaluated, and the topological constraint matrix is ​​generated; Based on the topological constraint matrix, the surrounding feature distribution is analyzed, the regional descriptor is calculated, and the context matching score is generated; Based on the geometric similarity score, topological constraint matrix and context matching score, the matching pairs are screened and a set of line segment matching pairs is output; S44, based on the line segment matching pair set, the direction distribution map and the optimized unified depth map, evaluate the shape features and generate basic model parameters; Based on the basic model parameters, calculate the local deformation characteristics, evaluate the elastic deformation, and generate the deformation function; Based on the deformation function, the measurement error is analyzed, the uncertainty is calculated, and the noise model is generated; Based on the basic model parameters, deformation function and noise model, optimization reconstruction is performed to output the three-dimensional model of power lines; S45, based on the three-dimensional model of the power line and the optimized camera posture, calculating the sag change rate to obtain sag data; Based on the sag data, the lateral offset is measured to obtain the offset data; based on the offset data, the vibration frequency is analyzed to obtain the vibration data; based on the vibration data, the tension value is estimated to obtain the tension data; the sag data, offset data, vibration data and tension data are time-series filtered and weighted, and a power line dynamic characteristics report is output.

6. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 5 is characterized in that: Step S5 is further as follows: S51, based on the compensated image sequence, the optimized unified depth map and the power line 3D model, evaluate the height stratification and generate the spatial obstacle features; Based on the spatial obstacle characteristics, motion characteristics are calculated, change patterns are evaluated, and dynamic obstacle characteristics are generated; Based on the dynamic obstacle characteristics, the geometric shape is analyzed, the texture features are calculated, and the obstacle morphological features are generated; Combine spatial obstacle features, dynamic obstacle features and obstacle morphological features to identify and segment obstacles and output obstacle feature data; S52. Based on the obstacle feature data and the power line dynamic characteristic report, calculate the shortest distance between the obstacle and the power line to obtain vertical distance data; based on the vertical distance data, analyze the obstacle movement trend to obtain dynamic approach data; based on the dynamic approach data, evaluate the environmental impact to obtain environmental impact data; Based on vertical distance data, dynamic approach data and environmental impact data, an obstacle distance assessment model is constructed to output obstacle distance data; S53, calculating the static distance risk based on the obstacle feature data, the obstacle distance data and the three-dimensional model of the power line to obtain the static threat degree; Based on the static threat level, the dynamic collision risk is evaluated to obtain the dynamic threat level; based on the dynamic threat level, the obstacle growth trend is analyzed to obtain the development threat level; based on the development threat level, the environmental impact is calculated to obtain the environmental threat level; the static threat level, dynamic threat level, development threat level and environmental threat level are weighted and combined to output the obstacle risk level table; S54. Calculate obstacle type weights based on the obstacle risk level table, obstacle feature data, and power line dynamic characteristic report to obtain an obstacle weight matrix; Based on the obstacle weight matrix, the weather impact is evaluated and the weather impact factor is obtained; Based on the obstacle risk level table, obstacle weight matrix and weather impact factors, obstacle warning classification is carried out and obstacle warning report is output; S55. Calculate obstacle monitoring coverage based on the obstacle warning report, the obstacle risk level table and the pre-stored historical inspection data to obtain monitoring coverage data; Based on the monitoring coverage data, key areas are evaluated and regional priorities are obtained; Based on regional priority, the monitoring frequency is analyzed to obtain frequency optimization data; based on the monitoring coverage data, regional priority and frequency optimization data, optimization calculations are performed to generate an obstacle monitoring plan that includes obstacle key monitoring paths, sampling frequencies, priority processing areas and monitoring cycles.

7. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 6 is characterized in that: Step S13 is further as follows: S131, based on the image quality score matrix, using an adaptive grid partitioning algorithm for each image in the original image sequence to obtain an image region set; based on the image region set, calculating the brightness mean of each region to obtain a region brightness matrix; Analyze the distribution characteristics of the regional brightness matrix to obtain a brightness distribution map; Applying Gaussian smoothing to the brightness distribution map to obtain a smoothed brightness map; calculating the difference between the smoothed brightness map and the preset standard brightness to obtain a brightness deviation map; generating a local mapping matrix based on the brightness deviation map and the image quality score matrix; S132, calculating the edge strength of each region based on the local mapping matrix and the image region set to obtain an edge strength map; Based on the edge intensity map, the regional texture features are extracted to obtain a texture feature map; The edge intensity map and the texture feature map are fused to obtain a regional importance map; the regional importance map is normalized to obtain a weight mask matrix; Performing element-wise multiplication of the local mapping matrix and the weight mask matrix to obtain a weighted mapping matrix; performing spatial domain filtering on the weighted mapping matrix to obtain a smoothed mapping matrix; S133. Based on the smooth mapping matrix, brightness mapping is performed on each image in the original image sequence to obtain a mapped image sequence; the local contrast of the mapped image sequence is calculated to obtain a contrast map; based on the contrast map, the degree of image detail preservation is analyzed to obtain a detail preservation map; the contrast map and the detail preservation map are fused to obtain a quality assessment map; based on the quality assessment map, local fine-tuning is performed on the mapped image sequence to output a compensated image sequence.

8. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 6 is characterized in that: Step S22 is further as follows: S221. Based on the multi-scale feature tensor set, construct a covariance matrix for each scale layer to obtain a covariance tensor set; calculate the eigenvalues ​​of the covariance tensor set to obtain an eigenvalue sequence; Based on the eigenvalue sequence, the anisotropy measure is calculated to obtain the anisotropy intensity vector; Based on the anisotropic intensity vector, the self-attention mechanism is used to calculate the correlation between feature points and obtain the attention weight map; The anisotropic intensity vector is fused with the attention weight map to obtain an enhanced feature vector; S222, constructing a local coordinate system based on the enhanced feature vector to obtain a local coordinate map; calculating the directional gradient in the local coordinate map to obtain a gradient amplitude map and a gradient directional map; performing directional quantization on the gradient directional map to obtain a directional histogram; Calculate the statistical characteristics of the gradient amplitude map to obtain the amplitude feature vector; combine the directional histogram and the amplitude feature vector to obtain the directional gradient information; S223. Based on the directional gradient information and the enhanced feature vector, a multi-head attention network is constructed to obtain an attention feature map; Based on the attention feature map, the semantic relevance of the feature point neighborhood is calculated to obtain the semantic association map; the attention feature map and the semantic association map are fused to obtain the context encoding data; the anisotropic intensity vector, directional gradient information and context encoding data are feature spliced, and the dimension is reduced through a multi-layer perceptron to output a set of feature descriptors.

9. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 6 is characterized in that: Step S32 is further as follows: S321, based on the initialization pose sequence and the enhanced feature matching set, construct a projection model to obtain a projection matrix set; based on the projection matrix set, calculate the reprojection coordinates of the feature points to obtain a projection coordinate set; calculate the Euclidean distance between the projection coordinate set and the actual coordinates to obtain a distance error map; Analyze the statistical characteristics of the distance error map to obtain the reprojection error data; Based on the reprojection error data, an error weight matrix is ​​constructed to obtain an error weight map; S322, constructing a scene structure graph based on the error weight map and the enhanced feature matching set to obtain a structure atlas; analyzing the local rigidity of the structure atlas to obtain a rigid constraint graph; calculating the geometric consistency between feature points based on the rigid constraint graph to obtain a geometric constraint graph; fusing the rigid constraint graph and the geometric constraint graph to obtain structural constraint data; Based on the structural constraint data, the constraint weights are calculated to obtain the structural weight matrix; S323. Analyze the temporal changes of the initialization pose sequence to obtain a motion trajectory graph; based on the motion trajectory graph and the structural weight matrix, calculate the motion continuity between adjacent frames to obtain a continuity score graph; based on the continuity score graph, evaluate the smoothness of the scene changes to obtain a smoothness graph; fuse the continuity score graph and the smoothness graph to obtain continuity constraint data; substitute the reprojection error data, structural constraint data and continuity constraint data into the optimization solver, and output the optimized camera pose and three-dimensional point cloud data.

10. The monocular camera-based UAV inspection route adaptive obstacle detection system is characterized by: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the monocular camera-based adaptive obstacle detection method for drone inspection routes as described in any one of claims 1 to 9.

Images