Unmanned aerial vehicle routing inspection line adaptive obstacle detection method and system based on monocular camera

Abstract

The invention discloses an unmanned aerial vehicle routing inspection line adaptive obstacle detection method and system based on a monocular camera. The method comprises the steps of obtaining initial flight parameters, evaluating and optimizing the flight parameters based on scene complexity, collecting an original image sequence and carrying out quality evaluation and exposure compensation. Constructing a multi-resolution feature pyramid, extracting feature descriptors, and performing feature matching and verification; performing multi-constraint pose optimization and depth estimation based on a feature matching result, and reconstructing a three-dimensional point cloud; calculating direction enhancement response to extract a power line candidate area, performing multi-view line segment matching and three-dimensional reconstruction, and analyzing dynamic characteristics of the power line; and based on the power line three-dimensional model and the depth map, carrying out obstacle segmentation, calculating an obstacle distance, evaluating a risk level, and generating early warning information and a monitoring scheme. According to the invention, different scenes can be adaptively processed, the obstacle risk can be accurately identified and evaluated, and the inspection efficiency and reliability are improved.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and in particular to a method and system for detecting obstacles of a wire assembly based on a monocular camera. Background Art

[0002] In the maintenance and monitoring of power facilities, the safe operation of power lines is directly related to the reliability and stability of power supply. Traditional manual inspection methods are not only time-consuming and labor-intensive, but also inefficient in complex terrain or severe weather conditions and pose safety risks. With the rapid development of computer vision algorithms, automatic inspection systems based on monocular cameras have gradually become an important means of intelligent monitoring of power lines. This method can not only improve inspection efficiency, but also obtain more comprehensive and objective monitoring data, providing important support for preventive maintenance of power lines.

[0003] At present, domestic and foreign research mainly focuses on two directions: target detection based on deep learning and feature matching based on traditional image processing. In terms of target detection, algorithms such as improved YOLO or Faster R-CNN are mainly used for obstacle recognition, but these methods often require a large amount of labeled data for training and are difficult to adapt to different scenes and lighting conditions. In terms of feature matching, algorithms such as SIFT or SURF are often used to extract image features, and matching and screening are performed through methods such as RANSAC, but these methods have high computational complexity and lack real-time performance. In terms of three-dimensional reconstruction, traditional methods mainly rely on SfM (Structure from Motion) technology, which optimizes and restores the three-dimensional structure of the scene through feature point matching and Bundle Adjustment, but when dealing with slender targets such as power lines, problems such as incomplete reconstruction or insufficient accuracy often occur.

[0004] However, the existing technical solutions still have the following key problems: First, the acquisition process of power line images lacks scene adaptation capabilities, and fixed flight parameters are difficult to cope with scenes of different complexities, resulting in unstable data quality; Second, in the feature extraction and matching process, the existing methods fail to fully utilize the directional and continuous characteristics of power lines, resulting in a significant decrease in matching accuracy when the feature distribution is uneven or there is local occlusion; Third, in the three-dimensional reconstruction process, there is a lack of modeling of the unique deformation characteristics and dynamic characteristics of power lines, making it impossible for the reconstruction results to accurately reflect the actual state of the power lines; Fourth, the existing obstacle risk assessment methods are mainly based on static distance measurement, which fails to consider the growth trend of obstacles and the dynamic influence of environmental factors, making it difficult to achieve forward-looking warnings; Fifth, in terms of image quality assessment, existing methods often use a single clarity or brightness index, lack a comprehensive assessment of scene integrity and feature reliability, and affect the accuracy of subsequent processing. These technical problems seriously restrict the actual application effect of the monocular camera inspection system. Summary of the invention

[0005] The purpose of the invention is to propose a method and system for detecting obstacles in wire components based on a monocular camera, in order to solve at least one technical problem existing in the prior art.

[0006] The technical solution is a method for detecting obstacles in a wire assembly based on a monocular camera, comprising the following steps:

[0007] S1. Obtain initial flight parameters, and calculate optimized flight parameters through scene complexity evaluation function; based on the optimized flight parameters, collect original image sequence, calculate image quality index, and obtain image quality score; based on the image quality score, perform exposure compensation processing on the original image sequence to obtain compensated image sequence; based on the compensated image sequence, calculate the spatial similarity and motion continuity between images, and generate an image association matrix; based on the image association matrix, screen and obtain the optimal image combination; wherein the initial flight parameters include initial flight altitude, initial flight speed, and initial sampling interval;

[0008] S2. Based on the optimal image combination and image quality score, a multi-resolution feature pyramid is constructed to obtain a multi-scale feature tensor set; the direction descriptor is calculated for the multi-scale feature tensor set to generate a feature descriptor set; based on the feature descriptor set and the image association matrix, the feature matching score is calculated and the initial feature matching pair set is output; the initial feature matching pair set is verified at multiple levels to obtain a reliable matching set; the feature distribution of the reliable matching set is analyzed, supplementary features are generated, and an enhanced feature matching set is obtained;

[0009] S3. Based on the enhanced feature matching set, the pre-stored camera parameter matrix and the GPS raw data, the initialization pose is calculated to obtain a pose sequence; multi-view geometry optimization is performed on the pose sequence to output the optimized camera pose; based on the optimized camera pose, the depth estimation is calculated to generate a multi-scale depth map set; the consistency of the multi-scale depth map set is optimized to obtain an optimized unified depth map; the reconstruction quality is evaluated by combining the optimized unified depth map and the optimized camera pose, and a quality distribution map is output;

[0010] S4. Based on the compensated image sequence and the quality distribution map, the directional enhancement response is calculated to obtain a power line candidate area map; the power line candidate area map is segmented with topological constraints to output a segmented line segment set; based on the segmented line segment set and the optimized camera pose, multi-view matching is performed to generate a segment matching pair set; based on the segment matching pair set and the optimized unified depth map, three-dimensional reconstruction is performed to obtain a three-dimensional model of the power line; the timing characteristics of the three-dimensional model of the power line are analyzed, and a report on the dynamic characteristics of the power line is output;

[0011] S5. Based on the compensated image sequence, the optimized unified depth map and the power line three-dimensional model, the scene is segmented to obtain obstacle feature data; based on the obstacle feature data and the power line dynamic characteristics report, the distance between the power line and the obstacle is calculated, and the obstacle distance data is output; based on the obstacle feature data, the obstacle distance data and the power line three-dimensional model, the obstacle risk assessment is performed to generate an obstacle risk level table; based on the obstacle risk level table and the power line dynamic characteristics report, obstacle warning information is generated and an obstacle warning report is output; based on the obstacle warning report and the obstacle risk level table, the inspection strategy is optimized and an obstacle monitoring plan is output.

[0012] The monocular camera-based wire assembly obstacle detection system includes:

[0013] at least one processor; and,

[0014] a memory communicatively connected to at least one of the processors; wherein,

[0015] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the monocular camera-based wire assembly obstacle detection method.

[0016] Beneficial effects: The present invention ensures the quality of data collection, realizes the accurate identification and feature expression of power lines; ensures the accuracy and reliability of reconstruction results, realizes the comprehensive monitoring of power line operation status and surrounding environmental risks; improves the automation level and accuracy of power line inspection, and realizes early warning of potential risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flow chart of the method of the present invention.

[0018] Figure 2 This is a flow chart of step S1 of the present invention.

[0019] Figure 3 This is a flow chart of step S2 of the present invention.

[0020] Figure 4 This is a flow chart of step S3 of the present invention.

[0021] Figure 5 This is a flow chart of step S4 of the present invention.

[0022] Figure 6 This is a flow chart of step S5 of the present invention. DETAILED DESCRIPTION

[0023] like Figure 1 As shown, the present application proposes a method for detecting obstacles of a wire assembly based on a monocular camera, comprising the following steps:

[0024] S1. Obtain initial flight parameters, and calculate optimized flight parameters through scene complexity evaluation function; based on the optimized flight parameters, collect original image sequence, calculate image quality index, and obtain image quality score; based on the image quality score, perform exposure compensation processing on the original image sequence to obtain compensated image sequence; based on the compensated image sequence, calculate the spatial similarity and motion continuity between images, and generate an image association matrix; based on the image association matrix, screen and obtain the optimal image combination; wherein the initial flight parameters include initial flight altitude, initial flight speed, and initial sampling interval;

[0025] S2. Based on the optimal image combination and image quality score, a multi-resolution feature pyramid is constructed to obtain a multi-scale feature tensor set; the direction descriptor is calculated for the multi-scale feature tensor set to generate a feature descriptor set; based on the feature descriptor set and the image association matrix, the feature matching score is calculated and the initial feature matching pair set is output; the initial feature matching pair set is verified at multiple levels to obtain a reliable matching set; the feature distribution of the reliable matching set is analyzed, supplementary features are generated, and an enhanced feature matching set is obtained;

[0026] S3. Based on the enhanced feature matching set, the pre-stored camera parameter matrix and the GPS raw data, the initialization pose is calculated to obtain a pose sequence; multi-view geometry optimization is performed on the pose sequence to output the optimized camera pose; based on the optimized camera pose, the depth estimation is calculated to generate a multi-scale depth map set; the consistency of the multi-scale depth map set is optimized to obtain an optimized unified depth map; the reconstruction quality is evaluated by combining the optimized unified depth map and the optimized camera pose, and a quality distribution map is output;

[0027] S4. Based on the compensated image sequence and the quality distribution map, the directional enhancement response is calculated to obtain a power line candidate area map; the power line candidate area map is segmented with topological constraints to output a segmented line segment set; based on the segmented line segment set and the optimized camera pose, multi-view matching is performed to generate a segment matching pair set; based on the segment matching pair set and the optimized unified depth map, three-dimensional reconstruction is performed to obtain a three-dimensional model of the power line; the timing characteristics of the three-dimensional model of the power line are analyzed, and a report on the dynamic characteristics of the power line is output;

[0028] S5. Based on the compensated image sequence, the optimized unified depth map and the power line three-dimensional model, the scene is segmented to obtain obstacle feature data; based on the obstacle feature data and the power line dynamic characteristics report, the distance between the power line and the obstacle is calculated, and the obstacle distance data is output; based on the obstacle feature data, the obstacle distance data and the power line three-dimensional model, the obstacle risk assessment is performed to generate an obstacle risk level table; based on the obstacle risk level table and the power line dynamic characteristics report, obstacle warning information is generated and an obstacle warning report is output; based on the obstacle warning report and the obstacle risk level table, the inspection strategy is optimized and an obstacle monitoring plan is output.

[0029] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0030] S11, obtaining initial flight parameters, calculating the edge density value, local variance value and depth change rate of the image, and substituting them into the scene complexity evaluation function for calculation to obtain a scene complexity score; based on the scene complexity score, optimizing the initial flight parameters and outputting the optimized flight parameters;

[0031] S12. Based on the optimized flight parameters, collect the original image sequence; obtain the pre-configured camera parameter matrix, perform wavelet transform calculation on each image in the original image sequence, and obtain the clarity index; calculate the brightness distribution of each area in the original image sequence to obtain the illumination uniformity index; analyze the blur degree of the original image sequence to obtain the motion blur index; evaluate the scene coverage in the original image sequence to obtain the scene integrity index; combine the clarity index, illumination uniformity index, motion blur index and scene integrity index to form an image quality scoring matrix;

[0032] S13, based on the image quality score matrix, dividing the original image sequence into a predetermined number of regions, calculating the local brightness mapping value of each region, and generating a regional weight mask; multiplying the local brightness mapping value and the regional weight mask and superimposing them to obtain an exposure compensation value; based on the exposure compensation value, adjusting the brightness of the original image sequence, and outputting a compensated image sequence;

[0033] S14. Based on the compensated image sequence and the optimized flight parameters, calculate the spatial similarity value between adjacent images; analyze the motion continuity of the compensated image sequence to obtain a continuity score; combine the spatial similarity value and the continuity score to generate a spatiotemporal correlation matrix; based on the spatiotemporal correlation matrix, screen the optimal image combination.

[0034] In one embodiment of the present application, edge density value calculation includes the following steps: calculating image gradient ED = ∑∑sqrt[(ΨI / Ψx) 2 + (ΨI / Ψy) 2] / (M·N); where ED is the edge density value; I is the image grayscale value; M and N are the image height and width; ΨI / Ψx is the x-direction gradient; ΨI / Ψy is the y-direction gradient; (ΨI / Ψx) 2 is the square of the gradient in the x direction; (ΨI / Ψy) 2 is the square of the gradient in the y direction, and Ψ is the partial derivative.

[0035] The local variance calculation includes the following steps: Calculate the local area variance LV = ∑∑(I(x, y) -μ(x, y)) 2 / k 2 ; Where LV is the local variance value; I(x, y) is the pixel value at position (x, y); μ(x, y) is the mean value in a k×k window centered at (x, y); k is the window size; divided by k 2 Perform normalization.

[0036] The depth change rate calculation includes the following steps: calculating the depth gradient DR = ∑∑|D(x+1, y) - D(x, y)| +|D(x, y+1) - D(x, y)| / (M·N); where DR is the depth change rate; D(x, y) is the depth value at the position (x, y); M and N are the depth map sizes; |D(x+1, y) - D(x, y)| is the depth difference in the horizontal direction; |D(x, y+1) - D(x, y)| is the depth difference in the vertical direction.

[0037] The calculation of the scene complexity score includes the following steps: calculating the comprehensive score SC = w1·ED + w2·LV +w3·DR + w4·∑(EDi·LVi·DRi) / n; where SC is the scene complexity score; ED is the edge density value; LV is the local variance value; DR is the depth change rate; EDi, LVi, DRi are the corresponding index values ​​of the i-th sub-region; n is the number of sub-regions; w1, w2, w3, w4 are weight coefficients, satisfying ∑wi = 1.

[0038] The clarity index calculation includes the following steps: Calculate the wavelet transform coefficient SI =∑|Wψ(s, p)| 2 / N; where SI is the clarity index; Wψ(s, p) is the wavelet coefficient at position p at scale s; N is the total number of coefficients; |Wψ(s, p)| 2 is the energy of the wavelet coefficient; ψ is the wavelet basis function.

[0039] The calculation of the illumination uniformity index includes the following steps: calculating the brightness distribution LI=1-∑|B(x, y)-Bm| / (255·M·N); where LI is the illumination uniformity index; B(x, y) is the brightness value at the position (x, y); Bm is the average brightness of the image; M and N are the image sizes; and 255 is the maximum brightness.

[0040] The motion blur index calculation includes the following steps: Calculate the frequency domain energy ratio MI = ∑|F(u, v)| 2 _high / ∑|F(u,v)| 2 _total; MI is the motion blur index; F(u, v) is the image Fourier transform coefficient; |F(u, v)| 2 _high is the energy of high-frequency component; |F(u,v)| 2 _total is the total energy; u and v are the frequency domain coordinates.

[0041] The calculation of the scene integrity index includes the following steps: calculating the effective coverage CI =∑V(x, y) / (M·N)·(1- O(x, y) / A); where CI is the scene integrity index; V(x, y) is the valid pixel mark at the position (x, y); O(x, y) is the occlusion area mark; A is the total area; M and N are the image sizes.

[0042] The quality score matrix generation includes the following steps: Calculate the comprehensive quality score QM = α1·SI + α2·LI + α3·MI + α4·CI -β·∑|grad 2 I(x, y)| / N; where QM is the quality score; SI is the clarity index; LI is the illumination uniformity index; MI is the motion blur index; CI is the scene integrity index; grad 2 I(x, y) is the Laplace operator response; α1, α2, α3, α4, β are weight coefficients, satisfying ∑αi = 1.

[0043] The calculation of the local brightness mapping value includes the following steps: calculating the brightness mapping function L(x, y) = k·log(1 +σ·I(x, y)) / (μ+σ·I(x, y)); where L(x, y) is the brightness value after mapping; I(x, y) is the original brightness value; k is the gain coefficient; σ is the contrast parameter; and μ is the local mean.

[0044] The regional weight mask generation includes the following steps: Calculate the weight value W(x, y) = exp(-((x-xc) 2 + (y-yc) 2 ) / (2σ 2 ))·(1 +γ·|grad I(x,y)|); where W(x,y) is the weight value at the position (x,y); xc, yc are the coordinates of the region center; σ is the Gaussian kernel parameter; |grad I(x,y)| is the gradient amplitude; γ is the edge enhancement coefficient.

[0045] The construction of the scene complexity evaluation model includes the following steps: calculating the scene complexity score SC = w1·ED +w2·LV + w3·DR; where SC is the comprehensive score of scene complexity; ED is the edge density value, ED = ∑|grad I(x, y)| / N, grad I(x, y) is the gradient value of the image (x, y) position, N is the total number of image pixels; LV is the local variance value, LV = ∑(I(x, y) -μ) 2 / N, I(x, y) is the grayscale value of the image position (x, y), μ is the average grayscale value of the local area; DR is the depth change rate, DR =∑|D(x+1, y) - D(x, y)| / N, D(x, y) is the depth value of the position (x, y); w1, w2, w3 are adaptive weight coefficients, which are dynamically adjusted by the neural network according to the scene characteristics to satisfy w1 + w2 + w3 = 1; the weight is updated using the gradient descent method, w_k(t+1) = w_k(t) - η·ΨL / Ψw_k, where η is the learning rate, L is the loss function, w_k(t+1) is the weight coefficient value at time t+1, w_k(t) is the weight coefficient value at time t, and w_k is the general symbol of the weight coefficient, which is used to represent the weight value at any time.

[0046] This embodiment achieves precise control of inspection tasks by constructing a multi-dimensional scene evaluation system and an adaptive flight parameter optimization mechanism. Specifically, firstly, by calculating the three key indicators of the edge density value, local variance value and depth change rate of the image, the scene complexity is comprehensively evaluated to avoid the one-sidedness that may be caused by a single indicator; then, based on the scene complexity score, an adaptive optimization algorithm is used to dynamically adjust the flight parameters so that the flight altitude, speed and sampling interval can be adjusted in real time according to the characteristics of the scene. The quality of the collected images is evaluated and screened through the image quality scoring mechanism to ensure the data quality of subsequent processing; finally, the continuity and quality consistency of the image sequence are ensured through exposure compensation processing and spatial similarity calculation. This embodiment improves the adaptability and reliability of power line image acquisition, and lays a solid foundation for subsequent feature extraction and matching. In practical applications, it can effectively cope with power line inspection scenes under different weather conditions and different terrain features, and improve inspection efficiency and data quality.

[0047] According to one aspect of the present application, step S11 is further:

[0048] S111. Obtain the initial flight altitude, initial flight speed and initial sampling interval, use the Sobel operator to calculate the image gradient amplitude to obtain an edge intensity map, use adaptive thresholding to process the edge intensity map to obtain an edge binary map, calculate the proportion of non-zero pixels in the edge binary map to obtain an edge density value; process the image in blocks, calculate the grayscale variance of each image block to obtain a local variance matrix, and perform weighted averaging on the local variance matrix to obtain a local variance value; calculate the depth maps of adjacent image frames based on disparity estimation to obtain a depth sequence, calculate the spatial gradient of the depth sequence to obtain a depth gradient map, and calculate the rate of change of the depth gradient map to obtain a depth change rate.

[0049] S112. According to the edge density value, local variance value and depth change rate, the structural complexity weight is calculated to obtain the structural weight vector, the texture complexity weight is calculated to obtain the texture weight vector, and the depth complexity weight is calculated to obtain the depth weight vector; the structural weight vector, texture weight vector and depth weight vector are substituted into the multi-layer perceptron for nonlinear mapping to obtain the complexity mapping value; the complexity mapping value is normalized and the scene complexity score is output.

[0050] S113, input the scene complexity score, initial flight altitude, initial flight speed and initial sampling interval, construct a parameter mapping matrix based on the scene complexity to obtain the parameter mapping matrix, multiply the scene complexity score by the parameter mapping matrix to obtain the parameter adjustment vector; perform weighted fusion of the initial flight altitude, initial flight speed and initial sampling interval with the parameter adjustment vector to obtain the optimized flight altitude, optimized flight speed and optimized sampling interval.

[0051] This embodiment realizes precise control of flight parameters by constructing an adaptive scene complexity evaluation system and a multi-level parameter optimization mechanism. The edge density value is calculated by multi-scale decomposition and Sobel operator, which can accurately capture the structural features in the scene; at the same time, the texture complexity and spatial variation characteristics of the scene are comprehensively evaluated by local variance calculation and depth estimation. In the parameter optimization link, a nonlinear mapping method based on a multi-layer perceptron is adopted to convert complex scene features into specific flight parameter adjustment values, taking into account the relationship and influence between scene features. By constructing a parameter mapping matrix, the precise conversion from scene complexity to specific flight parameters is realized, ensuring the rationality and reliability of the adjustment. This embodiment not only improves the setting accuracy of flight parameters, but also enhances the system's adaptability to different scenarios. In actual power line inspection, the flight parameters can be automatically adjusted according to the characteristics of different line sections, effectively avoiding the limitations of traditional fixed parameter solutions in complex scenes, and improving the quality and efficiency of data acquisition.

[0052] According to one aspect of the present application, step S13 is further:

[0053] S131. Based on the image quality score matrix, an adaptive grid partitioning algorithm is used for each image in the original image sequence to obtain an image region set; based on the image region set, a brightness mean of each region is calculated to obtain a regional brightness matrix; distribution characteristics of the regional brightness matrix are analyzed to obtain a brightness distribution map; Gaussian smoothing is applied to the brightness distribution map to obtain a smoothed brightness map; the difference between the smoothed brightness map and a preset standard brightness is calculated to obtain a brightness deviation map; based on the brightness deviation map and the image quality score matrix, a local mapping matrix is ​​generated;

[0054] S132, based on the local mapping matrix and the image region set, calculate the edge strength of each region to obtain an edge strength map; based on the edge strength map, extract the regional texture features to obtain a texture feature map; fuse the edge strength map and the texture feature map to obtain a regional importance map; normalize the regional importance map to obtain a weight mask matrix; perform element-wise multiplication of the local mapping matrix and the weight mask matrix to obtain a weighted mapping matrix; perform spatial domain filtering on the weighted mapping matrix to obtain a smoothing mapping matrix;

[0055] S133. Based on the smooth mapping matrix, brightness mapping is performed on each image in the original image sequence to obtain a mapped image sequence; the local contrast of the mapped image sequence is calculated to obtain a contrast map; based on the contrast map, the degree of image detail preservation is analyzed to obtain a detail preservation map; the contrast map and the detail preservation map are fused to obtain a quality assessment map; based on the quality assessment map, local fine-tuning is performed on the mapped image sequence to output a compensated image sequence.

[0056] This embodiment effectively solves the problem of uneven exposure that is common in power line inspection images by constructing an adaptive grid division and a multi-level exposure compensation mechanism. The image is segmented into regions through an adaptive grid division algorithm, and the segmentation granularity is automatically adjusted according to the complexity of the image content, avoiding the boundary effect that may occur in fixed grid division; by calculating the local brightness map and the weight mask, differentiated processing of different regions is achieved. In particular, in the process of generating the weight mask, by fusing the edge strength and texture features, the details of key target areas such as power lines are ensured to be maintained, while avoiding image distortion caused by overcompensation. In the final exposure compensation stage, the adaptive adjustment of the compensation intensity is achieved through the local contrast and detail retention evaluation, ensuring the overall visual quality of the image. This embodiment not only effectively balances the light and dark contrast of the image, but also maintains the detailed features of the power line and its surrounding environment, and improves the accuracy of subsequent feature extraction and matching. In practical applications, it can effectively process images under complex lighting conditions such as backlight and strong light, providing high-quality image data support for power line detection.

[0057] like Figure 3 As shown, according to one aspect of the present application, step S2 is further:

[0058] S21. Based on the optimal image combination and the image quality score, the scale parameter of each level is calculated; based on the scale parameter of each level, the image in the optimal image combination is subjected to tensor transformation and weight kernel function calculation to generate a single-layer feature tensor; the single-layer feature tensors of all levels are combined to generate a multi-scale feature tensor set;

[0059] S22, based on the multi-scale feature tensor set, calculating the anisotropic intensity value of each feature point to obtain an anisotropic intensity vector; based on the anisotropic intensity vector, generating a directional gradient histogram to obtain directional gradient information; based on the directional gradient information, calculating context correlation to obtain context coding data; combining the anisotropic intensity vector, the directional gradient information and the context coding data to generate a feature descriptor set;

[0060] S23, based on the feature descriptor set and the image association matrix, calculate the neighborhood constraints of the feature points to obtain neighborhood constraint data; based on the neighborhood constraint data, calculate the matching probability between the feature points; based on the matching probability and a preset threshold, generate an initial feature matching pair set;

[0061] S24, based on the initial feature matching pair set and the image association matrix, calculate the local structure consistency score, evaluate the global topological consistency, analyze the temporal continuity, and generate a multi-level verification score; based on the multi-level verification score, screen the initial feature matching pair set and output a reliable matching set;

[0062] S25. Based on the reliable matching set and the feature descriptor set, analyze the feature space distribution, identify the feature sparse area, and generate supplementary feature points in the feature sparse area; calculate the descriptor for the supplementary feature point, merge the descriptor with the reliable matching set, and output the enhanced feature matching set.

[0063] In one embodiment of the present application, the calculation of the anisotropic strength value includes the following steps: calculating the eigenvalue ratio of the feature point covariance matrix AI = (λ1-λ2) / (λ1+λ2); wherein AI is the anisotropic strength value; λ1 and λ2 are the eigenvalues ​​of the covariance matrix, and λ1≥λ2; the covariance matrix C = [∑Ix 2 ,∑IxIy; ∑IxIy,∑Iy 2 ]; Ix, Iy are image gradients.

[0064] The generation of the histogram of oriented gradients includes the following steps: calculating the histogram of oriented gradients HOG = ∑w(x, y)·δ(θ(x, y) - θbin); where HOG is the histogram of oriented gradients; w(x, y) is the gradient amplitude at the position (x, y); θ(x, y) is the gradient direction; θbin is the center angle of the directional bin; δ is the Dirac function.

[0065] The context relevance calculation includes the following steps: Calculate the attention weight matrix A(i, j) = softmax(Q·K T / sqrt(d))·V; where A(i, j) is the correlation between feature points i and j; Q is the query matrix; K is the key matrix; V is the value matrix; d is the feature dimension; and softmax is the normalization function.

[0066] The calculation of the feature point neighborhood constraint includes the following steps: Calculate the neighborhood consistency score NC = ∑exp(-||di,j -dk,l|| 2 / (2σ 2 ))·δ(||pi - pk|| < ​​r); where NC is the neighborhood consistency score; di,j is the descriptor distance between feature points i and j; dk,l is the descriptor distance between corresponding matching points k and l; pi,pk are the spatial positions of feature points; r is the neighborhood radius; σ is the Gaussian kernel parameter; δ is the indicator function.

[0067] The matching probability calculation model includes the following steps: Calculate the matching probability P(mi, j) = exp(-λ1·d(fi, fj) 2 -λ2·||xi - xj|| 2 ) / (Z·(1 + exp(-γ·NC))); where P(mi, j) is the matching probability of feature point pair (i, j); d(fi, fj) is the descriptor distance; xi, xj are the feature point positions; NC is the neighborhood consistency score; λ1, λ2, γ are weight coefficients; and Z is the normalization factor.

[0068] The calculation of multi-level verification score includes the following steps: calculating the verification score VS = w1·LS + w2·GS +w3·TS; where VS is the multi-level verification score; LS is the local structure consistency score, LS=∑||Ri,j - Rk,l||F / n; GS is the global topological consistency score, GS = exp(-||Fi - Fj|| 2 / σ 2 );TS is the temporal continuity score, TS = 1 -|vi,t - vi,t-1| / vmax;Ri,j is the local relative transformation matrix; Fi, Fj are global feature descriptors; vi,t is the motion speed at time t; w1, w2, w3 are weight coefficients, and vmax is the maximum motion speed.

[0069] The feature space distribution analysis includes the following steps: Calculate the feature distribution density D(x, y) = ∑K(||pi - (x, y)|| / h) / (n·h 2); where D(x, y) is the density value at position (x, y); K is the kernel function, K(u) = (1 - u 2 )·I(|u|≤ 1); pi is the position of the feature point; h is the bandwidth parameter; n is the total number of feature points; I is the indicator function.

[0070] In another embodiment of the present application, the timing constraint TC is calculated =∑||M(xt)-x(t+1)|| 2 ·exp(-||ft-f(t+1)|| 2 / σ 2 ), where TC is the timing constraint value; M is the motion prediction function; xt is the state at time t; ft is the feature at time t; σ is the feature similarity parameter.

[0071] This embodiment realizes accurate identification and reliable matching of power line features by constructing a multi-level feature extraction and matching optimization framework. First, multi-scale features are extracted through a multi-resolution feature pyramid, which can capture the characteristic performance of power lines at different scales; in the feature description link, a multi-head attention network and a context-aware mechanism are used to effectively extract key features such as the directionality and continuity of power lines. In particular, in the feature matching process, by constructing a multi-level verification mechanism and a spatial distribution analysis framework, high-precision screening of feature matching is achieved. At the same time, feature enhancement networks are used to supplement feature points in sparse areas to ensure the uniformity and integrity of feature distribution. This embodiment not only improves the accuracy of power line feature extraction, but also enhances the robustness of feature matching. In actual inspection tasks, it can effectively handle the difficulties in feature extraction caused by factors such as illumination changes and partial occlusion, and provide high-quality feature matching data support for subsequent three-dimensional reconstruction.

[0072] According to one aspect of the present application, step S22 is further:

[0073] S221. Based on the multi-scale feature tensor set, construct a covariance matrix for each scale layer to obtain a covariance tensor set; calculate the eigenvalues ​​of the covariance tensor set to obtain an eigenvalue sequence; based on the eigenvalue sequence, calculate the anisotropy metric to obtain an anisotropy intensity vector; based on the anisotropy intensity vector, use the self-attention mechanism to calculate the correlation between feature points to obtain an attention weight map; fuse the anisotropy intensity vector with the attention weight map to obtain an enhanced feature vector;

[0074] S222, based on the enhanced feature vector, construct a local coordinate system to obtain a local coordinate map; calculate the directional gradient in the local coordinate map to obtain a gradient amplitude map and a gradient direction map; perform direction quantization on the gradient direction map to obtain a directional histogram; calculate the statistical characteristics of the gradient amplitude map to obtain an amplitude feature vector; combine the directional histogram and the amplitude feature vector to obtain directional gradient information;

[0075] S223. Based on the directional gradient information and enhanced feature vectors, a multi-head attention network is constructed to obtain an attention feature map; based on the attention feature map, the semantic relevance of the feature point neighborhood is calculated to obtain a semantic association map; the attention feature map and the semantic association map are fused to obtain contextual coding data; the anisotropic intensity vector, directional gradient information and contextual coding data are feature concatenated, and dimension reduction is performed through a multi-layer perceptron to output a feature descriptor set.

[0076] This embodiment achieves accurate description and robust expression of power line features by designing a multi-head attention feature extraction network and a context-aware feature description mechanism. First, by calculating the anisotropic strength of the feature points, the directional characteristics of the power lines are accurately captured; in the feature extraction process, the self-attention mechanism is used to calculate the correlation between the feature points, which can effectively suppress background interference and highlight the main features of the power lines. In the process of extracting directional gradient information, by constructing a local adaptive coordinate system, the rotation invariance of the feature description is improved, so that the system can accurately process power line images at different angles. In particular, in the context encoding link, the multi-head attention network and semantic correlation analysis are used to achieve the effective fusion of multi-scale expression of features and context information, and improve the discriminability and robustness of feature description. This embodiment not only improves the extraction accuracy of power line features, but also enhances the system's resistance to interference factors such as occlusion and illumination changes. In the actual power line detection task, the key features of the power lines can be accurately extracted and described, providing reliable feature data support for subsequent matching and three-dimensional reconstruction.

[0077] According to one aspect of the present application, step S25 is further:

[0078] S251. Obtain a reliable matching set and a feature descriptor set, calculate the spatial distribution density of feature points to obtain a density distribution map, segment the density distribution map using an adaptive threshold to obtain a region segmentation map, extract low-density regions in the region segmentation map to obtain a sparse region marker map; perform morphological processing on the sparse region marker map to obtain an optimized region map, calculate the geometric features of each region in the optimized region map to obtain a region feature vector.

[0079] S252. Based on the regional feature vector and the sparse region labeling map, a multi-layer attention feature generation network is constructed to obtain an attention feature map, and candidate feature points are generated under the guidance of the attention feature map to obtain a candidate point set; the local response of each point in the candidate point set is calculated to obtain a response intensity map, and the feature points are screened according to the response intensity map to obtain a supplementary feature point set; the local image blocks of the supplementary feature point set are extracted to obtain a feature image block set.

[0080] S253. Apply a multi-scale feature extraction network to the feature image block set to obtain a multi-scale feature map, calculate the local descriptor of the multi-scale feature map to obtain a local description subset; use a contextual attention module to process the local description subset to obtain a contextual feature map, fuse the local description subset and the contextual feature map to obtain a supplementary description subset; perform feature fusion on the supplementary description subset and the reliable matching set, and output an enhanced feature matching set.

[0081] In one embodiment of the present application, the generation of multi-layer attention features includes the following steps: calculating the attention feature A = ∑Heads(Q, K, V) W; where A is the attention feature; Heads is the multi-head attention function, Heads = Concat(head1, ..., headh); headi = Attention(QWi Q , KWi K , VWi V );Attention(Q,K,V) =softmax(QK T / sqrt(dk))V; Wi is the learnable parameter matrix, Q is the query matrix, K is the key matrix, V is the value matrix, W is the learnable parameter matrix, Concat is the concatenation operation of the outputs of multiple heads, Attention is the core calculation formula of the attention mechanism, and dk is the dimension of the key matrix K.

[0082] The specific process of feature fusion is as follows: calculate the fused feature F = ∑αk·Φk(fk)·Wk(x, y); where F is the fused feature; Φk is the feature transformation function of the k-th layer; fk is the feature map of the k-th layer; Wk(x, y) is the spatial attention weight, Wk = softmax(θ(fk)); αk is the scale weight, which is obtained through network learning.

[0083] This embodiment effectively solves the problem of uneven distribution of power line features by constructing feature spatial distribution analysis and multi-layer attention feature generation network. First, by calculating the spatial distribution density of feature points and adaptive threshold segmentation, the feature sparse area is accurately identified; in the feature supplementation process, a multi-layer attention network is used to generate candidate feature points, which can adaptively generate supplementary points that conform to the distribution law of power line features according to contextual information. Especially in the feature verification stage, the quality and reliability of the supplementary features are ensured by the multi-scale feature extraction network and the contextual attention module. The system realizes the effective screening of supplementary features by analyzing the matching of local response intensity and feature descriptors, and avoids the introduction of noise feature points. This embodiment not only fills the blank areas of feature distribution, but also maintains the continuity and consistency of features, and improves the integrity and accuracy of subsequent feature matching.

[0084] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0085] S31. Based on the enhanced feature matching set and the pre-stored camera parameter matrix, the projection error value is calculated, and the time series smoothing constraint, including the speed constraint term and the acceleration constraint term, is generated; based on the pre-stored GPS raw data, the GPS constraint term is constructed; based on the time series smoothing constraint and the GPS constraint term, the camera position and attitude are calculated, motion prediction is performed, and the initialization posture sequence is output;

[0086] Calculating the projection error value includes calculating the projection coordinates of the feature points based on the reprojection model, calculating the Euclidean distance from the actual coordinates, and generating a projection error matrix; generating the time series smoothing constraint includes analyzing the motion continuity between adjacent frames, constructing the speed constraint term, and generating the acceleration constraint term; constructing the GPS constraint term includes analyzing the GPS signal quality, generating the position weight matrix, and constructing the GPS error term;

[0087] S32, based on the initialization pose sequence and the enhanced feature matching set, calculate the reprojection error to obtain reprojection error data; based on the reprojection error data, evaluate the structural consistency to obtain structural constraint data; based on the structural constraint data, analyze the scene continuity to obtain continuity constraint data; based on the reprojection error data, the structural constraint data and the continuity constraint data, perform optimization calculation, and output the optimized camera pose;

[0088] S33, based on the optimized camera pose and compensated image sequence, divide the local image blocks, calculate the local depth probability, and generate a depth estimation probability distribution; based on the depth estimation probability distribution, perform depth estimation calculation on each scale level, combine multi-scale results, and output a multi-scale depth map set;

[0089] S34, based on the multi-scale depth map set and the optimized camera pose, calculate the depth data item to obtain depth consistency data; based on the depth consistency data, analyze the depth gradient to obtain gradient constraint data; based on the gradient constraint data, evaluate the surface smoothness to obtain smoothness constraint data; combine the depth consistency data, the gradient constraint data and the smoothness constraint data for optimization, and output the optimized unified depth map;

[0090] S35. Based on the optimized unified depth map and the optimized camera pose, calculate the reconstruction completeness score, evaluate the geometric consistency and scale accuracy, and generate a reconstruction quality score; based on the reconstruction quality score, calculate the spatial distribution and obtain a quality distribution map.

[0091] In one embodiment of the present application, the reprojection error calculation includes the following steps: calculating the reprojection error RE = ∑||π(K[R|t]X) - x|| 2w(x); where RE is the reprojection error; π is the projection function; K is the camera intrinsic parameter matrix; [R|t] is the rotation and translation matrix; X is the 3D point coordinate; x is the corresponding image coordinate; w(x) is the feature point reliability weight.

[0092] The structural consistency assessment includes the following steps: Calculate the structural similarity SS = ∑||Si - Sj|| 2 ·exp(-di, j / σ) / (∑exp(-di, j / σ)); where SS is the structural similarity; Si, Sj are local structure descriptors; di, j are the distances between feature points; σ is the distance attenuation coefficient.

[0093] Calculate the depth probability P(d|I) = softmax(Φ(fθ(I),d))·∏P(d|Ii); where P(d|I) is the probability distribution of depth d under the given image I; Φ is the deep network feature extraction function; fθ is a neural network with parameter θ; P(d|Ii) is the depth probability predicted for each view; ∏ represents probability multiplication.

[0094] Calculate the consistency energy function E(D) = ∑(D - Di) 2 ·wi +λ·∑|grad D|·exp(-β·|grad I|); where E(D) is the energy function of the depth map D; Di is the initial depth estimate; wi is the depth confidence weight; gradD is the depth gradient; grad I is the image gradient; λ and β are smoothing parameters.

[0095] The specific process of optimization calculation is as follows: construct the total error energy function E = λ1·Er + λ2·Es + λ3·Ec; where Er is the reprojection error, Er = ∑||π(KTP) - p|| 2 , π is the projection function, K is the camera intrinsic parameter matrix, T is the pose transformation matrix, P is the three-dimensional point coordinate, p is the corresponding two-dimensional image coordinate; Es is the structural consistency error, Es = ∑||S(Pi) - S(Pj)|| 2 , S(P) is the local structure descriptor of point P; Ec is the continuity constraint error, Ec =∑||T(t) - T(t-1)|| 2 , T(t) is the pose matrix at time t; λ1, λ2, λ3 are weight coefficients, which are dynamically adjusted according to the scene characteristics through an adaptive algorithm; the LM algorithm is used for optimization and the pose parameters are iteratively updated until convergence.

[0096] The smoothness constraint function includes the following steps: calculating the smoothness constraint SC = ∑∑wij·ρ(di - dj); where SC is the smoothness constraint value; wij is the weight between pixels i and j, wij = exp(-α·||Ii - Ij|| 2 ), di and dj are the depth values ​​of pixels i and j, ρ is the Huber loss function, Ii and Ij are the color values ​​of pixels i and j, and α is the color similarity parameter.

[0097] This embodiment achieves high-precision reconstruction of the three-dimensional structure of power lines by designing a multi-constraint joint optimization framework and a depth estimation network. First, an accurate initialization pose estimation is established through the fusion optimization of GPS data and visual features; in the multi-view geometry optimization process, the triple constraints of reprojection error, structural consistency and scene continuity are used to improve the accuracy of camera pose estimation. Especially in the depth estimation link, the depth discontinuity and occlusion problems in the power line scene are effectively solved through multi-scale depth map generation and consistency optimization. By establishing a complete reconstruction quality evaluation system, the reliability verification of the reconstruction results is achieved. This embodiment not only improves the accuracy and completeness of the reconstruction, but also enhances the system's adaptability to complex scenes. In practical applications, the three-dimensional structure of power lines in various complex environments can be accurately reconstructed, laying a solid foundation for subsequent state monitoring and risk assessment.

[0098] According to one aspect of the present application, step S32 is further:

[0099] S321, based on the initialization pose sequence and the enhanced feature matching set, construct a projection model to obtain a projection matrix set; based on the projection matrix set, calculate the reprojection coordinates of the feature points to obtain a projection coordinate set; calculate the Euclidean distance between the projection coordinate set and the actual coordinates to obtain a distance error map; analyze the statistical characteristics of the distance error map to obtain reprojection error data; based on the reprojection error data, construct an error weight matrix to obtain an error weight map;

[0100] S322, based on the error weight map and the enhanced feature matching set, construct a scene structure map to obtain a structure atlas; analyze the local rigidity of the structure atlas to obtain a rigid constraint map; based on the rigid constraint map, calculate the geometric consistency between feature points to obtain a geometric constraint map; fuse the rigid constraint map and the geometric constraint map to obtain structural constraint data; based on the structural constraint data, calculate the constraint weights to obtain a structural weight matrix;

[0101] S323. Analyze the temporal changes of the initialization pose sequence to obtain a motion trajectory graph; based on the motion trajectory graph and the structural weight matrix, calculate the motion continuity between adjacent frames to obtain a continuity score graph; based on the continuity score graph, evaluate the smoothness of the scene changes to obtain a smoothness graph; fuse the continuity score graph and the smoothness graph to obtain continuity constraint data; substitute the reprojection error data, structural constraint data and continuity constraint data into the optimization solver, and output the optimized camera pose and three-dimensional point cloud data.

[0102] This embodiment achieves high-precision optimization of camera pose and three-dimensional point cloud by constructing a multi-constraint joint optimization mechanism and a scene structure consistency analysis framework. First, a reprojection error evaluation system is established through projection model and Euclidean distance calculation. By analyzing the deviation between the projection coordinates and the actual coordinates, the accuracy of pose estimation is accurately quantified; in the process of structural constraint construction, local rigidity analysis and geometric consistency evaluation are used to effectively maintain the structural integrity of the scene and avoid structural deformation caused by pose optimization. Especially in terms of continuity constraints, a complete motion continuity evaluation mechanism is established by analyzing the timing characteristics of the camera motion trajectory and the smoothness of scene changes, ensuring the timing consistency of the pose estimation results. This embodiment not only improves the accuracy of pose estimation, but also ensures the quality of three-dimensional point cloud reconstruction, and shows excellent robustness when processing complex power line scenes. In actual inspection tasks, the three-dimensional structure of the power line can be accurately reconstructed, providing reliable spatial information support for subsequent defect detection and risk assessment.

[0103] According to one aspect of the present application, step S34 is further:

[0104] S341, input the multi-scale depth map set and the optimized camera pose, calculate the depth consistency between adjacent views to obtain the view Figure 1 The complementarity of multi-scale depth data is analyzed to obtain a scale complementary map, and the fusion view Figure 1 The consistency map and the scale complementary map are combined to obtain the consistency weight matrix; the depth data is weightedly fused based on the consistency weight matrix to obtain the depth consistency data.

[0105] S342. Calculate the spatial gradient of the multi-scale depth map set to obtain a gradient amplitude map, analyze the gradient direction distribution to obtain a gradient direction map, calculate the local continuity of the gradient to obtain a continuity score map; construct a gradient constraint model by combining the gradient amplitude map, the gradient direction map and the continuity score map, and output gradient constraint data; generate edge preservation weights based on the gradient constraint data to obtain an edge weight map.

[0106] S343. Calculate local surface features based on depth consistency data to obtain a surface feature map, analyze surface normal distribution to obtain a normal distribution map, and evaluate local smoothness to obtain a local smooth map; fuse the surface feature map, normal distribution map and local smooth map to obtain smoothness constraint data; input the depth consistency data, gradient constraint data and smoothness constraint data into the optimization solver to generate an optimized unified depth map.

[0107] This embodiment achieves high-quality unified depth map reconstruction by constructing a multi-scale depth consistency optimization framework and an edge-preserving smoothness constraint mechanism. First, by analyzing the depth consistency between adjacent views and the complementarity of multi-scale data, a complete depth fusion evaluation system is established; in the gradient constraint construction process, spatial gradient analysis and local continuity evaluation are used to effectively maintain the edge features of the depth map and avoid excessive smoothing during the depth optimization process. Especially in terms of surface smoothness constraints, an adaptive smoothness constraint model is constructed by analyzing local surface features and normal distribution to ensure that the depth map has good continuity while maintaining details. This embodiment not only improves the accuracy of depth estimation, but also enhances the system's robustness to occlusion and noise. In the actual task of three-dimensional reconstruction of power lines, the spatial position and morphological characteristics of power lines can be accurately reconstructed, providing accurate depth information support for subsequent line monitoring and risk assessment.

[0108] like Figure 5 As shown, according to one aspect of the present application, step S4 is further:

[0109] S41. Based on the compensated image sequence and the quality distribution map, a multi-scale directional filter group is constructed to calculate the point cloud projection consistency score; based on the point cloud projection consistency score, an anisotropic Gaussian filter is used to calculate the local directionality score and generate a directional response tensor; based on the directional response tensor, edge enhancement processing is performed to obtain an enhanced response map; directionality analysis is performed on the enhanced response map, and a power line candidate area map and a directional distribution map are output;

[0110] S42. Based on the power line candidate area map and the direction distribution map, calculate the local direction consistency, evaluate the line segment ductility, and generate the linear feature weight; based on the linear feature weight, calculate the curvature change rate, evaluate the segment smoothness, and generate the continuity score; based on the continuity score, analyze the line segment intersection state, calculate the parallelism index, and generate the topological relationship matrix; combine the linear feature weight, the continuity score and the topological relationship matrix to perform segmentation optimization and output the segmented line segment set;

[0111] S43. Based on the segmented line segment set and the optimized camera pose, calculate the direction similarity, evaluate the length ratio consistency, and generate a geometric similarity score; based on the geometric similarity score, calculate the relative position relationship, evaluate the spatial distribution pattern, and generate a topological constraint matrix; based on the topological constraint matrix, analyze the surrounding feature distribution, calculate the regional descriptor, and generate a context matching score; based on the geometric similarity score, the topological constraint matrix and the context matching score, filter the matching pairs, and output the line segment matching pair set;

[0112] S44, based on the line segment matching pair set, the direction distribution map and the optimized unified depth map, evaluate the shape features and generate basic model parameters; based on the basic model parameters, calculate the local deformation features, evaluate the elastic deformation, and generate the deformation function; based on the deformation function, analyze the measurement error, calculate the uncertainty, and generate the noise model; based on the basic model parameters, the deformation function and the noise model, perform optimization reconstruction and output the three-dimensional model of the power line;

[0113] S45. Based on the three-dimensional model of the power line and the optimized camera posture, the sag change rate is calculated to obtain the sag data; based on the sag data, the lateral offset is measured to obtain the offset data; based on the offset data, the vibration frequency is analyzed to obtain the vibration data; based on the vibration data, the tension value is estimated to obtain the tension data; the sag data, offset data, vibration data and tension data are time-series filtered and weighted, and a report on the dynamic characteristics of the power line is output.

[0114] In one embodiment of the present application, the direction enhancement response calculation includes the following steps: calculating the direction response intensity DR = ∑G(θk)·F(I,θk) / (1 + λ·|▽ 2 I|); where DR is the directional response intensity; G(θk) is the Gaussian weight of direction θk; F(I, θk) is the filter response of direction θk; ▽ 2 I is the Laplace operator response; λ is the smoothing coefficient; I is the input image.

[0115] The topological constraint segmentation model includes the following steps: calculating the segmentation energy E(S) = Ed(S) +αEc(S) +βEt(S); where E(S) is the total energy of the segmentation result S; Ed is the data item, Ed =∑|I(x) -μk| 2 ; Ec is the continuity term, Ec = ∑|sk – s(k+1)|; Et is the topological term, Et = ∑exp(-||di - dj|| 2 / σ 2 )·δ(li ≠ lj); μk is the mean of region k; sk is line segment k; di and dj are feature descriptors; li and lj are labels; α and β are weight coefficients.

[0116] The calculation of line segment geometric similarity includes the following steps: Calculate geometric similarity GS = exp(-w1·θ 2 / σ1 2 -w2·(l1 / l2 - 1) 2 / σ2 2 - w3·d 2 / σ3 2 );GS is the geometric similarity; θ is the direction difference of the line segment; l1 and l2 are the lengths of the line segment; d is the distance between the midpoints of the line segment; w1, w2, w3 are weight coefficients; σ1, σ2, σ3 are scale parameters.

[0117] The calculation of the sag change rate includes the following steps: Calculate the sag change rate CS = (h(t) - h(t-1)) / h(t-1); where CS is the sag change rate; h(t) is the sag value at time t, h(t) = h0 +αL 2 / (8T(t)); h0 is the initial sag; α is the temperature coefficient; L is the span; T(t) is the tension function.

[0118] The vibration frequency analysis model includes the following steps: calculating the vibration characteristic VF = FFT(y(t))·W(ω); where VF is the vibration frequency characteristic; y(t) is the displacement time series; FFT is the fast Fourier transform; W(ω) is the frequency weight function, W(ω) = exp(-|ω-ω0| 2 / σ 2 ), ω0 is the fundamental frequency, and σ is the bandwidth parameter.

[0119] The tension estimation function includes the following steps: Calculate the tension value T = mgL 2 / (8h)·(1 + kv 2 / g 2 ) where T is the estimated value of tension; m is the mass per unit length; g is the acceleration of gravity; L is the gear spacing; h is the sag value; v is the lateral vibration velocity; k is the correction coefficient, k = exp(-βT / T0); β is the material characteristic parameter; and T0 is the reference tension.

[0120] This embodiment realizes the accurate identification and status assessment of power lines by constructing a direction-enhanced power line detection network and a dynamic characteristics analysis framework. First, the spatial structural characteristics of the power lines are accurately extracted through direction-enhanced response and topological constraint segmentation; in the multi-view matching process, geometric similarity and topological relationship analysis are used to establish a reliable segment matching mechanism. Especially in the three-dimensional reconstruction and dynamic characteristics analysis links, the accurate description of the morphology and motion state of the power lines is achieved through multi-head attention networks and spatiotemporal characteristics analysis. This embodiment not only improves the accuracy of power line detection, but also can effectively identify the abnormal state of the line. In actual inspection tasks, it can accurately monitor the sag changes, vibration characteristics and tension state of the power lines, providing comprehensive technical support for safe operation and maintenance of the lines.

[0121] In another embodiment of the present application, a power line dynamic characteristic description DM = {C(t), O(t), V(t), T(t)} is constructed; wherein C(t) is the sag function, C(t) = h0 + αL 2 / (8T(t)), h0 is the initial sag height, α is the temperature coefficient, L is the span, and T(t) is the tension; O(t) is the offset function, O(t) = A·sin(ωt +φ), A is the amplitude, ω is the angular frequency, and φ is the phase; V(t) is the vibration characteristic, V(t) = FFT(O(t)), and the spectrum characteristics are obtained by fast Fourier transform; T(t) is the tension function, T(t) = mg·L 2 / (8h), m is the mass per unit length, g is the gravitational acceleration, and h is the real-time sag value; each parameter is obtained by least squares fitting.

[0122] According to one aspect of the present application, step S44 is further:

[0123] S441. Input the line segment matching pair set and the direction distribution map, build a multi-head attention network to process the line segment features to obtain the attention feature map, calculate the geometric properties of the line segments to obtain the geometric feature map, analyze the topological relationship between the line segments to obtain the topological relationship map; fuse the attention feature map, the geometric feature map and the topological relationship map, extract the basic shape parameters of the power lines, and output the basic shape data.

[0124] S442. Based on the basic shape data and the optimized unified depth map, the elastic deformation characteristics of the line segment are calculated to obtain an elastic deformation map, the influence of environmental factors is analyzed to obtain an environmental impact map, and the load distribution is evaluated to obtain a load distribution map; a deformation model is established by combining the elastic deformation map, the environmental impact map and the load distribution map, and the deformation function data is output.

[0125] S443. Analyze the uncertainty of the measurement data to obtain an uncertainty map, evaluate the system error distribution to obtain an error distribution map, calculate the random noise characteristics to obtain a noise characteristic map; fuse the uncertainty map, error distribution map and noise characteristic map to obtain noise parameters; substitute the basic shape data, deformation function data and noise parameters into the optimizer to output the three-dimensional model of the power line.

[0126] This embodiment realizes the accurate reconstruction of the three-dimensional model of the power line by constructing a multi-head attention feature fusion network and a deformation parameter adaptive estimation framework. First, the line segment features and topological relationships are analyzed through the multi-head attention network, which can accurately capture the spatial structural characteristics of the power line; in the deformation parameter estimation process, elastic deformation analysis and environmental factor impact assessment are used to establish a complete deformation model, which effectively considers the influence of external factors such as temperature changes and wind loads on the shape of the power line. Especially in terms of noise processing, a reliable noise model is constructed by analyzing the uncertainty of the measurement data and the distribution of system errors to ensure the accuracy and reliability of the reconstruction results. This embodiment not only realizes the accurate reconstruction of the geometric shape of the power line, but also can reflect its dynamic change characteristics under different environmental conditions. In practical applications, it can accurately reconstruct power lines of different types and in different states, providing reliable three-dimensional model support for line operation status monitoring and fault diagnosis.

[0127] According to one aspect of the present application, step S45 is further:

[0128] S451. Obtain a three-dimensional model of the power line, construct a multi-scale tensor network to extract shape features to obtain a shape feature map, calculate the sag value of each segment to obtain a sag sequence, analyze the sag change trend to obtain a change trend map; establish a deformation mapping relationship based on the shape feature map and the sag sequence to obtain a deformation mapping matrix; calculate the dynamic characteristics in combination with the change trend map and the deformation mapping matrix, and output the sag data.

[0129] S452. Based on the three-dimensional model of power lines and the optimized camera posture, a spatiotemporal attention network is constructed to analyze the motion features to obtain a motion feature map, the horizontal displacement is calculated to obtain a displacement sequence, and the vertical deviation is evaluated to obtain a deviation sequence; the motion feature map, displacement sequence and deviation sequence are integrated to construct a motion model to obtain a motion state map; the timing characteristics of the motion state map are analyzed, and the offset data and vibration data are output.

[0130] S453, input sag data, offset data and vibration data, construct a multi-constraint state estimator to obtain a state estimation diagram, calculate the material stress distribution to obtain a stress distribution diagram, evaluate the external load to obtain a load distribution diagram; integrate the state estimation diagram, stress distribution diagram and load distribution diagram to perform mechanical analysis to obtain tension data; perform time series filtering on the sag data, offset data, vibration data and tension data, and output a power line dynamic characteristics report.

[0131] This embodiment realizes the accurate evaluation of the dynamic characteristics of power lines by constructing a multi-scale tensor feature extraction network and a spatiotemporal attention motion analysis framework. First, the shape features and sag change features are extracted through a multi-scale tensor network, which can fully capture the morphological changes of power lines under different load conditions; in the process of motion characteristic analysis, the spatiotemporal attention network is used to analyze the displacement and vibration features, and a complete motion state evaluation system is established to effectively identify the abnormal swing and vibration of power lines. In particular, in the tension estimation link, the multi-constrained state estimator and mechanical analysis model are used to accurately evaluate the stress state of the power line, comprehensively consider the influence of multiple factors such as material properties, environmental loads and installation status, and improve the accuracy of tension estimation. This embodiment can not only monitor the operating status of power lines in real time, but also predict potential safety hazards. In actual line inspection tasks, safety risks caused by abnormal tension, excessive vibration, etc. can be discovered in a timely manner, providing reliable data support for line maintenance decisions.

[0132] like Figure 6 As shown, according to one aspect of the present application, step S5 is further:

[0133] S51. Based on the compensated image sequence, the optimized unified depth map and the power line three-dimensional model, evaluate the height stratification and generate the spatial obstacle features; based on the spatial obstacle features, calculate the motion features, evaluate the change mode and generate the dynamic obstacle features; based on the dynamic obstacle features, analyze the geometric shape, calculate the texture features and generate the obstacle morphological features; combine the spatial obstacle features, the dynamic obstacle features and the obstacle morphological features to perform obstacle recognition and segmentation, and output the obstacle feature data;

[0134] S52. Based on the obstacle feature data and the power line dynamic characteristic report, calculate the shortest distance between the obstacle and the power line to obtain vertical distance data; based on the vertical distance data, analyze the obstacle movement trend to obtain dynamic approach data; based on the dynamic approach data, evaluate the environmental impact to obtain environmental impact data; based on the vertical distance data, the dynamic approach data and the environmental impact data, construct an obstacle distance evaluation model and output the obstacle distance data;

[0135] S53. Based on the obstacle feature data, obstacle distance data and the three-dimensional model of the power line, the static distance risk is calculated to obtain the static threat degree; based on the static threat degree, the dynamic collision risk is evaluated to obtain the dynamic threat degree; based on the dynamic threat degree, the obstacle growth trend is analyzed to obtain the development threat degree; based on the development threat degree, the environmental impact is calculated to obtain the environmental threat degree; the static threat degree, the dynamic threat degree, the development threat degree and the environmental threat degree are weighted and combined to output an obstacle risk level table;

[0136] S54. Based on the obstacle risk level table, obstacle feature data and power line dynamic characteristic report, calculate obstacle type weights to obtain an obstacle weight matrix; based on the obstacle weight matrix, evaluate weather impact to obtain weather impact factors; based on the obstacle risk level table, obstacle weight matrix and weather impact factors, perform obstacle warning classification and output an obstacle warning report;

[0137] S55. Based on the obstacle warning report, obstacle risk level table and pre-stored historical inspection data, calculate the obstacle monitoring coverage to obtain monitoring coverage data; based on the monitoring coverage data, evaluate key areas to obtain regional priorities; based on regional priorities, analyze monitoring frequencies to obtain frequency optimization data; based on the monitoring coverage data, regional priorities and frequency optimization data, perform optimization calculations to generate an obstacle monitoring plan that includes obstacle key monitoring paths, sampling frequencies, priority processing areas and monitoring cycles.

[0138] In one embodiment of the present application, the static threat level calculation includes the following steps: Calculate the static threat level ST = (d0 / d) α ·exp(-v·t / τ); where ST is the static threat level; d0 is the safety threshold distance; d is the actual distance; α is the distance decay exponent; v is the relative speed; t is the observation time; and τ is the time constant.

[0139] The calculation of dynamic threat degree includes the following steps: calculating dynamic threat degree DT = ∑wi·[vi(t) / v0i]·exp(-|θi(t) - θp(t)| / σθ); where DT is the dynamic threat degree; vi(t) is the movement speed of obstacle i at time t; v0i is the corresponding speed threshold; θi(t) is the movement direction; θp(t) is the predicted direction; wi is the obstacle type weight; σθ is the direction similarity parameter.

[0140] The calculation of environmental threat degree includes the following steps: Calculate environmental threat degree ET = w1·R(t) + w2·V(t) +w3·G(t); where ET is environmental threat degree; R(t) is the rainfall influencing factor, R(t) = (r(t) / r0) α ; V(t) is the visibility influencing factor, V(t) = exp(-v0 / v(t)); G(t) is the terrain influencing factor, G(t) = 1 + kg·∑|▽h(x, y)| / A; r(t) is the rainfall; v(t) is the visibility; h(x, y) is the terrain elevation; w1, w2, w3 are weight coefficients; α and kg are adjustment parameters.

[0141] The comprehensive risk level assessment model includes the following steps: calculating the comprehensive risk level R = f(ST, DT, ET)·(1+ γ·dR / dt); where R is the comprehensive risk level; f(ST, DT, ET) is the risk fusion function, f = (w1·ST p +w2 DT p + w3 ET p ) 1 / p ; dR / dt is the risk change rate; γ is the time series influence coefficient; p is the nonlinear fusion parameter; w1, w2, w3 are adaptive weights, which are dynamically adjusted by the scene type and environmental conditions.

[0142] This embodiment realizes intelligent monitoring and risk management of the environment around power lines by establishing a multi-dimensional obstacle analysis and risk warning system. First, various potential obstacles are accurately identified through scene segmentation and feature extraction; in the risk assessment process, a multi-layer position encoding network and temporal attention mechanism are used to achieve accurate quantification of static and dynamic threats. Especially in the early warning strategy formulation link, a complete risk level assessment system and monitoring scheme optimization mechanism are established through comprehensive analysis of environmental impacts and development trends. This embodiment can not only detect current safety hazards in a timely manner, but also predict potential risk development trends. In practical applications, it can accurately assess the risks of different types of obstacles and automatically generate targeted monitoring plans, thereby improving the efficiency and reliability of line safety management.

[0143] In another embodiment of the present application, the comprehensive risk level R = w1·Rs + w2·Rd + w3·Rg+ w4·Re is calculated; wherein Rs is the static threat level, Rs = d0 / d(t), d0 is the safety threshold distance, and d(t) is the real-time measurement distance; Rd is the dynamic threat level, Rd = v(t) / v0, v(t) is the relative movement speed of the obstacle, and v0 is the speed threshold; Rg is the development threat level, Rg = g(t) / g0, g(t) is the growth rate of the obstacle (such as trees), and g0 is the growth rate threshold; Re is the environmental threat level, Re = e(t) / e0, e(t) is the comprehensive impact of environmental factors, and e0 is the environmental impact threshold; w1, w2, w3, and w4 are dynamic weight coefficients, which are adaptively adjusted according to factors such as season and weather to satisfy ∑wi = 1.

[0144] According to one aspect of the present application, step S53 is further:

[0145] S531. Obtain obstacle feature data and obstacle distance data, construct a multi-layer position coding network to analyze the spatial relationship to obtain a spatial relationship diagram, calculate the minimum safe distance to obtain a safe distance diagram, and evaluate the degree of spatial interference to obtain an interference degree diagram; integrate the spatial relationship diagram, the safe distance diagram, and the interference degree diagram to establish a risk assessment model, and output the static threat degree; calculate the relative motion characteristics based on the three-dimensional model of the power line to obtain motion feature data.

[0146] S532. Based on the motion feature data and obstacle feature data, a temporal attention network is constructed to analyze the dynamic features to obtain a dynamic feature map, the collision probability is calculated to obtain a collision probability map, and the motion trend is evaluated to obtain a trend prediction map; a dynamic risk model is constructed by combining the dynamic feature map, the collision probability map and the trend prediction map, and the dynamic threat degree and the development threat degree are output.

[0147] S533. Input obstacle feature data and a three-dimensional model of power lines, analyze the impact of meteorological conditions to obtain a meteorological impact map, calculate terrain constraints to obtain a terrain constraint map, and evaluate seasonal changes to obtain a seasonal change map; integrate the meteorological impact map, terrain constraint map, and seasonal change map to construct an environmental impact model, and output the environmental threat level; use an adaptive weight network to perform weighted integration of the static threat level, dynamic threat level, development threat level, and environmental threat level, and output an obstacle risk level table.

[0148] This embodiment achieves accurate quantification of the threat level of obstacles by constructing a multi-layer position coding network and a temporal attention risk assessment framework. First, the spatial relationship between obstacles and power lines is analyzed through a multi-layer position coding network, and a complete static threat assessment system is established; in the dynamic risk analysis process, a temporal attention network is used to predict the movement trend and potential collision risk of obstacles, which can accurately assess the potential threats posed by dynamic obstacles such as tree growth and construction. Especially in terms of environmental impact assessment, a complete environmental impact model is constructed by analyzing meteorological conditions, terrain constraints and seasonal changes, which fully considers the impact of various environmental factors on obstacle risks. This embodiment can not only accurately identify current safety hazards, but also predict potential risk development trends. In practical applications, it is possible to accurately classify different types of obstacles into risk levels, providing a scientific decision-making basis for line safety management and preventive maintenance.

[0149] According to one aspect of the present application, a method for detecting obstacles in a wire assembly based on a monocular camera comprises the following steps:

[0150] S1. Multiple images taken at different positions and angles, combined with GPS and camera intrinsic parameter data, provide high-quality raw data for subsequent feature matching and 3D reconstruction.

[0151] Fly along the vertical wire direction and use a monocular camera to take an image at a certain distance. Collect multiple images (at least 3) of the same scene as processing data. Each image needs to contain the GPS data of the current camera position and the internal parameter matrix information of the current camera.

[0152] S2. Apply advanced feature detection algorithms such as SIFT to stably detect feature points from images, find corresponding points between different images through matching algorithms, and use multi-view geometry principles to calculate the camera's posture changes, laying the foundation for 3D reconstruction.

[0153] Feature detection and matching are performed on multiple images captured by the same camera. The means of feature detection include but are not limited to using feature detectors such as the scale-invariant feature transform (SIFT) operator. This embodiment takes the SIFT detection operator as an example, performs SIFT feature detection on each image, and generates feature points and feature descriptors. The Euclidean distance is used as a measure of the similarity of the feature points, and the Euclidean distance of the key point descriptors in the two images is calculated.

[0154] After feature extraction and matching, a series of matching feature point pairs of the two images are obtained. Based on the series of matching point pairs detected in the two images, the pose transformation rotation matrix R and translation matrix t of the two images can be solved. The basic principle is as follows: a point P in the camera coordinate system C =(x C ,y C , z C ) projected to pixel point p = (u, v) can be expressed as:

[0155] ;

[0156] Where u and v are pixel coordinates respectively;

[0157] Then, the pixel points on the second image can be expressed as follows:

[0158] ;

[0159] After substitution, conversion and normalization, it is expressed as: K -1 p2 = RK -1 p1+t;

[0160] After substituting a series of pixel coordinates, the rotation matrix and translation matrix can be solved, and the camera pose can be further optimized through algorithms such as Bundle Adjustment.

[0161] S3. Combine GPS data to perform scale correction to ensure the accuracy and reliability of 3D reconstruction.

[0162] Since the camera system only retains 2D information, the depth obtained at this time is a normalized data. Obstacle analysis requires 3D coordinate judgment in the actual scene, and the scale correction of the posture is required. The camera posture can be calibrated using the GPS data in the image EXIF ​​to restore the actual position information of the point cloud.

[0163] Assume that the camera pose obtained in the previous step is T cal , the camera position obtained by GPS is T GPS There is a rotation R, a translation matrix t and a scale transformation s between the two postures. R, t and the scale factor s can be solved by minimizing f. f represents the difference between the scale-transformed posture and the actual GPS posture, expressed as: min||sRT cal +tT GPS || F 2 ;

[0164] After taking the derivative of the translation matrix t and setting it equal to 0, we can solve for t:

[0165] ;

[0166] in represents the center of N poses, that is, the average of N poses. Substituting the analytical result of the translation matrix t into the difference f, the difference f can be converted:

[0167] ;

[0168] Furthermore, we differentiate s and set it equal to 0 to solve for the scale factor s:

[0169] ;

[0170] At this time, the minimum transformation of the difference f is f R Find the maximum: .

[0171] Pair Matrix Perform SVD decomposition to solve the rotation matrix R: M = UΣV T , R=VU T .

[0172] By using the above formula, the scale factor is calculated for N poses to obtain accurate 3D reconstruction points.

[0173] S4. Find the matching points of the wire through wire segmentation and epipolar geometry constraints.

[0174] Due to the characteristics of wires, it is difficult to find feature matching points using traditional feature detection algorithms, resulting in general 3D reconstruction that can only reconstruct the scene but cannot match the wire information. Therefore, the wire data is processed separately.

[0175] First, some image segmentation algorithms are used to segment the wires on the image, and the wire segmentation results in the two images are obtained and the matching relationship of the wires is obtained. For the reference image, a series of points on the wires are extracted according to the segmentation map, and the matching points of these points on the other image can be obtained through the epipolar geometry relationship and the wire matching relationship.

[0176] The camera internal parameter is K, and the coordinate of a guide point P in the world coordinate system is X W , the projection coordinates of the point in the two images are p0, p1, and the transformation matrices of the two camera positions are R0, t0 and R1, t1 respectively. According to the expression in step S2, it can be deduced that: z0p0= K(R0 X W +t0); z1p1= K(R1 X W +t1), where z0 and z1 represent the normalized constants of P projection in the two camera coordinate systems. When P is at different depths such as P1, P2, and P3, its image on the second image must be located on the epipolar line. In the extreme case, when z0=0: X W =-R0 T t0, at this time P z0=0 The image formed at camera C1 is: e0=-K(R1R0 T t0+t1); when z0=∞: X W =R0 T K -1 p0, at this time P z0=∞ The image formed at camera C1 is: e1=-K(R1R0 T K -1 p0); the epipolar line can be represented as a straight line connecting points e0 and e1. Furthermore, the expression of the epipolar line is obtained as e0×e1, which can be converted to epipolar line:e0×e1= skew(e0)∙e1 through the antisymmetric matrix, where epipolar line is the epipolar line and skew is the antisymmetric matrix. After obtaining the point of the wire on the first image, the position of p1 is solved through the epipolar line and the wire segmentation mask.

[0177] S5. Triangulate the wire matching points and densely reconstruct the wire to obtain three-dimensional information.

[0178] Due to the existence of error, position X W Cannot be solved exactly, that is, x=PX W and x'=P' X WThe triangulation positioning problem of the wire point is transformed into an optimization problem, namely: min XW |x-PX W |+|x'-P' X W |.

[0179] The projection of the wire point at camera 1 can be expressed as:

[0180] ;

[0181] Can be broken down into:

[0182] ;

[0183] After transforming the above formula, we can get two constraints:

[0184] ;

[0185] After introducing the projection point of the second camera, we can get an overdetermined equation:

[0186] ;

[0187] Solve for X by SVD W By reconstructing a series of matching points on the wire, a dense reconstruction result of the wire can be obtained.

[0188] S6. Through dense 3D reconstruction technology, the 3D model of the scene is restored from multiple images, and the distance and obstacles are determined through the point cloud of the wires and the background, providing accurate spatial information for the maintenance of power facilities.

[0189] Perform 3D reconstruction of scene points, including but not limited to dense 3D reconstruction of scenes using a network such as MVSNet. Obstacles are identified by measuring the distance of scene points near the wire points.

[0190] This embodiment uses image feature matching to accurately detect obstacles near the wires, calculate the shortest distance between the obstacles and the wires, and provide accurate data for the safe operation of power facilities; combines GPS data with camera posture information, and calibrates the camera posture through an optimization algorithm to ensure the accuracy of the three-dimensional reconstruction model and enhance the robustness of the system; generates a three-dimensional model of the wire and its environment through multi-view stereo vision technology and a dense three-dimensional reconstruction algorithm, which helps to evaluate the impact of obstacles on the line.

[0191] According to one aspect of the present application, a monocular camera-based wire assembly obstacle detection system includes:

[0192] at least one processor; and,

[0193] a memory communicatively connected to at least one of the processors; wherein,

[0194] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the wire assembly obstacle detection method based on a monocular camera as described in any of the above embodiments.

[0195] The present invention realizes the intelligent processing of the whole process from data collection to risk warning by constructing a complete adaptive power line inspection and risk assessment system. First, the quality of data collection is ensured by optimizing the scene adaptive parameters, and the accurate identification and feature expression of power lines are realized by adopting multi-level image processing and feature extraction technology; in the three-dimensional reconstruction link, the accuracy and reliability of the reconstruction results are guaranteed by multi-constraint optimization and deep learning methods; especially in the dynamic characteristic analysis and risk assessment, the spatiotemporal attention network and multi-dimensional risk assessment framework are adopted to realize the comprehensive monitoring of the power line operation status and surrounding environmental risks. The present invention not only improves the automation level and accuracy of power line inspection, but also realizes the early warning of potential risks. In practical applications, it can effectively cope with the challenges brought by complex meteorological conditions, diverse terrain features and various obstacles, provide all-round technical support for the safe operation and maintenance of power lines, and improve the operation reliability and maintenance efficiency of the power system.

[0196] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. The method of adaptive obstacle detection on the inspection route of UAV based on monocular camera is characterized by: The steps include: S1. Obtain initial flight parameters, and calculate optimized flight parameters through scene complexity evaluation function; based on the optimized flight parameters, collect original image sequences, calculate image quality indicators, and obtain image quality scores; Based on the image quality score, the original image sequence is subjected to exposure compensation to obtain a compensated image sequence; Based on the compensated image sequence, the spatial similarity and motion continuity between images are calculated to generate an image correlation matrix; based on the image correlation matrix, the optimal image combination is screened; the initial flight parameters include the initial flight altitude, the initial flight speed and the initial sampling interval; S2. Based on the optimal image combination and image quality score, a multi-resolution feature pyramid is constructed to obtain a multi-scale feature tensor set; a direction descriptor is calculated for the multi-scale feature tensor set to generate a feature descriptor set; Based on the feature descriptor set and the image association matrix, the feature matching score is calculated and the initial feature matching pair set is output; the initial feature matching pair set is verified at multiple levels to obtain a reliable matching set; the feature distribution of the reliable matching set is analyzed, and supplementary features are generated to obtain an enhanced feature matching set; S3, based on the enhanced feature matching set, the pre-stored camera parameter matrix and the GPS raw data, calculate the initialization pose to obtain a pose sequence; Perform multi-view geometry optimization on the pose sequence and output the optimized camera pose; calculate the depth estimate based on the optimized camera pose and generate a multi-scale depth map set; perform consistency optimization on the multi-scale depth map set to obtain an optimized unified depth map; Combine the optimized unified depth map and the optimized camera pose to evaluate the reconstruction quality and output the quality distribution map. S4. Based on the compensated image sequence and the quality distribution map, the directional enhancement response is calculated to obtain a power line candidate area map; the power line candidate area map is segmented by topological constraints to output a segmented line segment set; based on the segmented line segment set and the optimized camera pose, multi-view matching is performed to generate a segment matching pair set; based on the segment matching pair set and the optimized unified depth map, three-dimensional reconstruction is performed to obtain a three-dimensional model of the power line; Analyze the timing characteristics of the three-dimensional model of the power line and output the power line dynamic characteristics report; S5. Based on the compensated image sequence, the optimized unified depth map and the power line 3D model, scene segmentation is performed to obtain obstacle feature data; Based on obstacle feature data and power line dynamic characteristics report, calculate the distance between power lines and obstacles and output obstacle distance data; Combine obstacle feature data, obstacle distance data and power line 3D model to conduct obstacle risk assessment and generate obstacle risk level table; Generate obstacle warning information and output obstacle warning report based on obstacle risk level table and power line dynamic characteristics report; Based on the obstacle warning report and obstacle risk level table, the inspection strategy is optimized and the obstacle monitoring plan is output.

2. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 1 is characterized in that: Step S1 is further as follows: S11, obtaining initial flight parameters, calculating the edge density value, local variance value and depth change rate of the image, and substituting them into the scene complexity evaluation function for calculation to obtain the scene complexity score; Based on the scene complexity score, the initial flight parameters are optimized and calculated, and the optimized flight parameters are output; S12, based on the optimized flight parameters, collecting the original image sequence; obtaining the pre-configured camera parameter matrix, performing wavelet transform calculation on each image in the original image sequence, and obtaining a clarity index; Calculate the brightness distribution of each area in the original image sequence to obtain the illumination uniformity index; analyze the blur degree of the original image sequence to obtain the motion blur index; evaluate the scene coverage in the original image sequence to obtain the scene integrity index; combine the clarity index, illumination uniformity index, motion blur index and scene integrity index to form an image quality scoring matrix; S13, based on the image quality score matrix, dividing the original image sequence into a predetermined number of regions, calculating the local brightness mapping value of each region, and generating a regional weight mask; Multiply the local brightness map value with the regional weight mask and superimpose them to obtain the exposure compensation value; Based on the exposure compensation value, the brightness of the original image sequence is adjusted, and the compensated image sequence is output; S14, calculating the spatial similarity value between adjacent images based on the compensated image sequence and the optimized flight parameters; Analyze the motion continuity of the compensated image sequence to obtain a continuity score; The spatial similarity values ​​and continuity scores are combined to generate a spatiotemporal association matrix; Based on the spatiotemporal correlation matrix, the optimal image combination is screened.

3. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 2 is characterized in that: Step S2 is further as follows: S21. Based on the optimal image combination and the image quality score, the scale parameter of each level is calculated; based on the scale parameter of each level, the image in the optimal image combination is subjected to tensor transformation and weight kernel function calculation to generate a single-layer feature tensor; the single-layer feature tensors of all levels are combined to generate a multi-scale feature tensor set; S22. Based on the multi-scale feature tensor set, the anisotropic intensity value of each feature point is calculated to obtain an anisotropic intensity vector; based on the anisotropic intensity vector, a directional gradient histogram is generated to obtain directional gradient information; Based on the directional gradient information, the context correlation is calculated to obtain context coding data; the anisotropic intensity vector, the directional gradient information and the context coding data are combined to generate a feature descriptor set; S23, based on the feature descriptor set and the image association matrix, calculate the neighborhood constraints of the feature points to obtain neighborhood constraint data; based on the neighborhood constraint data, calculate the matching probability between the feature points; based on the matching probability and a preset threshold, generate an initial feature matching pair set; S24, based on the initial feature matching pair set and the image association matrix, calculate the local structure consistency score, evaluate the global topological consistency, analyze the temporal continuity, and generate a multi-level verification score; Based on the multi-level verification scores, the initial feature matching pair set is screened and a reliable matching set is output; S25. Based on the reliable matching set and the feature descriptor set, analyze the feature space distribution, identify the feature sparse area, and generate supplementary feature points in the feature sparse area; calculate the descriptor for the supplementary feature point, merge the descriptor with the reliable matching set, and output the enhanced feature matching set.

4. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 3 is characterized in that: Step S3 is further as follows: S31. Based on the enhanced feature matching set and the pre-stored camera parameter matrix, the projection error value is calculated, and the time series smoothing constraint, including the speed constraint term and the acceleration constraint term, is generated; based on the pre-stored GPS raw data, the GPS constraint term is constructed; based on the time series smoothing constraint and the GPS constraint term, the camera position and attitude are calculated, motion prediction is performed, and the initialization posture sequence is output; S32, based on the initialization pose sequence and the enhanced feature matching set, calculating the reprojection error to obtain reprojection error data; based on the reprojection error data, evaluating the structural consistency to obtain the structural constraint data; Based on the structural constraint data, the scene continuity is analyzed to obtain the continuity constraint data; Based on the reprojection error data, structural constraint data and continuity constraint data, optimization calculation is performed to output the optimized camera pose; S33, based on the optimized camera pose and compensated image sequence, divide the local image blocks, calculate the local depth probability, and generate a depth estimation probability distribution; based on the depth estimation probability distribution, perform depth estimation calculation on each scale level, combine multi-scale results, and output a multi-scale depth map set; S34, based on the multi-scale depth map set and the optimized camera pose, calculate the depth data item to obtain depth consistency data; based on the depth consistency data, analyze the depth gradient to obtain gradient constraint data; based on the gradient constraint data, evaluate the surface smoothness to obtain smoothness constraint data; combine the depth consistency data, the gradient constraint data and the smoothness constraint data for optimization, and output the optimized unified depth map; S35. Based on the optimized unified depth map and the optimized camera pose, calculate the reconstruction completeness score, evaluate the geometric consistency and scale accuracy, and generate a reconstruction quality score; Based on the reconstruction quality score, the spatial distribution is calculated to obtain a quality distribution map.

5. The method for adaptive obstacle detection of unmanned aerial vehicle inspection routes based on a monocular camera according to claim 4 is characterized in that: Step S4 is further as follows: S41. Based on the compensated image sequence and the quality distribution map, a multi-scale directional filter group is constructed to calculate the point cloud projection consistency score; based on the point cloud projection consistency score, an anisotropic Gaussian filter is used to calculate the local directionality score and generate a directional response tensor; Based on the directional response tensor, edge enhancement processing is performed to obtain an enhanced response map; the enhanced response map is subjected to directionality analysis to output a power line candidate area map and a directional distribution map; S42, based on the power line candidate area map and the direction distribution map, calculate the local direction consistency, evaluate the line segment ductility, and generate the linear feature weight; Based on the linear feature weights, the curvature change rate is calculated, the piecewise smoothness is evaluated, and a continuity score is generated; Based on the continuity score, the intersection status of the line segments is analyzed, the parallelism index is calculated, and the topological relationship matrix is ​​generated; combined with Linear feature weights, continuity scores and topological relationship matrices are used to optimize segmentation and output a set of segmented line segments; S43, based on the segmented line segment set and the optimized camera pose, calculating the direction similarity, evaluating the length ratio consistency, and generating a geometric similarity score; Based on the geometric similarity score, the relative position relationship is calculated, the spatial distribution pattern is evaluated, and the topological constraint matrix is ​​generated; Based on the topological constraint matrix, the surrounding feature distribution is analyzed, the regional descriptor is calculated, and the context matching score is generated; Based on the geometric similarity score, topological constraint matrix and context matching score, the matching pairs are screened and a set of line segment matching pairs is output; S44, based on the line segment matching pair set, the direction distribution map and the optimized unified depth map, evaluate the shape features and generate basic model parameters; Based on the basic model parameters, calculate the local deformation characteristics, evaluate the elastic deformation, and generate the deformation function; Based on the deformation function, the measurement error is analyzed, the uncertainty is calculated, and the noise model is generated; Based on the basic model parameters, deformation function and noise model, optimization reconstruction is performed to output the three-dimensional model of power lines; S45, based on the three-dimensional model of the power line and the optimized camera posture, calculating the sag change rate to obtain sag data; Based on the sag data, the lateral offset is measured to obtain the offset data; based on the offset data, the vibration frequency is analyzed to obtain the vibration data; based on the vibration data, the tension value is estimated to obtain the tension data; the sag data, offset data, vibration data and tension data are time-series filtered and weighted, and a power line dynamic characteristics report is output.

6. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 5 is characterized in that: Step S5 is further as follows: S51, based on the compensated image sequence, the optimized unified depth map and the power line 3D model, evaluate the height stratification and generate the spatial obstacle features; Based on the spatial obstacle characteristics, motion characteristics are calculated, change patterns are evaluated, and dynamic obstacle characteristics are generated; Based on the dynamic obstacle characteristics, the geometric shape is analyzed, the texture features are calculated, and the obstacle morphological features are generated; Combine spatial obstacle features, dynamic obstacle features and obstacle morphological features to identify and segment obstacles and output obstacle feature data; S52. Based on the obstacle feature data and the power line dynamic characteristic report, calculate the shortest distance between the obstacle and the power line to obtain vertical distance data; based on the vertical distance data, analyze the obstacle movement trend to obtain dynamic approach data; based on the dynamic approach data, evaluate the environmental impact to obtain environmental impact data; Based on vertical distance data, dynamic approach data and environmental impact data, an obstacle distance assessment model is constructed to output obstacle distance data; S53, calculating the static distance risk based on the obstacle feature data, the obstacle distance data and the three-dimensional model of the power line to obtain the static threat degree; Based on the static threat level, the dynamic collision risk is evaluated to obtain the dynamic threat level; based on the dynamic threat level, the obstacle growth trend is analyzed to obtain the development threat level; based on the development threat level, the environmental impact is calculated to obtain the environmental threat level; the static threat level, dynamic threat level, development threat level and environmental threat level are weighted and combined to output the obstacle risk level table; S54. Calculate obstacle type weights based on the obstacle risk level table, obstacle feature data, and power line dynamic characteristic report to obtain an obstacle weight matrix; Based on the obstacle weight matrix, the weather impact is evaluated and the weather impact factor is obtained; Based on the obstacle risk level table, obstacle weight matrix and weather impact factors, obstacle warning classification is carried out and obstacle warning report is output; S55. Calculate obstacle monitoring coverage based on the obstacle warning report, the obstacle risk level table and the pre-stored historical inspection data to obtain monitoring coverage data; Based on the monitoring coverage data, key areas are evaluated and regional priorities are obtained; Based on regional priority, the monitoring frequency is analyzed to obtain frequency optimization data; based on the monitoring coverage data, regional priority and frequency optimization data, optimization calculations are performed to generate an obstacle monitoring plan that includes obstacle key monitoring paths, sampling frequencies, priority processing areas and monitoring cycles.

7. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 6 is characterized in that: Step S13 is further as follows: S131, based on the image quality score matrix, using an adaptive grid partitioning algorithm for each image in the original image sequence to obtain an image region set; based on the image region set, calculating the brightness mean of each region to obtain a region brightness matrix; Analyze the distribution characteristics of the regional brightness matrix to obtain a brightness distribution map; Applying Gaussian smoothing to the brightness distribution map to obtain a smoothed brightness map; calculating the difference between the smoothed brightness map and the preset standard brightness to obtain a brightness deviation map; generating a local mapping matrix based on the brightness deviation map and the image quality score matrix; S132, calculating the edge strength of each region based on the local mapping matrix and the image region set to obtain an edge strength map; Based on the edge intensity map, the regional texture features are extracted to obtain a texture feature map; The edge intensity map and the texture feature map are fused to obtain a regional importance map; the regional importance map is normalized to obtain a weight mask matrix; Performing element-wise multiplication of the local mapping matrix and the weight mask matrix to obtain a weighted mapping matrix; performing spatial domain filtering on the weighted mapping matrix to obtain a smoothed mapping matrix; S133. Based on the smooth mapping matrix, brightness mapping is performed on each image in the original image sequence to obtain a mapped image sequence; the local contrast of the mapped image sequence is calculated to obtain a contrast map; based on the contrast map, the degree of image detail preservation is analyzed to obtain a detail preservation map; the contrast map and the detail preservation map are fused to obtain a quality assessment map; based on the quality assessment map, local fine-tuning is performed on the mapped image sequence to output a compensated image sequence.

8. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 6 is characterized in that: Step S22 is further as follows: S221. Based on the multi-scale feature tensor set, construct a covariance matrix for each scale layer to obtain a covariance tensor set; calculate the eigenvalues ​​of the covariance tensor set to obtain an eigenvalue sequence; Based on the eigenvalue sequence, the anisotropy measure is calculated to obtain the anisotropy intensity vector; Based on the anisotropic intensity vector, the self-attention mechanism is used to calculate the correlation between feature points and obtain the attention weight map; The anisotropic intensity vector is fused with the attention weight map to obtain an enhanced feature vector; S222, constructing a local coordinate system based on the enhanced feature vector to obtain a local coordinate map; calculating the directional gradient in the local coordinate map to obtain a gradient amplitude map and a gradient directional map; performing directional quantization on the gradient directional map to obtain a directional histogram; Calculate the statistical characteristics of the gradient amplitude map to obtain the amplitude feature vector; combine the directional histogram and the amplitude feature vector to obtain the directional gradient information; S223. Based on the directional gradient information and the enhanced feature vector, a multi-head attention network is constructed to obtain an attention feature map; Based on the attention feature map, the semantic relevance of the feature point neighborhood is calculated to obtain the semantic association map; the attention feature map and the semantic association map are fused to obtain the context encoding data; the anisotropic intensity vector, directional gradient information and context encoding data are feature spliced, and the dimension is reduced through a multi-layer perceptron to output a set of feature descriptors.

9. The monocular camera-based adaptive obstacle detection method for unmanned aerial vehicle inspection routes according to claim 6 is characterized in that: Step S32 is further as follows: S321, based on the initialization pose sequence and the enhanced feature matching set, construct a projection model to obtain a projection matrix set; based on the projection matrix set, calculate the reprojection coordinates of the feature points to obtain a projection coordinate set; calculate the Euclidean distance between the projection coordinate set and the actual coordinates to obtain a distance error map; Analyze the statistical characteristics of the distance error map to obtain the reprojection error data; Based on the reprojection error data, an error weight matrix is ​​constructed to obtain an error weight map; S322, constructing a scene structure graph based on the error weight map and the enhanced feature matching set to obtain a structure atlas; analyzing the local rigidity of the structure atlas to obtain a rigid constraint graph; calculating the geometric consistency between feature points based on the rigid constraint graph to obtain a geometric constraint graph; fusing the rigid constraint graph and the geometric constraint graph to obtain structural constraint data; Based on the structural constraint data, the constraint weights are calculated to obtain the structural weight matrix; S323. Analyze the temporal changes of the initialization pose sequence to obtain a motion trajectory graph; based on the motion trajectory graph and the structural weight matrix, calculate the motion continuity between adjacent frames to obtain a continuity score graph; based on the continuity score graph, evaluate the smoothness of the scene changes to obtain a smoothness graph; fuse the continuity score graph and the smoothness graph to obtain continuity constraint data; substitute the reprojection error data, structural constraint data and continuity constraint data into the optimization solver, and output the optimized camera pose and three-dimensional point cloud data.

10. The monocular camera-based UAV inspection route adaptive obstacle detection system is characterized by: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the monocular camera-based adaptive obstacle detection method for drone inspection routes as described in any one of claims 1 to 9.

Images