Autonomous inspection wire detection method and system for UAV based on Bezier curve modeling

Abstract

The present invention discloses a method and system for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling. The method includes collecting binocular image sequences, GPS positioning and IMU attitude data through unmanned aerial vehicles, synchronously aligning multi-source data, calculating scene dynamic evaluation values, performing adaptive image enhancement, extracting and fusing spatiotemporal features; constructing a multi-scale feature pyramid, extracting wire topological features, and generating enhanced features in combination with environmental context information; generating a control point candidate set based on the enhanced features, optimizing Bezier curve parameters and performing temporal consistency optimization. The present invention realizes high-precision wire detection and robust tracking in complex scenes, improves the reliability of autonomous inspection by unmanned aerial vehicles, and can solve key technical problems such as scale change, feature matching, dynamic modeling, data fusion and parameter optimization in the process of wire detection.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and in particular to a method and system for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling. Background Art

[0002] The safe operation of power transmission lines is an important guarantee for the reliable operation of power grids. With the continuous expansion of the scale of power grids and the continuous increase in the total length of transmission lines, the traditional manual inspection method can no longer meet the growing inspection needs. UAV inspection has gradually become the main means of intelligent inspection of transmission lines due to its advantages such as strong mobility, wide coverage and flexible operation. Especially in complex terrain areas such as mountains and hills, UAV inspection can overcome terrain restrictions and realize all-round detection of transmission lines. At the same time, the multi-sensor system carried by the UAV can obtain rich inspection data, which provides an important basis for transmission line status assessment and fault diagnosis. Therefore, the research on autonomous inspection technology of transmission lines based on UAVs has important engineering application value and economic benefits.

[0003] At present, the inspection of transmission lines by UAV mainly adopts the wire detection method based on image processing. Traditional methods mainly include straight line detection based on Hough transform, line segment extraction based on LSD algorithm, contour tracking based on edge detection and other technologies. These methods usually adopt a single feature extraction strategy in the wire detection process, such as gradient-based edge features or grayscale-based texture features. In terms of feature matching, classic algorithms such as template matching or local descriptor matching are mainly used. For wire trajectory modeling, common methods include straight line fitting, multi-segment line fitting and spline curve fitting. In terms of wire tracking, traditional tracking algorithms such as Kalman filtering or particle filtering are mainly used. These methods can achieve basic wire detection and tracking functions under ideal conditions.

[0004] However, there are still many specific problems that need to be solved in the practical application of the existing technology: First, in terms of feature extraction, the existing methods fail to effectively deal with the scale change of wire images, especially in the process of rapid movement of drones, the imaging scales of wires at different distances are significantly different, resulting in unstable feature extraction; second, the traditional feature matching algorithm is prone to mismatching when dealing with highly similar wire textures, especially in dense areas with multiple wires, the matching accuracy is significantly reduced; third, the existing wire trajectory modeling method often uses a curve model with fixed parameters, which is difficult to adapt to the dynamic changes of the wire shape, especially when the wire swings under strong wind conditions, the model accuracy is significantly reduced; fourth, the traditional tracking algorithm fails to make full use of the complementarity of multi-source data, and only relies on a single data source for state estimation, resulting in insufficient tracking robustness; fifth, the existing methods lack a quantitative evaluation mechanism for scene dynamics, and cannot adaptively adjust the algorithm parameters according to scene changes, affecting the performance of the system in complex environments; finally, the existing feature verification methods usually use simple geometric constraints or similarity thresholds, lack a consistency verification mechanism for multi-scale features, and are easily affected by background interference. These technical problems seriously restrict the practicality and reliability of the UAV autonomous inspection system. Summary of the invention

[0005] The purpose of the invention is to propose a method and system for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling to solve the above-mentioned problems existing in the prior art.

[0006] The technical solution is a method for autonomous inspection of wires by drones based on Bezier curve modeling, which includes the following steps:

[0007] S1. Collect multi-source data through UAV, including binocular image sequence data, GPS positioning data and IMU attitude data; perform synchronous alignment processing on the multi-source data to obtain an aligned multi-modal data stream; calculate the scene dynamics based on the aligned multi-modal data stream to obtain a scene dynamics evaluation value; perform adaptive enhancement processing on the binocular image sequence data based on the scene dynamics evaluation value to obtain an enhanced image sequence; extract spatiotemporal features based on the enhanced image sequence to obtain a fused spatiotemporal feature map;

[0008] S2. Based on the fused spatiotemporal feature graph, a multi-scale feature pyramid is constructed to obtain a multi-level feature pyramid set; based on the multi-level feature pyramid set, the topological features of the wire are extracted to generate a wire topological feature graph; based on the wire topological feature graph, environmental context information is extracted and feature enhancement is performed to output an enhanced feature graph;

[0009] S3. Based on the enhanced feature map, a candidate set of wire control points is generated to obtain a candidate set of control points; based on the candidate set of control points, initial Bezier curve parameters are generated; based on the initial Bezier curve parameters and pre-stored historical parameters, time consistency optimization is performed to generate optimized Bezier curve parameters;

[0010] S4. Based on the optimized Bezier curve parameters, geometric constraints are constructed and the degree of constraint satisfaction is evaluated to obtain a constraint satisfaction score. Based on the constraint satisfaction score and the multi-level feature pyramid set, multi-scale feature verification is performed to generate a feature consistency score. Based on the constraint satisfaction score and the feature consistency score, the wire trajectory is optimized and the final wire trajectory parameters are output.

[0011] The UAV autonomous inspection wire detection system based on Bezier curve modeling includes:

[0012] at least one processor; and,

[0013] a memory communicatively connected to at least one of the processors; wherein,

[0014] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the UAV autonomous inspection wire detection method based on Bezier curve modeling.

[0015] Beneficial effects: The present invention establishes a foundation for time consistency and scene adaptability of data processing, realizes multi-level representation of wire features, and ensures the accuracy and continuity of trajectory modeling; in challenging scenarios such as changing lighting, complex backgrounds, and dense wires, it can still maintain stable detection performance, achieve high precision and high reliability of wire detection in complex scenarios, and provide reliable technical support for autonomous UAV inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flow chart of the method of the present invention.

[0017] Figure 2 This is a flow chart of step S1 of the present invention.

[0018] Figure 3 This is a flow chart of step S2 of the present invention.

[0019] Figure 4 This is a flow chart of step S3 of the present invention.

[0020] Figure 5 This is a flow chart of step S4 of the present invention. DETAILED DESCRIPTION

[0021] like Figure 1As shown, the present application proposes a UAV autonomous inspection wire detection method based on Bezier curve modeling, comprising the following steps:

[0022] S1. Collect multi-source data through UAV, including binocular image sequence data, GPS positioning data and IMU attitude data; perform synchronous alignment processing on the multi-source data to obtain an aligned multi-modal data stream; calculate the scene dynamics based on the aligned multi-modal data stream to obtain a scene dynamics evaluation value; perform adaptive enhancement processing on the binocular image sequence data based on the scene dynamics evaluation value to obtain an enhanced image sequence; extract spatiotemporal features based on the enhanced image sequence to obtain a fused spatiotemporal feature map;

[0023] S2. Based on the fused spatiotemporal feature graph, a multi-scale feature pyramid is constructed to obtain a multi-level feature pyramid set; based on the multi-level feature pyramid set, the topological features of the wire are extracted to generate a wire topological feature graph; based on the wire topological feature graph, environmental context information is extracted and feature enhancement is performed to output an enhanced feature graph;

[0024] S3. Based on the enhanced feature map, a candidate set of wire control points is generated to obtain a candidate set of control points; based on the candidate set of control points, initial Bezier curve parameters are generated; based on the initial Bezier curve parameters and pre-stored historical parameters, time consistency optimization is performed to generate optimized Bezier curve parameters;

[0025] S4. Based on the optimized Bezier curve parameters, geometric constraints are constructed and the degree of constraint satisfaction is evaluated to obtain a constraint satisfaction score. Based on the constraint satisfaction score and the multi-level feature pyramid set, multi-scale feature verification is performed to generate a feature consistency score. Based on the constraint satisfaction score and the feature consistency score, the wire trajectory is optimized and the final wire trajectory parameters are output.

[0026] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0027] S11, acquiring multi-source data from the UAV in real time, including binocular image sequence data, GPS positioning data and IMU attitude data; calculating the time deviation between each data source in the multi-source data based on a preset data timestamp; aligning the sampling time of all data sources in the multi-source data to a unified time reference based on the time deviation, and generating an aligned multimodal data stream, including image data, position data and attitude data at the same time;

[0028] S12, reading the position data and posture data in the aligned multimodal data stream, calculating the position change and posture change at adjacent moments; dynamically adjusting the weight coefficient based on the position change and posture change to obtain the adjusted weight coefficient; based on the adjusted weight coefficient, mapping the weighted sum of the position change and posture change to a predetermined interval through a sigmoid function to obtain a scene dynamics evaluation value representing the degree of scene dynamics;

[0029] S13, inputting the image data and the scene dynamic evaluation value in the aligned multimodal data stream into the image enhancement network, processing them using a static scene enhancement function and a dynamic scene enhancement function respectively, to obtain an enhanced result; based on the scene dynamic evaluation value, adaptively fusing the enhanced result, and outputting an enhanced image sequence;

[0030] S14. Based on the enhanced image sequence, a convolutional neural network is used to extract the spatial features of the current frame, and the historical frame features are processed through a recurrent neural network to obtain the temporal features; based on the scene dynamic evaluation value, the fusion weights of the spatial features and the temporal features are calculated; based on the fusion weights, the spatial features and the temporal features are weightedly fused to obtain a fused spatiotemporal feature map.

[0031] This embodiment realizes the time consistency of data stream and the scene adaptability of feature extraction through the synchronous alignment processing of multi-source data and the dynamic evaluation mechanism of the scene. Specifically, firstly, the image is preprocessed by using adaptive histogram equalization and bilateral filtering, and the sensor data is denoised and smoothed by combining Kalman filtering, thereby improving the data quality; secondly, the precise alignment of different data sources is realized by the dynamic time warping algorithm, and the time deviation problem caused by the asynchronous acquisition of sensor data is solved; thirdly, the weight coefficient is dynamically adjusted based on the position and attitude change, and the scene dynamic evaluation value is obtained by sigmoid function mapping, which provides a scene adaptability basis for subsequent processing; finally, the spatiotemporal features are extracted and fused by adaptive enhancement network and recurrent neural network, and the temporal consistency of feature representation is realized. This embodiment not only improves the adaptability of the system to complex scenes, but also enhances the robustness of feature representation through the organic fusion of spatiotemporal features, laying a reliable foundation for subsequent wire detection. Especially in the scene where the drone is rapidly maneuvering or the lighting conditions change drastically, it can maintain stable feature extraction performance and improve the environmental adaptability of the system.

[0032] In another embodiment of the present application, it also includes:

[0033] S151, analyzing the dynamic characteristics of the scene. Input the scene dynamic evaluation value w(t), the multimodal data stream D(t) and the feature pyramid set {P_k(t)} into the scene dynamic characteristics analysis module, use the optical flow estimation algorithm to calculate the scene motion vector field; use the region segmentation algorithm to divide the scene into multiple sub-regions; calculate the dynamic index for each sub-region, including motion amplitude, directional consistency and time continuity; output the scene dynamic characteristics map D_m(t) and the regional dynamic weight matrix W_d(t);

[0034] S152, evaluate feature reliability. Based on the scene dynamic characteristic map D_m(t) and the regional dynamic weight matrix W_d(t), perform feature reliability evaluation: analyze the stability of feature points under different dynamic levels through scale space theory; use the information entropy criterion to evaluate the significance and distinguishability of feature points; establish a traceability evaluation model for feature points based on temporal consistency analysis; output feature reliability map R_f(t) and feature weight matrix W_f(t);

[0035] S153, dynamically adjust adaptive parameters. Input the feature reliability map R_f(t) and the feature weight matrix W_f(t) into the parameter adjustment module: build a parameter adjustment rule base based on fuzzy logic; update the threshold parameters of feature extraction and matching through an adaptive learning algorithm; dynamically adjust the search range and matching strategy of feature matching; output the adaptive parameter set A_p(t) and the adjustment strategy matrix S_a(t).

[0036] According to one aspect of the present application, step S11 is further:

[0037] S111, acquiring multi-source data from the UAV in real time, including binocular image sequence data, GPS positioning data and IMU attitude data; inputting the binocular image sequence data into an image preprocessing unit, correcting the image brightness distribution by using an adaptive histogram equalization algorithm, removing image noise by bilateral filtering, performing geometric correction based on camera distortion parameters, and outputting the preprocessed image sequence;

[0038] S112, based on the GPS positioning data and IMU attitude data, the Kalman filter algorithm is used for noise reduction and smoothing, the abnormal data points are eliminated by the outlier detection algorithm, the data is calibrated in combination with the sensor calibration parameters, and the calibrated position data and calibrated attitude data are output;

[0039] S113, input the preprocessed image sequence, the calibrated position data and the calibrated posture data into the time synchronization module, extract the timestamp information of each data stream; based on the timestamp information, calculate the time deviation by a dynamic time warping algorithm; based on the time deviation, resample all data to a unified time reference by a linear interpolation method, and output the time-aligned image sequence, position data and posture data;

[0040] S114. Input the time-aligned image sequence, position data and posture data into the data fusion module, and calculate the reliability weight of each data source through the information entropy weight method; based on the reliability weight, use the multi-source data fusion algorithm to integrate the time-aligned image sequence, position data and posture data, and finally output the aligned multimodal data stream containing synchronized image, position and posture information.

[0041] In one embodiment of the present application, the dynamic time warping algorithm is specifically: measure the multi-feature distance D(i, j) = ω1(t)·d_img(i, j) + ω2(t)·d_gps(i, j) + ω3(t)·d_imu(i, j); where: d_img(i, j)= ||I(i) - I(j)|| 2 ; d_img(i, j) is the image feature distance; d_gps(i, j) = ||G(i) - G(j)|| 2 ; d_gps(i, j) is the GPS position distance; d_imu(i, j) = arccos(q(i)·q(j)*); d_imu(i, j) is the IMU quaternion distance; ω_k(t) is the adaptive weight coefficient of the kth data source at time t; i and j are the time indexes of the two time series to be aligned; I(i) is the image feature vector at time i; G(i) is the GPS position vector at time i; q(i) is the IMU attitude quaternion at time i; ||·|| 2 represents the square of the Euclidean distance; arccos( ) represents the arccosine function; q(j)* represents the conjugate of the quaternion q(j). Weight adaptive update formula ω_k(t+1) = ω_k(t) + η·▽L_k(t); where: ▽L_k(t) = ΨL / Ψω_k is the gradient of the loss function with respect to the weight ω_k, Ψ is the partial derivative; η=η0 / (1 + αt) is the adaptive learning rate; η0 is the initial learning rate; α is the learning rate attenuation coefficient; t is the number of iterations; L = Σ(D_aligned - D_target) 2 +λ·R(ω) is the total loss function; D_aligned is the aligned distance; D_target is the target alignment distance; λ is the regularization coefficient; R(ω) = ||ω|| 1 + β||ω|| 2 is the regularization term; β is the L2 regularization coefficient; ||ω|| 1 is the L1 norm of the weight vector; ||ω|| 2is the L2 norm of the weight vector. The path cost calculation formula is: C(i, j) = D(i, j) + min{γ_h·C(i-1, j), γ_d·C(i-1, j-1), γ_v·C(i, j-1)}; where: γ_h = exp(-|v_h| / σ 2 ) is the horizontal shift weight; γ_d = exp(-|v_d| / σ 2 ) is the diagonal shift weight; γ_v = exp(-|v_v| / σ 2 ) is the vertical movement weight; v_h, v_d, v_v are the velocity estimates in the horizontal, diagonal and vertical directions respectively; σ 2 is the direction weight coefficient; exp(·) is the exponential function. The optimal path search formula is: P(t) = argmin_P Σ[C(i, j) + λ·S(i, j)]; where: S(i, j) = |Ψ 2 P / Ψt 2 | is the path smoothness constraint; Ψ 2 P / Ψt 2 represents the second-order derivative of the path; λ is the smoothness weight coefficient; argmin_P represents the path P that minimizes the objective function.

[0042] The specific process of the outlier detection algorithm is as follows: calculate the sample local density ρi = Σj χ(dij - dc); where: dij = ||xi - xj|| is the Euclidean distance between sample points i and j; dc is the cutoff distance, which is determined adaptively by the data distribution; χ(x) is the indicator function, which takes 1 when x < 0 and takes 0 otherwise; xi, xj are the feature vectors of the data sample points; calculate the relative distance δi = min j:ρj>ρi (dij); when sample point i has the maximum local density, δi = max j (dij); calculate the anomaly score Si = ρi·δi·ψ(ρi); where: ψ(ρi) = exp(-ρi / ρmax) is the density adjustment factor; ρmax is the maximum local density. Construct the anomaly judgment criterion γi = Si / σs > θ; where: σs is the standard deviation of the anomaly score; θ is the adaptive threshold, θ = μs + k·σs; μs is the mean of the anomaly score; k is the adjustment coefficient.

[0043] The specific process of the information entropy weighting method is as follows: calculate the enhanced information entropy Ei = -Σj (pij·ln(pij) +qij·ln(qij)) / ln(n); where: pij = (xij + ε) / (Σj (xij + ε)) is the standardized probability; qij = 1- pij is the complementary probability; xij is the value of the jth sample of the ith indicator; ε is a small positive number to prevent zero values; n is the number of samples; calculate the coefficient of variation di = 1 - Ei; calculate the weight wi = (di·vi) / (Σk (dk·vk)); where: vi = Σj |xij – x*i| / n is the variability; x*i is the average value of the ith indicator; perform dynamic weight adjustment w'i(t) = wi·(1 +λ·σi(t)); where: σi(t) is the indicator volatility at time t; λ is the adjustment coefficient.

[0044] This embodiment realizes the precise timing alignment and data quality optimization of heterogeneous data through the synchronous alignment processing mechanism of multi-source data. Specifically, firstly, in the image preprocessing link, an adaptive histogram equalization algorithm is used to correct the image brightness distribution, and bilateral filtering is used to effectively remove noise while maintaining edge features, and then geometric correction is performed through camera distortion parameters to improve image quality; secondly, in sensor data processing, the Kalman filter algorithm is used to reduce noise and smooth GPS and IMU data, and the abnormal data points are removed by combining the outlier detection algorithm, and the sensor calibration parameters are used for accurate calibration to ensure the reliability of position and posture data; thirdly, the time deviation between different data sources is calculated by the dynamic time warping algorithm, and all data are resampled to a unified time base by the linear interpolation method, so as to realize the precise synchronization of multi-source data; finally, the reliability of each data source is evaluated by the information entropy weight method, and the organic integration of data is realized based on the multi-source data fusion algorithm. This embodiment not only solves the problem of inconsistent timing in the process of multi-source data acquisition, but also improves the data quality through multi-level data optimization processing, laying a reliable data foundation for subsequent feature extraction and trajectory detection.

[0045] According to one aspect of the present application, step S13 is further:

[0046] S131, based on the aligned multimodal data stream, extract image data and input it into the illumination assessment module, calculate the brightness distribution characteristics of each area of ​​the image through multi-scale brightness histogram analysis; based on the brightness distribution characteristics, extract shadow and highlight areas through local contrast mapping; based on the shadow and highlight areas, use the image block adaptive threshold method to detect abnormal illumination areas, and output the illumination feature vector and regional illumination weight matrix;

[0047] S132, inputting the image data and the illumination feature vector into the noise assessment module, extracting the high-frequency noise component of the image by wavelet transform decomposition; estimating the noise level of each area by local variance analysis based on the high-frequency noise component, and obtaining an estimation result; based on the estimation result, analyzing the noise spatial distribution characteristics by using the Markov random field model, and outputting the noise level spectrum and the noise distribution characteristic matrix;

[0048] S133, constructing a motion compensation model based on the scene dynamic evaluation value and the noise level map; calculating the pixel-level motion vector by optical flow estimation based on the motion compensation model; suppressing motion blur by using adaptive time domain filtering; aligning the features of the motion region by using a block matching algorithm based on the pixel-level motion vector, and outputting motion compensation parameters and a feature alignment mapping matrix;

[0049] S134, inputting the illumination feature vector, the regional illumination weight matrix, the noise distribution feature matrix and the motion compensation parameters into the parameter optimization module, solving the optimal combination of enhancement parameters by using the particle swarm optimization algorithm, and dynamically adjusting the enhancement intensity by using the fuzzy logic controller, and outputting an adaptive enhancement parameter set;

[0050] S135, input the image data, feature alignment mapping matrix and adaptive enhancement parameter set into the image enhancement module, use non-local mean filtering to suppress noise, and improve image clarity through a detail enhancement network, improve image contrast based on contrast-limited adaptive histogram equalization, and finally output an enhanced image sequence.

[0051] In one embodiment of the present application, the process of the image block adaptive threshold method is specifically as follows: calculate the block-level threshold Ti(x, y) = μi(x, y) + ki·σi(x, y)·(1 + φ(gi(x, y))); wherein: μi(x, y) is the local mean of the i-th block; σi(x, y) is the local standard deviation; ki is the adaptive coefficient; gi(x, y) is the local gradient strength; φ(g) = 1 / (1+ exp(-αg + β)) is the gradient adjustment function; α and β are control parameters; perform inter-block smoothing constraint Si,j = ωs·exp(-||Ti - Tj|| 2 / 2σs 2 );where: ωs is the smoothing weight; σs is the smoothing coefficient; construct the global optimization objective J(T) = Σi(Ji(Ti) + λ·Σ j∈Ni Si, j); where: Ji(Ti) is the intra-block cost function; Ni is the set of adjacent blocks; λ is the smoothing constraint weight.

[0052] The specific process of the wavelet transform decomposition algorithm is as follows: construct an adaptive wavelet basis function ψa, b(t) = |a| -1 / 2·ψ((tb) / a)·w(t); where: a is the scale parameter; b is the translation parameter; w(t) = exp(-γ·|f'(t)|) is the adaptive weight function; f'(t) is the signal derivative; γ is the sensitivity parameter; perform continuous wavelet transform CWTx(a,b) = ∫ x(t)·ψa,b*(t)dt; where: x(t) is the input signal; ψa,b*(t) are the conjugates of the wavelet basis functions; perform coefficient threshold processing λj,k= σj·(ln(N) / N) 1 / 2 ·(1 + α·|dj,k| / σj); where: σj is the standard deviation of the noise at the jth layer; N is the signal length; dj,k are the wavelet coefficients; α is the adjustment parameter; construct the reconstruction formula x'(t) = C -1 ·Σj,k dj,k'·ψj,k(t); where: C is the normalization constant; dj,k' is the coefficient after threshold processing; ψj,k(t) is the reconstructed wavelet basis function.

[0053] The specific process of the block matching algorithm is as follows: construct an adaptive cost function C(x, y, d) = ωs · SSSD(x, y, d) + ωt · CTMP(x, y, d); where: SSSD(x, y, d) = Σ ij [(I1(x+i,y+j) – I2(x+id,y+j)) 2 ]·W(i, j) is the weighted square difference; CTMP(x, y, d) = exp(-|▽I1(x, y) - ▽I2(xd, y)| / σ) is the gradient similarity; W(i, j) = exp(-(i 2 + j 2 ) / 2σ k 2 ) is the adaptive weight kernel; ωs, ωt are the adaptive weight coefficients; construct the disparity update equation d'(x, y) = d(x, y) + λ·ΨC / Ψd; where: λ is the learning rate; ΨC / Ψd is the cost function gradient; calculate the confidence conf(x, y) = exp(-C(x, y, d') / C max )·(1 - |Ψ 2 d' / Ψx 2 | / κ); where: C max is the maximum cost value; κ is the curvature threshold, σ k represents the standard deviation of the spatial kernel function, I1 and I2 represent the reference image and the target image.

[0054] The specific process of particle swarm optimization algorithm is as follows: construct the speed update equation v i t+1 = ω·v i t+ c1r1(pbest i -x i t ) + c2r2 (gbest t -x i t ) + c3r3(nbest - x i t ), where: ω = ω max - (ω max -ω min )·t / T is the adaptive inertia weight; v i t is the velocity of the ith particle at time t; x i t is the location; pbest i is the individual optimal position; gbest t is the global optimal position; nbest is the neighborhood optimal position; c1, c2, c3 are learning factors; r1, r2, r3 are random numbers. Update position x i t+1 = x i t + v i t+1 + μ·▽f(x i t ), where: μ is the gradient guidance coefficient; ▽f(x i t ) is the objective function gradient. Perform fitness evaluation F(x i ) = f(x i )·exp(-α·Σ j ||x i -x j || 2 / σ 2 ); where: f(x i ) is the basic fitness function; α is the diversity adjustment parameter. Where T is the maximum number of iterations; ω max ,ω min are the upper and lower limits of the inertia weight; σ is the diversity evaluation parameter.

[0055] This embodiment realizes adaptive optimization of image quality in complex scenes through multi-dimensional image enhancement and dynamic compensation mechanism. In the specific implementation process, firstly, the brightness distribution characteristics of each area of ​​the image are calculated through multi-scale brightness histogram analysis, the shadow and highlight areas are accurately identified by local contrast mapping, and the abnormal illumination area is detected based on the image block adaptive threshold method, which provides a reliable basis for subsequent image enhancement; secondly, the high-frequency noise components of the image are extracted by wavelet transform decomposition, the noise level of each area is estimated by combining local variance analysis, and the noise spatial distribution characteristics are analyzed by Markov random field model to achieve accurate noise evaluation; thirdly, a motion compensation model is constructed based on the scene dynamic evaluation value and the noise level map, motion blur is suppressed by optical flow estimation and adaptive time domain filtering, and feature alignment of motion areas is realized by block matching algorithm; finally, the optimal combination of enhancement parameters is solved by particle swarm optimization algorithm, the enhancement strength is dynamically adjusted by fuzzy logic controller, and the image quality is improved by non-local mean filtering and detail enhancement network. This embodiment not only effectively improves the image quality under different lighting conditions, but also improves the image clarity in motion scenes through dynamic compensation mechanism.

[0056] like Figure 3 As shown, according to one aspect of the present application, step S2 is further:

[0057] S21, inputting the fused spatiotemporal feature map into the feature pyramid generation network, extracting multi-scale features through feature extraction functions of at least four different scales; calculating the dynamic attention mask corresponding to each scale in the multi-scale features, multiplying the fused spatiotemporal feature map by the corresponding dynamic attention mask, and obtaining a multi-level feature pyramid set containing at least four levels;

[0058] S22. Based on a multi-level feature pyramid set, the feature map of each level is processed by a direction-sensitive feature transformation function to extract the directional features of the conductor; the adaptive weight coefficients of the directional features of each level are calculated, and based on the adaptive weight coefficients, the directional features of all levels are weighted and superimposed to generate a conductor topology feature map reflecting the spatial distribution of the conductor;

[0059] S23, extracting background information of the area around the conductor topology feature map; based on the background information, analyzing the relationship between the conductor and the surrounding environment through the environmental feature extraction function to obtain environmental features; calculating the feature fusion coefficient based on the conductor topology feature map and the environmental features; based on the feature fusion coefficient, adaptively fusion the conductor topology feature map and the environmental features to output an enhanced feature map. During the feature fusion process, the fusion coefficient μ is dynamically adjusted according to the complexity of the environment to ensure that the fusion result contains sufficient environmental context information.

[0060] In one embodiment of the present application, the feature extraction module uses resnet34 as the basic network, and replaces the convolution in resnet34 with the hole convolution with a hole rate of [4, 8], which improves the reasoning speed while ensuring accuracy. After the drone inspection picture passes through the feature extraction module, it outputs 8x, 16x, and 32x downsampled feature maps F1, F2, and F3. The feature map F3 is input into the vertical attention module to extract the wire information and obtain the feature map F3'; the feature fusion module is a commonly used FPN module, and the feature maps F1, F2, and F3' are input into FPN together for feature fusion, and new feature maps F1*, F2*, and F3* are output, and finally the 8x downsampled feature map F1* is output.

[0061] The prediction module includes wire starting point prediction, wire key point bias prediction, and control point bias prediction. The feature map F1* is subjected to three layers of convolution to output a wire starting point heat map of size n×1×(H / 8)×(W / 8), where n is the number of wires, H is the height of the input image, and W is the width of the input image; the feature map F1* is subjected to three layers of convolution to output a wire key point bias map of n×2×(H / 8)×(W / 8); the wire starting point heat map and the wire key point bias map are combined to obtain the starting positions (X, Y) of all wires, and the corresponding features in the feature map F1* are obtained according to the starting position (X, Y), and these features are input into the three-layer fully connected output control point bias, and the bias is encoded into a control point:

[0062] ctrpts=((X,Y)+(predx,predy)) ×stride;

[0063] Wherein, (X, Y) represents the position coordinates of the corresponding feature in the feature map, (predx, predy) represents the predicted control point bias value, and stride represents the multiple of the feature map downsampling, which is set to an integer 8 in this embodiment.

[0064] This embodiment is based on a multi-scale feature pyramid and a wire topology feature extraction mechanism to achieve a multi-level representation of wire features and effective use of environmental context information. The feature pyramid is constructed through feature extraction functions of different scales, and the key feature area is highlighted in combination with a dynamic attention mask, which effectively solves the problem of scale changes in the wire in the image; secondly, a direction-sensitive feature transformation function and an adaptive weight coefficient are used to extract the directional features of the wire, and a wire topology feature map is generated by feature weighted superposition, which improves the accuracy of wire detection; finally, the relationship between the wire and the surrounding environment is analyzed through the environmental feature extraction function, and adaptive fusion of features is achieved. This embodiment can not only accurately capture the multi-scale features of the wire, but also improve the robustness of the system to complex backgrounds by integrating environmental context information. Especially in scenes with dense wire areas or complex backgrounds, it can effectively suppress background interference, accurately extract wire features, and provide reliable feature support for subsequent trajectory modeling.

[0065] According to one aspect of the present application, step S22 is further:

[0066] S221. Based on a multi-level feature pyramid set, anisotropic diffusion filtering is used to enhance the linear structure to obtain an enhanced feature pyramid set; based on the enhanced feature pyramid set, a Gabor filter group is used to extract multi-directional texture features; based on the multi-directional texture features, a principal curvature analysis is used to calculate the local direction field; based on the local direction field, a non-maximum suppression algorithm is used to screen the dominant direction, and a direction feature set and a local direction confidence matrix are output;

[0067] S222, based on the directional feature set and the local directional confidence matrix, the regional structural features are encoded by the local binary pattern operator; based on the regional structural features, the edge continuity information is extracted by phase consistency analysis; based on the edge continuity information, the local anisotropy features are calculated by the structural tensor; based on the local anisotropy features, the local shape descriptor is extracted by the adaptive kernel function, and the structural feature map and the continuity measurement matrix are output;

[0068] S223, inputting the structural feature map and the continuity measure matrix into the global topology analysis module, inferring the connection relationship between the feature points through the tensor voting algorithm; based on the connection relationship, using the minimum spanning tree algorithm to construct an initial topological structure; using the graph partitioning algorithm to optimize the connection relationship of the initial topological structure to obtain an optimized topological structure; based on the optimized topological structure, using the path optimization algorithm to adjust and output a topological relationship diagram;

[0069] S224. Input the directional feature set, structural feature map and topological relationship map into the feature fusion module, calculate the correlation weights between the features through the self-attention mechanism; based on the correlation weights, optimize the feature combination through the preconfigured conditional random field model; based on the optimized feature combination, build a multi-level feature fusion network through residual learning; based on the multi-level feature fusion network, use an adaptive gating mechanism to control the information flow, and finally output the wire topology feature map.

[0070] In one embodiment of the present application, the process of the non-maximum suppression algorithm is specifically as follows: Calculate the response strength S ij =R ij ·exp(-|θ ij – θ* ij | 2 / 2σθ 2 );where: R ij is the original response value of position (i, j); θ ij The main direction angle; θ* ij is the neighborhood average direction angle; σθ is the angle variance parameter; adaptive suppression radius r(i, j) = r min + (r max -r min )·exp(-||▽I(i,j)|| 2 / 2σ r 2 );where: r min 、r max are the minimum and maximum suppression radii; σ r is the radius adjustment parameter; ▽I(i, j) is the image gradient; the local maximum value M(i, j) = [S ij > S kl + δ(k,l)]; where: δ(k,l) =α·exp(-d ijkl / r(i, j)) is the adaptive threshold function; d ijkl is the spatial distance; α is the control parameter.

[0071] The specific process of the local binary pattern operator is as follows: the adaptive sampling radius is R(x, y) = R min + (R max -R min )·(1 - exp(-V(x, y) / σ v )); where: V(x, y) is the local variance; σ v is the variance scale parameter; weighted binary coding LBP is performed rv w = Σ k w(k)·s(g k – gc )·2 k ; where: w(k) = exp(-d k 2 / 2R(x,y) 2 ) is the spatial weight; s(x) is the threshold function; g k , g c is the gray value of the sampling point and the center point; construct the histogram feature H i = Σ Xγ δ(LBP rv w (x, y), i)·G(x, y); where: G(x, y) = exp(-||▽I(x, y)|| 2 / 2σg 2 ) is the gradient weight. min , R max are the upper and lower limits of the sampling radius; σg is the gradient weight parameter.

[0072] The specific process of the tensor voting algorithm is: calculate the tensor field T(x) = Σ i DF(xx i ,θ i )·exp(-||xx i || 2 / 2σ t 2 )·T i ; Where: DF(x, θ) is the attenuation field function; θ i is the direction angle; T i is the initial tensor; construct the voting strength V(x, s) = tr(T(x))·exp(-κ·|λ1-λ2| / λ1); where: λ1, λ2 are eigenvalues; κ is the anisotropy parameter; σ t is the spatial attenuation parameter; tr( ) is the trace of the matrix; the curvature estimation is k(x) = -▽·(T(x)·n(x)) / ||T(x)||; where n(x) is the normal vector. ▽ is the gradient operator.

[0073] The specific process of the graph segmentation algorithm is as follows: Calculate the edge weight w(v i , v j ) = exp(-||F(v i )-F(v j || 2 / 2σf 2 )·exp(-d ij 2 / 2σd 2 )·ψ(v i , v j ), where F(v) is the node feature vector; dij is the spatial distance; ψ(v i , v j ) is the boundary consistency term; construct the region merging criterion D(R i , R j ) = D int (R i , R j ) + λ·D ext (R i , R j );Among them: D int is the similarity within the region; D ext is the external difference of the region; segmentation optimization target E(S) = Σ ij w(v i , v j )·[1-δ(l i , l j )] +μ·Σ i B(l i );where: l i is the label; B(·) is the boundary smoothing term.

[0074] In another embodiment of the present application, in order to solve the problem of feature extraction of slender wires, a self-attention module specifically for wires is designed. The wires are relatively slender in the picture, and neither the traditional convolutional neural network nor the transformer network can effectively extract features of such slender objects. In traditional convolutional neural networks, the receptive field of each pixel is in a very small area around the center of its corresponding pixel, and the receptive field is too small. The wire itself is relatively slender, and its range usually spans a large range of the entire image; on the other hand, the receptive field of the transformer-based structure is too large, and each pixel can see the full image information. Such a receptive field is too large for relatively slender wires. In the actual drone inspection process, the wires are arranged vertically along the image. Therefore, this embodiment designs a feature extraction module with a global receptive field in the longitudinal direction and a local receptive field in the lateral direction, which is called VerticalAttention, and the implementation process is as follows:

[0075] Assume that the input feature X is of size B×C×H×W, i.e., batch size (batch_size), number of channels, height, and width. Adjust the input shape to B*W×H×C, input it into the self-attention module, and finally restore the output to the original size. The self-attention module is implemented as follows: xi=Flatten(Xi)W+b; where b is the bias vector.

[0076] To preserve the position information of the feature in the image, the position code PE is added to each block vector. This embodiment designs a rotation position code. Let the feature x∈R d The position of is p, then the rotation position encoding recodes x as:

[0077] Rot(x)=R(p)x;

[0078] in: ; θ i =b i T p; R(p) is the rotation matrix; θ is the rotation angle;

[0079] Add the rotation position code PE to each vector: z0=[x1+PE1, x2+PE2, …, xN+PEN]; where z0 is the initial feature vector set after adding the position code; xN is the Nth feature vector, and PEN is the Nth position code;

[0080] The multi-head attention mechanism allows the model to focus on different parts in different representation spaces. For each vector zi, calculate the Query, key and Value: Qi=ziW Q , Ki = ziW K ,Vi=ziW V ;

[0081] The calculation of attention score and output is: Attention(Q, K, V) = softmax(QK T / sqrt(d k ))V;

[0082] The outputs of multiple heads are concatenated and transformed linearly: MultiHead(Q, K, V) = Concat(head1, head2, ..., head h )W O ;Head h is the hth attention head; W O is the output weight matrix;

[0083] The Transformer encoder layer also includes a feed-forward neural network module: (FFN(z))=max(0,zW1+b1)W2+b2;

[0084] After each attention layer and feed-forward layer, residual links and layer normalization are applied, and the final attention model expression is: z'=LayerNorm(z+ MultiHead(Q, K, V)); z"= LayerNorm(z'+ FFN(z')).

[0085] This embodiment realizes the accurate extraction and reliable representation of the wire topological features through the direction-sensitive feature extraction and global topological analysis mechanism. In the implementation process, firstly, the linear structure is enhanced by anisotropic diffusion filtering, the multi-directional texture features are extracted by using the Gabor filter group, the local direction field is calculated based on the principal curvature analysis, and the dominant direction is screened by the non-maximum suppression algorithm, thereby improving the directional sensitivity of the feature; secondly, the regional structural features are encoded by the local binary pattern operator, the edge continuity information is extracted by phase consistency analysis, the local anisotropic features are calculated by the structural tensor, and the local shape descriptor is extracted by combining the adaptive kernel function, thereby realizing the accurate representation of the structural features; thirdly, the connection relationship between the feature points is inferred by the tensor voting algorithm, the initial topological structure is constructed by the minimum spanning tree algorithm, and the connection relationship is optimized based on the graph segmentation algorithm; finally, the correlation weights between the features are calculated by the self-attention mechanism, the feature combination is optimized by the conditional random field model, and the multi-level feature fusion is realized based on residual learning. This embodiment not only accurately captures the directional characteristics of the wire, but also improves the integrity and reliability of the feature representation through global topological optimization.

[0086] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0087] S31, inputting the enhanced feature map into the control point generation network, extracting significant points through feature map convolution and non-maximum suppression, and calculating the confidence score of each significant point; screening the point set whose confidence score exceeds the preset threshold, combining the spatial coordinates of the points in the point set and the corresponding confidence score, and generating a control point candidate set;

[0088] S32, based on the control point candidate set, constructing a Bezier curve parameter optimization objective function, including a fitting error term between the curve and the candidate points and a regularization term of the curve shape; minimizing the Bezier curve parameter optimization objective function through an iterative optimization algorithm to obtain initial Bezier curve parameters;

[0089] S33, extract the Bezier curve parameters of the previous moment, calculate the Euclidean distance between the initial Bezier curve parameters of the current moment and the Bezier curve parameters of the previous moment; based on the Euclidean distance, calculate the smoothing weight coefficient through the sigmoid function; based on the smoothing weight coefficient, perform weighted average on the initial Bezier curve parameters of the current moment and the pre-stored historical parameters, and output the optimized Bezier curve parameters. In the process of parameter smoothing, the weight coefficient ω is adaptively adjusted as the parameter changes to ensure the temporal continuity of the curve trajectory.

[0090] In one embodiment of the present application, a Bezier curve is a single-parameter curve defined by a series of control points, which can be defined by n+1 control points: B(t)=∑ i=0n b i，n (t)p i , 0≤t≤1; where p i is the i-th control point, b i，n is a Bernstein polynomial of order n; b i，n =C n i t i (1-t) n-i , i=0,…,n; it has been verified that the classic cubic Bezier curve is sufficient to fit the wire, with a total of 4 control points.

[0091] Since the annotation of the wire is to mark the key points on the wire, and the model of this embodiment directly outputs the control points of the wire, it is necessary to generate control points to calculate the loss. xi , k yi )} i=1 m , (k xi , k yi ) represents the coordinates of the i-th point. The goal is to obtain the control point {p i (x i ,y i )} i=1 n , so the least squares method is used to solve the equation to obtain the control points:

[0092]

[0093] Among them, {t i} i=1 m ∈[0,1], considering the linear dependence of m, n, and b, the equation can be efficiently solved by using the pseudo-inverse of the matrix on the left side of the equation.

[0094] This embodiment achieves accurate modeling and temporal continuity of the wire trajectory through the generation of control point candidate sets and Bezier curve parameter optimization mechanism. In the specific implementation, firstly, the significant points are extracted through feature map convolution and non-maximum suppression, and the control point candidate set is screened in combination with confidence score. This candidate point generation method based on significance improves the reliability of the control points; secondly, by constructing an optimization objective function including fitting error terms and shape regularization terms, the precise optimization of Bezier curve parameters is achieved, ensuring the accuracy of trajectory modeling; finally, the temporal consistency optimization of trajectory parameters is achieved through Euclidean distance calculation and sigmoid function adaptive adjustment of smoothing weight coefficients. This embodiment not only ensures the accurate representation of the wire trajectory, but also effectively suppresses parameter jitter and improves the stability of trajectory tracking through temporal smoothing processing. Especially when the wire morphology changes dynamically or there is occlusion, the continuity and accuracy of trajectory modeling can be maintained, which improves the robustness of the system.

[0095] According to one aspect of the present application, step S32 is further:

[0096] S321. Input the candidate set of control points {p_i(t)} into the control point evaluation module, calculate the spatial distribution characteristics of the point set through principal component analysis, identify key control areas using density clustering algorithm, screen reliable control points based on geometric invariant analysis, remove abnormal points using spatial consistency test, and output the screened control point set P_f(t) and point set reliability score R_p(t).

[0097] S322. Based on the screened control point set P_f(t) and the point set reliability score R_p(t), a local Bezier curve is constructed by segmented curve fitting. The curvature continuity constraint is used to optimize the smoothness of the segmented connections. The elastic deformation model is used to adjust the shape of the local curve. The curve coverage is expanded in combination with the regional growing algorithm. The local curve parameter set L_b(t) and the segmented connection relationship matrix J(t) are output.

[0098] S323. Input the local curve parameter set L_b(t) and the segment connection relationship matrix J(t) into the global optimization module, construct the global energy function through the variational inference algorithm, optimize the curve parameters using the gradient projection method, process the geometric constraints based on the Lagrange multiplier method, solve the optimization problem using the alternating direction multiplier method, and output the global curve parameters G_b(t) and the constraint satisfaction vector C_s(t).

[0099] S324. Based on the global curve parameter G_b(t) and the constraint satisfaction vector C_s(t), the geometric characteristics of the curve are verified through differential geometry analysis, the structural rationality of the curve is evaluated using the topological consistency test, the shape similarity metric is used to compare the shape characteristics of adjacent curve segments, the stability of the curve is analyzed based on the energy functional, and the verification result matrix V(t) and the optimization direction vector D_o(t) are output.

[0100] S325. Input the global curve parameters G_b(t), the verification result matrix V(t) and the optimization direction vector D_o(t) into the iterative optimization module, control the parameter update rate through adaptive step size, accelerate the optimization convergence by using momentum method, avoid overfitting based on early stopping strategy, use multi-starting point search to improve the probability of obtaining the global optimal solution, and finally output the initial Bezier curve parameters B(t).

[0101] In one embodiment of the present application, the process of the density clustering algorithm is specifically as follows: calculating the local density ρ i = Σ j K(||x i -x j || 2 / h 2 )·w(x j ), where K(·) is the kernel function, h is the bandwidth parameter, w(x) is the sample weight, and the relative distance δ is calculated. i = min{d ij |ρ j >ρ i} + α·exp(-ρ i / ρ max );where: d ij is the sample distance; α is the density compensation parameter; γ is used to determine the cluster center i = ρ i ·δ i ·(1-exp(-σ i / σ max )); where: σ i is the local variance. max is the maximum local density; σ max is the maximum local variance.

[0102] The specific process of the region growing algorithm is as follows: Construct the growth criterion G(x, R) = ω t ·T(x,R) + ω s· S(x,R)+ω e E(x, R); where T(x, R) = exp(-|I(x)-μ r | 2 / 2σ r 2) is the grayscale similarity term; S(x, R) = exp(-||▽I(x)|| 2 / 2σ s 2 ) is the boundary smoothness term; E(x, R) = exp(-|H(x)-H* r | / σ e ) is the entropy difference term; μ r is the regional mean; H(x) is the local entropy; H* r is the average entropy of the region; dynamic threshold update τ(t) = τ0·exp(-βt)·(1+γ·v(t)); where v(t) is the growth rate; β and γ are control parameters; τ0 is the initial threshold; regional merging judgment M(R i , R j ) = [D(R i , R j ) < τ(t)]·C(R i , R j ), where: D(·) is the regional distance metric; C(·) is the connectivity constraint, σ r is the regional similarity parameter; σ s is the boundary smoothness parameter; σ e is the entropy difference parameter.

[0103] The process of variational inference algorithm is as follows: estimate the posterior probability P(θ|X) = q(θ) exp(-KL(q||p) + L(q)); where: q(θ) is the variational distribution; KL(q||p) is the KL divergence; L(q) is the lower bound of evidence; update equation q*(θ k ) = exp(E_{q(θ -k )}[ln p(X,θ)])·Z -1 ; where: θ -k Indicates that the k All parameters except ; Z is a normalization constant; optimization objective function F(q) = L(q) - λ·Σ i H(q i ); where H(q i ) is the entropy of the variational distribution; λ is the regularization parameter; θ is the set of model parameters; X is the observed data.

[0104] The specific process of the gradient projection method is: iteratively update x k+1 = P_C[x k - α k ▽f(x k )]·exp(-β k ||▽f(x k )||); where: P_C is the projection operator; αk is the adaptive step size; β k is the damping factor; adjust the step size α k = α0·(1-exp(-γ·k)) / (1+||▽f(x k )||); where α0 is the initial step size; γ is the decay rate; perform convergence judgment||x k+1 -x k ||≤ε·(1-exp(-δ·k)); where ε is the basic threshold and δ is the adjustment parameter.

[0105] The specific process of the momentum method is: perform momentum update v t+1 = μ t ·v t - η·(▽f(x t ) + λ·R'(x t )); where: μ t = μ0·(1-exp(-αt)) is the adaptive momentum coefficient; η is the learning rate; R'(x) is the derivative of the regularization term; perform parameter update x t+1 = x t + v t+1 ·exp(-β·||v t+1 ||); where β is the speed decay coefficient; adjust the learning rate η(t) = η0 / (1+γ·t)·(1+κ·||▽f(x t )||); where γ and κ are control parameters, η0 is the initial learning rate, and μ0 is the initial momentum coefficient.

[0106] This embodiment realizes the accurate estimation and reliability guarantee of Bezier curve parameters through a multi-stage parameter optimization and verification mechanism. In the specific implementation, firstly, the spatial distribution characteristics of the point set are calculated by principal component analysis, the key control area is identified by density clustering algorithm, reliable control points are screened based on geometric invariant analysis, and abnormal points are removed by spatial consistency test, thereby improving the reliability of the initial control points; secondly, the local Bezier curve is constructed based on segmented curve fitting, the smoothness of the segmented connection is optimized by curvature continuity constraint, the local curve shape is adjusted by elastic deformation model, and the curve coverage is expanded by combining regional growth algorithm, thereby realizing the accurate modeling of the local curve; thirdly, the global energy function is constructed by variational inference algorithm, the curve parameters are optimized by gradient projection method, the geometric constraints are processed based on Lagrange multiplier method, and the optimization problem is solved by alternating direction multiplier method; finally, the geometric characteristics of the curve are verified by differential geometry analysis, the structural rationality of the curve is evaluated by topological consistency test, and the shape similarity metric is used to compare the shape characteristics of adjacent curve segments. This embodiment not only ensures the accuracy of Bezier curve parameters, but also improves the reliability of parameter estimation through a multi-level verification mechanism.

[0107] like Figure 5 As shown, according to one aspect of the present application, step S4 is further:

[0108] S41. Based on the optimized Bezier curve parameters, calculate the second-order derivative value and the curve length of the curve; obtain obstacle information, and based on the optimized Bezier curve parameters and the obstacle information, calculate the minimum distance between the curve and the obstacle; based on the second-order derivative value of the curve, the curve length, the minimum distance between the curve and the obstacle and a preset threshold, respectively evaluate the satisfaction degree of the constraints of curvature continuity, length change and spatial distribution, and obtain an evaluation result; based on the evaluation result, generate a constraint satisfaction score;

[0109] S42, projecting the optimized Bezier curve parameters onto each scale level of the multi-level feature pyramid set, calculating the matching degree between the curve and the feature map at each scale through a feature matching function, and obtaining a matching result; performing weighted fusion on the matching results based on the importance weights of each level, and generating a feature consistency score;

[0110] S43, substituting the optimized Bezier curve parameters, constraint satisfaction scores and feature consistency scores into a pre-configured joint optimization function, finding the optimal solution through the optimization algorithm, and outputting the final wire trajectory parameters. The joint optimization function simultaneously considers the three aspects of parameter change, constraint satisfaction and feature consistency.

[0111] This embodiment realizes the reliability verification and optimization adjustment of the wire trajectory based on the geometric constraint evaluation and multi-scale feature verification mechanism. By evaluating the three types of constraints, namely curvature continuity, length change and spatial distribution, the constraint satisfaction score is obtained, thereby ensuring the geometric rationality of the trajectory model; by projecting the trajectory parameters to each scale level of the multi-level feature pyramid, combined with the feature matching function and importance weight, the multi-scale verification of feature consistency is realized, thereby improving the reliability of the verification result; by comprehensively considering the parameter change, constraint satisfaction and feature consistency through the joint optimization function, the precise optimization of the trajectory parameters is realized. This embodiment not only ensures the geometric validity of the trajectory model, but also improves the reliability of the model through multi-scale feature verification. Especially in complex environments, it can effectively identify and correct erroneous trajectory estimates, thereby improving the accuracy and reliability of the system.

[0112] In another embodiment of the present application, the process of generating a feature consistency score can also be: processing a multi-level feature pyramid set {P_k(t)}, extracting multi-scale invariant features through a deep learning network; using an attention mechanism to calculate the spatial correlation between features; establishing a topological relationship of features based on a graph convolutional network; outputting a feature association graph C_f(t) and a spatial dependency matrix D_s(t). Based on the scene dynamic characteristic graph D_m(t) and the feature association graph C_f(t), a spatiotemporal feature matching network is constructed, taking into account the impact of scene dynamics; processing the temporal evolution characteristics of features through a recursive neural network; using adversarial learning to improve the robustness of feature matching; outputting the optimized feature matching result M_f(t) and the matching confidence matrix C_m(t). Comprehensively considering all evaluation indicators, a multi-objective evaluation model is established to integrate dynamics and consistency evaluation; optimizing the scoring weight distribution through reinforcement learning; using a Bayesian reasoning framework to fuse multi-source evaluation results; outputting a revised consistency score S_c(t) and a scoring reliability index R_s(t).

[0113] According to one aspect of the present application, step S42 is further:

[0114] S421. Obtain the optimized Bezier curve parameters B*(t) and the multi-level feature pyramid set {P_k(t)} and perform feature space mapping processing. Construct a multi-resolution representation through scale space theory. Calculate the cross-scale mapping relationship using affine invariant feature transformation. Generate dense corresponding points based on an adaptive interpolation algorithm. Use a bilinear resampling method to align feature maps. Output multi-scale mapping features M_f(t) and scale transformation matrix T_s(t).

[0115] S422. Based on the multi-scale mapping features M_f(t) and the scale transformation matrix T_s(t), the feature contrast is enhanced by local response normalization, the local feature similarity is calculated using the adaptive receptive field, the feature matching relationship is constructed using the non-local self-similarity metric, and the scale difference is processed by combining the spatial pyramid matching strategy. The feature matching map S_m(t) and the matching credibility matrix R_m(t) are output.

[0116] S423. Input the feature matching graph S_m(t) and the matching credibility matrix R_m(t) into the consistency verification module, analyze the feature distribution pattern through spectral clustering, use the geometric verification algorithm to screen reliable matching pairs, evaluate the feature correspondence based on random sampling consistency, use the Markov random field model to optimize the global consistency, and output the consistency evaluation result E(t) and the feature weight vector W_f(t).

[0117] S424, input the multi-scale mapping feature M_f(t), consistency evaluation result E(t) and feature weight vector W_f(t) into the feature fusion module, screen the discriminative features through the adaptive feature selection mechanism, integrate the multi-scale responses using the weighted voting strategy, optimize the feature combination weights based on the conditional entropy criterion, normalize the fusion results using the soft maximum function, and finally output the feature consistency score C_m(t).

[0118] In one embodiment of the present application, the process of the geometric verification algorithm is specifically as follows: estimating the transformation matrix H = argmin Σ i ρ(||x' i -Hx i || 2 )·w(x i , x' i ), where ρ(·) is the robust loss function; w(x i , x' i ) = exp(-d(x i , x' i ) 2 / 2σ 2 )·s(x i , x' i ) is the adaptive weight; s(·) is the feature similarity; the consistency score is S(x, x') = exp(-E(x, x') / σ e )·(1-exp(-O(x, x') / σ0)); where E(·) is the reprojection error; O(·) is the directional consistency; the model selection criterion M(H) = S(H)·exp(-λ·C(H)); where S(H) is the support; C(H) is the complexity penalty.

[0119] The specific process of the conditional entropy criterion is: Calculate the conditional entropy H(Y|X) = -Σ ij p(x i ,y j )·log(p(y j |x i ))·w(x i ,y j ); where: w(x i ,y j ) = exp(-d(x i ,y j ) 2 / 2σ 2 ) is the adaptive weight; d(x i ,y j ) is the characteristic distance; p(x i ,y j) is the joint probability; perform feature selection score S(f) = MI(X, Y) - λ·H(Y|X)·exp(-α·V(f)); where MI(X, Y) is the mutual information; V(f) is the feature variance; λ and α are control parameters; perform dynamic threshold update τ(t) =τ0·(1-exp(-βt))·(1+γ·R(t)); where R(t) is the feature redundancy; β and γ are adjustment parameters; σ is the distance metric parameter; τ0 is the initial threshold value.

[0120] This embodiment realizes reliable evaluation and dynamic optimization of feature consistency through multi-scale feature verification and adaptive weight optimization mechanism. Multi-resolution representation is constructed through scale space theory, cross-scale mapping relationship is calculated by affine invariant feature transformation, dense corresponding points are generated based on adaptive interpolation algorithm, and feature map is aligned by bilinear resampling method, which effectively solves the scale change problem; feature contrast is enhanced by local response normalization, local feature similarity is calculated by adaptive receiving field, feature matching relationship is constructed by non-local self-similarity measurement, and scale difference is processed by combining spatial pyramid matching strategy to realize reliable feature matching; feature distribution pattern is analyzed by spectral clustering, reliable matching pairs are screened by geometric verification algorithm, feature correspondence is evaluated based on random sampling consistency, and global consistency is optimized by Markov random field model; discriminative features are screened by adaptive feature selection mechanism, multi-scale responses are integrated by weighted voting strategy, and feature combination weights are optimized based on conditional entropy criterion. This embodiment not only ensures the accuracy of feature matching, but also improves the reliability of feature verification through adaptive weight optimization.

[0121] According to one aspect of the present application, it also includes:

[0122] S5. Perform multi-dimensional quality assessment based on the final wire trajectory parameters, multi-level feature pyramid set and enhanced feature map to obtain a quality assessment vector; based on the quality assessment vector and historical parameters, update the system control parameters and output an updated parameter set; based on the quality assessment vector and the updated parameter set, construct a feedback control signal and generate a system feedback signal. Specifically:

[0123] S51. Input the final wire trajectory parameters into the geometric quality assessment module to calculate the curvature distribution and length characteristics of the trajectory; at the same time, substitute the trajectory parameters at adjacent moments into the timing stability assessment function to analyze the time continuity of the trajectory; extract the feature response value corresponding to the trajectory position from the enhanced feature map to evaluate the degree of feature matching; combine the evaluation results of the above three dimensions and output the quality assessment vector.

[0124] S52, extract the historical parameter set of the system, including the control parameters of modules such as feature extraction, parameter optimization and constraint evaluation, calculate the gradient direction of each parameter according to the quality evaluation vector, update the parameters in combination with the adaptive learning rate, and generate an updated parameter set. During the parameter update process, the learning rate is dynamically adjusted according to the quality evaluation results to ensure that the parameters converge to the optimization direction.

[0125] S53, input the quality assessment vector and the update parameter set into the feedback control module, calculate the update amount of the dynamic weight, feature fusion coefficient and context enhancement weight respectively, combine these update amounts into feedback control instructions, and output the system feedback signal. The system feedback signal contains the weight and coefficient adjustment values ​​required by each module in the next processing cycle, which is used to guide the dynamic optimization of system parameters.

[0126] According to one aspect of the present application, step S51 is further:

[0127] S511. Input the final wire trajectory parameter B_opt(t) into the geometric characteristic evaluation module, calculate the curvature distribution characteristics of the curve through differential geometry analysis, evaluate the smoothness and continuity of the curve using tangent space analysis, calculate the spatial relationship between curve segments based on geodesic distance measurement, and use shape complexity analysis to evaluate the rationality of the curve structure. Output the geometric characteristic score G_s(t) and the geometric constraint deviation matrix D_g(t).

[0128] S512. The final wire trajectory parameter B_opt(t) and the historical trajectory sequence cached by the system are input into the timing stability analysis module. The timing consistency of the trajectory is calculated by the dynamic time warping algorithm. The trajectory evolution characteristics are analyzed using the state space model. The trajectory change trend is predicted based on recursive Bayesian estimation. The entropy method is used to evaluate the temporal stability of the trajectory. The timing stability index T_s(t) and the change prediction vector P_t(t) are output.

[0129] S513. The final wire trajectory parameters B_opt(t) and the enhanced feature map C(t) are input into the feature matching calculation module. The directional consistency of the feature response is calculated by structural tensor analysis. The feature correspondence is evaluated by local descriptor matching. The feature distribution similarity is measured based on the mutual information criterion. The feature probability model is constructed using the kernel density estimation method. The feature matching score F_s(t) and the feature consistency matrix C_f(t) are output.

[0130] S514. Based on the geometric characteristic score G_s(t), the temporal stability index T_s(t) and the feature matching score F_s(t), an environmental adaptability evaluation model is constructed through an adaptive weight network, the fuzzy reasoning system is used to analyze the combination relationship of quality indicators, the multi-source evaluation results are integrated based on evidence theory, the hierarchical analysis method is used to determine the indicator importance weights, and the environmental adaptability score E_s(t) and the weight adjustment vector W_a(t) are output.

[0131] S515. Input the geometric characteristic score G_s(t), timing stability index T_s(t), feature matching score F_s(t), environmental adaptability score E_s(t) and weight adjustment vector W_a(t) into the comprehensive scoring module, build a scoring fusion model through a multi-criteria decision-making method, calculate the correlation between indicators using grey correlation analysis, optimize the indicator combination method based on an adaptive weighted algorithm, generate a normalized score using a nonlinear mapping function, and finally output a quality assessment vector Q(t).

[0132] In one embodiment of the present application, the recursive Bayesian process is as follows: prediction step p(x t |y 1：t-1 ) =∫p(x t |x t-1 )·p(x t-1 |y 1：t-1 )·w(x t-1 )dx t-1 ; where w(x) = exp(-||xx*|| 2 / 2σ x 2 ) is the state weight; update step p(x t |y 1：t-1 ) = η·p(y t |x t )·p(x t |y 1：t-1 )·exp(-λ·D(x t )); where D(x t ) is the dynamic uncertainty measure; η is the normalization constant; the adaptive noise covariance R t = R0·exp(-αt)·(1+β·||y t -y* t ||); where y* t is the predicted observation value; α and β are control parameters; x* is the state mean; σ x is the state uncertainty; R0 is the initial noise covariance.

[0133] The specific process of structural tensor analysis is as follows: Calculate the structural tensor J(x,σ) = Gσ*(▽I·▽I T)·exp(-κ·tr(▽I·▽I T )); where Gσ is the Gaussian kernel; κ is the gradient decay parameter; perform eigenvalue decomposition λi(x) = eig(J(x,σ))·w(x); where w(x) = 1-exp(-||▽ 2 I(x)|| / σ h ) is the adaptive weight; Directional consistency measurement C(x) = (λ1-λ2) / (λ1+λ2)·exp(-μ·|θ(x)-θ - |); where θ(x) is the main direction; θ - is the neighborhood average direction; μ is the angle-sensitive parameter; tr( ) is the trace of the matrix; σ h is the Hessian matrix parameter; 2 is the Hessian operator.

[0134] The process of kernel density estimation method is as follows: estimate the adaptive bandwidth kernel f*(x) = (nh(x)) -1 ·Σ i K((xX i ) / h(x))·w(X i ), where h(x) = h0·(1+α·v(x)) is the local bandwidth, v(x) is the local variance, and w(X i ) is the sample weight; construct the bandwidth selection criterion CV(h) =∫f*(x) 2 dx - 2n -1 ·Σ i f* -i (X i )·exp(-β·r(X i )); where f* -i is the leave-one-out estimate; r(X) is the outlier measure; the estimated density gradient ▽f*(x) = -(nh(x) 2 ) -1 Σ i (xX i )K'((xX i ) / h(x))·exp(-γ·||xX i ||); K' is the derivative of the kernel function; h0 is the basic bandwidth parameter; n is the number of samples; K( ) is the kernel function.

[0135] The specific process of the hierarchical analysis method is: to calculate the elements of the judgment matrix a ij = (w i / w j )·exp(-α·|s i -s j |); where w i 、wj is the weight; s i 、s j is the index score; α is the sensitivity parameter; calculate the eigenvalue λ max = Σ i (AW) i / (n·w i )·exp(-β·CI); CI is the consistency index; β is the penalty coefficient; update weight w i ' = w i ·exp(γ·(CR-CR0)); where CR is the consistency ratio; CR0 is the target threshold; γ is the learning rate; A is the judgment matrix; W is the weight vector,

[0136] The specific process of grey correlation analysis is as follows: Calculate the correlation coefficient ξ ij (k)= (min i min j |x0 j (k)-x ij (k)|+ρ·max i max j |x0 j (k)-x ij (k)|) / (|x0 j (k)-x ij (k)|+ρ·max i max j |x0 j (k)-x ij (k)|)·w(k); where w(k) = exp(-λ·k / n) is the time series weight; calculate the correlation r i = Σ k ξ ij (k)·exp(-μ·v(i)); where v(i) is the index variability; calculate the comprehensive correlation R = Σ i w i ·r i ·exp(-ν·|r i -r*|); where r* is the average correlation; ν is the difference penalty coefficient; ρ is the resolution coefficient; and n is the data length.

[0137] The specific process of the adaptive weighted algorithm is: update the weight w i (t+1) = w i (t)·exp(η·▽L(w i ))·(1+α·p(w i )); where p(w i ) is the performance evaluation function; η is the learning rate; α is the reward coefficient; w iis the weight; L is the loss function; the adaptive learning rate is η(t) = η0 / (1+βt)·exp(-γ·||▽L||); where β is the decay rate; γ is the gradient sensitive parameter; weight normalization w* i = w i ·exp(-δ·|w i -w*|) / Σ j w j ·exp(-δ·|w j -w*|); where w* is the weight mean; δ is the distribution adjustment parameter; η0 is the initial learning rate; L( ) is the loss function; ▽L is the gradient of the loss function.

[0138] This embodiment achieves comprehensive evaluation and reliability quantification of trajectory detection results through multi-dimensional quality evaluation and adaptive fusion mechanism. The curvature distribution characteristics of the curve are calculated by differential geometry analysis, the smooth continuity of the curve is evaluated by tangent space analysis, the spatial relationship between curve segments is calculated based on geodesic distance measurement, and the rationality of the curve structure is evaluated by shape complexity analysis, which comprehensively quantifies the geometric characteristics of the trajectory; the temporal consistency of the trajectory is calculated by the dynamic time warping algorithm, the trajectory evolution characteristics are analyzed by the state space model, the trajectory change trend is predicted based on recursive Bayesian estimation, and the temporal stability of the trajectory is evaluated by the entropy method, which realizes the temporal stability evaluation of the trajectory; the directional consistency of the feature response is calculated by structural tensor analysis, the feature correspondence is evaluated by local descriptor matching, the feature distribution similarity is measured based on the mutual information criterion, and the feature probability model is constructed by the kernel density estimation method; the environmental adaptability evaluation model is constructed by the adaptive weight network, the combination relationship of quality indicators is analyzed by the fuzzy reasoning system, and the multi-source evaluation results are fused based on the evidence theory. This embodiment not only achieves a comprehensive evaluation of the trajectory detection results, but also improves the reliability of the evaluation results through the adaptive fusion mechanism.

[0139] According to one aspect of the present application, it also includes:

[0140] S6. Establish a unified evaluation framework based on multi-dimensional evaluation indicators, implement an adaptive evaluation mechanism in dynamic scenarios, and build a feedback optimization mechanism. Specifically:

[0141] S61. Perform multi-dimensional indicator fusion. Input the scene dynamic evaluation value w(t), feature consistency score C_m(t), constraint satisfaction score S_g(t) and quality evaluation vector Q(t) into the indicator fusion module: construct an indicator evaluation system through a hierarchical analysis model and establish a hierarchical relationship between indicators; use the principal component analysis method to reduce the correlation between the evaluation indicators; use deep neural network learning to establish a nonlinear mapping relationship between indicators; output a unified evaluation indicator U_e(t) and an indicator importance vector W_i(t).

[0142] S62. Perform dynamic evaluation optimization. Perform dynamic optimization based on the unified evaluation index U_e(t) and the index importance vector Wi(t): construct a temporal Bayesian network model to analyze the time evolution characteristics of the evaluation index; optimize the time consistency of the evaluation results through the variational inference algorithm; use an integrated learning method to improve the stability and reliability of the evaluation results; output the optimized evaluation result O_e(t) and the optimized confidence C_o(t).

[0143] S63, perform adaptive feedback control. Input the optimized evaluation result O_e(t) and the optimized confidence C_o(t) into the feedback control module: establish an adaptive PID controller to dynamically adjust the system parameters; optimize the dynamic response characteristics of the system through the model predictive control method; use the robust control theory to improve the stability of the system in complex environments; output the control instruction set C_i(t) and the adjustment parameter matrix P_a(t).

[0144] S64. Verify the evaluation results. Verify the results based on the control instruction set C_i(t) and the adjustment parameter matrix P_a(t): build a cross-validation mechanism to evaluate the reliability of each indicator; analyze the stability of the evaluation results through the Monte Carlo method; verify the statistical significance of the evaluation results using the hypothesis testing method; output the verification report V_r(t) and the reliability index R_i(t).

[0145] In one embodiment of the present application, a method for autonomous inspection of wires by a drone based on Bezier curve modeling includes the following steps:

[0146] S1. Construct a UAV channel tree barrier measurement dataset to provide data support for wire detection and tree barrier measurement. Specifically: use UAVs to collect distribution network line inspection data, and use labelme annotation software to annotate the wires in the image. Each wire in the image contains several points on the wire. At the same time, the group of points can fully describe the entire wire.

[0147] S2. Construct a Bezier curve control point regression network, improve the attention mechanism module to address the problem of slender wire characteristics, and design control point loss and distance loss based on Bezier control point regression;

[0148] Traditional segmentation-based or keypoint-based methods usually require decoding prediction results or setting a large number of anchor points. In order to solve the problem of wire detection difficulties, a parameterizable Bezier curve fitting wire detection method is designed. It has the characteristics of small computational complexity, strong stability, and high degree of transformation freedom, including feature extraction module, vertical attention mechanism module, feature fusion module, and prediction module.

[0149] S3. Model training and results. The marked wires were preprocessed, including image brightness change, image color change, image random cropping, image normalization, key point interpolation, each wire was interpolated to 100 key points, and the control point production method was used to generate wire control points; the model and loss function were built, and the training optimization of the model was completed using the constructed training set. The optimizer was Adam, the learning rate was set to 0.0001, and the number of iterations was set to 300 rounds.

[0150] The model of this embodiment can effectively detect the starting point of the conductor in the distribution network line and fit the shape of the conductor well, providing strong support for drone tree barrier analysis. This embodiment proposes a distribution network conductor detection method based on Bezier curve fitting for drone binocular ranging channel tree barrier measurement, including constructing a distribution network channel inspection tree barrier measurement data set, designing an end-to-end network structure to achieve the output of the Bezier control point of the conductor, and improving the attention mechanism module for the slender characteristics of the conductor, and designing the control point loss and distance loss according to the Bezier control point regression. This embodiment can effectively fit the conductor in the inspection image, providing strong support for channel tree barrier analysis.

[0151] In one embodiment of the present application, multiple loss functions are used for supervised training of the Bessel control point regression network, including the Heatmap loss L at the starting point of the wire. point , bias loss of key points of conductor L offset , control point L1 loss L ctrlpts , the distance loss L between the wire and the Bezier fitting curve dist .

[0152] Taking into account the imbalance between the starting point area and the non-starting point area, the heatmap loss function of the wire starting point adopts the improved focal loss:

[0153] L point =(-1 / H'×W')∑ yx (1-Y* yx ) α log Y* yx ; Y yx =1;

[0154] L point =(-1 / H'×W')∑ yx (1-Y yx ) β Y* yx α log (1-Y* yx );other;

[0155] Among them, α and β are the hyperparameters of focal loss, and H'×W' is H / r×W / r;

[0156] During the inspection of the distribution network channel, the collected image is the front view of the wire. The lower end point of the wire in the image is selected as the starting point of the wire. In order to distinguish different wires, this embodiment proposes to use the starting point of the wire to represent each independent wire. Because there is a maximum spacing between the wires at the beginning, in order to better supervise the entire wire, the deviation from the key point of the wire to the starting point is added to the loss function. The mathematical relationship is as follows:

[0157] (Vx i j , Vy i j )=(sx i ,sy i )-(x i j ,y i j );

[0158] Generate bias regression map O using annotated key points yx , the size is H / r×W / r×C, yx means (x i j ,y i j ) coordinates are equal to (Vx i j , Vy i j ), the values ​​of other positions are 0, and C=2 represents the bias in the X-axis and Y-axis directions respectively. In order to evaluate the bias error, L1 loss is used:

[0159] L offset =(-1 / H'×W')∑ yx ∣O* yx -O yx ∣;

[0160] The key to network learning Bezier curves is to define a good distance evaluation index to evaluate the distance between the real wire and the predicted wire. The most direct method is to calculate the L1 distance between the control points. However, even when the L1 distance between the control points is large, the visual distance error between the Bezier curves may be small, especially on wires with small curvature. Since the Bezier curve can be parameterized by t∈[0,1], this embodiment constructs multiple losses to supervise the Bezier curve, including the L1 loss of the control points and the distance loss from the predicted curve points to the real curve.

[0161] Assume that the control points of the Bezier curve are P0, P1, ..., Pn. The points on the Bezier curve can be represented by a single parameter t: Pt = Bn (t, P0, P1, ..., Pn). The distance from point q to the Bezier curve can be expressed as:

[0162] dq =min t ∣∣q-pt∣∣2;

[0163] However, the above formula does not have a simple analytical solution, so multiple dense points t1, t2, ..., tn are sampled on the Bezier curve, and the shortest distance between point q and these sampled points is regarded as the distance from the point to the Bezier curve:

[0164] d q : min t ∣∣q-pti∣∣2; i=1, 2,...,N;

[0165] The average distance between the wire point and the predicted Bezier curve is recorded as L dist :L dist =1 / n∑ i=1 n d qi 2 ;

[0166] To perform direct regression of control points, we must first obtain the Bezier curve corresponding to the marked wire, and use the above Bezier control point generation method to generate control points. Suppose the control points obtained by network regression are P1, P2, ..., PN, and the control points in the marked wire are Q1, Q2, ..., QN, then the control point regression loss is expressed as:

[0167] L ctrlpts =1 / N||Pi-Qi||1;

[0168] The final loss is:

[0169] Loss = λ1 L point +λ2 L offset +λ3 L dist +λ4 L ctrlpts ;

[0170] Among them, λ1=1.0, λ2=1.0, λ3=10.0, λ4=10.0.

[0171] According to one aspect of the present application, a UAV autonomous inspection wire detection system based on Bezier curve modeling includes:

[0172] at least one processor; and,

[0173] a memory communicatively connected to at least one of the processors; wherein,

[0174] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the drone autonomous inspection wire detection method based on Bezier curve modeling described in any of the above embodiments.

[0175] The present invention realizes high precision and high reliability of wire detection in complex scenes through the organic combination of multi-source data fusion, multi-scale feature extraction, Bezier curve modeling and multi-level verification optimization. First, through the synchronous alignment of multi-source data and dynamic evaluation of scenes, the time consistency and scene adaptability foundation of data processing are established; secondly, through multi-scale feature pyramid and topological feature extraction, the multi-level representation of wire features is realized; again, through Bezier curve parameter optimization and time consistency processing, the accuracy and continuity of trajectory modeling are ensured; finally, through geometric constraint evaluation and multi-scale feature verification, the reliability of trajectory model is guaranteed. The present invention not only solves the key problems of scale change, feature matching, dynamic modeling and so on in wire detection, but also improves the robustness and reliability of the system in complex environments through the organic cooperation of various modules. In particular, in challenging scenes such as lighting changes, complex backgrounds, and dense wires, it can still maintain stable detection performance, providing reliable technical support for autonomous inspection of drones.

[0176] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A UAV autonomous inspection wire detection method based on Bezier curve modeling, characterized in that: The steps include: S1. Collect multi-source data through UAV, including binocular image sequence data, GPS positioning data and IMU attitude data; perform synchronous alignment processing on the multi-source data to obtain aligned multi-modal data stream; Based on the aligned multimodal data stream, the scene dynamics is calculated to obtain a scene dynamics evaluation value; Based on the scene dynamic evaluation value, the binocular image sequence data is adaptively enhanced to obtain an enhanced image sequence; based on the enhanced image sequence, the spatiotemporal features are extracted to obtain a fused spatiotemporal feature map; S2. Based on the fusion of spatiotemporal feature maps, a multi-scale feature pyramid is constructed to obtain a multi-level feature pyramid set; Based on a multi-level feature pyramid set, the topological features of the wire are extracted to generate a wire topological feature map; based on the wire topological feature map, the environmental context information is extracted and the features are enhanced, and the enhanced feature map is output; S3, generating a candidate set of wire control points based on the enhanced feature map to obtain a candidate set of control points; Based on the control point candidate set, initial Bezier curve parameters are generated; based on the initial Bezier curve parameters and pre-stored historical parameters, time consistency optimization is performed to generate optimized Bezier curve parameters; S4. Based on the optimized Bezier curve parameters, geometric constraints are constructed and the degree of constraint satisfaction is evaluated to obtain a constraint satisfaction score; based on the constraint satisfaction score and the multi-level feature pyramid set, multi-scale feature verification is performed to generate a feature consistency score; Based on the constraint satisfaction score and feature consistency score, the wire trajectory is optimized and the final wire trajectory parameters are output.

2. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 1 is characterized in that: Step S1 is further as follows: S11, acquiring multi-source data from the UAV in real time, including binocular image sequence data, GPS positioning data and IMU attitude data; calculating the time deviation between each data source in the multi-source data based on a preset data timestamp; Based on the time deviation, the sampling time of all data sources in the multi-source data is aligned to a unified time base to generate an aligned multimodal data stream, including image data, position data and posture data at the same time; S12, reading the position data and posture data in the aligned multimodal data stream, calculating the position change and posture change at adjacent moments; dynamically adjusting the weight coefficient based on the position change and posture change to obtain the adjusted weight coefficient; Based on the adjusted weight coefficient, the weighted sum of the position change and the posture change is mapped to a predetermined interval through a sigmoid function to obtain a scene dynamics evaluation value representing the degree of scene dynamics; S13, inputting the image data and the scene dynamic evaluation value in the aligned multimodal data stream into the image enhancement network, and processing them using a static scene enhancement function and a dynamic scene enhancement function respectively to obtain an enhancement result; Based on the scene dynamic evaluation value, the enhancement results are adaptively fused and the enhanced image sequence is output; S14, based on the enhanced image sequence, using a convolutional neural network to extract the spatial features of the current frame, and processing the historical frame features through a recurrent neural network to obtain the temporal features; Based on the dynamic evaluation value of the scene, the fusion weight of spatial features and temporal features is calculated; Based on the fusion weight, the spatial features and temporal features are weightedly fused to obtain a fused spatiotemporal feature map.

3. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 2 is characterized in that: Step S2 is further as follows: S21, inputting the fused spatiotemporal feature map into the feature pyramid generation network, extracting multi-scale features through feature extraction functions of at least four different scales; calculating the dynamic attention mask corresponding to each scale in the multi-scale features, multiplying the fused spatiotemporal feature map by the corresponding dynamic attention mask, and obtaining a multi-level feature pyramid set containing at least four levels; S22. Based on a multi-level feature pyramid set, the feature map of each level is processed by a direction-sensitive feature transformation function to extract the directional features of the wire; Calculate the adaptive weight coefficient of the directional characteristics of each level, and based on the adaptive weight coefficient, weightedly superimpose the directional characteristics of all levels to generate a conductor topology feature map reflecting the spatial distribution of the conductors; S23, extracting background information of the area surrounding the conductor topology feature map; Based on the background information, the relationship between the conductor and the surrounding environment is analyzed through the environmental feature extraction function to obtain the environmental features; Calculate the feature fusion coefficient based on the wire topology feature map and environmental features; Based on the feature fusion coefficient, the wire topology feature map is adaptively fused with the environmental features to output an enhanced feature map.

4. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 3 is characterized in that: Step S3 is further as follows: S31, inputting the enhanced feature map into the control point generation network, extracting significant points through feature map convolution and non-maximum suppression, and calculating the confidence score of each significant point; screening the point set whose confidence score exceeds the preset threshold, combining the spatial coordinates of the points in the point set and the corresponding confidence score, and generating a control point candidate set; S32, based on the control point candidate set, constructing a Bezier curve parameter optimization objective function, including a fitting error term between the curve and the candidate points and a regularization term of the curve shape; minimizing the Bezier curve parameter optimization objective function through an iterative optimization algorithm to obtain initial Bezier curve parameters; S33, extracting the Bezier curve parameters of the previous moment, calculating the Euclidean distance between the initial Bezier curve parameters of the current moment and the Bezier curve parameters of the previous moment; based on the Euclidean distance, calculating the smoothing weight coefficient through the sigmoid function; based on the smoothing weight coefficient, performing weighted averaging on the initial Bezier curve parameters of the current moment and the pre-stored historical parameters, and outputting the optimized Bezier curve parameters.

5. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 4 is characterized in that: Step S4 is further as follows: S41. Based on the optimized Bezier curve parameters, calculate the second-order derivative value and the curve length of the curve; obtain obstacle information, and based on the optimized Bezier curve parameters and the obstacle information, calculate the minimum distance between the curve and the obstacle; based on the second-order derivative value of the curve, the curve length, the minimum distance between the curve and the obstacle and a preset threshold, respectively evaluate the satisfaction degree of the constraints of curvature continuity, length change and spatial distribution, and obtain an evaluation result; based on the evaluation result, generate a constraint satisfaction score; S42, projecting the optimized Bezier curve parameters onto each scale level of the multi-level feature pyramid set, calculating the matching degree between the curve and the feature map at each scale through a feature matching function, and obtaining a matching result; performing weighted fusion on the matching results based on the importance weights of each level, and generating a feature consistency score; S43, substituting the optimized Bezier curve parameters, constraint satisfaction score and feature consistency score into the preconfigured joint optimization function, finding the optimal solution through the optimization algorithm, and outputting the final wire trajectory parameters.

6. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 5 is characterized in that: Step S11 is further as follows: S111, acquiring multi-source data from the UAV in real time, including binocular image sequence data, GPS positioning data and IMU attitude data; inputting the binocular image sequence data into an image preprocessing unit, correcting the image brightness distribution by using an adaptive histogram equalization algorithm, removing image noise by bilateral filtering, performing geometric correction based on camera distortion parameters, and outputting the preprocessed image sequence; S112, based on the GPS positioning data and IMU attitude data, the Kalman filter algorithm is used for noise reduction and smoothing, the abnormal data points are eliminated by the outlier detection algorithm, the data is calibrated in combination with the sensor calibration parameters, and the calibrated position data and calibrated attitude data are output; S113, inputting the preprocessed image sequence, the calibrated position data and the calibrated posture data into a time synchronization module, and extracting the timestamp information of each data stream; Based on the timestamp information, the time deviation is calculated by the dynamic time warping algorithm; based on the time deviation, all data are resampled to a unified time base using a linear interpolation method, and time-aligned image sequences, position data, and posture data are output; S114, inputting the time-aligned image sequence, position data and posture data into the data fusion module, and calculating the reliability weight of each data source by using the information entropy weight method; Based on the reliability weight, a multi-source data fusion algorithm is used to integrate the time-aligned image sequence, position data and posture data, and finally an aligned multimodal data stream containing synchronized image, position and posture information is output.

7. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 5 is characterized in that: Step S13 is further as follows: S131, based on the aligned multimodal data stream, extracting image data and inputting it into the illumination assessment module, and calculating the brightness distribution characteristics of each area of ​​the image through multi-scale brightness histogram analysis; Based on the brightness distribution characteristics, the shadow and highlight areas are extracted through local contrast mapping. Based on the shadow and highlight areas, the image block adaptive threshold method is used to detect abnormal lighting areas and output the lighting feature vector and regional lighting weight matrix. S132, inputting the image data and the illumination feature vector into the noise assessment module, and extracting the high-frequency noise component of the image by wavelet transform decomposition; Based on the high-frequency noise component, the noise level of each area is estimated through local variance analysis to obtain the estimation result; based on the estimation result, the Markov random field model is used to analyze the noise spatial distribution characteristics, and the noise level spectrum and noise distribution characteristic matrix are output; S133, constructing a motion compensation model based on the scene dynamic evaluation value and the noise level map; and calculating a pixel-level motion vector through optical flow estimation based on the motion compensation model; Based on pixel-level motion vectors, a block matching algorithm is used to align the features of the moving area, and the motion compensation parameters and feature alignment mapping matrix are output; S134, inputting the illumination feature vector, the regional illumination weight matrix, the noise distribution feature matrix and the motion compensation parameters into the parameter optimization module, solving the optimal combination of enhancement parameters by using the particle swarm optimization algorithm, and dynamically adjusting the enhancement intensity by using the fuzzy logic controller, and outputting an adaptive enhancement parameter set; S135, input the image data, feature alignment mapping matrix and adaptive enhancement parameter set into the image enhancement module, use non-local mean filtering to suppress noise, and improve image clarity through a detail enhancement network, improve image contrast based on contrast-limited adaptive histogram equalization, and finally output an enhanced image sequence.

8. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 5 is characterized in that: Step S22 is further as follows: S221. Based on a multi-level feature pyramid set, anisotropic diffusion filtering is used to enhance the linear structure to obtain an enhanced feature pyramid set; based on the enhanced feature pyramid set, a Gabor filter group is used to extract multi-directional texture features; based on the multi-directional texture features, a principal curvature analysis is used to calculate the local direction field; based on the local direction field, a non-maximum suppression algorithm is used to screen the dominant direction, and a direction feature set and a local direction confidence matrix are output; S222, based on the directional feature set and the local directional confidence matrix, encoding the regional structural features through the local binary pattern operator; based on the regional structural features, extracting edge continuity information using phase consistency analysis; based on the edge continuity information, using the structure tensor to calculate the local anisotropy features; Based on the local anisotropy features, the adaptive kernel function is used to extract the local shape descriptor, and the structural feature map and continuity measure matrix are output; S223, inputting the structural feature map and the continuity measure matrix into the global topology analysis module, and inferring the connection relationship between the feature points through the tensor voting algorithm; Based on the connection relationship, the minimum spanning tree algorithm is used to construct the initial topological structure; the graph partitioning algorithm is used to optimize the connection relationship of the initial topological structure to obtain the optimized topological structure; based on the optimized topological structure, the path optimization algorithm is used to make adjustments and output the topological relationship diagram; S224, inputting the directional feature set, the structural feature map and the topological relationship map into the feature fusion module, calculating the correlation weights between the features through the self-attention mechanism; optimizing the feature combination through a preconfigured conditional random field model based on the correlation weights; and constructing a multi-level feature fusion network through residual learning based on the optimized feature combination; Based on a multi-level feature fusion network, an adaptive gating mechanism is used to control the information flow and finally output the wire topology feature map.

9. The method for autonomous inspection of wires by unmanned aerial vehicles based on Bezier curve modeling according to claim 5 is characterized in that: Also includes: S5. Based on the final wire trajectory parameters, the multi-level feature pyramid set and the enhanced feature map, a multi-dimensional quality assessment is performed to obtain a quality assessment vector; based on the quality assessment vector and the historical parameters, the system control parameters are updated and an updated parameter set is output; based on the quality assessment vector and the updated parameter set, a feedback control signal is constructed to generate a system feedback signal.

10. The UAV autonomous inspection wire detection system based on Bezier curve modeling is characterized by: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the drone autonomous inspection wire detection method based on Bezier curve modeling as described in any one of claims 1 to 9.

Images