Visual intelligent identification method and system for unmanned aerial vehicle inspection depth of power transmission line

Abstract

The invention discloses a power transmission line unmanned aerial vehicle inspection depth visual intelligent identification method and system, and relates to the field of power transmission line inspection, and the method comprises the steps: dynamically adjusting imaging parameters based on the voltage grade and environment complexity of a power transmission line, collecting the multispectral visual data of a line body, a metal fitting for line connection and fixation, and a gallery environment, according to the invention, the method can dynamically adapt to the voltage grade of the power transmission line and the environmental complexity, optimizes the imaging effect, accurately collects the multi-dimensional visual data, and associates the key environmental parameters. The data standardization level is improved through adaptive noise reduction, accurate splicing and distortion correction, defect features and environment association attributes are efficiently mined through an attention guidance mechanism and a dynamic convolutional network, and accurate judgment of defect types and levels and future evolution trend prediction are performed in combination with historical data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power transmission line inspection, in particular to a power transmission line unmanned aerial vehicle inspection depth vision intelligent identification method and system. BACKGROUND

[0002] Power transmission line inspection is a combination of unmanned aerial vehicle autonomous cruise and image recognition technology, which carries out intelligent inspection of key components such as ground wires and insulators of power transmission lines. It can automatically identify defects such as rust and damage, transmit data in real time and generate analysis reports, improve inspection efficiency and defect identification accuracy, and thus ensure the safe and stable operation of the power grid.

[0003] The invention patent application with application number 202111642756.2 discloses a kind of unmanned aerial vehicle intelligent image acquisition method of power transmission line, which aims to solve the problem of "heavy task in actual use of unmanned aerial vehicle, unmanned aerial vehicle task allocation efficiency is not high, at the same time, traditional image processing algorithm including simulated annealing algorithm, edge extraction, object detection method based on brightness and spatial information, etc. are used to identify defects in collected images, which leads to low accuracy of image type recognition and complex algorithm, and the universality is not strong enough".

[0004] However, the existing research and development direction of power transmission line unmanned aerial vehicle inspection technology mostly stays at the level of image acquisition accuracy and shallow image information acquisition, and it is difficult to identify power transmission line defects using image depth vision information and make evolution trend prediction.

[0005] Therefore, we propose a power transmission line unmanned aerial vehicle inspection depth vision intelligent identification method and system. SUMMARY

[0006] In view of the above shortcomings of the prior art, the present application provides a power transmission line unmanned aerial vehicle inspection depth vision intelligent identification method and system, which can effectively solve the problems of the prior art.

[0007] To achieve the above purpose, the present application is realized by the following technical scheme;

[0008] The present application discloses a power transmission line unmanned aerial vehicle inspection depth vision intelligent identification system, which comprises:

[0009] The collection module is used for dynamically adjusting imaging parameters based on the voltage grade of the power transmission line and the environmental complexity, collecting multispectral visual data of the line body, metal fittings for connecting and fixing the line, and the corridor environment, and binding the collected data with the inspection pose, environmental temperature and humidity, and electromagnetic interference parameters; the preprocessing module is used for receiving the bound collected data, performing data denoising through an environment parameter adaptive electromagnetic denoising logic, performing image stitching and distortion correction in combination with the inspection pose, so as to construct a standardized three-dimensional data matrix with environmental attribute labels; the mining module is used for constructing a dynamic convolution extraction network based on a defect feature attention guide mechanism, mining corresponding visual features and environmental associated attributes of each type of defect from the three-dimensional data matrix, and synchronously generating a feature vector set with scene adaptation coefficients; the recognition module is used for classifying and matching the feature vector set to complete defect type determination and grade evaluation, and constructing a growth rule model based on historical defect data of the power transmission line, and applying the model to output evolution trend prediction results of the defect in a preset time interval in the future; the verification module is used for receiving the defect determination, grade evaluation, and evolution trend prediction results output by the recognition module, eliminating misjudgment data caused by environmental interference through real-time feature backtracking verification, historical defect data comparison, and digital twin scene simulation, and synchronously supplementing geometric size and material loss rate quantitative parameters of the defect to form a verified feedback data package; and the feedback module is used for receiving the feedback data package output by the verification module and synchronously pushing it to a preset inspection terminal.

[0010] The collection module is interactively connected with the preprocessing module through a wireless network, the preprocessing module is interactively connected with the mining module through a wireless network, the mining module is interactively connected with the recognition module through a wireless network, the recognition module is interactively connected with the verification module through a wireless network, and the verification module is interactively connected with the feedback module through a wireless network.

[0011] Further, the imaging parameters in the collection module include exposure time, focal length, sensitivity, and multispectral channel gain.

[0012] When the imaging parameters are dynamically adjusted, they are subject to: ;

[0013] In the formula: is the adjusted exposure time; is the adjusted focal length; is the adjusted sensitivity; is the adjusted multispectral channel gain; , , , is the corresponding reference imaging parameter; is the parameter adjustment coefficient; is the voltage grade of the power transmission line; For standard voltage level; For environmental complexity;

[0014] When the collected data is associated with the inspection pose, the environmental temperature and humidity, and the electromagnetic interference parameter, the following calculation is performed: ;

[0015] In the formula: is the correlation degree; is the time difference of the collected data and the parameter record; is the maximum allowable time difference; is the core feature value of the collected data; is the feature threshold value corresponding to the associated parameter; is the standard feature reference value;

[0016] When is not less than the preset association threshold, it is determined that the binding is valid.

[0017] Further, the environmental parameter adaptive electromagnetic noise reduction logic of the preprocessing module is:

[0018] A noise reduction model is constructed based on the environmental temperature and humidity and the electromagnetic interference strength. The output of the original collected signal after the model noise reduction processing completes the noise reduction signal: ;

[0019] In the formula: is the original collected signal; , is the noise reduction coefficient; is the environmental temperature and humidity composite parameter; is the electromagnetic interference strength; is the reference temperature.

[0020] Further, in the image stitching stage, any one of the scale-invariant feature transform, the speeded up robust features, or the ORB feature extraction algorithm is used to perform the stitching operation.

[0021] After stitching, distortion correction is performed in combination with the pitch angle, roll angle, and yaw angle of the inspection pose for coordinate correction: ;

[0022] In the formula: is the original pixel coordinate; is the corrected coordinate; is the pitch angle of the inspection pose; is the yaw angle of the inspection pose; is the roll angle of the inspection pose;

[0023] The standardized three-dimensional data matrix M is a three-dimensional tensor structure, denoted as , wherein i∈[1,X] corresponds to the spatial dimension (x,y,z), j∈[1,Y] corresponds to the multi-spectral channel, and k∈[1,Z] corresponds to the time sequence, and the matrix element ;

[0024] In the formula: is the denoised signal corresponding to the spatial position, spectral channel and time point; , , is the weight coefficient of the spatial, spectral and time dimensions.

[0025] Further, the defect feature attention guide mechanism in the mining module performs feature enhancement by constructing a defect probability weight matrix ;

[0026] In the formula: , is the attention weight coefficient; is the feature intensity of the candidate defect region; is the total feature intensity of the whole image; is the correlation degree of the candidate region and the environmental parameter; is the global maximum environmental correlation degree;

[0027] The convolution kernel K(t) of the dynamic convolution extraction network is adaptively adjusted with the network iteration step number t: ;

[0028] In the formula: is the initial convolution kernel; is the convolution kernel adjustment increment; is the scene adaptation coefficient;

[0029] The feature vector set , m is the number of defect types, and each feature vector satisfies , represents the nth visual feature value of the mth defect type, represents the total dimension of the single-class defect feature.

[0030] Further, the classification matching operation in the recognition module is:

[0031] Construct a standard defect feature vector library , and calculate the matching similarity between the feature vector set and the standard defect feature vector library: ; ​

[0032] wherein: is the similarity degree; is the total dimension of the feature of the single-class defect; is the feature vector of the mth-class defect; is a preset minimum value; is a scene adaptation coefficient, when is not less than a preset matching threshold, it is determined as the corresponding defect type;

[0033] The growth law model is used to calculate the defect size, and the expression is: ;

[0034] wherein: is the defect size at time t; is the initial defect size, and the source verification module; , is the evolution coefficient; is the environmental influence factor at time t; is the defect size at time t; is the critical safety size of the defect;

[0035] The defect future evolution trend prediction result includes the size growth rate and the critical time: ;

[0036] wherein: is the size growth rate; is the critical time; is the current inspection time point.

[0037] Further, the stage of quantifying the geometric size of the supplemented defect and the material loss rate includes: ;

[0038] wherein: is the defect geometric size; is the material loss rate; is a set of viewing angles; is the measurement value of the ith viewing angle; is the measurement error of the ith viewing angle; is a preset minimum value; is the multispectral signal intensity of the defect area; is the standard signal intensity of the intact area; is the defect area; is the total area of the detection area.

[0039] In another aspect, the power transmission line unmanned aerial vehicle inspection deep vision intelligent identification method comprises:

[0040] Based on the voltage level and environmental complexity of the power transmission line, the imaging parameters are dynamically adjusted, the multispectral vision data of the line body, metal accessories and corridor environment are collected, and are associated and bound with the inspection pose, temperature and humidity and electromagnetic interference parameters;Through the adaptive electromagnetic noise reduction logic processing of the bound data, the image stitching and distortion correction are completed in combination with the inspection pose, and a standardized three-dimensional data matrix with environmental attribute labels is constructed;Based on the defect feature attention guide mechanism, a dynamic convolution extraction network is constructed to mine the defect visual features and environmental correlation attributes, and a feature vector set with scene adaptation coefficients is generated;The feature vector set is classified and matched with the standard defect feature vector library to complete the defect type determination and grade evaluation, and a growth rule model is constructed based on historical defect data to predict the defect evolution trend;Through real-time feature backtracking verification, historical defect data comparison and digital twin scene simulation, false positive data is removed, and verified feedback data packets are formed by supplementing the defect geometric size and material loss rate quantitative parameters;The verified feedback data packets are synchronously pushed to the preset inspection terminal.

[0041] Compared with the known prior art, the technical scheme provided by the present application has the following beneficial effects:

[0042] The present application provides a power transmission line unmanned aerial vehicle inspection deep vision intelligent identification method and system, which can dynamically adapt to the voltage level and environmental complexity of the power transmission line to optimize the imaging effect, accurately collect multi-dimensional vision data and associate key environmental parameters, improve the data standardization level through adaptive noise reduction, accurate stitching and distortion correction, efficiently mine defect features and environmental correlation attributes through attention guide mechanism and dynamic convolution network, accurately determine the defect type and grade and predict the future evolution trend in combination with historical data, remove false positive data through multiple verifications and supplement quantitative parameters, greatly improve the inspection efficiency and defect recognition accuracy, provide accurate data support for power transmission line operation and maintenance, effectively reduce the operation and maintenance cost, and ensure the stable operation of the power transmission line. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical scheme in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0044] Figure 1 The structure diagram of the power transmission line unmanned aerial vehicle inspection deep vision intelligent identification system;

[0045] Figure 2A flowchart of a depth vision intelligent recognition method for unmanned aerial vehicle inspection of a power transmission line. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0047] The present application will be further described below with reference to the embodiments.

[0048] Embodiment 1

[0049] The power transmission line unmanned aerial vehicle inspection depth vision intelligent recognition system of the present embodiment, as shown in Figure 1 , comprises:

[0050] The acquisition module is configured to dynamically adjust imaging parameters based on the voltage level of the power transmission line and the environmental complexity, acquire multispectral vision data of the line body, metal fittings for connecting and fixing the line, and the corridor environment, and bind the acquired data with the inspection pose, environmental temperature and humidity, and electromagnetic interference parameters.

[0051] The imaging parameters in the acquisition module include exposure time, focal length, light sensitivity, and multispectral channel gain.

[0052] When the imaging parameters are dynamically adjusted, they are subject to:

[0053] In the formula: is the adjusted exposure time; is the adjusted focal length; is the adjusted light sensitivity; is the adjusted multispectral channel gain; , , , is the corresponding reference imaging parameter; is the parameter adjustment coefficient; is the voltage level of the power transmission line; is the standard voltage level; is the environmental complexity;

[0054] ​The above formula combines the core influence of the voltage level of the power transmission line and the environmental complexity on the imaging quality, and based on the reference imaging parameters, the parameter adaptation is realized through the adjustment coefficients k1 to k5 set in a targeted manner, so that the exposure time, focal length, sensitivity and multi-spectral channel gain can dynamically match different inspection scenes, ensuring that clear and effective visual data can be collected under different voltage levels and environmental complexities;

[0055] When the collected data is associated and bound with the inspection pose, environmental temperature and humidity and electromagnetic interference parameters, the following is calculated: ;

[0056] In the formula: is the correlation degree; is the time difference between the collected data and the parameter record; is the maximum allowable time difference; is the core feature value of the collected data; is the feature threshold value corresponding to the associated parameter; is the standard feature reference value;

[0057] The above formula takes into account the time consistency and feature matching degree of the collected data and the associated parameters, and through the comparison of the time difference and the maximum allowable time difference, the data timing is synchronized, and with the aid of the correlation check of the core feature value of the collected data and the feature threshold value and the standard feature reference value of the associated parameters, the feature fit degree is determined. Only when the correlation degree is not less than the preset threshold value, the binding is determined to be valid, thereby avoiding the problem of data failure caused by timing misalignment or feature inconsistency;

[0058] When is not less than the preset correlation threshold value, the binding is determined to be valid;

[0059] wherein, The voltage induction sensor carried by the unmanned aerial vehicle collects the electromagnetic induction signal of the power transmission line in real time, and is quantitatively determined based on the corresponding calibration relationship between the electromagnetic induction strength and the voltage level;

[0060] The value range is a preset interval (0, 1), the greater the deviation of the voltage level of the power transmission line from the standard voltage level, the greater the value, and the closer the voltage level to the standard voltage level, the smaller the value; The value range is a preset interval (0, 0.8), the higher the environmental complexity, the greater the value, and the lower the environmental complexity, the smaller the value; The value range is a preset interval (0, 0.6), the higher the environmental complexity, the greater the value, and the lower the environmental complexity, the smaller the value; The value range is a preset interval (0, 0.9), the higher the environmental complexity, the greater the value, and the lower the environmental complexity, the smaller the value; The value range is a preset interval (0, 1.2), the product of the voltage level of the power transmission line and the environmental complexity is greater, and the value is greater, and the product is smaller, and the value is smaller;

[0061] , The environmental factor adaptation weight is represented by a and b, The visibility, wind power and environmental humidity are represented by a and b, The standard visibility reference value, the safety wind power threshold and the standard humidity reference value are represented by a and b, The values are all in the range of (0, 1), and the sum is 1;

[0062] The preprocessing module is used for receiving the collected data after binding, and the data is denoised through the environmental parameter adaptive electromagnetic denoising logic, combined with the inspection pose for image stitching and distortion correction, to construct a standardized three-dimensional data matrix with environmental attribute labels.

[0063] The environmental parameter adaptive electromagnetic denoising logic of the preprocessing module is:

[0064] The denoising model is constructed based on environmental temperature and humidity and electromagnetic interference strength, and the output of the original collected signal after model denoising processing is the signal after denoising: ;

[0065] In the formula, The original collected signal is represented by a and b, , The denoising coefficient is represented by a and b, The environmental temperature and humidity composite parameter is represented by a and b, The electromagnetic interference strength is represented by a and b, The reference temperature is represented by a and b,

[0066] The above formula is based on the interference characteristics of environmental temperature and humidity and electromagnetic interference on the original collected signal, introduces the denoising coefficients a and b to adapt to the linear component proportion of the voltage level of the power transmission line and the electromagnetic interference, and the nonlinear component proportion of the electromagnetic interference, and combines the temperature and humidity composite parameter and the electromagnetic interference strength to denoise the original signal. It can accurately cope with noise interference in different electromagnetic environments, realize adaptive denoising by distinguishing the types of interference components, effectively improve the signal purity, and lay a foundation for subsequent data processing;

[0067] Among them, ;

[0068] In the formula, The temperature and humidity fusion weight is represented by a and b, The actual temperature of the environment is represented by a and b, The actual relative humidity of the environment is represented by a and b, The standard reference humidity is represented by a and b, The standard reference temperature is represented by a and b,

[0069] The above formula considers that the influence degree of temperature and humidity on electromagnetic noise is different, dynamically adjusts the influence proportion of the two by fusing weights, combines the correlation calculation of the actual temperature and humidity with the standard reference value to break the traditional mode of equal weighting of temperature and humidity, and makes the composite parameter more truly reflect the comprehensive effect of the actual environment on electromagnetic noise.

[0070] > 0, the value is larger when the voltage level of the power transmission line is higher and the linear component of the environmental electromagnetic interference intensity accounts for a larger proportion of the total electromagnetic interference, and the value is smaller when the voltage level of the power transmission line is lower and the linear component of the environmental electromagnetic interference intensity accounts for a smaller proportion of the total electromagnetic interference; > 0, the value is larger when the nonlinear component of the environmental electromagnetic interference intensity accounts for a larger proportion of the total electromagnetic interference, and the value is smaller when the nonlinear component of the environmental electromagnetic interference intensity accounts for a smaller proportion of the total electromagnetic interference; ∈ (0, 1), the value is larger when the influence degree of the temperature of the inspection area on electromagnetic noise is greater than the influence degree of humidity on electromagnetic noise, and the value is smaller when the influence degree of the humidity of the inspection area on electromagnetic noise is greater than the influence degree of temperature on electromagnetic noise;

[0071] In the image stitching stage, a scale-invariant feature transform, an accelerated robust feature or an ORB feature extraction algorithm is adopted to extract key feature points in adjacent inspection images, initial matching is performed based on the Euclidean distance of the feature point descriptors, and then the random sample consensus algorithm is used to eliminate the mis-matching feature points in the initial matching result to obtain a pair of accurately matched feature points. A homography transformation matrix between images is solved based on the pair of accurately matched feature points, the spatial coordinate mapping and alignment of adjacent images are completed through the matrix, and finally a multi-band pixel fusion strategy is adopted to perform pixel-level fusion processing on the overlapping region of the aligned images. In the fusion process, the pixel weight of the overlapping region is dynamically allocated based on the image signal-to-noise ratio to eliminate the stitching seam, so as to complete the image stitching.

[0072] After the stitching is completed, the distortion correction is combined with the pitch angle, roll angle and yaw angle of the inspection pose to correct the coordinates: ;

[0073] In the formula: is the original pixel coordinate; is the corrected coordinate; is the pitch angle of the inspection pose; is the yaw angle of the inspection pose; is the roll angle of the inspection pose;

[0074] The above formula is directly related to the pitch angle, yaw angle and roll angle of the UAV inspection posture. In response to the image distortion caused by the attitude change during the flight of the UAV, the original pixel coordinates are accurately corrected through these three key angles, so as to achieve accurate matching between the image spatial coordinates and the actual spatial position of the power transmission line, effectively eliminate the distortion error caused by attitude change, and improve the spatial accuracy of the image and the reliability of subsequent stitching and analysis.

[0075] The standardized three-dimensional data matrix M is a three-dimensional tensor structure, denoted as M. Where i∈[1,X] corresponds to the spatial dimension (x,y,z), j∈[1,Y] corresponds to the multispectral channel, k∈[1,Z] corresponds to the time series, and the matrix elements are... ;

[0076] In the formula: The noise-reduced signal corresponds to the spatial location, spectral channel, and time point; , , These are the weighting coefficients for the spatial, spectral, and temporal dimensions;

[0077] The above formula is based on the denoised signal and introduces weight coefficients for three dimensions: space, spectrum, and time. The weights are assigned according to whether the spatial location is the core inspection area, the ability of the spectral channel to distinguish defect features and its anti-interference ability, the ability of the time point data to resist environmental fluctuations, and the signal-to-noise ratio. A standardized three-dimensional data matrix that takes into account key information in multiple dimensions is constructed, making the matrix more suitable for subsequent defect feature mining needs.

[0078] in, When the spatial location is within the core inspection area of ​​the transmission line The larger the value, the more likely the location is to be a background area or a non-critical inspection area. The smaller the value, the better the spectral channel can effectively distinguish the characteristic differences between defects and normal areas, and the less it is affected by environmental interference. The larger the value, the lower the spectral channel's ability to identify defect features and the greater its susceptibility to environmental interference. The smaller the value, the less the inspection data at that point in time is affected by environmental fluctuations and the higher the data signal-to-noise ratio. The larger the value, the greater the impact of environmental fluctuations on the data and the lower the signal-to-noise ratio. The smaller the value;

[0079] The mining module is used to construct a dynamic convolutional extraction network based on the defect feature attention guidance mechanism, and to mine the corresponding visual features and environmental association attributes of various types of defects from the three-dimensional data matrix, and simultaneously generate a feature vector set with scene adaptation coefficients.

[0080] The defect feature attention guiding mechanism in the mining module is guided by constructing a defect probability weight matrix Feature enhancement is performed: ;

[0081] In the formula: 、 is an attention weight coefficient; is the feature intensity of the candidate defect region; is the total feature intensity of the whole image; is the correlation degree of the candidate region and the environmental parameter; is the global maximum environmental correlation degree;

[0082] The above formula focuses on the defect feature recognition degree and the influence degree of the environmental parameter on the defect feature through the attention weight coefficient, combines the proportion of the feature intensity of the candidate defect region and the total intensity of the whole image, and the ratio of the correlation degree of the candidate region and the environmental parameter and the global maximum correlation degree, realizes the targeted enhancement of the defect feature, makes the defect feature more prominent in the complex environmental background, and thus improves the precision of feature extraction;

[0083] The convolution kernel K(t) of the dynamic convolution extraction network is adaptively adjusted with the network iteration step number t: ;

[0084] In the formula: is the initial convolution kernel; is the convolution kernel adjustment increment; is a scene adaptation coefficient;

[0085] Feature vector set , m is the number of defect types, and each element of the feature vector satisfies , represents the nth visual feature value of the mth defect type, represents the total dimension of the single defect feature;

[0086] The above formula is adaptively adjusted with the network iteration step number, based on the initial convolution kernel, combines the normalized output value of the defect probability weight matrix, the environmental complexity, and the ratio of the voltage level of the power transmission line to the standard voltage level to calculate the adjustment increment, and the scene adaptation coefficient increases with the increase of the environmental complexity and the voltage level, so that the convolution kernel can be continuously optimized according to the real-time changes of the inspection scene, thereby improving the extraction adaptability of the defect feature in different scenes;

[0087] The visual feature value specifically represents the length-width ratio of the outer rectangle of the defect area, the area proportion of the defect area, the edge gradient amplitude mean value, the three-dimensional protrusion / recess depth, the gray level co-occurrence matrix entropy value, the local binary pattern mean value, the texture direction consistency, the feature spectral band reflectivity, the spectral band difference value, and the spectral time stability coefficient. ∈(0, 1), when the recognition degree of the visual feature of the power transmission line defect is higher, the value is larger, and when the defect feature is disturbed by the environment, the recognition degree is lower; ∈(0, 1), when the influence degree of the environmental parameter on the defect feature is higher, the value is larger, and when the influence of the environmental parameter on the defect feature is negligible, the value is smaller; The application candidate defect area corresponds to the environmental parameter set and the preset defect high-occurrence environmental parameter feature set. The normalized output value of the defect probability weight matrix, the environmental complexity, and the ratio of the voltage level of the power transmission line to the standard voltage level are weighted and calculated. > 0, and is subject to the environmental complexity and the voltage level of the power transmission line, and the value is larger when the environmental complexity is higher and the voltage level of the power transmission line is higher, and the value is smaller when the environmental complexity is lower and the voltage level of the power transmission line is lower.

[0088] The recognition module is used for classifying and matching the feature vector set to complete defect type determination and level evaluation, and constructing a growth rule model based on historical defect data of the power transmission line, and applying the model to output evolution trend prediction results of the defect in a preset time interval in the future;

[0089] The classification matching operation in the recognition module is:

[0090] A standard defect feature vector library is constructed The matching similarity between the feature vector set and the standard defect feature vector library is calculated:

[0091] In the formula: is the matching similarity; is the total dimension of the features of a single defect; is the feature vector of the mth defect; is a preset minimum value; is a scene adaptation coefficient, when When the matching similarity is not less than a preset matching threshold, it is determined as the corresponding defect type.

[0092] ​​The formula is based on the total dimension of the characteristics of single-class defects, calculates the correlation degree of the feature vector to be identified and the corresponding vector in the standard defect feature vector library, introduces a preset minimum value to avoid calculation anomalies, and integrates scene adaptation coefficient to associate the early inspection scene information, so as to no longer rely on fixed matching standards, but dynamically adjust the matching judgment basis according to the actual scene, thereby improving the accuracy and adaptability of defect type judgment in different inspection scenes;

[0093] The growth law model is used to calculate the defect size, and the expression is: ;

[0094] In the formula, is the defect size at time t; is the initial defect size, and the source verification module; 、 is the evolution coefficient; is the environmental influence factor at time t; is the defect size at time t; is the critical safety size of the defect;

[0095] The above formula comprehensively considers the initial defect size, environmental influence factor and defect size feedback effect, and the evolution coefficient According to the driving effect value of the environmental influence factor on the evolution of the defect, then according to the distance adjustment between the current defect size and the critical safety size, the inhibition effect of the defect size on the evolution is embodied, and the dynamic change of the environmental influence factor is combined to realize the accurate prediction of the defect size evolution;

[0096] The future evolution trend prediction result of the defect includes the size growth rate and the critical time: ;

[0097] In the formula, is the size growth rate; is the critical time; is the current inspection time point;

[0098] The above formula is based on the current inspection time point, the real-time size of the defect and the critical safety size, calculates the growth rate through the change trend of the defect size, and then deduces the critical time when the defect reaches the critical safety size, which extends the defect evolution prediction from simple size estimation to the clear key operation and maintenance time node, and provides specific time reference for operation and maintenance decision;

[0099] wherein, The value range of is preset as [0, 1], and the higher the environmental temperature, the greater the electromagnetic interference intensity and the higher the environmental humidity, the greater the value, and vice versa. ∈(0,1], the stronger the driving effect of the electromagnetic interference intensity, temperature and humidity and other environmental factors of the power transmission line on the defect evolution, the larger the value, the weaker the driving effect of the environmental factors on the defect evolution, the smaller the value; ∈(0,0.8], the closer the current size of the defect to the critical safe size, the stronger the inhibition of the defect size feedback on the evolution, the smaller the value, the farther the current size of the defect from the critical safe size, the weaker the inhibition of the defect size feedback on the evolution, the larger the value;

[0100] The verification module is used to receive the defect judgment, level evaluation and evolution trend prediction results output by the identification module, and through real-time feature backtracking verification, historical defect data comparison and digital twin scene simulation, it eliminates misjudgment data caused by environmental interference, synchronously supplements the geometric size and material loss rate quantitative parameters of the defect to form a verified feedback data package;

[0101] The real-time feature backtracking verification operation in the verification module is: ;

[0102] The above formula calculates the consistency coefficient by comparing the differences between the current identification feature vector, the backtracking extracted feature vector and the corresponding feature values in the standard defect feature vector library, and combining the verification coefficient. When the coefficient does not exceed the preset threshold, it passes the verification, otherwise it triggers feature re-extraction. Through triple feature comparison, the accuracy of feature extraction is ensured, and invalid features caused by environmental interference or extraction deviation are effectively eliminated.

[0103] In the formula: is the feature consistency coefficient; is the total number of feature dimensions; is the nth feature value of the current identification feature vector; is the nth feature value of the backtracking extracted feature vector; is the nth standard feature value of the mth defect in the standard defect feature vector library; is the verification coefficient, when does not exceed the preset consistency threshold, it is determined that the current feature vector passes the first verification and enters the historical defect data comparison link; if exceeds the preset consistency threshold, it is determined that the current feature has environmental interference or extraction deviation, and the feature re-extraction process is triggered, that is, the standardized three-dimensional data matrix in the preprocessing module is secondarily mined for features until the feature passes the verification or reaches the preset extraction upper limit;

[0104] The historical defect data comparison operation in the verification module is: ;

[0105] The formula assigns different weights to different defect parameters. The weight coefficient is set according to the influence degree of the parameter on defect type judgment and evolution prediction. By calculating the weighted Euclidean distance between the current defect parameter set and the historical defect parameter set, it is determined whether they are the same type or homologous defects, which not only ensures the dominant role of key parameters, but also realizes the effective reuse of historical data, accurately identifies new defects or abnormal defects, and perfects the defect database;

[0106] In the formula: is the weighted Euclidean distance between the current defect parameter set and the historical defect parameter set; is the total number of defect parameter dimensions participating in comparison; is the weight coefficient of the i th defect parameter; is the measured quantitative value of the i th dimensional parameter of the defect detected by the current inspection; is the i th dimensional parameter record value of the candidate matching defect in the preset historical defect database. If the preset threshold is not exceeded, it is determined that the current defect and the corresponding historical defect are the same type / homologous defects, and the evolution law model parameter, disposal priority and operation and maintenance strategy of the historical defect are directly reused. If the preset threshold is exceeded, it is determined to be a new defect or an abnormal defect, triggering the digital twin scene deep simulation verification of the verification module, and the current defect parameter set is labeled and supplemented to the historical defect database;

[0107] The digital twin scene simulation operation in the verification module is: ;

[0108] The formula integrates electromagnetic characteristic coefficients, line operating current, defect geometric size estimated value, environmental temperature, humidity, mechanical characteristic coefficients and line stress, comprehensively considers the comprehensive influence of electromagnetic characteristics, mechanical properties and environmental factors on defect characteristics, constructs a complete theoretical feature vector by calculating the theoretical characteristic value of each dimension, compares and calculates the simulation error with the actual recognition feature vector, verifies the feature effectiveness or optimizes the parameters according to the error result, improves the reliability of defect recognition and enriches the feature library;

[0109] In the formula: is the n th dimensional value of the theoretical feature vector of the defect; is the n th dimensional electromagnetic characteristic coefficient; is the line operating current; is the current defect geometric size estimated value; , is the environmental attenuation coefficient; is the environmental temperature; is the environmental humidity; is the n th dimensional mechanical characteristic coefficient; is the line stress, and all dimensions are completed After calculation, the complete defect theoretical feature vector is constructed, and a simulation error is calculated based on the theoretical feature vector and the actually identified feature vector. If the simulation error does not exceed a preset simulation threshold, it is determined that the deviation between the theoretical feature vector and the actually identified feature vector is within an allowable range, the feature extraction result of the current defect is valid, the third recheck is passed, and the theoretical feature vector is supplemented to the defect feature library as a reference for subsequent feature optimization of the same type of defect; if the simulation error exceeds the preset simulation threshold, the 、 simulation error is recalculated and the simulation error is solved again. If the adjusted simulation error still exceeds the preset simulation threshold, a comprehensive determination is made in combination with the value of the real-time feature backtracking check: if the difference between the and the preset consistency threshold does not exceed a preset deviation value, it is determined that the current defect is a new type of defect, and the complete defect theoretical feature vector and the actually identified feature vector are fused and recorded in the defect feature library; if the deviates from the preset consistency threshold by a large margin, it is determined to be misjudgment data caused by environmental interference and is rejected;

[0110] The geometric size and material loss rate quantitative parameter of the supplemented defect are subjected to: ;

[0111] In the formula: is the defect geometric size; is the material loss rate; is the set of viewing angles; is the measurement value of the i-th viewing angle; is the measurement error of the i-th viewing angle; is the preset minimum value; is the multispectral signal intensity of the defect area; is the standard signal intensity of the intact area; is the defect area; is the total area of the detection area;

[0112] The above formula is calculated by weighting the measurement values of multiple viewing angles and the measurement errors of each viewing angle, reducing the deviation of single viewing angle measurement, and comprehensively deducing based on the multispectral signal intensity ratio of the defect area to the intact area and the ratio of the defect area to the total area of the detection area. The design of multi-view fusion and multispectral signal combination makes the quantitative parameter more truly reflect the actual condition of the defect;

[0113] wherein, ∈(0,1], the value is larger when the feature backtracking time difference is smaller and the defect feature recognition degree is higher, and the value is smaller when the feature backtracking time difference is larger and the defect feature recognition degree is lower; E (0, 1), and the sum of all weight coefficients is 1, and the value is larger when the influence degree of the corresponding defect parameter on the defect type judgment and evolution prediction is higher, and the value is smaller when the influence degree of the corresponding defect parameter on the defect type judgment and evolution prediction is lower; (0, 2.0], the higher the voltage level of the power transmission line, the higher the defect recognition degree of the n-dimensional electromagnetic feature and the smaller the environmental interference, the stronger the correlation with defect evolution, the larger the value, and vice versa; E (0, 0.8], the n-dimensional feature is more sensitive to temperature, the environmental temperature deviates from the standard value more, the defect material is more sensitive to temperature, the larger the value, and vice versa; E (0, 0.7], the n-dimensional feature is more sensitive to humidity, the environmental humidity deviates from the standard value more, the defect material is more sensitive to humidity, the larger the value, and vice versa; E (0, 1.5], the closer the defect is related to the mechanical properties, the greater the line operating stress, the higher the feature measurement accuracy and the smaller the interference, the larger the value, and vice versa;

[0114] The feedback module is used for receiving the feedback data packet output by the verification module and synchronously pushing to the preset inspection terminal.

[0115] The acquisition module is connected with the preprocessing module through wireless network interaction, the preprocessing module is connected with the mining module through wireless network interaction, the mining module is connected with the identification module through wireless network interaction, the identification module is connected with the verification module through wireless network interaction, and the verification module is connected with the feedback module through wireless network interaction.

[0116] In this embodiment, the acquisition module runs dynamic adjustment of imaging parameters based on the voltage level of the power transmission line and the environmental complexity, acquires multispectral visual data of the line body, metal fittings for line connection and fixation, and the corridor environment, and associates and binds the acquired data with the inspection pose, environmental temperature and humidity, and electromagnetic interference parameters. The preprocessing module runs behind to receive the completed bound acquired data, performs data denoising through the environmental parameter adaptive electromagnetic denoising logic, performs image stitching and distortion correction in combination with the inspection pose, to construct a standardized three-dimensional data matrix with environmental attribute labels. The mining module further constructs a dynamic convolution extraction network based on a defect feature attention guide mechanism, mines corresponding visual features and environmental associated attributes of each type of defect from the three-dimensional data matrix, synchronously generates a feature vector set with scene adaptation coefficients, and classifies and matches the feature vector set through the recognition module to complete defect type determination and grade evaluation. A growth rule model is constructed based on historical defect data of the power transmission line, and the model output is used to predict the evolution trend of the defect in a preset time interval in the future. The verification module receives the defect determination, grade evaluation, and evolution trend prediction results output by the recognition module, removes misjudgment data caused by environmental interference through real-time feature backtracking verification, historical defect data comparison, and digital twin scene simulation, synchronously supplements the geometric size and material loss rate quantitative parameters of the defect to form a verified feedback data package. Finally, the feedback module receives the feedback data package output by the verification module and synchronously pushes it to the preset inspection terminal.

[0117] In the above embodiment, the system can adaptively adjust the imaging according to the voltage level and environmental complexity of the power transmission line in the power transmission line inspection scene, improve the effectiveness of the acquired data, reduce electromagnetic interference, correct image deviation, accurately determine the defect type and grade, scientifically predict the future evolution trend of the defect, effectively remove misjudgments caused by environmental interference, supplement the geometric size and material loss rate parameters of the defect, provide accurate support for operation and maintenance, greatly improve the inspection efficiency and accuracy, and effectively ensure the safe and stable operation of the power transmission line.

[0118] Embodiment 2:

[0119] In the specific implementation layer, based on embodiment 1, this embodiment refers to Figure 2 Further specific description is made to the unmanned aerial vehicle inspection deep visual intelligent recognition system of the power transmission line in embodiment 1:

[0120] The unmanned aerial vehicle inspection deep visual intelligent recognition method of the power transmission line comprises:

[0121] Dynamic adjustment of imaging parameters based on the voltage level of the power transmission line and the environmental complexity, acquisition of multispectral visual data of the line body, metal fittings, and the corridor environment, and association and binding with the inspection pose, temperature and humidity, and electromagnetic interference parameters;

[0122] The binding data is processed by an environmental parameter adaptive electromagnetic noise reduction logic, image stitching and distortion correction are completed in combination with the inspection pose, and a standardized three-dimensional data matrix with an environmental attribute label is constructed;

[0123] A dynamic convolution extraction network is constructed based on a defect feature attention guide mechanism, defect visual features and environmental correlation attributes are mined, and a feature vector set with a scene adaptation coefficient is generated;

[0124] The feature vector set is classified and matched with a standard defect feature vector library, defect type determination and grade evaluation are completed, and a growth rule model is constructed based on historical defect data to predict defect evolution trends;

[0125] Through real-time feature backtracking verification, historical defect data comparison and digital twin scene simulation, false positive data is removed, and verified feedback data packets are formed by supplementing defect geometric size and material loss rate quantitative parameters;

[0126] The verified feedback data packets are synchronously pushed to the preset inspection terminal.

[0127] In summary, the system and method in the above embodiments can dynamically adapt to the voltage level of the power transmission line and the environmental complexity to optimize the imaging effect, accurately collect multi-dimensional visual data and associate key environmental parameters, improve the data standardization level through adaptive noise reduction, accurate stitching and distortion correction, efficiently mine defect features and environmental correlation attributes through the attention guide mechanism and dynamic convolution network, accurately determine the defect type and grade and predict the future evolution trend in combination with historical data, remove false positive data through multiple verifications and supplement quantitative parameters, greatly improve the inspection efficiency and defect recognition accuracy, provide accurate data support for power transmission line operation and maintenance, effectively reduce operation and maintenance costs, and ensure the stable operation of the power transmission line.

[0128] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for part of the technical features; and these modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines, characterized in that: include: The acquisition module is used to dynamically adjust imaging parameters based on the voltage level of the transmission line and the complexity of the environment, acquire multispectral visual data of the line body, metal fittings for line connection and fixing, and corridor environment, and associate and bind the acquired data with inspection posture, environmental temperature and humidity and electromagnetic interference parameters. The preprocessing module receives the collected data after binding, performs data denoising through environmental parameter adaptive electromagnetic noise reduction logic, and performs image stitching and distortion correction in combination with the inspection pose to construct a standardized three-dimensional data matrix with environmental attribute labels. The mining module is used to construct a dynamic convolutional extraction network based on the defect feature attention guidance mechanism, and to mine the corresponding visual features and environmental association attributes of various types of defects from the three-dimensional data matrix, and simultaneously generate a feature vector set with scene adaptation coefficients. The identification module is used to classify and match the feature vector set to complete the defect type determination and level assessment, and to build a growth law model based on the historical defect data of the transmission line. The model is used to output the prediction results of the evolution trend of the defect within the future preset time interval. The verification module is used to receive the defect judgment, level assessment and evolution trend prediction results output by the identification module. It eliminates misjudged data caused by environmental interference through real-time feature backtracking verification, comparison of historical defect data and digital twin scene simulation. It also supplements the geometric size and material loss rate quantification parameters of the defect to form a verified feedback data packet. The feedback module is used to receive the feedback data packets output by the verification module and push them synchronously to the preset inspection terminal.

2. The deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines according to claim 1, characterized in that, The imaging parameters in the acquisition module include exposure time, focal length, sensitivity, and multispectral channel gain. When imaging parameters are dynamically adjusted, they follow the following rules: ； In the formula: This is the adjusted exposure time; To adjust the focal length; For the adjusted ISO; Adjusted multispectral channel gain; , , , For the corresponding reference imaging parameters; Adjustment coefficients for parameters; Voltage level of the transmission line; Standard voltage level; For environmental complexity; When the collected data is associated with and bound to the inspection posture, ambient temperature and humidity, and electromagnetic interference parameters, the following calculation is performed: ； In the formula: For relevance; The time difference between data collection and parameter recording; To allow the maximum time difference; These are the core feature values ​​of the collected data; The feature threshold corresponding to the associated parameter; Standard feature reference value; when The binding is considered valid if the value is not less than the preset association threshold.

3. The deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines according to claim 1, characterized in that, The environmental parameter adaptive electromagnetic noise reduction logic of the preprocessing module is as follows: A noise reduction model is constructed based on ambient temperature and humidity and electromagnetic interference intensity. The output of the original acquired signal after model noise reduction processing is the denoised signal: ； In the formula: The original acquired signal; , The noise reduction coefficient; This refers to a composite parameter of ambient temperature and humidity. Electromagnetic interference intensity; This is a reference temperature.

4. The deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines according to claim 1, characterized in that, During the image stitching stage, the stitching operation is performed using any one of the following algorithms: scale-invariant feature transformation, accelerated robust feature extraction, or ORB feature extraction algorithm. After the stitching is completed, distortion correction is performed by combining the pitch angle, roll angle, and yaw angle of the inspection position to correct the coordinates: ； In the formula: These are the original pixel coordinates; The coordinates are after correction; The pitch angle for the inspection position; The yaw angle for the inspection position; The roll angle for the inspection position; The standardized three-dimensional data matrix M is a three-dimensional tensor structure, denoted as... Where i∈[1,X] corresponds to the spatial dimension (x,y,z), j∈[1,Y] corresponds to the multispectral channel, k∈[1,Z] corresponds to the time series, and the matrix elements are... ; In the formula: The noise-reduced signal corresponds to the spatial location, spectral channel, and time point; , , These are the weighting coefficients for the spatial, spectral, and temporal dimensions.

5. The deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines according to claim 1, characterized in that, The defect feature attention guidance mechanism in the mining module constructs a defect probability weight matrix. Perform feature enhancement: ； In the formula: , This refers to the attention weighting coefficient. The feature intensity of the candidate defect region; The total intensity of features across the entire image; The correlation between candidate regions and environmental parameters; This represents the maximum global environmental correlation. The convolution kernel K(t) of the dynamic convolution extraction network is adaptively adjusted with the number of network iterations t: ； In the formula: Use the initial convolution kernel; Adjust the increment for the convolution kernel; For scene adaptation coefficients; The feature vector set m is the number of defect types, and each feature vector The elements satisfy , This represents the nth visual feature value of the m-th type of defect. This represents the total dimension of features for a single type of defect.

6. The deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines according to claim 1, characterized in that, The classification and matching operation in the recognition module is as follows: Construct a standard defect feature vector library Then calculate the matching similarity between the feature vector set and the standard defect feature vector library: ； In the formula: To match similarity; The total dimension of features for a single type of defect; Let be the feature vector of the m-th type of defect; This is the preset minimum value; For scene adaptation coefficient, when If the value is not less than the preset matching threshold, it is determined to be the corresponding defect type; The growth law model is used to calculate the defect size, and its expression is: ； In the formula: Let be the defect size at time t; Initial defect size, source verification module; , Evolution coefficient; for Environmental influencing factors at any given time; for The defect size at any given moment; This is the critical safety size for defects; The predicted future evolution trend of the defect includes the size growth rate and the critical time: ； In the formula: This represents the growth rate of the size. This is the critical time. This is the current inspection time.

7. The deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines according to claim 1, characterized in that, The quantitative parameters for the geometric dimensions and material loss rate of the supplementary defects shall conform to: ； In the formula: The defect geometry; Material loss rate; A set of perspectives; The measurement value is for the i-th viewpoint; Let be the measurement error of the i-th viewpoint; This is the preset minimum value; The multispectral signal intensity of the defect region; Standard signal strength for intact areas; The defect area; This represents the total area of ​​the detection zone.

8. The deep vision intelligent recognition system for unmanned aerial vehicle (UAV) inspection of power transmission lines according to claim 1, characterized in that, The acquisition module is interconnected with a preprocessing module via a wireless network. The preprocessing module is interconnected with a mining module via a wireless network. The mining module is interconnected with an identification module via a wireless network. The identification module is interconnected with a verification module via a wireless network. The verification module is interconnected with a feedback module via a wireless network.

9. A method for deep visual intelligent recognition of UAV inspection of power transmission lines, wherein the method is an implementation method of the deep visual intelligent recognition system for UAV inspection of power transmission lines as described in any one of claims 1-8, characterized in that, include: Based on the voltage level and environmental complexity of the transmission line, the imaging parameters are dynamically adjusted to collect multispectral visual data of the line body, metal accessories and corridor environment, and these data are associated with the inspection posture, temperature and humidity and electromagnetic interference parameters. By processing the bound data through environmental parameter adaptive electromagnetic noise reduction logic, and combining the inspection pose to complete image stitching and distortion correction, a standardized three-dimensional data matrix with environmental attribute labels is constructed. A dynamic convolutional extraction network is constructed based on a defect feature attention guidance mechanism to mine visual features of defects and environmental association attributes, and generate a feature vector set with scene adaptation coefficients. The feature vector set is classified and matched with the standard defect feature vector library to complete the defect type determination and level assessment. Based on historical defect data, a growth law model is constructed to predict the defect evolution trend. By using real-time feature backtracking verification, historical defect data comparison, and digital twin scene simulation to eliminate misjudged data, and supplementing defect geometric dimensions and material loss rate quantitative parameters to form a verification feedback data package; The feedback data packet that has completed verification will be pushed synchronously to the preset inspection terminal.

Images