Unmanned aerial vehicle inspection image enhancement method based on motion blur restoration

Abstract

The application discloses an unmanned aerial vehicle inspection image enhancement method based on motion blur restoration, and relates to the technical field of unmanned aerial vehicle image processing.The method first collects and pre-processes multi-source sensing data and aerial photography images, and constructs a sliding window;calculates a flight stability index based on an entropy weight stability algorithm, and completes three-level flight state division;then adaptively estimates a blur kernel according to the state classification, matches a corresponding de-blurring enhancement strategy, and realizes differentiated image quality restoration;meanwhile, the on-board load and battery capacity are monitored, the algorithm complexity is dynamically adjusted, and finally a clear inspection image is output;through flight state classification and adaptive blur kernel estimation, the application realizes accurate restoration of motion blur, effectively improves image detail and texture recognition degree;meanwhile, the on-board load and capacity are dynamically monitored, the algorithm complexity is cooperatively adjusted, the processing effect and energy consumption are balanced, real-time performance and endurance are guaranteed, and the adaptability and reliability of the unmanned aerial vehicle inspection are enhanced.

Description

Technical Field

[0001] This invention relates to the field of UAV image processing technology, specifically to a method for enhancing UAV inspection images based on motion blur restoration. Background Technology

[0002] With its advantages of high efficiency, flexibility, and non-contact operation, drone inspection has been widely applied in many industrial inspection fields such as power transmission, oil and gas pipelines, infrastructure, and new energy power plants. Aerial images are the core data carrier for target observation, defect identification, and status determination. However, in actual outdoor inspection operations, drones are easily affected by multiple factors such as natural airflow disturbances, flight maneuvers, and gimbal attitude fluctuations. Aerial images are prone to varying degrees of motion blur, resulting in loss of image details, blurred textures, and unclear target features, directly reducing the effectiveness and reliability of inspection data. At the same time, there are problems such as sampling frequency differences, noise interference, abnormal data, and temporal misalignment between the various types of sensor data on the drone and the aerial images. The real-time fluctuations in flight status are closely related to the imaging quality. How to achieve accurate motion blur restoration and improve the quality of inspection images based on existing onboard resources has become a core technical problem that urgently needs to be solved in the field of drone-based automated inspection, and it is also a key link to ensure the accurate and efficient conduct of inspection operations.

[0003] Existing UAV inspection image enhancement and blur restoration technologies mostly employ fixed processing schemes without dynamically adapting to the real-time flight status of the UAV. This results in significant limitations in image restoration effects for different degrees and distributions of blur. Traditional blur kernel estimation methods are simplistic and cannot adapt to various imaging scenarios such as uniform blur, locally differentiated blur, and severe motion blur, easily leading to incomplete blur restoration or image distortion. Most schemes do not perform systematic preprocessing and temporal coordination of multi-source sensor data, and data noise and temporal deviations further reduce the accuracy of blur restoration. Furthermore, they do not integrate airborne load, battery power consumption status, and algorithm operation, resulting in fixed algorithm complexity. This can easily lead to problems such as airborne resource overload, reduced endurance, and insufficient real-time processing. Traditional technologies have poor adaptability to complex outdoor flight conditions, making it difficult to stably output high-quality inspection images and failing to meet the actual operational needs of refined, long-duration, and automated inspections. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a UAV inspection image enhancement method based on motion blur restoration. This method first collects multi-source sensor data and aerial images, and constructs a sliding window after denoising and synchronization. It then uses the entropy weight stability algorithm to calculate the flight stability index, divides the state into three levels: stable, slightly jittery, and severely jittery. Based on the state, it adaptively estimates the global, regional, or spatiotemporal joint blur kernel and matches a differentiated deblurring enhancement strategy. At the same time, it monitors the airborne load and battery power, and dynamically adjusts the algorithm complexity through resource coupling parameter tuning.

[0005] To solve the above-mentioned technical problems, this invention provides the following technical solution: a method for enhancing UAV inspection images based on motion blur restoration, the specific steps of which are as follows: S100, Data Acquisition and Preprocessing: Real-time acquisition of multi-source sensor data and aerial image data required for UAV inspection, noise reduction, abnormal data removal and time axis alignment processing of various types of data, and construction of a unified time sequence sensor data sliding window; S200 Flight Status Classification: Based on preprocessed multi-source sensor data, the real-time flight stability index is calculated using the entropy weight stability algorithm, and the three-level flight status classification is completed according to the preset threshold. S300, Fuzzy Kernel Adaptive Estimation: Based on the graded flight stability state, the corresponding fuzzy kernel estimation method is used to complete the fuzzy kernel solution and generate fuzzy kernel parameters that are adapted to the current flight state; S400, Graded Image Enhancement: Based on the flight status graded results and combined with the corresponding blur kernel parameters obtained by solving, the corresponding deblurring enhancement strategy is matched to perform differentiated image quality restoration and optimization on the inspection image; S500, Dynamic Algorithm Complexity Adjustment: Real-time monitoring of UAV onboard load and battery level, calculation of comprehensive resource consumption value, combined with flight stability index, and calculation of algorithm complexity adjustment coefficient using resource coupling parameter tuning algorithm. Based on this coefficient, relevant algorithm parameters are adjusted, and finally, clear UAV inspection images are output.

[0006] Furthermore, the multi-source sensor data includes triaxial acceleration data and triaxial angular velocity data collected by the airborne IMU, longitude data, latitude data, flight altitude data and three-dimensional flight speed data collected by the airborne GPS, pitch angle data, roll angle data and yaw angle data collected by the gimbal encoder; the aerial image data is the inspection target scene image data captured by the airborne visible light camera. The IMU data is continuously collected at a fixed high-frequency sampling rate of 100-200Hz, the GPS data is synchronously collected at a set sampling frequency of 5-10Hz, the gimbal encoder data is collected in real time at a frequency of 20-50Hz as the gimbal attitude changes, and the visible light camera collects aerial images frame by frame at a fixed frame rate of 25-30 frames per second.

[0007] Furthermore, when denoising various types of data, Kalman filtering is used for attitude data denoising of IMU data, moving average filtering is used for GPS data to eliminate positioning drift noise, mean filtering is used for gimbal encoder data to smooth angle fluctuation noise, and Gaussian filtering is used for aerial image data to remove imaging noise. The 3σ criterion is used to identify and remove discrete abnormal numerical points in various types of sensor data, and abnormal image frames with overexposure and underexposure problems in aerial images are screened out. Using the acquisition timestamp of the aerial image frame as a unified benchmark, multi-source sensor data with different sampling frequencies are calibrated and matched to the acquisition time of the corresponding image frame through linear interpolation. After completing the time synchronization of all data, a unified time-series sensor data sliding window is constructed.

[0008] Furthermore, the mathematical expression for the entropy weight stability algorithm is: in, Let be the flight stability index of the UAV at time t, with a value range of [0,1]. The larger the value, the better the stability. Let be the information entropy of the i-th type of sensor data at time t, used to characterize the disorder or fluctuation complexity of the corresponding sensor data, where i is the sensor category number, i=1 corresponds to IMU acceleration data, i=2 corresponds to GPS speed data, and i=3 corresponds to gimbal angle data. Let be the variance of the i-th type of sensor data within the sliding window at time t, which is used to reflect the dispersion or fluctuation of the corresponding sensor data within the sliding window; The maximum permissible variance of the i-th type of sensor data under extreme shaking flight conditions is obtained by collecting sample data and conducting statistical analysis during offline calibration under various extreme flight scenarios such as strong wind interference, sharp turns, and sudden ascents and descents.

[0009] Furthermore, the three-level flight state classification uses the real-time flight stability index calculated by the entropy-weighted stability algorithm. Using this as the core criterion, preset thresholds are set by collecting sensor data from the UAV offline under various inspection scenarios, including stable cruise, gust disturbances, and maneuvering, and then statistically analyzing the data. The distribution range was then calibrated and determined, with the specific division criteria being as follows: When the value is ≥0.8, the drone is considered to be in a stable flight state; when 0.5≤ When the value is less than 0.8, the drone is determined to be in a state of slight shaking during flight; when... When the value is less than 0.5, the UAV is determined to be in a state of severe shaking during flight, and the flight state classification result is adjusted accordingly. The real-time calculation updates frame by frame to match the corresponding fuzzy kernel estimation method and image enhancement processing strategy.

[0010] Furthermore, the fuzzy kernel estimation method is matched one-to-one with the three flight states. In the stable flight state, the fuzzy characteristics of the entire inspection image are spatially uniformly distributed. Global single fuzzy kernel estimation is adopted, and the entire inspection image is treated as a unified processing object to complete the solution of a single fuzzy kernel. In the slight jitter state, there are differential fuzziness in different regions of the inspection image. Region adaptive fuzzy kernel estimation is adopted, and the inspection image is divided into multiple non-overlapping square image blocks. The local fuzzy kernel is solved for each image block, and the fuzzy kernels of adjacent image blocks are smoothly fused to generate a spatially continuous region adaptive fuzzy kernel. In the severe jitter state, the inspection image has severe motion fuzziness. Spatiotemporal joint fuzzy kernel estimation is adopted, and the motion correlation features of the current frame and the previous and next frames are combined to construct a feature tensor that fuses spatial image features and temporal motion features to complete the solution of the final fuzzy kernel.

[0011] Furthermore, the deblurring and enhancement strategy is executed based on the flight state classification results and the corresponding blur kernel parameters. Under stable flight conditions, motion blur removal of the entire inspection image is completed using a global single blur kernel, while simultaneously performing basic noise reduction and color correction, preserving the original imaging features of the inspection scene, and completing basic image quality optimization. Under slight shaking conditions, differentiated deblurring processing is completed using a region adaptive blur kernel, while simultaneously performing edge enhancement and detail sharpening, repairing local edge blur caused by shaking, improving image detail recognition, and completing regional image quality restoration. Under severe shaking conditions, full-image depth deblurring is completed using a spatiotemporal joint blur kernel, while simultaneously performing texture reconstruction and feature enhancement, restoring target details and texture information lost due to severe shaking, completing differentiated image quality restoration and optimization of the inspection image, and ensuring the image quality adaptability of the inspection image under different flight conditions.

[0012] Furthermore, the real-time monitored UAV onboard load specifically includes the real-time utilization rate of the UAV's onboard central processing unit, the real-time utilization rate of the onboard graphics processor, the real-time occupancy rate of the onboard running memory, the task queue length, and the task execution response time. These data are collected at fixed intervals to monitor the occupancy of onboard resources in real time. The real-time monitored UAV battery power specifically includes the remaining battery power percentage, battery output voltage, and battery temperature. These battery-related data are collected in real time, and the battery charging and discharging status is recorded synchronously. This provides accurate basic data for the resource coupling parameter tuning algorithm, ensuring the accuracy of the algorithm complexity adjustment coefficient calculation, and thus enabling precise adjustment of relevant algorithm parameters.

[0013] Furthermore, the mathematical expression of the resource coupling parameter tuning algorithm is: in, The algorithm complexity adjustment coefficient at time t is in the range of [0,1]. The higher the coefficient value, the higher the complexity of the algorithm processing at the corresponding stage. The flight stability index at time t is the output of the entropy-weighted stability algorithm. The comprehensive resource consumption value at time t is obtained by weighted summation of load and battery power, specifically as follows: =0.6L+0.4P, where L is the comprehensive normalized value of the airborne load at time t, which is calculated by weighting the utilization rate of the airborne central processing unit, the utilization rate of the graphics processing unit, and the running memory usage rate; P is the comprehensive normalized value of the battery power at time t, which is calculated by weighting the remaining battery power percentage, the real-time discharge current, and the battery core temperature. This value is an intermediate variable used to characterize the current resource usage status of the UAV's airborne platform. The maximum resource carrying capacity threshold of the airborne platform is obtained through offline data collection and calibration under extreme conditions such as full-load operation and extreme endurance consumption.

[0014] Compared with existing technologies, this UAV inspection image enhancement method based on motion blur restoration has the following advantages: I. This invention achieves efficient and accurate restoration of motion blur in UAV inspection images through multi-source sensor data preprocessing and flight state classification, combined with fuzzy kernel adaptive estimation and a graded image enhancement process. First, it performs denoising, anomaly removal, and time-series alignment on various types of sensor data and aerial images to ensure data consistency. Then, it relies on stable state classification logic to distinguish different flight conditions, avoiding the adaptability issues caused by uniform processing. It matches specific fuzzy kernel solution methods to different flight states, aligning with the actual blur distribution characteristics of the images. Finally, it combines corresponding enhancement strategies to achieve differentiated image quality optimization, accurately repairing motion blur problems of varying degrees. This preserves the original imaging features of the inspection scene while improving image details and texture recognition, making the clarity of inspection images adaptable to various flight conditions. This provides high-quality visual evidence for target identification and fault determination, enhancing the accuracy and practicality of UAV inspection operations.

[0015] II. This invention achieves efficient collaboration between image enhancement processing and airborne platform resources by real-time monitoring of the UAV's onboard load and battery power status, combined with a resource coupling parameter tuning mechanism to dynamically adjust algorithm complexity. It collects onboard hardware operation and battery power consumption data in real time, calculates adjustment coefficients based on flight stability, and precisely controls algorithm operating parameters to balance processing effectiveness and resource consumption. This avoids excessive load and energy waste on the airborne platform. The design requires no additional hardware load, adapts to the lightweight computing needs of UAVs, ensures the real-time and continuous nature of image enhancement processing, and extends the endurance of inspection operations. Even under resource-constrained conditions, it maintains stable image enhancement effects, improves the environmental adaptability and continuous operation capability of the UAV inspection system, and provides reliable image output for long-term, complex outdoor inspections.

[0016] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0018] Figure 1 This is a flowchart of a UAV inspection image enhancement method based on motion blur restoration; Figure 2 This is a schematic diagram of data transmission for an image enhancement method for UAV inspection based on motion blur restoration. Figure 3 This is a schematic diagram of the data transmission process in the dynamic adjustment step of the algorithm complexity of this invention. Detailed Implementation

[0019] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0020] Example 1: This embodiment applies to autonomous drone inspection of outdoor high-voltage power lines. High-voltage power lines are mostly distributed in high-altitude areas in the wild. During inspection, drones are susceptible to attitude fluctuations caused by strong high-altitude airflow, flight attitude fine-tuning, and line crossing maneuvers, resulting in varying degrees of motion blur in aerial images. Furthermore, high-voltage power line inspection requires continuous long-term operation, placing stringent demands on the onboard platform's resource consumption and endurance. Traditional image enhancement methods cannot adapt to changes in flight status and onboard resource constraints, making it difficult to output clear images that meet fault detection requirements. This embodiment fully adopts a drone inspection image enhancement method based on motion blur restoration. Through end-to-end collaborative processing, it accurately solves the motion blur problem in power line inspection images while optimizing onboard resource utilization efficiency, providing high-quality visual data for hazard identification, equipment status monitoring, and fault location of high-voltage power lines. Figure 1 As shown, the specific implementation steps are as follows: S100, Data Acquisition and Preprocessing: Real-time acquisition of multi-source sensor data and aerial image data required for UAV inspection of high-voltage power lines. Multi-source sensor data includes three-axis acceleration and angular velocity data acquired by the airborne IMU, longitude, latitude, flight altitude, and three-dimensional flight speed data acquired by the airborne GPS, and pitch, roll, and yaw angle data acquired by the gimbal encoder. Aerial image data consists of images of power lines, towers, fittings, and other inspection targets captured by the airborne visible light camera. IMU data is continuously acquired at a fixed high-frequency sampling rate of 100-200Hz, GPS data is synchronously acquired at a set sampling frequency of 5-10Hz, gimbal encoder data is acquired in real-time at a frequency of 20-50Hz as the gimbal attitude changes, and the visible light camera acquires aerial images frame by frame at a fixed frame rate of 25-30 frames per second. Noise reduction, outlier removal, and time axis alignment are performed on all types of data. Kalman filtering is used for attitude data noise reduction on the IMU data. GPS data is filtered using a moving average to eliminate positioning drift noise, gimbal encoder data is filtered using a mean to smooth angle fluctuation noise, and aerial image data is filtered using a Gaussian filter to remove imaging noise. The 3σ criterion is used to identify and remove discrete outlier values ​​from various sensor data sets. Simultaneously, outlier image frames with overexposure or underexposure issues are filtered out. Using the acquisition timestamp of the aerial image frame as a unified benchmark, multi-source sensor data with different sampling frequencies are calibrated and matched to the acquisition time of the corresponding image frame using linear interpolation. After completing the time-series synchronization of all data, a unified time-series sensor data sliding window is constructed, allowing multi-source sensor data and aerial image data to form a precise time-series correlation. This filters out various noises and outlier data interference, removes invalid image frames, and provides a clean, regular, and synchronized data source for subsequent flight status determination. This avoids deviations in flight status judgment due to data noise, time-series misalignment, and outlier interference, thus solidifying the data foundation of the entire image enhancement process and ensuring the accuracy and stability of subsequent processing stages.

[0021] S200 Flight Status Classification: Based on preprocessed multi-source sensor data, the real-time flight stability index is calculated using the entropy-weighted stability algorithm. The mathematical expression for the entropy-weighted stability algorithm is: in, Let be the flight stability index of the UAV at time t; Let be the information entropy of the data of the i-th type of sensor at time t; where i is the sensor category number. Let be the variance of the i-th type of sensor data within the sliding window at time t; To determine the maximum permissible variance of the i-th type of sensor data under extreme jitter flight conditions, a three-level flight state classification is completed based on a preset threshold. The flight state classification results are updated frame by frame with the real-time calculation of the flight stability index. The entropy weight stability algorithm can comprehensively consider the fluctuation characteristics and information disorder of multiple types of sensor data, objectively reflecting the real-time stable flight state of the UAV. The three-level flight state classification can accurately distinguish different working conditions of the UAV in power line inspection, such as stable cruise, gust disturbance, and maneuvering. This allows the subsequent fuzzy kernel estimation and image enhancement strategies to accurately match the actual flight state, abandoning the drawbacks of traditional unified processing. It fits the dynamic change characteristics of the flight state in the complex high-altitude environment of high-voltage power line inspection, providing a clear judgment basis for fuzzy kernel adaptive solution and hierarchical image enhancement, ensuring the pertinence and adaptability of subsequent processing links.

[0022] S300, Adaptive Blur Kernel Estimation: Based on the graded flight stability state, corresponding fuzzy kernel estimation methods are used to solve the fuzzy kernel, generating fuzzy kernel parameters adapted to the current flight state. In stable flight state, global single fuzzy kernel estimation is used, treating the entire inspection image as a unified processing object to solve the single fuzzy kernel. In slight jitter state, regional adaptive fuzzy kernel estimation is used, dividing the inspection image into multiple non-overlapping square image blocks. After solving the local fuzzy kernel for each image block, the fuzzy kernels of adjacent image blocks are smoothly fused to generate a spatially continuous regional adaptive fuzzy kernel. In severe jitter state, spatiotemporal joint fuzzy kernel estimation is used, combining the motion correlation features of the current frame and the previous and next frames to construct a feature tensor that fuses spatial image features and temporal motion features to solve the final fuzzy kernel. The three fuzzy kernel estimation methods correspond one-to-one with the three flight states, accurately capturing the fuzzy distribution features of the inspection image under different flight states. This ensures that the fuzzy kernel parameters are highly consistent with the actual fuzziness degree and fuzziness range of the image, avoiding fuzzy kernel estimation deviations that lead to poor deblurring effects. This provides core and accurate parameter support for graded image enhancement, ensuring the effectiveness of the motion blur restoration process.

[0023] S400, Graded Image Enhancement: Based on the graded flight status results and the corresponding blur kernel parameters obtained from the solution, a corresponding deblurring enhancement strategy is matched to perform differentiated image quality restoration and optimization on the inspection images. Under stable flight conditions, motion blur removal of the entire inspection image is completed by relying on a global single blur kernel, while simultaneously performing basic noise reduction and color correction, preserving the original imaging features of the power line inspection scene, and completing basic image quality optimization. Under slight shaking conditions, differentiated deblurring processing is completed by relying on a regional adaptive blur kernel, while simultaneously performing edge enhancement and detail sharpening, repairing the local edge blurring of power lines and tower components caused by shaking, and improving the image detail recognition. Under severe shaking conditions, full-image depth deblurring is completed by relying on a spatiotemporal joint blur kernel, while simultaneously performing texture reconstruction and feature enhancement, restoring the line texture and hardware details lost due to severe shaking. Through differentiated enhancement strategies, power inspection images under different flight conditions can be optimized in a targeted manner, clearly presenting key hidden danger features such as line damage, loose components, and foreign object attachment, meeting the visual requirements of high-precision fault detection and status recognition of high-voltage power lines.

[0024] S500, Dynamic Algorithm Complexity Adjustment: Real-time monitoring of the UAV's onboard load and battery level. Onboard load includes real-time utilization of the onboard CPU, onboard GPU, and onboard RAM, as well as task queue length and task execution response time. Battery level includes remaining battery percentage, battery output voltage, and battery temperature. These data are collected at fixed intervals, and the battery charging / discharging status is recorded synchronously. The comprehensive resource consumption value is calculated, and combined with the flight stability index, a resource-coupled parameter tuning algorithm is used to calculate the algorithm complexity adjustment coefficient. The mathematical expression for the resource-coupled parameter tuning algorithm is: in, This is the algorithm complexity adjustment factor at time t; The flight stability index at time t is the output of the entropy-weighted stability algorithm. Let be the total resource consumption value at time t; The maximum resource carrying capacity threshold of the airborne platform is used as the basis for adjusting relevant algorithm parameters, ultimately outputting clear UAV inspection images. The resource-coupled parameter tuning algorithm balances flight stability and airborne resource consumption, dynamically controlling algorithm operating parameters to avoid excessive load, computational lag, and energy waste on the airborne platform. This ensures the real-time and continuous nature of image enhancement processing, adapting to the operational needs of long-distance, long-duration inspections of high-voltage power lines, extending the single-inspection endurance of the UAV, and ensuring stable output of clear images that meet inspection requirements throughout the entire process. Figure 3 As shown.

[0025] This embodiment applies a motion blur-based UAV image enhancement method to high-voltage power line inspection. Through multi-source data acquisition and preprocessing, entropy weight stability algorithm for classifying flight states, adaptive fuzzy kernel estimation, graded image enhancement, and resource coupling parameter tuning algorithm to dynamically adjust complexity, the entire process is adapted to high-altitude power line inspection conditions. This method effectively solves the image blurring problem caused by UAV flight jitter, accurately restores the details of power lines and equipment, optimizes onboard resource utilization, ensures real-time performance and endurance for long-term inspections, provides high-quality visual data for high-voltage power line hazard investigation and fault detection, and significantly improves the accuracy and efficiency of power line inspections.

[0026] Example 2: This embodiment is applied to UAV inspection operations of long-distance oil and gas pipelines in the field. These pipelines span vast areas and traverse complex terrains. During UAV inspections, they are susceptible to severe motion blur due to external factors such as wind, sandstorms, undulating terrain, and sudden gusts. Furthermore, field inspections lack temporary resupply, placing extremely high demands on the utilization of UAV onboard resources and endurance. Traditional image enhancement methods are ill-suited to complex field conditions and resource constraints, failing to clearly present critical details such as pipeline corrosion damage, valve malfunctions, and leaks. This embodiment employs a motion blur-based UAV inspection image enhancement method throughout the process. Through multi-source data collaborative processing, dynamic flight status determination, adaptive fuzzy kernel solving, hierarchical image optimization, and dynamic resource parameter adjustment, it accurately solves the image blur problem in field oil and gas pipeline inspections while maximizing the utilization of onboard resources. This provides reliable visual assurance for the safe inspection, hazard identification, and status monitoring of long-distance oil and gas pipelines. Figure 2 The specific implementation steps are as follows: S100. Data Acquisition and Preprocessing: Real-time acquisition of multi-source sensor data and aerial image data during UAV inspection of long-distance oil and gas pipelines. Multi-source sensor data includes three-axis acceleration and angular velocity data acquired by the airborne IMU, latitude and longitude, flight altitude, and 3D flight speed data acquired by the airborne GPS, and pitch, roll, and yaw angle data acquired by the gimbal encoder. Aerial image data consists of images of target scenes such as oil and gas pipelines, valves, anti-corrosion coatings, and auxiliary facilities captured by the airborne visible light camera. IMU data is acquired at a high frequency of 100-200Hz, GPS data is acquired synchronously at 5-10Hz, gimbal encoder data is acquired in real-time at 20-50Hz following the gimbal's attitude, and the visible light camera acquires images frame by frame at 25-30 frames per second. Noise reduction, anomaly removal, and time-series alignment are performed on all types of data. IMU data undergoes attitude denoising through Kalman filtering, GPS data is filtered using moving average to eliminate positioning drift noise, gimbal encoder data is smoothed by mean filtering to smooth angle fluctuations, and aerial image data is filtered using Gaussian filtering to remove imaging noise. Abnormal sensor data values ​​are eliminated using the 3σ criterion, and overexposed or underexposed abnormal image frames are filtered out. Using the aerial image timestamp as a reference, multi-source sensor data is calibrated and matched to the corresponding image frame time through linear interpolation. After completing the time synchronization, a unified time sequence sensor data sliding window is constructed, allowing multi-source data to form a precise correspondence with the aerial image. This completely eliminates interference from noise, abnormal data, and invalid image frames, providing clean and synchronized data support for subsequent flight status determination. This prevents errors in status judgment caused by data problems and lays a stable data foundation for the entire image enhancement process.

[0027] S200 Flight State Classification: Based on preprocessed multi-source sensor data, the entropy weight stability algorithm is used to calculate the real-time flight stability index. According to the preset threshold, the flight state is divided into three levels: stable flight, slight shaking, and severe shaking. The classification result is updated in real time with each frame of image. The entropy weight stability algorithm can integrate the data fluctuation characteristics of IMU, GPS, and gimbal encoder to objectively quantify the stability of UAV flight in the field. The three-level flight state classification can accurately identify different flight conditions such as stable cruise, wind and sand disturbance, and terrain avoidance maneuvers in oil and gas pipeline inspection. This allows the subsequent fuzzy kernel estimation and image enhancement strategies to dynamically adapt to changes in flight state, get rid of the limitations of traditional fixed processing mode, fit the flight characteristics of complex field environments, and provide accurate state guidance for fuzzy kernel solution and image enhancement, ensuring that subsequent processing links are highly matched with actual working conditions.

[0028] S300, Adaptive Fuzzy Kernel Estimation: Based on the real-time flight status classification results of oil and gas pipeline inspection, a fuzzy kernel estimation method matching the flight status is used to solve the fuzzy kernel parameters. In stable flight conditions, a global single fuzzy kernel estimation is used, taking the entire pipeline inspection image as the object to complete the single fuzzy kernel solution. In slightly jittery conditions, a regional adaptive fuzzy kernel estimation is used, dividing the image into non-overlapping square image blocks, solving the local fuzzy kernels separately, and then smoothly fusing adjacent fuzzy kernels to generate a spatially continuous regional adaptive fuzzy kernel. In severe jittery conditions, a spatiotemporal joint fuzzy kernel estimation is used, combining the motion correlation features between image frames and fusing spatial image and temporal motion features to complete the fuzzy kernel solution. The three fuzzy kernel estimation methods can accurately adapt to the fuzzy distribution pattern of pipeline images under different jitter levels, ensuring that the fuzzy kernel parameters completely match the actual fuzzy features of the image, avoiding the impact of fuzzy kernel estimation errors on the deblurring effect, providing core parameter guarantees for graded image enhancement, and ensuring the accuracy of motion fuzz restoration.

[0029] S400, Hierarchical Image Enhancement: Combining the hierarchical flight status results with the solved fuzzy kernel parameters, a dedicated deblurring enhancement strategy is matched to complete the image quality restoration and optimization of pipeline inspection images. Under stable flight conditions, the global single fuzzy kernel is used to remove motion blur of the entire image, while simultaneously performing basic noise reduction and color correction, preserving the original imaging features of pipeline facilities and achieving basic image quality optimization. Under slight shaking conditions, the region adaptive fuzzy kernel is used to perform regional deblurring processing, while simultaneously performing edge enhancement and detail sharpening, repairing the problem of local edge blurring of pipelines, and improving the recognition of details such as pipeline welds, anti-corrosion layers, and valves. Under severe shaking conditions, the spatiotemporal joint fuzzy kernel is used to complete full-image depth deblurring, while simultaneously performing texture reconstruction and feature enhancement, restoring the pipeline surface texture and facility details lost due to severe shaking. Through differentiated enhancement processing, the oil and gas pipeline inspection images under different flight conditions can clearly present the status of key facilities, accurately capture safety hazards such as anti-corrosion layer damage, pipeline deformation, and valve abnormalities, and meet the visual recognition requirements of high-precision inspection of oil and gas pipelines in the field.

[0030] S500, Dynamic Algorithm Complexity Adjustment: Real-time monitoring of the UAV's onboard central processing unit, graphics processing unit, and RAM usage, as well as onboard load information such as task queue length and response time. Simultaneously, it collects battery status data such as remaining battery percentage, output voltage, and temperature. It calculates the comprehensive resource consumption value and, combined with the flight stability index, calculates the algorithm complexity adjustment coefficient through a resource coupling parameter tuning algorithm. Based on this coefficient, it adjusts the algorithm's operating parameters, ultimately outputting clear oil and gas pipeline inspection images. The resource coupling parameter tuning algorithm dynamically balances image enhancement effects with onboard resource consumption, flexibly adjusting algorithm complexity according to the resource status of field inspections. This avoids onboard hardware overload and excessive power consumption, ensuring the real-time performance and stability of image enhancement processing. It adapts to the continuous, unreplenished operation requirements of long-distance oil and gas pipeline inspections in the field, extending the UAV's inspection endurance and ensuring stable, clear, and usable inspection images throughout the entire process.

[0031] This embodiment applies the image enhancement method to the inspection of long-distance oil and gas pipelines in the field. Relying on multi-source data collaborative processing, dynamic flight status determination, adaptive fuzzy kernel solving, hierarchical image quality optimization, and dynamic resource parameter tuning, it perfectly adapts to the complex field inspection environment. This method effectively eliminates motion blur caused by wind, sand, and terrain disturbances, clearly presenting pipeline facility details and safety hazards. Simultaneously, by dynamically adjusting the algorithm complexity to balance processing effectiveness and onboard resource consumption, it meets the needs of long-distance, unreplenished field inspections, providing stable and reliable visual support for the safe operation and maintenance of oil and gas pipelines and hazard identification, thus improving the intelligence and efficiency of field pipeline inspections.

[0032] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for enhancing UAV inspection images based on motion blur restoration, characterized in that, The specific steps of this method are as follows: S100, Data Acquisition and Preprocessing: Real-time acquisition of multi-source sensor data and aerial image data required for UAV inspection, noise reduction, abnormal data removal and time axis alignment processing of various types of data, and construction of a unified time sequence sensor data sliding window; S200 Flight Status Classification: Based on preprocessed multi-source sensor data, the real-time flight stability index is calculated using the entropy weight stability algorithm to complete the three-level flight status classification. S300, Fuzzy Kernel Adaptive Estimation: Based on the graded flight stability state, the corresponding fuzzy kernel estimation method is used to complete the fuzzy kernel solution and generate fuzzy kernel parameters that are adapted to the current flight state; S400, Graded Image Enhancement: Based on the flight status graded results and combined with the corresponding blur kernel parameters obtained by solving, the corresponding deblurring enhancement strategy is matched to perform differentiated image quality restoration and optimization on the inspection image; S500, Dynamic Algorithm Complexity Adjustment: Real-time monitoring of UAV onboard load and battery power, calculation of comprehensive resource consumption value, combined with flight stability index, and the use of resource coupling parameter tuning algorithm to calculate algorithm complexity adjustment coefficient and adjust relevant algorithm parameters, ultimately outputting clear UAV inspection images.

2. The UAV inspection image enhancement method based on motion blur restoration according to claim 1, characterized in that, In step S100, the multi-source sensor data includes triaxial acceleration data and triaxial angular velocity data collected by the airborne IMU, longitude data, latitude data, flight altitude data and three-dimensional flight speed data collected by the airborne GPS, pitch angle data, roll angle data and yaw angle data collected by the gimbal encoder; the aerial image data is the inspection target scene image data captured by the airborne visible light camera. The IMU data is continuously collected at a fixed high-frequency sampling rate of 100-200Hz, the GPS data is synchronously collected at a set sampling frequency of 5-10Hz, the gimbal encoder data is collected in real time as the gimbal attitude changes, and the visible light camera collects aerial images frame by frame at a fixed frame rate of 25-30 frames per second.

3. The UAV inspection image enhancement method based on motion blur restoration according to claim 1, characterized in that, In step S100, when denoising various types of data, Kalman filtering is used to denoise the attitude data of IMU data, moving average filtering is used to eliminate positioning drift noise of GPS data, mean filtering is used to smooth angle fluctuation noise of gimbal encoder data, and Gaussian filtering is used to remove imaging noise of aerial image data. The 3σ criterion is used to identify and remove discrete abnormal numerical points in various types of sensor data, and abnormal image frames with overexposure and underexposure problems in aerial images are screened out. Using the acquisition timestamp of the aerial image frame as a unified benchmark, multi-source sensor data with different sampling frequencies are calibrated and matched to the acquisition time of the corresponding image frame through linear interpolation. After completing the time synchronization of all data, a unified time-series sensor data sliding window is constructed.

4. The UAV inspection image enhancement method based on motion blur restoration according to claim 1, characterized in that, In step S200, the mathematical expression of the entropy weight stability algorithm is: in, Let be the flight stability index of the UAV at time t; Let be the information entropy of the data of the i-th type of sensor at time t; where i is the sensor category number. Let be the variance of the i-th type of sensor data within the sliding window at time t; Let be the maximum permissible variance of the data from the i-th type of sensor under extreme jitter flight conditions.

5. The UAV inspection image enhancement method based on motion blur restoration according to claim 1 or 4, characterized in that, In step S200, the three-level flight state division is based on the real-time flight stability index calculated using the entropy weight stability algorithm. Using this as the core criterion, preset thresholds are set by collecting sensor data from the UAV offline under various inspection scenarios, including stable cruise, gust disturbances, and maneuvering, and then statistically analyzing the data. The distribution range was then calibrated and determined, with the specific division criteria being as follows: When the value is ≥0.8, the drone is considered to be in a stable flight state; when 0.5≤ When the value is less than 0.8, the drone is determined to be in a state of slight shaking during flight; when... When the value is less than 0.5, the UAV is determined to be in a state of severe shaking during flight, and the flight state classification result is adjusted accordingly. The real-time calculation updates frame by frame to match the corresponding fuzzy kernel estimation method and image enhancement processing strategy.

6. The UAV inspection image enhancement method based on motion blur restoration according to claim 1, characterized in that, In step S300, the fuzzy kernel estimation method is matched one-to-one with the three flight states. In the stable flight state, the fuzzy characteristics of the entire inspection image are spatially uniformly distributed. Global single fuzzy kernel estimation is adopted, and the entire inspection image is used as a unified processing object to complete the solution of the single fuzzy kernel. In the slight jitter state, regional adaptive fuzzy kernel estimation is adopted. The inspection image is divided into multiple non-overlapping square image blocks. The local fuzzy kernel is solved for each image block. Then, the fuzzy kernels of adjacent image blocks are smoothly fused to generate a spatially continuous regional adaptive fuzzy kernel. In the severe jitter state, spatiotemporal joint fuzzy kernel estimation is adopted. Combining the motion correlation features of the current frame and the previous and next frames, a feature tensor that fuses spatial image features and temporal motion features is constructed to complete the solution of the final fuzzy kernel.

7. The UAV inspection image enhancement method based on motion blur restoration according to claim 1, characterized in that, In step S400, the deblurring enhancement strategy is executed based on the flight state classification results and the corresponding blur kernel parameters. Under stable flight conditions, motion blur removal of the entire inspection image is completed using a global single blur kernel, while simultaneously performing basic noise reduction and color correction, preserving the original imaging features of the inspection scene, and completing basic image quality optimization. Under slight shaking conditions, differentiated deblurring processing is completed using a region adaptive blur kernel, while simultaneously performing edge enhancement and detail sharpening, repairing local edge blur caused by shaking, and completing regional image quality restoration. Under severe shaking conditions, full-image depth deblurring is completed using a spatiotemporal joint blur kernel, while simultaneously performing texture reconstruction and feature enhancement, restoring target details and texture information lost due to severe shaking, and completing differentiated image quality restoration and optimization of the inspection image.

8. The UAV inspection image enhancement method based on motion blur restoration according to claim 1, characterized in that, In step S500, the real-time monitored UAV onboard load specifically includes the real-time utilization rate of the UAV's onboard central processing unit, the real-time utilization rate of the onboard graphics processor, the real-time occupancy rate of the onboard running memory, the task queue length, and the task execution response time. The above data are collected at fixed intervals to monitor the occupancy of onboard resources in real time. The real-time monitored UAV battery power specifically includes the remaining battery power percentage, battery output voltage, and battery temperature. The above battery-related data are collected in real time, and the battery charging and discharging status is recorded synchronously.

9. The UAV inspection image enhancement method based on motion blur restoration according to claim 1, characterized in that, In step S500, the mathematical expression of the resource coupling parameter tuning algorithm is: in, This is the algorithm complexity adjustment factor at time t; The flight stability index at time t is the output of the entropy-weighted stability algorithm. Let be the total resource consumption value at time t; This represents the maximum resource carrying capacity threshold for the airborne platform.

Images