An unmanned aerial vehicle night flight image enhancement and recovery system based on multi-degradation identification and adaptive algorithm scheduling

Abstract

The application discloses a kind of based on multiple degradation identification and adaptive algorithm scheduling unmanned aerial vehicle night flight image enhancement and recovery method.Method is composed of image acquisition, multiple degradation identification, algorithm scheduling, image enhancement and recovery algorithm library, edge adaptation control etc.Modular.First, multiple degradation identification module automatically distinguishes the degradation type (such as low brightness, high noise, motion blur etc.) in night image;Second, algorithm scheduling module is according to the degradation condition of discrimination, from algorithm library adaptive selection and call corresponding image enhancement and recovery algorithm processing, algorithm library integrates low light enhancement, noise reduction, deblurring, defogging etc.Multiple algorithms;Then, edge adaptation control mechanism will dynamically adjust the execution order and parameter of algorithm, to give consideration to processing effect and real-time performance.Through the above-mentioned modular cooperation, this method can significantly improve the brightness, definition and signal-to-noise ratio of unmanned aerial vehicle night shooting image, effectively restore image details, improve image usability.The application has the characteristics of modular structure and high resource utilization rate, can be deployed on a variety of unmanned aerial vehicle platform, is applicable to security monitoring, automatic driving night vision, aviation remote sensing etc.Scenes, for enhancing night image quality, improving environmental perception and target recognition ability, has broad engineering application prospect.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision, and in particular to a nighttime image enhancement and restoration technology solution for unmanned aerial vehicle (UAV) systems. Specifically, it relates to a system that improves the quality of UAV nighttime images by identifying multiple image degradation factors and using adaptive algorithm scheduling, and belongs to the fields of UAV visual perception and embedded image processing technology. Background Technology

[0002] Drones are widely used in nighttime security, patrol, traffic control, and remote sensing tasks. However, due to insufficient lighting, the acquired images often suffer from low brightness, high noise, blurred details, and poor contrast, severely impacting recognition and monitoring effectiveness. Traditional algorithms (such as histogram equalization and Retinex) can only improve brightness or contrast, struggling to address both noise and blur. While deep learning methods can improve image quality, they often target single degradations, resulting in large, computationally intensive models unsuitable for real-time embedded operation on drones. Concatenating multiple models can handle multiple degradation problems, but this increases system complexity and energy consumption.

[0003] Furthermore, due to limitations in computing power and power consumption, drone platforms struggle to run complex algorithms in real time, often requiring reliance on ground stations for processing, which introduces latency and uncertainty. Existing systems lack intelligent scheduling mechanisms, often employing fixed processing procedures and failing to dynamically select the optimal algorithm combination based on image degradation types, resulting in resource waste or unsatisfactory enhancement effects.

[0004] In summary, existing drone nighttime image enhancement technologies mainly suffer from three problems: (1) Insufficient capacity for comprehensive treatment of multiple degradations; (2) The algorithm is complex and embedded deployment is difficult; (3) Lacking an adaptive algorithm scheduling mechanism, it is difficult to balance real-time performance and effectiveness. Summary of the Invention

[0005] Objective: To address the aforementioned problem of image quality degradation in UAV nighttime flight, this invention provides a method for enhancing and restoring UAV nighttime flight images based on multi-degradation identification and adaptive algorithm scheduling. By introducing an automatic identification mechanism for multiple image degradation factors and dynamically allocating appropriate subsets of image enhancement and restoration algorithms based on the identification results, this invention aims to overcome the shortcomings of existing technologies, such as poor performance of single degradation processing, difficulties in embedded real-time processing, and lack of intelligent algorithm scheduling. This achieves enhanced clarity and detail restoration of UAV nighttime images, significantly improving image quality while ensuring real-time performance.

[0006] Technical solution: 1. This invention addresses the nighttime flight scenarios of unmanned aerial vehicles (UAVs) and proposes a nighttime image enhancement and restoration method based on "degradation identification → adaptive scheduling → resource-constrained enhancement and restoration." The method comprises an image acquisition module, a multi-degradation identification module, an algorithm scheduling module, an image enhancement and restoration algorithm library, and an edge adaptation control module. These modules work collaboratively online to achieve brightness enhancement, noise suppression, deblurring, haze removal, and detail restoration of nighttime images, while ensuring the real-time performance and stability of the onboard processing platform. The following step-by-step explanation is provided:

[0007] For ease of explanation, the following steps will be presented: 1-1) System module composition and data flow An airborne image processing system is constructed, comprising: an image acquisition module for acquiring raw images during nighttime flight of a UAV, denoted as ; a multi-degradation recognition module for analyzing the degradation features of the raw images; an algorithm scheduling module for generating subsequent processing flows based on the degradation recognition results; an image enhancement and restoration algorithm library for providing processing operators such as low-light enhancement, noise reduction, deblurring, dehazing, and detail sharpening; and an edge adaptation control module for real-time adjustment of the process flow based on computing power and power consumption. The overall system data flow can be formally represented as . Where is the degradation intensity vector, is the algorithm call order, is the running parameters of each algorithm module, and is the final output image.

[0008] 1-2) Degradation Characterization and Multidimensional Description The system defines a set of typical degradation types for night flight images. The multi-degradation recognition module outputs a degradation intensity vector from the original image. Here, represents the degree of brightness degradation, represents the noise intensity, represents the degree of sharpness degradation (motion blur), and represents the intensity of the impact of haze / atmospheric scattering. This vector will serve as the driving signal for subsequent algorithm scheduling, enabling the system to selectively select or skip certain processing steps and to set different intensities of processing for different degrees of degradation.

[0009] 1-3) Generation of Adaptive Image Enhancement and Restoration Workflow The algorithm scheduling module selects a set of algorithm modules to be executed from the image enhancement and restoration algorithm library, and determines the execution order and parameter set of these modules. The relationship between this processing pipeline and the original image can be written as follows: Through this adaptive sequence generation mechanism, the system avoids using a fixed, single enhancement process. Instead, it performs differentiated and customized processing based on the actual degradation situation, significantly improving robustness in complex night flight scenarios.

[0010] 1-4) Modeling of real-time performance and stability constraints Unmanned aerial vehicles (UAVs) have limited onboard computing resources, especially during nighttime flights where high requirements are placed on endurance and flight control stability. Therefore, the system must adhere to latency and energy consumption budgets when generating and executing processing flows. Let the single-frame computation latency of the algorithm module under certain parameters be *l*, and the energy consumption be *r*, then the constraints are: The reciprocal of the target frame rate corresponds to the allowable average energy consumption threshold. The edge adaptation control module monitors resource usage and dynamically intervenes during operation to ensure that the above constraints are continuously met, thereby guaranteeing the stable online operation of the visual perception link during flight missions.

[0011] 2. The multi-degradation identification module utilizes a multi-task deep neural network model to simultaneously estimate multiple degradation indicators, such as brightness degradation, noise intensity, motion blur, and haze impact, on the input night flight image. The output includes continuous intensity values ​​and discrete level labels. These indicators provide direct decision-making basis for the subsequent algorithm scheduling module.

[0012] For ease of explanation, the following steps will be presented: 2-1) Multi-task network architecture design A neural network with a shared feature extraction layer and multiple dedicated branches is constructed. The shared backbone network extracts features from the input image to obtain a multi-scale low-light feature description; for each degradation type, a regression or classification branch is set separately. The network outputs a continuous intensity estimate for each degradation type. Here, represents the parameters of the shared feature extraction layer, and represents the parameters of the corresponding degradation branch. The advantage of this structure is that it can obtain multiple degradation information simultaneously with only one forward inference, making it suitable for real-time operation on the UAV's onboard device.

[0013] 2-2) Output Indicators and Level Labels For each degradation type, the network outputs not only continuous intensity values ​​but also discrete level labels, representing "no obvious," "mild," "moderate," and "severe," respectively. The labels can be obtained by setting a threshold for the continuous intensity values. Here, the threshold is configurable or learnable within the system. This labeling result is used to drive rule-based scheduling logic (e.g., when scheduling must insert a noise reduction step).

[0014] 2-3) Training objective and joint loss function To ensure that the multi-degradation recognition module maintains its estimation reliability in multi-task scenarios, a joint loss is used during training. Here, and represent the degradation intensity and rank label of the labeled or pseudo-labeled data, is the cross-entropy loss, and is the weight. Through this joint supervision of "intensity value + rank label", the model possesses both fine-grained numerical estimation capabilities and provides decision-friendly discrete rank signals, thereby supporting rapid decision-making in subsequent modules.

[0015] 2-4) Explainable physical and statistical auxiliary features To improve the robustness and interpretability of degradation identification under extreme conditions (e.g., extremely low light, strong noise, jitter), manually interpretable metrics can be introduced during the network input or post-processing stages. Typical metrics include: Brightness degradation indicators, such as average brightness and the proportion of dark pixels Used to measure the overall degree of underexposure.

[0016] The noise intensity index can be obtained by using a smoothing operator to obtain the residual. And its variance is calculated as the noise level.

[0017] Blur level index, which can be used to calculate image gradient energy Or the variance of the Laplace response; a low variance usually corresponds to stronger motion blur.

[0018] The degree of haze can be estimated based on the distribution range of atmospheric light and transmittance using an atmospheric scattering model.

[0019] These metrics can enhance the stability of network output and also help to perform security redundancy checks in subsequent scheduling, avoiding network misjudgments that lead to incorrect processing links.

[0020] 3. The algorithm scheduling module automatically selects and prioritizes image enhancement and restoration algorithm modules based on the multi-degradation identification results and onboard resource conditions, while setting appropriate operating parameters and execution intensity for each module. This module can use rule-based decision-making logic and continuously optimize rule weights and thresholds through self-learning during operation to achieve a balance between image quality and real-time performance under different flight conditions.

[0021] For ease of explanation, the following steps will be presented: 3-1) Rule-based process template matching A rule base is established, defining several typical degradation scenarios and corresponding algorithm processing templates. Examples include: when image brightness is severely insufficient, the low-light enhancement module is executed first, followed by noise reduction and sharpening; when noise is significant, noise reduction is executed first to avoid premature brightening that could amplify the noise; when significant motion blur exists, a deblurring module is inserted into the enhancement process. The above mapping relationship can be represented as follows: The degradation level label mentioned in Article 2 indicates the initial algorithm call order.

[0022] 3-2) Adaptive adjustment of sequence and intensity The process template provided by the rule base is not the final process. The algorithm scheduling module will combine it with the continuous degradation intensity vector. In addition to the current resource status, the algorithm order, whether it is enabled or not, and the parameter strength are refined. Formally, the process optimization can be modeled as... Among them is the image quality index, which can be composed of weighted factors such as no-reference quality score, contrast improvement, and edge sharpness improvement. is the total latency estimate for executing this process; is the latency penalty coefficient. The goal of this optimized representation system is to achieve the highest possible visual quality while ensuring real-time constraints.

[0023] 3-3) Self-learning update of rule weights and thresholds During flight, the system continuously records the quality of processing results and actual execution latency under different scenarios. Using this feedback, the system can update thresholds in the rule base (such as the threshold for determining "severe noise") and processing priorities. For example, the system can treat different process templates as candidate actions in a contextual bandit, using degradation vectors and resource states as context, and actual output quality minus latency penalties as rewards, thereby gradually increasing the probability of selecting templates that perform better under specific flight conditions. This mechanism enables the system to have online adaptive capabilities, continuously optimizing decisions as factors such as task, lighting, and flight speed change.

[0024] 3-4) Dependencies and legal order constraints between modules Different algorithm modules are constrained by the order in which they are executed. For example, strong sharpening should usually be performed after noise reduction to avoid amplifying noise as detail; dehazing should usually be performed after brightness enhancement to obtain a more reliable atmospheric light estimate; and deblurring should be performed before final sharpening to avoid secondary amplification of the blur kernel. This dependency can be described using a partial order relation: Here, the input assumptions of the algorithm (e.g., "image brightness is basically visible and noise is moderate") represent the image feature state after the algorithm runs. The scheduling module will eliminate illegal processes that violate this partial order during the search, ensuring that the final output image is visually consistent and physically reliable.

[0025] 4. The image enhancement and restoration algorithm library consists of multiple independently callable algorithm modules, including low-light enhancement, image denoising, image deblurring, image dehazing, and detail sharpening modules. Each module can be implemented using either classic image processing algorithms or deep learning models; each module exposes configurable runtime parameters, allowing the scheduling module to call and tune them as needed under different scenarios and resource conditions.

[0026] For ease of explanation, the following steps will be presented: 4-1) Low-light enhancement module Low-light enhancement modules are used to improve the overall brightness, dynamic range, and discernible details of low-light images at night. Typical methods include: (1) Gamma correction model, which enhances the brightness of dark areas through nonlinear mapping: The smaller the value, the brighter the output image.

[0027] (2) Retinex decomposition model, which assumes that an image can be represented as the product of the reflection component and the illumination component: Optimization is possible The enhanced result is obtained after uniform illumination.

[0028] (3) Zero-reference depth brightening networks (Zero-DCE-type methods) directly adjust brightness and contrast by learning a set of pixel-level brightening curve parameters, while suppressing overexposure and enhanced noise. This type of method is suitable for restoring the outline of visible targets in extremely low-light scenes.

[0029] 4-2) Image Denoising Module The image denoising module primarily targets the strong random noise caused by high ISO imaging at night. Noise can be addressed using a Gaussian additive model. Or use the Poisson-Gaussian mixture model under extremely low light conditions. This represents the Poisson perturbation associated with the incident photon count.

[0030] Noise reduction methods can be traditional block matching-3D transform thresholding (BM3D) approaches, or deep learning noise reduction networks. The latter can be trained by minimizing the following objective: The variable is a total variational regularization term used to smooth noise while preserving edges. The location of the noise reduction module (before or after brightness enhancement) is determined by the scheduling module based on the noise intensity and the degree of brightness degradation to avoid the problem of "noise being amplified first and then difficult to suppress".

[0031] 4-3) Image Deblurring Module The image deblurring module addresses image blurring or slight defocusing caused by drone motion. Common imaging degradation can be represented by a convolutional model. Here, is the fuzzy kernel, is the convolution term, and is the noise term. If the fuzzy kernel is known or can be estimated, then non-blind deconvolution can be performed: The regularization term is used to suppress ringing artifacts and maintain sharp edges. For complex non-uniform motion blur, an end-to-end deblurring network based on deep learning can be used to directly output a sharp image or simultaneously estimate the sharp image and the blur kernel of spatial variation.

[0032] 4-4) Image Dehazing / De-Atmospheric Scattering Module To address the reduced visibility during night flights caused by fog, haze, long-distance light scattering, and strong backlighting, the system employs a defogging method based on an atmospheric scattering model. The atmospheric scattering model can be expressed as follows: Where is the observed image pixel value, is the desired haze-free pixel value, and is atmospheric light. Let be the transmittance, be the scattering coefficient, and represent the scene depth or equivalent distance. By estimating and , a fog-free image can be recovered. This includes a preset lower limit to prevent numerical instability. This module can significantly enhance long-distance road contours, landmark lights, and obstacle boundaries, contributing to nighttime flight safety.

[0033] 4-5) Detail Sharpening Module The detail sharpening module further enhances edge sharpness and texture clarity after brightness enhancement, noise reduction, and deblurring. A commonly used contrast enhancement (UnsharpMask) can be represented as... Here, is the Gaussian smoothing operator, and is the sharpening intensity coefficient. A lightweight depth model can also be used to selectively enhance high-frequency regions, thereby highlighting edge structures without significantly amplifying noise. This module is typically located at the back end of the processing chain, and its purpose is to generate a clear image at the output stage that can be directly used for human visual observation or object detection algorithms.

[0034] 4-6) Parameter Configurability and Adaptive Control Each algorithm module exposes configurable parameters, such as parameters related to brightness enhancement, noise reduction intensity, deconvolution regularization weights, and sharpening intensity. The scheduling module automatically sets these parameters for each module based on the current degradation intensity vector. For example, brightness enhancement can employ a linear adjustment strategy. This means that a stronger brightening effect is used when the brightness degradation is more severe (larger). Through parameter adaptation, the system can achieve a trade-off between optimal image quality and real-time performance based on environmental changes without real-time human intervention.

[0035] 5. The edge adaptation control module monitors the resource status of the UAV's onboard processing platform in real time, including CPU / GPU usage, memory usage, processing frame rate, and power status. When resources are scarce, it proactively reduces the image processing flow (e.g., reducing resolution, switching to lightweight models, skipping non-critical steps, and reducing the number of iterations). When resources are restored, it restores high precision and the complete link, thereby maintaining real-time output and system stability during flight.

[0036] For ease of explanation, the following steps will be presented: 5-1) Status acquisition of resource monitoring unit The system continuously collects resource status vectors Where and represent processor and accelerator utilization, respectively; represent memory usage; represent the current image processing frame rate; and represent the current power / battery status. To avoid misjudgment due to instantaneous fluctuations, the system uses an exponential moving average filter on these signals: The degree of smoothing is determined by this factor. The resulting smoothed value is used as the basis for subsequent decisions.

[0037] 5-2) Load assessment and trigger condition setting The system sets thresholds for key performance indicators, such as the target frame rate lower limit and the allowable GPU usage threshold. When observed... When this occurs, the system is considered to be in a resource-scarce state, requiring the triggering of a load reduction process. On the other hand, when... Here, the hysteresis threshold is used to restore the target frame rate, allowing the system to gradually restore the complete processing chain. By setting the trigger threshold and hysteresis threshold, the system can avoid frequent mode switching in critical states.

[0038] 5-3) Adaptive Strategy Adjustment Content When resource constraints are detected, the strategy adjustment unit optimizes the current algorithm flow by reducing load. Typical strategies include: reducing the resolution or frame rate of the input image to proportionally decrease computational complexity (where represents the downsampling ratio); switching high-complexity deep learning models to lightweight versions, i.e., replacing the runtime parameters; prioritizing the disabling of computationally intensive steps with low marginal returns based on the marginal benefits of each processing step to the overall quality; and shortening the iteration count of iterative algorithms, for example, by scaling the iteration limit proportionally. Through these hierarchical strategies, the system can still output usable images under high load, rather than becoming completely disabled.

[0039] 5-4) Stability and Rollback Mechanism The system defines a cost function for the running state. Here, represents the no-reference quality score of the current output image, and represents the current frame rate. During operation, the system attempts to minimize resource usage and records the system configuration at its lowest point (including selected flow, resolution, model version, etc.). When resources are restored, the system rolls back to the recorded high-quality configuration, achieving a smooth transition from "download mode" to "high-quality mode." Therefore, the edge adaptation control module not only prevents system crashes during resource-constrained periods but also restores image quality as much as possible when resources are restored.

[0040] 6. The system adopts a hardware-software co-deployment approach: the multi-degradation recognition module, algorithm scheduling module, and edge adaptation control module mainly run in software on the UAV's onboard processor, while the image acquisition module and various image enhancement / restoration algorithm modules can be executed by a general-purpose processor or accelerated using dedicated hardware such as DSPs, FPGAs, or AI acceleration chips. The entire system is deployed within the UAV platform, allowing the enhanced images to be directly used by the flight control system or transmitted to the ground station via a data link.

[0041] For ease of explanation, the following steps will be presented: 6-1) Unmanned Aerial Vehicle Platform Integration The system is deployed on the UAV platform, including the UAV itself, onboard camera devices, and onboard computing devices. The image acquisition module transmits the raw image signals captured at night to the onboard computing devices; the onboard computing devices run multi-degradation recognition, algorithm scheduling, and edge adaptation control logic, manage the calling process of the image enhancement and restoration algorithm library, and obtain enhanced and restored images.

[0042] 6-2) Data Flow and Delay Control Upon receiving the data, the onboard computing unit executes the relevant programs in the order of "degradation identification → scheduling process → enhancement / recovery processing → resource adaptation and callback," and outputs the data under clock synchronization constraints (e.g., based on a unified timestamp or frame count). The system requires that the end-to-end latency not exceed a preset upper limit to ensure that the flight control system and ground station obtain near real-time visual information during nighttime flights. Excessive image processing latency will weaken the reliability of critical downstream tasks such as obstacle avoidance, identification, and navigation.

[0043] 6-3) Heterogeneous computing power scheduling and operator mapping For each algorithm module in the image enhancement and restoration algorithm library, its internal operator graph is represented as follows: Nodes represent operators (such as convolution, upsampling, FFT deconvolution steps, etc.), and edges represent data dependencies between operators. The system uses a set of devices... This means that both operator execution latency and data transmission latency between different hardware units are considered simultaneously. Through this mapping strategy, the system can fully utilize hardware acceleration capabilities while ensuring real-time performance, thereby reducing overall energy consumption.

[0044] 6-4) Outputs and Interfaces The enhanced and restored output image, along with related quality metrics (such as degradation intensity vector, resource status, and current quality score), can be directly fed back to the UAV's flight control system for nighttime navigation, obstacle avoidance, and target recognition. It can also be transmitted via a link to a ground station interface for operator observation, and stored to form mission documentation or subsequent training data. This integrated hardware and software deployment scheme elevates image enhancement from simple visual enhancement to a crucial link in the flight control perception chain.

[0045] 7. The system's operational steps during UAV night flight missions include image acquisition, degradation recognition, scheduling decision-making, enhancement and restoration processing, resource adaptive control, and result output. These steps are designed to ensure that high-quality, usable images can still be output in scenarios with limited resources, extremely low illumination, and complex interference such as jitter and atmospheric scattering.

[0046] For ease of explanation, the following steps will be presented: 7-1) Nighttime Raw Image Acquisition Steps The drone's onboard camera acquires raw images in real time during nighttime flight and performs necessary preprocessing (including depigmentation, white balance, and linearization response correction) to obtain image frames that can be used for subsequent algorithm processing. This step ensures the basic availability of the input signal.

[0047] 7-2) Multiple Degradation Identification Steps The acquired images are input into a multi-degradation recognition module. This module, based on a shared feature extraction network and a multi-branch output structure, estimates the degradation intensity vector and discrete level labels. Simultaneously, the system can also calculate auxiliary statistical indicators, such as average brightness, noise residual variance, gradient energy, and atmospheric scattering parameters. These results collectively constitute a quantitative description of image degradation.

[0048] 7-3) Adaptive Algorithm Scheduling Steps The algorithm scheduling module selects candidate processing flow templates based on the degradation information obtained in step 7-2 and a pre-built rule base, and then optimizes the problem based on these templates. The final processing flow and parameter set are generated. This scheduling not only determines which algorithm modules will be activated, but also their execution order and intensity, ensuring that the highest possible image quality improvement is achieved while maintaining real-time performance.

[0049] 7-4) Perform the image enhancement and restoration steps in sequence. Based on the process and parameters obtained in step 7-3, the system sequentially performs low-light enhancement, noise reduction, deblurring, dehazing, and detail sharpening on the image. For example, low-light enhancement can be based on gamma correction. Or based on the Retinex decomposition model Noise reduction can be achieved by smoothing random noise using BM3D or deep noise reduction networks; deblurring can be achieved using non-blind deconvolution. Improve edge sharpness. After the above sequence processing, the enhanced and restored output image is obtained. If a reference ground truth image exists (such as during calibration or simulation), the peak signal-to-noise ratio can be calculated. And structural similarity. In actual flight, when there is no reference ground truth, the usability of the image can be evaluated using no-reference quality metrics (such as negative values ​​of NIQE and BRISQUE), local contrast enhancement, and gradient energy enhancement.

[0050] 7-5) Resource Adaptive Control Steps During the processing of step 7-4, the edge adaptation control module continuously monitors the resource status. When high load is detected or the frame rate drops below a threshold (e.g., or ), the system automatically initiates a load reduction strategy, including reducing resolution or frame rate, switching to a lightweight model, reducing the number of iterations, and skipping processing steps with low marginal returns. Conversely, when resources recover (load decreases and frame rate recovers), the system gradually restores the complete processing chain and high-precision parameter configuration. Through this closed-loop control strategy, the system maintains real-time output throughout the flight, avoiding visual interruptions due to overload.

[0051] 7-6) Output and Task-Level Usage Steps The enhanced and restored output images can be used for subsequent UAV missions, including but not limited to: nighttime environmental perception, navigation, and obstacle avoidance in the flight control system; transmission to the ground station to provide operators with clear visual information of the nighttime environment; and as input to target detection, recognition, and tracking algorithms to improve recognition confidence. The recognition results from the target detection module can also serve as feedback signals in the weighting of image quality scores, thereby gradually guiding the system's scheduling module to favor processing strategies that improve detection performance in subsequent frames. This feedback forms a closed loop of perception-decision-re-perception.

[0052] 8. The computer-readable storage medium stores program instructions that can be loaded and executed on the UAV's onboard processor. These instructions are used to perform all steps of the above-described night flight image enhancement and restoration method, namely, from image acquisition, degradation identification, adaptive scheduling, serialized enhancement and restoration, to online resource adaptive control and result output.

[0053] For ease of explanation, the following steps will be presented: 8-1) Functional Scope of Program Instructions When the program instructions are loaded and run by the UAV's onboard processor, they execute the steps described in Section 7, including nighttime raw image acquisition (step 7-1), multi-degradation identification (step 7-2), adaptive algorithm scheduling (step 7-3), sequential image enhancement and restoration (step 7-4), resource adaptive control (step 7-5), and enhanced image output and task use (step 7-6). In other words, the program instructions realize a complete closed-loop process from "perceiving raw images" to "outputting clear images that can be directly used for flight control and ground station decision-making."

[0054] 8-2) Platform Adaptation and Portability To adapt to different drone platforms and hardware acceleration solutions, this program can work with the Platform Abstraction Layer (PAL) and Hardware Adaptation Layer (HAL) to ensure consistent behavior across different CPU / GPU / DSP / FPGA / AI acceleration chip combinations, using the same scheduling, load reduction, quality assessment, and rollback strategies. The configuration file can include initial rules for the rule base, resource thresholds (e.g., target frame rate lower limit, latency budget, energy budget), weight files for each enhancement and recovery module, and their lightweight version parameters, enabling rapid deployment of the system across different drone platforms.

[0055] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are mainly reflected in the following aspects: (1) Intelligent recognition of multiple degradation factors for targeted enhancement: This invention overcomes the limitation of traditional methods that can only passively handle a single known degradation by introducing an automatic recognition mechanism for multiple degradation factors in images. The system can distinguish between different degradation conditions such as low brightness, noise, blur, and haze, and select optimization algorithms accordingly to achieve "targeted treatment". This intelligent recognition and decision-making makes image enhancement processing no longer blind in complex nighttime environments, and significantly improves the overall enhancement effect in scenarios with multiple degradation combinations.

[0056] (2) Adaptive algorithm scheduling optimizes processing order and resource utilization: The algorithm scheduling module of this invention dynamically determines the combination and order of algorithm calls based on the degradation type and degree, breaking through the rigid mode of fixed pipelines. For example, the processing order of "enhancement priority" or "noise reduction priority" can be flexibly adjusted according to image characteristics, avoiding problems such as amplifying noise if enhancement is performed before noise reduction or insufficient brightness if noise reduction is performed before enhancement. At the same time, the scheduling strategy fully considers the computational cost of the algorithm and the real-time requirements of the hardware, skipping redundant steps or adopting a lightweight mode while ensuring the key enhancement effect, maximizing the use of limited computing power to achieve the best image quality improvement. This adaptive collaborative mechanism effectively improves the efficiency and robustness of system processing.

[0057] (3) Integrating the advantages of multiple algorithms to comprehensively improve the quality of nighttime images: This invention integrates multiple image enhancement and restoration algorithm modules, which can take the most appropriate processing measures for different degradation factors, and make the algorithms work together to output images that are significantly improved in terms of brightness, contrast, sharpness and noise control. Compared with a single algorithm scheme, this system enhances the overall usability of nighttime images while minimizing side effects (such as loss of detail or noise residue caused by over-enhancement). The nighttime images processed by this system have clear details in dark areas, significantly reduced image noise, and corrected motion blur. The overall visual effect is close to the level of normal daytime shooting, which greatly improves the nighttime visual perception capability of UAVs.

[0058] (4) Supports embedded real-time deployment, lightweight and efficient system: This invention fully considers the resource limitations of UAV embedded hardware in its system architecture and algorithm design. Through modular design and edge-adaptive control strategies, the system can tailor algorithms as needed and dynamically reduce computational load, ensuring near real-time operation on embedded processors (such as ARM CPUs or small GPUs). Compared to existing solutions that rely on ground stations or high-power equipment, this system can directly complete image enhancement processing on the UAV itself, achieving true airborne real-time closed-loop processing. This not only reduces communication latency and improves the autonomy and robustness of processing, but also expands the application scenarios for UAV nighttime operations.

[0059] (5) Strong engineering versatility and broad application prospects: This invention adopts an open modular architecture, which has good scalability and versatility. Each functional module can be integrated into a single embedded platform or deployed in a distributed manner on UAVs and ground servers to operate collaboratively, as needed. In addition, the system's multi-degradation recognition and adaptive enhancement mechanism can be generalized to various low-light imaging platforms, not limited to specific UAV models. This technology can be widely used in security monitoring, intelligent patrol, emergency search and rescue, night vision assistance for unmanned vehicles, and night imaging for aerial remote sensing, which can significantly improve the usability and information content of night images, and has extremely high engineering promotion value and commercial prospects. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the overall system structure of the present invention; Figure 2 This is a schematic diagram of the multi-degradation identification module structure of the present invention; Figure 3 This is the logical flowchart of the algorithm scheduling module of the present invention. Detailed Implementation

[0061] The present invention will be further described below through specific embodiments. It should be understood that these embodiments are only used to illustrate the technical solutions of the present invention, and not to limit its scope.

[0062] This invention provides a UAV nighttime image enhancement and restoration system based on multi-degradation recognition and adaptive algorithm scheduling. The system includes an image acquisition module, a multi-degradation recognition module, an algorithm scheduling module, an image enhancement and restoration algorithm library, and an edge adaptation control module. These modules work collaboratively to achieve real-time enhancement and restoration of UAV nighttime images. The system is deployed on an UAV-borne embedded platform, with a typical configuration of: a quad-core ARM processor with a main frequency of 1.8GHz, an embedded GPU with a computing power of no less than 1 TFLOPS, 4GB of memory, a target processing frame rate of no less than 25fps, and a single-frame processing latency controlled within 80ms.

[0063] First, (1-1) after the system starts, the image acquisition module collects raw image data of the UAV during night flight. This module uses a high-sensitivity camera, such as a 1 / 1.8-inch CMOS sensor with 1920×1080 effective pixels, supporting a 30fps frame rate, and a lens pixel size of approximately 2.9μm. The camera acquires visible light or infrared images in low-light environments (typically 0.01–5 lux), with an exposure time range of 5ms–40ms and a gain range of 0dB–24dB. The acquisition results are expressed as follows: Here, represents the brightness value at the pixel location in the original image, and represents the pixel coordinates. The image is then converted from analog to digital and fed into the multi-degradation recognition module for analysis.

[0064] Next, (2-1) the multi-degradation recognition module extracts multi-dimensional quality features of the image, including brightness histogram, texture sharpness, noise intensity, and local contrast, to form a feature set: Wherein, represents the image brightness distribution, preferably using a histogram with 256 gray levels; represents the variance based on the Laplacian operator, used to measure image sharpness, typically ranging from 50 to 150 in sharp images, and decreasing to below 10 in severely blurred images; represents the noise variance, obtained in this embodiment through spatial domain noise estimation or wavelet domain estimation, typically ranging from 5 to 30; and represents the local contrast.

[0065] (2-2) Then, the module uses a deep learning model or a rule classifier to identify the degradation type and outputs a degradation vector: These represent the confidence levels for low brightness, noise, blur, glare, and haze in the image, respectively, with values ​​ranging from [0,1]. In this embodiment, a value greater than 0.7 is considered significant degradation, while a value between 0.3 and 0.7 is considered moderate.

[0066] (2-3) Further calculate the degree of degradation of the signal-to-noise ratio and ambiguity index: Where is the average brightness of the image, typically between 20 and 80, is the noise variance, is the sharpness variance of the current image, is the standard variance of the reference sharp image, and represents the blur level index.

[0067] Subsequently, (3-1) the algorithm scheduling module dynamically determines the algorithm link based on the identified degradation information and system resource status: Here, represents the set of algorithm execution orders, and represents the system operating status (including CPU / GPU utilization, battery power, and frame rate). In this embodiment, when CPU utilization exceeds 80% or GPU utilization exceeds 85%, or the real-time processing frame rate is below 25fps, the system is considered to be in a resource-scarce state, requiring a simplified algorithm chain. When the battery power is below 20%, the image resolution is reduced to 1280×720, and the computationally complex deblurring module is disabled. When low brightness and noise are detected simultaneously (r_light>0.6 and r_noise>0.6), brightness enhancement is performed first, followed by noise reduction. If motion blur is detected (r_blur>0.7), a deblurring algorithm is inserted. If fog or glare exists (r_fog>0.6 or r_glare>0.6), defogging and HDR processing are enabled.

[0068] Module (3-2) can also optimize the scheduling strategy through reinforcement learning to maximize the reinforcement effect: Where is the optimal scheduling strategy, is the system state, is the quality score after image enhancement, and is the discount factor. In this embodiment, the discount factor γ is preferably 0.9, and the time step T is 5 to 10; the quality score R_t can be normalized by combining PSNR, structural similarity index SSIM and subjective evaluation, and the value range is [0,1].

[0069] Next, (4-1) the image enhancement and restoration algorithm library performs targeted processing. For low-light images, gamma correction or the Retinex algorithm is used for brightness enhancement: Wherein, is the Gamma correction coefficient. In this embodiment, when the average brightness μ_I < 30, the value of γ ranges from 1.8 to 2.4, typically γ = 2.0; when μ_I is between 30 and 60, the value of γ ranges from 1.2 to 1.6. is a Gaussian convolution kernel with a standard deviation of σ. In this embodiment, for 1080p images, σ is preferably between 1.5 and 3.0, corresponding to a kernel size of 5×5 or 7×7 pixels, representing the convolution operation.

[0070] (4-2) When noise is significant, the system performs bilateral filtering for noise reduction: Wherein, and are Gaussian functions in the spatial domain and pixel intensity domain, respectively, is the pixel neighborhood window, and in this embodiment, Ω is preferably a neighborhood with a diameter of 5 to 9 pixels, and is the normalization coefficient.

[0071] (4-3) If motion blur is detected, blind deconvolution is used for restoration: Where is the Fourier transform operator, is the point spread function (PSF), and is the regularization term to prevent the denominator from being zero. In this embodiment, the size of the PSF is set according to the estimated blur length, generally taken as 9×9 to 21×21 pixels, and the regularization term is set to 10^(-3) to 10^(-2). The PSF parameters can be obtained through edge direction analysis or a deep learning-based estimation network.

[0072] (4-4) For haze or glare images, a dark channel prior algorithm is used to recover transmittance and scene irradiance: Where is transmittance, is an empirical coefficient (usually taken as 0.95), is atmospheric light intensity, which can be estimated by selecting the maximum brightness value from the darkest 0.1% pixels of the image, is the value of the pixel in the channel, and is the restored image.

[0073] (4-5) While the image is still slightly smooth, perform detail sharpening: Here, is the sharpening intensity parameter, which controls the enhancement magnitude. In this embodiment, the value ranges from 0.3 to 0.8, with a typical value of 0.5; the standard deviation is 0.8 to 1.5, used to construct low-frequency components, thereby enhancing edge and texture details.

[0074] Meanwhile, (5-1) the edge adaptation control module monitors the CPU / GPU utilization, power consumption and frame rate of the UAV embedded computing platform in real time, with each monitoring cycle typically lasting 100ms.

[0075] (5-2) When resources are scarce, the system automatically simplifies the algorithm structure or reduces the model resolution: for example, when the CPU utilization is above 85% for three consecutive cycles, the input image resolution is reduced from 1920×1080 to 1280×720, and the blind deconvolution module is turned off; when the battery level is below 15%, the processing frame rate is reduced from 30fps to 20fps, and a lightweight denoising and brightness enhancement algorithm is adopted. When resources are sufficient (CPU utilization is below 70%, GPU utilization is below 75%, and battery level is above 40%), the complete high-precision pipeline is enabled, including dehazing, blind deconvolution, and detail sharpening modules, thereby achieving a balance between performance and quality.

[0076] Finally, (6-1) the enhanced image is output after processing: Here, represents the enhanced and restored output image, and represents the overall enhancement function. After configuring the parameters in this embodiment, the signal-to-noise ratio of the output image in a typical night flight scenario can be improved from 15-20dB to 25-30dB, the structural similarity index (SSIM) can be improved by 0.1-0.2, and the overall processing frame rate can be maintained above 25fps. The enhancement results can be used for UAV navigation, target recognition, and ground monitoring, achieving real-time and robust night flight image processing.

[0077] In summary, the system of this invention achieves night flight image enhancement with multi-degradation recognition and adaptive algorithm scheduling on an embedded platform, balancing image quality and real-time performance, and has broad application prospects. The above description is merely a preferred embodiment of this invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the technical principles of this invention. For example, updating and expanding the algorithm library: the image enhancement and restoration algorithm library can be updated or replaced with more advanced low-light enhancement, noise reduction, and deblurring models based on new research progress, or super-resolution, rain / snow removal, and other algorithm modules can be incorporated; optimizing the degradation recognition model: the multi-degradation recognition module can be upgraded to a deep learning model that recognizes more degradation types (such as optical distortion and artifact removal), or the confidence threshold and feature extraction method can be adjusted; adjusting the edge adaptation strategy: the edge adaptation control module can adjust the resource-constrained and sufficient judgment thresholds according to different hardware platforms (such as higher-performance NVIDIA Jetson series or low-power Movidius VPU), and optimize the simplified / complete switching logic of the algorithm link.

[0078] These improvements and variations, as long as their core principles remain based on "multi-degradation recognition" and "adaptive scheduling" to achieve real-time enhancement and restoration of night flight images, should all be considered within the scope of protection of this invention.

Claims

1. A method for enhancing and restoring UAV night flight images based on multi-degradation recognition and adaptive algorithm scheduling, characterized in that, For nighttime flight scenarios of UAVs, the method mainly includes an image acquisition module, a multi-degradation recognition module, an algorithm scheduling module, an image enhancement and restoration algorithm library, and an edge adaptation control module. First, the image acquisition module acquires raw images during nighttime flight and completes basic preprocessing. Then, the multi-degradation recognition module analyzes the raw images to obtain degradation intensity vectors and corresponding level labels that reflect the degree of degradation such as low brightness, noise, motion blur, haze, or atmospheric scattering. Then, the algorithm scheduling module selects and arranges a subset of image enhancement and restoration algorithms to be executed from the image enhancement and restoration algorithm library based on the degradation intensity vector, level label and current onboard computing resource status, and determines the running parameters of each algorithm; Under the constraints of the edge adaptation control module, the image enhancement and restoration algorithm library performs low-light enhancement, image noise reduction, image deblurring, image dehazing, and detail sharpening on the original image according to the algorithm sequence and running parameters, and outputs an enhanced and restored image. Among them, the edge adaptation control module monitors the resource status such as CPU and GPU utilization, power consumption, and frame rate in real time, and dynamically adjusts the algorithm sequence and running parameters according to the resource margin to maximize image quality while meeting real-time and energy consumption constraints.

2. The method for enhancing and restoring UAV night flight images based on multi-degradation recognition and adaptive algorithm scheduling according to claim 1, characterized in that, The multi-degradation identification module generates degradation intensity vectors and level labels by constructing a multi-task deep neural network. The key steps are as follows: 2.1 Construct a neural network with a shared feature extraction layer and multiple degenerate branches. The shared feature extraction layer extracts features from the input image to obtain a multi-scale low-light feature description. 2.2 For each degradation type, such as brightness degradation, noise intensity, motion blur, and the effects of haze or atmospheric scattering, a regression or classification branch is set up to output a continuous degradation intensity estimate. 2.3 Input the night flight image, pass through the shared feature extraction layer to obtain shared features, and then send them to each degradation branch head to obtain a degradation intensity vector composed of brightness degradation intensity, noise intensity, blur degree, haze intensity, etc. 2.4 For each type of degradation, multiple threshold levels are set for its continuous intensity, and the degradation intensity is divided into discrete levels such as "no obvious", "mild", "moderate" and "severe", to obtain the corresponding level labels; 2.5 The degradation intensity vector and the level label are used together as the decision input for subsequent algorithm scheduling. When the level label of a certain degradation type reaches the preset threshold, the algorithm scheduling module must insert the corresponding processing steps. 2.6 During the training phase, a joint loss function is used to simultaneously constrain the regression error and classification error of each degradation branch, so as to improve the overall accuracy and robustness of multi-degradation recognition.

3. The method for enhancing and restoring UAV night flight images based on multi-degradation recognition and adaptive algorithm scheduling according to claim 1, characterized in that, The edge adaptation control module divides the onboard computing resource status into several typical states, including one of the following: idle state, light load state, normal load state, high load state, and overload state. The onboard computing resource status set can be a subset of the states, or a superset with the states as the key states, and is used to constrain the process selection and parameter configuration of the algorithm scheduling module under different resource states.

4. The method according to claim 3, characterized in that, The determination of the status of airborne computing resources uses the following characteristic parameters: Let the current overall load rate of the airborne computing platform be , the current output frame rate be , the target frame rate be , and the remaining battery percentage be . ， The power safety threshold is, the upper load threshold is, the lower load threshold is, and the lower frame rate limit is. Then the state division method is as follows: Idle or light load state: , Normal load state: , High load state: or , Overload state: or .

5. The method for enhancing and restoring UAV night flight images based on multi-degradation recognition and adaptive algorithm scheduling according to claim 1, characterized in that, The algorithm scheduling module establishes a set of scheduling strategies based on business requirements and resource constraints, and constructs a mapping from "degradation level combination and airborne resource status" to "schedule strategy set power set". Each degradation level combination and resource status corresponds to a set of scheduling strategies. If a set of scheduling strategies contains only one algorithm flow template, then that template must be executed. If a scheduling strategy contains multiple candidate templates, one or more templates are selected to generate the final processing flow based on task type, visual quality preference, or energy consumption constraints.

6. The method for enhancing and restoring UAV night flight images based on multi-degradation recognition and adaptive algorithm scheduling according to claim 1, characterized in that, The algorithm scheduling module generates an image enhancement and restoration process based on degradation information and resource status. The key steps are as follows: 6.1 Obtain the degradation intensity vector and level label output by the multi-degradation identification module, as well as the current resource status provided by the edge adaptation control module; 6.2 In the pre-built rule base, a set of matching candidate process templates is found based on the combination of degradation levels. Each template specifies the type of algorithm to be called and its basic order constraints. 6.3 In the candidate process template set obtained in step 6.2, based on the current airborne resource status, templates that cannot meet the constraints in terms of computational latency or energy consumption are eliminated, and a feasible process set that meets the constraints is obtained; 6.4 For each feasible process, the execution order and parameters of the algorithm modules in the process are used as optimization variables to construct an optimization problem with the goal of improving the no-reference quality score and the frame rate constraint. The optimal process and its parameter set are obtained by solving the problem. 6.5 When there are multiple optimal processes that meet the conditions, select one as the final processing process for the current frame image based on task importance, target region weight, or historical performance. 6.6 Send the selected process and parameter set to the image enhancement and restoration algorithm library, and execute the corresponding algorithm modules in sequence.

7. The method according to claim 1, characterized in that, The image enhancement and restoration algorithm library processes images according to the described process and evaluates the processing effect based on quality indicators. The key steps are as follows: 7.1 For images with significant low-light degradation, low-light enhancement algorithms should be prioritized, including at least one of Gamma correction, Retinex decomposition, or zero-reference depth brightening network; 7.2 For images with high noise levels, perform image denoising algorithms, including BM3D-like methods based on Gaussian noise or Poisson Gaussian noise models or deep learning denoising networks; 7.3 When motion blur or defocus is detected, execute an image deblurring algorithm, including non-blind deconvolution or end-to-end depth deblurring network; 7.4 In the presence of haze, long-distance scattering, or strong backlight, execute a dehazing or de-atmospheric scattering algorithm based on an atmospheric scattering model to restore transmittance and scene irradiance; 7.5 After the above processing, perform detail sharpening on the image to improve edge sharpness and local texture contrast; 7.6 During processing, quality metrics such as no-reference quality, local contrast enhancement, and gradient energy enhancement are calculated. When a reference ground truth image exists, Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) are calculated, where: 。 8. The method for enhancing and restoring UAV night flight images according to claims 3 and 4, characterized in that, The edge adaptation control module determines the airborne resource status based on the algorithm configuration currently running in the system and real-time resource measurement values. The key steps are as follows: 8.1 Periodically collect resource indicators such as current CPU and GPU utilization, output frame rate F, average single-frame computation latency, and remaining power E, and compare them with preset thresholds; 8.2 Calculate the overall load rate, frame rate deviation, and power margin using the collected resource indicators; 8.3 If so, the airborne resource status will be determined as overloaded; 8.4 If so, it is determined to be a high load state; 8.5 If so, it is determined to be a normal load state; 8.6 If so, it is determined to be in an idle or lightly loaded state; 8.7 The resource status obtained from the determination is fed back to the algorithm scheduling module to constrain the subsequent process generation and parameter selection.

9. The method according to claim 1, characterized in that, The edge adaptation control module employs different load reduction or quality improvement strategies when detecting different resource states. The key steps are as follows: 9.1 When resources are idle or lightly loaded, enable the full high-precision image enhancement and restoration workflow, improve the processing intensity of low-light enhancement, noise reduction, deblurring and dehazing, and use complex deep learning models. 9.2 When the resource status is under normal load, under the premise of ensuring the performance of key algorithm modules such as brightness enhancement and noise reduction, the execution frequency or processing intensity of modules with relatively small benefits such as detail sharpening should be appropriately reduced. 9.3 When the resource status is high load, automatically start some load reduction strategies, including reducing image resolution or frame rate, switching to a lightweight model, reducing the number of iterations, or temporarily skipping processing steps with low marginal benefits. 9.4 When the resource status is overloaded, enforce the load reduction strategy, retain the necessary basic enhancement processes to maintain minimum visual availability, and record the current working configuration; 9.5 When the resource status recovers from high load or overload to idle, light load or normal load, the high-precision process and parameter configuration are gradually restored according to the preset rollback strategy, so that the system can smoothly transition from the load reduction mode to the high-quality mode, while avoiding system oscillation caused by frequent switching.

Images