A Monocular Vision Image Deblurring Method for Low-Altitude Micro UAVs

Abstract

This invention discloses a monocular vision image deblurring method for low-altitude micro unmanned aerial vehicles (UAVs), belonging to the field of micro UAV image processing. The method includes: preprocessing a blurred image to obtain a preprocessed blurred image and a blurred kernel estimation result; constructing a deep reinforcement learning model based on attention mechanism and memory playback, dividing the preprocessed blurred image into overlapping sub-image blocks to obtain several blurred image blocks and corresponding blurred kernel blocks; inputting the blurred image blocks and corresponding blurred kernel blocks into the deep reinforcement learning model, performing channel-level fusion of the attention weight map with the blurred image blocks and blurred kernel blocks, and then performing deblurring processing according to the dual-network structure to output deblurred sub-blocks with the same size as the input; and eliminating the discontinuity of the block boundaries through weighted fusion of overlapping regions based on the deblurred sub-blocks to obtain a reconstructed image. This invention solves the edge discontinuity problem that easily occurs in traditional block-segmentation methods, balancing processing speed and image quality in complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of micro unmanned aerial vehicle (UAV) image processing technology, and particularly relates to a monocular vision image deblurring method for low-altitude micro unmanned aerial vehicles (UAVs). Background Technology

[0002] Low-altitude micro drones have broad application prospects in fields such as aerial photography, environmental monitoring, agricultural plant protection, and emergency rescue. However, due to the complex and variable low-altitude environment, such as atmospheric turbulence, drone vibration, unstable lighting conditions, and rapid target movement, images acquired by monocular vision systems often exhibit blurring, which seriously affects image quality and the accuracy of subsequent target detection, recognition, and tracking tasks.

[0003] Traditional image deblurring methods mainly include model-based and statistical methods. Model-based methods typically assume that image blurring is caused by a linear time-invariant system. They estimate the blur kernel and then use inverse filtering, Wiener filtering, and other methods to restore the blurred image. However, in complex low-altitude environments, the blurring process is often nonlinear and time-varying, making it difficult for traditional model-based methods to accurately estimate the blur kernel, resulting in poor deblurring performance. Statistical methods, on the other hand, learn from a large number of sharp-blurred image pairs to establish a mapping relationship between blurred and sharp images. Early statistical methods, such as support vector machines and sparse coding, also had limited deblurring performance due to their limited feature extraction capabilities. With the development of deep learning technology, existing deep learning-based deblurring methods still have some problems: they usually require a large amount of training data, which is often difficult to obtain in practical applications of low-altitude micro UAVs; moreover, the performance of existing methods still needs improvement when dealing with complex blurring scenarios such as dynamic blur and multi-direction blur.

[0004] In summary, the existing monocular vision image deblurring methods for low-altitude micro UAVs still need improvement in terms of adaptability and deblurring effect in complex environments. Therefore, there is an urgent need for a new monocular vision image deblurring method for low-altitude micro UAVs that can effectively handle complex and blurred scenes and reduce dependence on training data. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a monocular vision image deblurring method for low-altitude micro unmanned aerial vehicles (UAVs) to solve the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides a monocular vision image deblurring method for low-altitude micro unmanned aerial vehicles, comprising:

[0007] The blurred images acquired by the low-altitude micro UAV are preprocessed to obtain the preprocessed blurred images and the blur kernel estimation results;

[0008] A deep reinforcement learning model based on attention mechanism and memory replay is constructed. The preprocessed blurred image is divided into overlapping sub-image blocks to obtain several blurred image blocks. The blurred kernel block corresponding to each blurred image block is determined based on the blurred kernel estimation result.

[0009] The blurred image patch and the corresponding blurred kernel block are input into the deep reinforcement learning model. The deep reinforcement learning model performs channel-level fusion of the attention weight map with the local features of the blurred image patch and the blurred kernel block, and then performs deblurring processing according to the dual network structure, outputting a deblurred sub-block with the same size as the input.

[0010] Based on the deblurred sub-blocks, the discontinuities at the block boundaries are eliminated by weighted fusion of overlapping regions to obtain the reconstructed image.

[0011] Optionally, the preprocessing of blurred images acquired by low-altitude micro UAVs includes:

[0012] The blurred image is normalized to obtain a normalized image;

[0013] The normalized image is subjected to median filtering for noise reduction to obtain a preprocessed blurred image;

[0014] The preprocessed blurred image is subjected to fuzzy kernel estimation based on the improved variational model to obtain the fuzzy kernel estimation result.

[0015] Optionally, the improved variational model is:

[0016] ;

[0017] In the formula, express L2 norm, and Let L1 norms represent the gradients of the sharp image and the blur kernel, respectively. The first regularization parameter is used. This is the second regularization parameter. For a clear image, For fuzzy kernel blocks, This represents the convolution operation. Represents the gradient operator, This is the blurred image after preprocessing.

[0018] Optionally, the deep reinforcement learning model performs channel-level fusion of the attention weight map with the local features of the blurred image patch and the blurred kernel patch, and the deblurring process based on the action selection of the policy network includes:

[0019] The blurred image patch and the corresponding blurred kernel block are input into the deep reinforcement learning model, and the deep reinforcement learning model extracts the features of the blurred image patch and the blurred kernel block through convolutional layers;

[0020] The normalized features of the attention weight map are obtained based on the attention weight map.

[0021] The features of the blurred image patch and the blurred kernel patch are concatenated with the normalized features of the attention weight map to obtain the state;

[0022] The dual-network structure performs action selection and value evaluation on the state to obtain deblurred sub-blocks.

[0023] Optionally, the dual-network structure includes: a policy network and a value network;

[0024] The process of obtaining deblurred sub-blocks by performing action selection and value evaluation on the state based on the dual-network structure includes:

[0025] The policy network outputs the action probability distribution based on the state;

[0026] The value network evaluates the long-term expected reward of the current state based on the state, and obtains the value evaluation result.

[0027] Based on the action probability distribution and value assessment results, the blurred image block is deblurred, and the deblurred sub-block with the same size as the input is output.

[0028] Optionally, the expression for calculating the probability distribution of actions is:

[0029] ;

[0030] In the formula, exp is an exponential function. In response to The convolutional feature extraction function, In response to The convolutional feature extraction function, For the action space, In the action space, except All other possible actions besides For state, According to The chosen action Let be the probability of the action.

[0031] Optionally, the deep reinforcement learning model is trained using a memory replay training mechanism; the process of training the model using the memory replay training mechanism includes updating the training loss function of the value network and optimizing the policy network.

[0032] The expression for calculating the training loss function of the value network is as follows:

[0033] ;

[0034] In the formula, Let be the training loss function for the value network. Represents the experience replay buffer Medium sampling The expected result The result of the state valuation. This is a discount reward.

[0035] Optionally, the objective of optimizing the policy network is to maximize the expected reward;

[0036] The objective function of the policy network is:

[0037] ;

[0038] In the formula, The objective function of the policy network is represented. For the parameters of the policy network, Representing state Obedience by strategy Induced state distribution , Indicates action Obedience Policy Network The probability distribution of action selection. for Expectations This is the reward function.

[0039] Optionally, several blurred image blocks and corresponding blurred kernel blocks are input into the deep reinforcement learning model to obtain the expression for the deblurred sub-blocks:

[0040] ;

[0041] In the formula, This represents the row index of the image patch in a two-dimensional grid. Indicates column index, For deep reinforcement learning networks based on attention and memory playback, For the action A defined set of network parameters To extract from the preprocessed blurred image The extracted first Overlapping sub-images, The input is used to obtain the deblurred sub-image from the ADRL network.

[0042] Optionally, based on the deblurred sub-blocks, the discontinuities at the block boundaries are eliminated through weighted fusion of overlapping regions, resulting in the following expression for the reconstructed image:

[0043] ;

[0044] In the formula, Represents the coordinates in the reconstructed image Pixel value at that location, This indicates that for all pixels... Image Patch Index Weighted fusion weight Summation, Relative coordinates within the block, and For the first The coordinates of the top left corner of each image patch.

[0045] Compared with the prior art, the present invention has the following advantages and technical effects:

[0046] This invention presents a monocular vision image deblurring method for blurry images acquired by low-altitude micro UAVs in complex environments, offering an efficient and adaptive deblurring solution. Preprocessing steps include normalization calibration, noise reduction, and blur kernel estimation, providing high-quality input for subsequent deblurring. A deep reinforcement learning model based on attention mechanisms and memory playback dynamically focuses on key regions in the image and intelligently adjusts the deblurring strategy according to local blur characteristics. Furthermore, overlapping block segmentation and weighted fusion techniques effectively address the problem of discontinuous block boundaries, improving the overall quality of the reconstructed image. This method not only significantly improves image deblurring performance but also enhances the model's adaptability and generalization ability in complex scenarios, providing clearer and more reliable visual data support for applications of low-altitude micro UAVs in fields such as marine ecological environment monitoring. Attached Figure Description

[0047] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0048] Figure 1 This is a flowchart of a monocular vision image deblurring method for low-altitude micro unmanned aerial vehicles according to an embodiment of the present invention. Detailed Implementation

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0051] Example 1

[0052] This invention relates to a monocular vision image deblurring method suitable for low-altitude micro unmanned aerial vehicles (UAVs), comprising: addressing image blurring caused by complex environments such as atmospheric turbulence and UAV vibration by normalizing and preprocessing the original blurred image with median filtering to eliminate noise interference and pixel value distribution deviations; employing an improved variational model to jointly constrain the gradient sparsity of the clear image and the blur kernel, thereby improving the accuracy of blur kernel estimation; constructing a deep reinforcement learning model (ADRL) based on attention mechanism and memory playback, fusing the attention weight map, image patch, and blur kernel block features into a composite state, and achieving adaptive optimization of the deblurring strategy through a dual-network structure with a policy network and a value network sharing a feature extraction layer, while utilizing the memory playback mechanism to improve training efficiency; finally, for large-size images, overlapping block processing is adopted, and after each sub-block is processed independently by the ADRL network, weighted fusion is performed to eliminate boundary discontinuities, achieving globally consistent clear image reconstruction. This invention, by combining attention focusing and dynamic policy adjustment, achieves accurate estimation of the blur kernel and efficient recovery of image details in complex environments.

[0053] like Figure 1 As shown in the figure, this embodiment provides a monocular vision image deblurring method for low-altitude micro UAVs. This method combines a deep reinforcement learning model (ADRL) based on attention mechanism and memory playback, which can effectively handle the monocular vision image blurring problem in complex low-altitude environments, improve the image deblurring effect and the adaptability of the model.

[0054] The process of image preprocessing to enhance image quality includes steps 100-102.

[0055] Step 100, Normalization Calibration: To avoid the impact of pixel value range differences on neural network training, the blurred images acquired by the low-altitude micro UAV are normalized, and the image pixel values ​​are normalized to the range of [0,1]. This ensures that the inputs of each layer of the neural network have similar numerical ranges, thereby accelerating convergence and improving training stability. The normalization formula is:

[0056] ;

[0057] in, Coordinates in the image Pixel value at that location, For normalized , The maximum pixel value in the image. This is the minimum pixel value.

[0058] Step 101, Median filtering noise reduction: A nonlinear filtering method is used to sort the pixel values ​​within the local window of the image and take the median value, which effectively suppresses impulse interference such as salt and pepper noise, and cleans up the image background while protecting edge details, thereby improving the input quality of fuzzy kernel estimation.

[0059] To address the noise commonly found in low-altitude environments, median filtering is used for noise reduction, resulting in a preprocessed blurred image. For a size of The rectangular filtering window noise filtering uses the median filtering method.

[0060] ;

[0061] in, Coordinates in the preprocessed image Pixel value at that location, This is the set obtained by sorting pixel values ​​in ascending order.

[0062] Step 102, variational model fuzz kernel estimation: Based on the fuzzy image formation model, the fuzzing process is modeled as the convolution of the clear image and the fuzz kernel with noise. By minimizing the L2 norm error to constrain data consistency, and combining L1 norm gradient regularization to suppress overfitting, the local smoothness of the clear image and the fuzz kernel is ensured.

[0063] An improved variational model is used to estimate the blur kernel of the preprocessed blurred image. The variational model is constructed as follows:

[0064] ;

[0065] in, express L2 norm, and Let L1 norms represent the gradients of the sharp image and the blur kernel, respectively. The first regularization parameter is used. This is the second regularization parameter. For a clear image, For fuzzy kernel blocks, This represents the convolution operation. This represents the gradient operator representation.

[0066] The process of constructing an ADRL-based intelligent deblurring network includes steps 200-204.

[0067] Step 200: Construct a deep reinforcement learning model (ADRL) based on attention mechanism and memory replay. In the state space construction, the attention weight graph is... By performing channel-level fusion with local features of image patches and fuzzy kernel blocks, a composite state representation containing spatial structure, fuzzy characteristics, and regional importance is formed, providing multi-dimensional information input for policy decision-making.

[0068] Specifically, for the preprocessed image patch And a fuzzy kernel block m, features are extracted through a convolutional layer. and Then it is concatenated with the normalized features of the attention weight map. , get the state :

[0069] ;

[0070] in, This is for image stitching operations.

[0071] Step 201, Dual Network Structure Design: Policy Network and value network The system employs an architecture where the policy network and value network share a feature extraction layer. The policy network outputs the action probability distribution based on the composite state, while the value network estimates the long-term expected reward of the current state. The shared feature extraction layer design reduces parameter redundancy and strengthens the correlation modeling between local features and the deblurring policy, thereby improving the model's ability to represent complex scenes.

[0072] Policy networks are used to determine the state Select Action Value networks are used to estimate states. The value of the policy network and the value network both adopt a multi-layer convolutional neural network structure. The policy network uses the Softmax activation function in the output layer to output the probability of each action.

[0073] ;

[0074] Where exp is an exponential function. In response to The convolutional feature extraction function, In response to The convolutional feature extraction function, For the action space, In the action space, except All other possible actions besides those mentioned above.

[0075] Step 202: The value network uses a linear activation function in the output layer to output the value of the state.

[0076] ;

[0077] in, The mapping function is used. The policy network and the value network share a convolutional feature extraction layer to improve parameter utilization efficiency. Subsequent branches process the policy output and value estimation respectively. This structural design can effectively capture the correlation between the local features of image patches and the deblurring policy.

[0078] Step 203, Memory Replay Training Mechanism: Historical experience tuples are stored through memory replay units, and batch data is randomly sampled during training to update parameters. This mechanism breaks the strong correlation between continuous data, avoids the model losing its generalization ability due to overfitting to the current scenario, and significantly improves training stability and policy optimization efficiency.

[0079] Memory playback unit Stored experience tuples Batch sampling is used during training, where... For the reward function, for The state at any given time. Training is performed using randomly sampled historical experience data to reduce correlations between data points and improve the model's training efficiency and stability. The training loss function for the value network. for:

[0080] ;

[0081] in, As a discount reward, Represents the experience replay buffer Medium sampling The expected outcome.

[0082] Step 204: The policy network is optimized using the policy gradient method, with the goal of maximizing the expected reward.

[0083] ;

[0084] in, The objective function of the policy network is represented. For the parameters of the policy network, Representing state Obedience by strategy Induced state distribution . Indicates action Obedience Policy Network The probability distribution of action selection. for The expectation.

[0085] The image deblurring process includes steps 300-301.

[0086] Step 300, Block Processing: The preprocessed blurred image is divided into overlapping sub-image blocks. Each sub-image block is independently input into the ADRL model for deblurring, and the output is a deblurred sub-block with the same size as the input. Block processing reduces computational complexity and allows the model to dynamically adjust its strategy for sub-blocks with different degrees of blur.

[0087] Considering the large image size and uneven blur level of low-altitude micro UAV images, the images are divided into overlapping image blocks, and the pre-processed blurred images are analyzed. The extracted first Extracting sub-images from overlapping sub-images The input to the ADRL network yields the deblurred sub-image. The sub-images output by the network have the same size as the input, maintaining the spatial correspondence.

[0088] ;

[0089] in, This represents the row index of the image patch in a two-dimensional grid. Indicates column index, For deep reinforcement learning networks based on attention and memory playback, For the action A defined set of network parameters.

[0090] Step 301, Overlap Fusion: Discontinuities at block boundaries are eliminated through weighted fusion of overlapping regions. During the fusion process, the final value of each pixel is obtained by a weighted sum of the deblurred results of all sub-blocks containing that pixel. The weights are dynamically adjusted based on the pixel's position within the sub-block (e.g., lower weights for edge regions and higher weights for center regions). This strategy ensures the global continuity and detail consistency of the reconstructed image.

[0091] After deblurring each image patch independently, the overlapping areas are merged to eliminate the discontinuity of the patch boundaries, thus obtaining a complete and clear reconstructed image. The specific formula is as follows:

[0092] ;

[0093] in, Represents the coordinates in the reconstructed image Pixel value at that location, For all pixels Image Patch Index Weighted fusion weight Perform summation. For the first Deblurring results for individual image patches Relative coordinates within the block, and For the first The coordinates of the top left corner of each image patch.

[0094] Through the above steps, this invention achieves efficient deblurring of monocular vision images of low-altitude micro UAVs. It can adaptively adjust the deblurring strategy in complex environments such as atmospheric turbulence and UAV vibration, effectively improving the accuracy of blur kernel estimation and the ability to restore image details.

[0095] This invention overcomes the limitations of traditional single-variable constraints by jointly constraining the gradient sparsity of the sharp image and the blur kernel through an improved variational model, accurately capturing the coupling relationship between the blur kernel and image content. This design effectively suppresses artifacts and constrains the structural complexity of the blur kernel, enabling the model to adaptively optimize blur kernel estimation in complex scenarios such as atmospheric turbulence and UAV vibration. This significantly improves the ability to recover details such as image edges and textures, providing a clearer and more realistic image foundation for subsequent visual tasks.

[0096] This invention addresses the large-size image characteristics of UAVs by employing an overlapping block input and weighted fusion strategy. This reduces computational complexity while eliminating discontinuous block boundaries. By combining an attention mechanism for dynamic focusing on highly blurred regions and adaptive adjustment of the ADRL strategy, the model can intelligently optimize deblurring operations for local regions with varying degrees of blur. This ensures processing efficiency while achieving a globally smooth transition in the reconstructed image, effectively solving the edge discontinuity problem common in traditional block-based methods, and balancing processing speed and image quality in complex scenes.

[0097] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle, characterized in that, The method comprises the following steps: The blurred image obtained by the low-altitude micro unmanned aerial vehicle is preprocessed to obtain a preprocessed blurred image and a blur kernel estimation result; A deep reinforcement learning model based on an attention mechanism and memory replay is constructed, the preprocessed blurred image is divided into overlapping sub-image blocks to obtain a plurality of blurred image blocks, and a blur kernel block corresponding to each blurred image block is determined based on the blur kernel estimation result; The blurred image blocks and the corresponding blur kernel blocks are input into the deep reinforcement learning model, the deep reinforcement learning model performs channel-level fusion of an attention weight map and local features of the blurred image blocks and the blur kernel blocks, and performs deblurring processing according to a double network structure to output deblurred subblocks with the same size as the input; Based on the deblurred subblocks, the discontinuity of the block boundary is eliminated by overlapping area weighted fusion to obtain a reconstructed image; The process of the deep reinforcement learning model performing channel-level fusion of the attention weight map and the local features of the blurred image blocks and the blur kernel blocks and performing deblurring processing according to the action selection of the policy network comprises: inputting the blurred image blocks and the corresponding blur kernel blocks into the deep reinforcement learning model, the deep reinforcement learning model extracting features of the blurred image blocks and the blur kernel blocks through a convolution layer; obtaining normalized features of the attention weight map based on the attention weight map; splicing the features of the blurred image blocks and the blur kernel blocks and the normalized features of the attention weight map to obtain a state; and the double network structure performing action selection and value evaluation on the state to obtain deblurred subblocks; The double network structure comprises a policy network and a value network; wherein the process of the double network structure performing action selection and value evaluation on the state to obtain deblurred subblocks comprises: the policy network outputting an action probability distribution based on the state; the value network evaluating a long-term expected reward of the current state based on the state to obtain a value evaluation result; and based on the action probability distribution and the value evaluation result, deblurring processing is performed on the blurred image blocks to output deblurred subblocks with the same size as the input. 2.The monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle according to claim 1, wherein, The process of preprocessing the blurred image obtained by the low-altitude micro unmanned aerial vehicle comprises: The blurred image is normalized and calibrated to obtain a normalized image; The normalized image is subjected to median filtering and noise reduction processing to obtain a preprocessed blurred image; The preprocessed blurred image is subjected to blur kernel estimation based on an improved variational model to obtain a blur kernel estimation result. 3.The monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle according to claim 2, characterized in that, The improved variational model is: ； wherein, denotes the L2 norm of and denote the L1 norm of the sharp image gradient and the blur kernel gradient, respectively, is a first regularization parameter, is a second regularization parameter, is a sharp image, is a blur kernel block, denotes a convolution operation, denotes a gradient operator, is a pre-processed blurred image. 4.The monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle according to claim 1, wherein, The expression for calculating the action probability distribution is: ； where exp is the exponential function, is a convolutional feature extraction function for is a convolutional feature extraction function for is the action space, is all other possible actions in the action space except is the state, is the action selected according to is the action probability.​​​​ 5. The monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle according to claim 4, wherein, The deep reinforcement learning model adopts a memory replay training mechanism for model training; in the process of model training using the memory replay training mechanism, a training loss function of the value network is updated and the policy network is optimized; The calculation expression of the training loss function of the value network is: ； In the formula, Let be the training loss function for the value network. Represents the experience replay buffer Medium sampling The expected result The result of the state valuation. This is a discount reward.

6. The monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle according to claim 5, wherein, The goal of optimizing the policy network is to maximize the expected reward; The objective function of the policy network is: ； wherein represents an objective function of the policy network, are parameters of the policy network, represents a state subject to a state distribution induced by a policy , , represents an action subject to an action selection probability distribution of the policy network , is an expectation of , is a reward function.

7. The monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle according to claim 1, wherein, The expression for inputting a plurality of blurred image blocks and corresponding blur kernel blocks into the deep reinforcement learning model to obtain deblurred subblocks is: ； wherein, represents a row index of the image block in a two-dimensional grid, represents a column index, is a deep reinforcement learning network based on attention and memory replay, is a set of network parameters determined by the action , is the i-th overlapping sub-image extracted from the pre-processed blurred image , is the i-th overlapping sub-image extracted from the pre-processed blurred image , is the deblurred sub-image obtained by inputting the ADRL network.

8. The monocular vision image deblurring method for low-altitude micro unmanned aerial vehicle according to claim 7, characterized in that, Based on the deblurring sub-block, the discontinuity of the block boundary is eliminated by overlapping region weighted fusion, and the expression of the reconstructed image is: ； wherein denotes the pixel value at the coordinates in the reconstructed image, denotes the weighted blending weight for all image blocks containing the pixel with index are summed up, is the relative coordinate within the block, and is the top-left corner coordinate of the th image block.​

Images