Remote sensing image deblurring method and device based on definition prior and storage medium
By constructing a variational model with clarity prior and iterative sub-problem decomposition, the method addresses optical and assembly errors in remote sensing, achieving clearer and robust deblurred images.
Patent Information
- Application Number
- CN202411293096.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2025-01-03
AI Technical Summary
Existing remote sensing image deblurring methods face challenges in achieving clear and robust results due to optical design issues and assembly errors, with coded aperture-based methods being hardware-dependent, variational-based methods struggling with complex simulations, and deep learning-based methods relying on data sets and hardware support.
A variational model incorporating clarity prior information is constructed, simplified into two sub-problems for iterative solution, and combined with non-blind estimation to estimate a blur kernel, iteratively refining the deblurring process across varying resolutions.
This approach yields clearer, more faithful, and robust deblurred remote sensing images by leveraging clarity prior information, enhancing edge details and structural information.
Smart Images
Figure CN119251094A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image enhancement, and in particular to a remote sensing image deblurring method, device and storage medium based on clarity prior. Background Art
[0002] As a common ground perception technology, remote sensing can effectively obtain diverse and rich perception information at low altitude, high altitude and space, and is widely used in military, agriculture, surveying and mapping, climate monitoring and other fields. However, since the design of the optical system is difficult to achieve perfect focusing results and there are certain errors in assembly, the sensors installed on the aircraft often obtain relatively blurred remote sensing images, which greatly hinders the subsequent interpretation and application work. Therefore, a large amount of work has been carried out on this issue, usually including: coded aperture-based methods, variational-based methods and deep learning-based methods.
[0003] The coded aperture-based method usually adds an additional mask to the front lens to add fixed coding information to the acquired image, and finally decodes the image using a post-processing method to obtain the result image. This method can usually effectively improve the high-frequency information contained in the image, but it is heavily dependent on the subsequent decoding module. Once there is a problem with the decoding, the result image will be abnormal. In addition, the decoding steps are relatively cumbersome and the calculations are relatively complex.
[0004] Variation-based methods usually contain strict mathematical models, which can effectively simulate the blur problem of remote sensing images, thereby effectively removing blur and obtaining clear restoration results. However, it usually uses the L1 norm of the image gradient as prior information for deblurring, which is difficult to simulate the prior information contained in complex clear images, and cannot obtain more robust and clear deblurring results.
[0005] The deep learning-based method performs deblurring based on data priors and can effectively obtain clearer deblurring results. However, it is heavily dependent on the data set, and the computing environment requires GPU support, and is also highly dependent on hardware. Summary of the invention
[0006] The purpose of the present invention is to propose a remote sensing image deblurring method, device and storage medium based on clarity prior, so as to solve the optical design problem of the remote sensing system in the prior art and the defocus blur technical problem of the remote sensing image caused by assembly error.
[0007] The present invention provides a remote sensing image deblurring method based on clarity prior, comprising the following steps:
[0008] S1. Construct a variational model with added clarity prior.
[0009] S2, simplifying the variational model into two sub-problems, and solving them iteratively to obtain a potential deblurred image;
[0010] S3, estimating blur kernel based on potential deblurred image;
[0011] S4, iteratively calculating steps S2 to S3 using images of different resolutions, and outputting a final blur kernel;
[0012] S5. Deblurring the remote sensing image using a non-blind estimation method according to the final blur kernel to obtain a deblurred image.
[0013] A storage medium stores instructions and data for implementing a remote sensing image deblurring method based on clarity prior.
[0014] A remote sensing image deblurring device based on clarity prior comprises: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a remote sensing image deblurring method based on clarity prior.
[0015] The beneficial effect provided by the present invention is that by adopting a variational method and introducing additional clarity prior information in the modeling of the problem, a clearer, more fidelity and more robust deblurred remote sensing image can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a schematic flow chart of the method of the present invention;
[0017] Figure 2 It is a detailed flow chart of the method of the present invention;
[0018] Figure 3 This is a graph showing the intermediate results of iteratively solving the blur kernel;
[0019] Figure 4 This is a diagram showing the results of remote sensing image deblurring;
[0020] Figure 5 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION
[0021] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0022] Before formally describing the present invention, the scheme of the present invention is first generally described for easy understanding.
[0023] Please refer to Figure 1 and Figure 2 , Figure 1It is a schematic flow diagram of the method of the present invention; Figure 2 It is a detailed flow chart of the method of the present invention.
[0024] The present invention provides a remote sensing image deblurring method based on clarity prior, comprising the following steps:
[0025] S1. Construct a variational model with added clarity prior.
[0026] It should be noted that the present invention adds additional clarity prior information in the variational model, which can provide clearer edges and real ground object detail information in the deblurred remote sensing image. The specific model is as follows:
[0027]
[0028] Among them, u and f are the potential clear remote sensing image and the damaged remote sensing image respectively, B is the blur kernel corresponding to the damaged remote sensing image; λ>0, μ>0, ξ>0 are the clarity prior coefficient, regularization coefficient, and sparsity coefficient of the blur kernel respectively, ‖·‖ is the L2 norm, and ‖·‖0 is the L0 norm; i,j are the pixel coordinates in the vertical and horizontal directions, P4 is the set of 4 neighborhood pixels around the pixel (i,j), is the gradient operator. In formula (1), the second term This adds clarity prior information to us.
[0029] S2, simplifying the variational model into two sub-problems, and solving them iteratively to obtain a potential deblurred image;
[0030] It should be noted that the variational model in step S1 is non-convex and non-smooth. The present invention decomposes the problem and simplifies it into two sub-problems for iterative solution, as shown below:
[0031]
[0032] By decomposing the above problem, the difficult-to-solve variational model can be transformed into two easy-to-solve sub-problems for rapid solution. Next, the present invention solves the first sub-problem.
[0033] It should be noted that the iterative solution process in step S2 is as follows:
[0034] S21. For the first sub-problem, two additional variables G and H are introduced to simplify the solution process as follows:
[0035]
[0036] Limited by
[0037] S22, converting the solution process in step S21 into an unrestricted problem:
[0038]
[0039] Among them, θ1>0, θ2>0 are the introduced compensation term coefficients;
[0040] S23, As an operation, it can be expressed as As follows:
[0041]
[0042] S24. When Q is known, solve u as follows:
[0043]
[0044] Among them, F(·) and F -1 (·) are fast Fourier transform and fast quasi-Fourier transform, respectively, B T , Q T , The corresponding B, Q, The matrix transpose of ;
[0045] S25. When u is known, solve Q as follows:
[0046]
[0047] It should be noted that the potential deblurred image u here is not the final deblurred result, but an intermediate deblurred image with detail information smoothed and structural information enhanced (see the attached figure). Figure 3 The purpose of solving the deblurred image u in this sub-iterative process is to solve the accurate blur kernel, and finally use the blur kernel to solve the final deblurring result by non-blind estimation method. The overall solution process method is shown in the attached Figure 2 As shown in , the final deblurring result is Figure 4 shown.
[0048] S26. Iteratively solve the expression of u and the expression of q until they are fitted, and obtain the potential deblurred image u during the fitting.
[0049] S3, estimating blur kernel based on potential deblurred image;
[0050] It should be noted that the blur kernel B is estimated based on the deblurred image obtained in the previous step. Specifically, the present invention uses fast Fourier transform to quickly solve, as shown below:
[0051]
[0052] Among them, F(·) and F -1 (·) are fast Fourier transform and fast quasi-Fourier transform, respectively.
[0053] S4, iteratively calculating steps S2 to S3 using images of different resolutions, and outputting a final blur kernel;
[0054] It should be noted that, through the above calculations, closed solutions of the potential clear image and blur kernel are obtained respectively. Then, the present invention iterates steps S2 and S3 until the obtained u reaches the maximum number of iterations or satisfies ||uu pre || 2 <0.001, the current B can be output as the final blur kernel. pre Compute the result for the previous iteration.
[0055] In the above steps, in order to reduce the amount of calculation, we can use the downsampled image to estimate the initial blur kernel in the initial iteration. Then, we gradually use the large-resolution image to estimate the precise blur kernel. Finally, we use the full-resolution image to estimate the final precise blur kernel.
[0056] As an embodiment, if the number of iterations is 5, the specific downsampling ratios are as follows: in the first iteration, the downsampling is 1 / 5; in the second iteration, the downsampling is 2 / 5; in the third iteration, the downsampling is 3 / 5; in the fourth iteration, the downsampling is 4 / 5; in the fifth iteration, no downsampling is performed.
[0057] As an example, please refer to Figure 3 , Figure 3 The intermediate results of iterative solution of blur kernel are shown. (a) represents the first iterative solution with a resolution of 254*254; (b) represents the second iterative solution with a resolution of 360*360; (c) represents the third iterative solution with a resolution of 500*500.
[0058] S5. Deblurring the remote sensing image using a non-blind estimation method according to the final blur kernel to obtain a deblurred image.
[0059] Through the above process, the present invention estimates an accurate blur kernel. At this time, the remote sensing image deblurring is transformed into a general non-blind deblurring problem. In this step, a general variational method is used for non-blind deblurring. The specific process and method are existing and will not be described in detail here.
[0060] Please refer to Figure 4 , Figure 4 The results of remote sensing image deblurring are shown, where (a) is the input blurry remote sensing image and (b) is the deblurred remote sensing image.
[0061] See also Figure 5 , Figure 5 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically comprises: a remote sensing image deblurring device 401 based on clarity prior, a processor 402 and a storage medium 403.
[0062] A remote sensing image deblurring device 401 based on clarity prior: The remote sensing image deblurring device 401 based on clarity prior implements the remote sensing image deblurring method based on clarity prior.
[0063] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the remote sensing image deblurring method based on clarity prior.
[0064] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the remote sensing image deblurring method based on clarity prior.
[0065] The beneficial effect of the present invention is that by adopting a variational method and introducing additional clarity prior information in the modeling of the problem, a clearer, more fidelity and more robust deblurred remote sensing image can be obtained.
[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A remote sensing image deblurring method based on clarity prior, characterized in that: The method comprises the following steps: S1. Construct a variational model with added clarity prior. S2, simplifying the variational model into two sub-problems, and iteratively solving the first sub-problem to obtain a potential deblurred image; S3, estimating blur kernel based on potential deblurred image; S4, iteratively calculating steps S2 to S3 using images of different resolutions, and outputting a final blur kernel; S5. Deblurring the remote sensing image using a non-blind estimation method according to the final blur kernel to obtain a deblurred image.
2. The remote sensing image deblurring method based on clarity prior according to claim 1, characterized in that: The variational model in step S1 is as follows: Among them, u and f are the potential clear remote sensing image and the damaged remote sensing image respectively, B is the blur kernel corresponding to the damaged remote sensing image; λ>0, μ>0, ξ>0 are the clarity prior coefficient, regularization coefficient, and sparsity coefficient of the blur kernel respectively, ‖·‖ is the L2 norm, and ‖·‖0 is the L0 norm; i,j are the pixel coordinates in the vertical and horizontal directions, P4 is the set of 4 neighborhood pixels around the pixel (i,j), is the gradient operator, where the second term Prior information for added clarity.
3. The remote sensing image deblurring method based on clarity prior according to claim 2, characterized in that: The first sub-problem in step S2 is:
4. The remote sensing image deblurring method based on clarity prior according to claim 3, characterized in that: The iterative solution process in step S2 is as follows: S21. Introduce two additional variables G and H to simplify the solution process as follows: S22, converting the solution process in step S21 into an unrestricted problem: Among them, θ1>0, θ2>0 are the introduced compensation term coefficients; S23, As an operation, it can be expressed as As follows: S24. When Q is known, solve u as follows: Among them, F(·) and F -1 (·) are fast Fourier transform and fast quasi-Fourier transform, respectively, B T , Q T , The corresponding B, Q, The matrix transpose of ; S25. When u is known, solve Q as follows: S26. Iteratively solve the expression of u and the expression of q until they are fitted, and obtain the potential deblurred image u during the fitting.
5. The remote sensing image deblurring method based on clarity prior according to claim 4, characterized in that: The blur kernel calculation in step S3 is as follows: Among them, F(·) and F -1 (·) are fast Fourier transform and fast quasi-Fourier transform, respectively.
6. The remote sensing image deblurring method based on clarity prior according to claim 5, characterized in that: Step S4 is specifically as follows: in the initial iteration, the downsampled image is used to estimate the initial blur kernel; Then, the image resolution is gradually increased to estimate the accurate blur kernel; finally, the full-resolution image is used to estimate the final accurate blur kernel.
7. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a remote sensing image deblurring method based on clarity prior as described in any one of claims 1 to 6.
8. A remote sensing image deblurring device based on clarity prior, characterized in that: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a remote sensing image deblurring method based on clarity prior as described in any one of claims 1 to 6.