Optical image restoration method and system based on composite point spread function directional prior

Abstract

The application provides an optical image restoration method and system based on a composite point spread function direction prior, and belongs to the technical field of computer vision and image processing. The method comprises the following steps: obtaining a degraded image, a composite point spread function, a scattering compensation parameter and a noise estimation value; extracting a data fidelity degradation kernel; obtaining a to-be-restored blurred image through radiation compensation; constructing a second-order central moment matrix by taking the energy distribution of the degradation kernel as a weight, determining a principal axis direction, a principal axis scale, a secondary axis scale and an anisotropy ratio; constructing a direction difference operator along the principal axis direction and the orthogonal direction thereof; determining two direction regularization weights according to the principal axis scale, the secondary axis scale, the anisotropy ratio and the noise estimation value; constructing a target function containing a data fidelity term and a direction prior total variation regularization term, and solving the target function to obtain a restored image. The application adaptively matches the regularization constraint to the direction structure of the degradation kernel, improves the stretching and blurring suppression effect in the direction, and preserves the real edge and texture details in the orthogonal direction.

Description

Technical Field

[0001] This application belongs to the field of computer vision and image processing technology, and in particular relates to an optical image restoration method and system based on the direction prior of the composite point spread function. Background Technology

[0002] During Earth observation and long-range target identification and detection, high-altitude, high-speed platforms (such as UAVs and reconnaissance aircraft) often experience optical images affected by a combination of factors, including aero-optical effects, platform vibration, atmospheric scattering, and sensor noise. This results in images exhibiting composite degradation characteristics such as directional stretching and blurring, decreased contrast, and reduced signal-to-noise ratio. For such degraded images, even if the composite point spread function corresponding to the current operating conditions has been obtained through system calibration, online estimation, or external simulation modules, subsequent restoration algorithms still need to address how to fully utilize the directional information in this degradation kernel.

[0003] Currently, methods for restoring degraded images mainly include Wiener filtering, Richardson-Lucy (RL) iterative deconvolution, and regularized restoration methods based on Total Variation (TV). Wiener filtering and RL deconvolution typically rely on a global degradation kernel for frequency domain compensation or iterative correction, minimizing the mean square error or likelihood function by estimating a clear image. The total variation method, on the other hand, introduces the L1 norm of the image gradient as a regularization term on top of the data fidelity term, achieving a balance between deblurring and denoising by solving a constrained optimization problem. In practice, these methods often employ numerical solutions using variable splitting algorithms such as Alternating Directional Multiplier Method (ADMM) and Split Bregman.

[0004] However, the existing technologies still have the following shortcomings: First, conventional deconvolution methods are prone to ringing effects or high-frequency noise amplification under high-noise conditions. Traditional total variation models typically use fixed horizontal and vertical difference operators to apply the same or empirical penalty weights to the image gradients in each direction. When the principal axis of the composite point spread function is inconsistent with the image coordinate axes, it is difficult to effectively suppress diagonal or arbitrary stretching blur. Second, isotropic total variation regularization cannot distinguish the degradation differences between the principal axis and orthogonal directions of the composite point spread function: the principal axis often corresponds to severe stretching blur and requires strong regularization constraints, while orthogonal directions may retain more real edges and texture details, and excessive smoothing can lead to information loss. Third, the regularization weight coefficients in existing methods often need to be manually adjusted image by image, lacking an adaptive correlation mechanism with the degradation kernel scale, directionality, and degree of anisotropy, making it difficult to maintain stable restoration results under different observation conditions. Summary of the Invention

[0005] This application provides an optical image restoration method and system based on the directional prior of the composite point spread function, which can be used to solve the problems of insufficient suppression of directional stretching blur, over-smoothing of orthogonal directional details, and reliance on human experience for regularization parameters due to the failure to fully utilize the directional characteristics of the composite point spread function.

[0006] In a first aspect, this application provides an optical image restoration method based on the direction prior of the composite point spread function, including: Obtain the point spread function, scattering compensation parameters, and noise estimate of the degraded image and its composite image; Extract the data fidelity degradation kernel for image restoration from the composite point spread function; Radiometric compensation is performed on the degraded image based on the scattering compensation parameters to obtain the blurred image to be restored. A second-order central moment matrix is ​​constructed using the energy distribution of the data-fidelity degenerate kernel as weights; The principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degenerate kernel are determined based on the second-order central moment matrix. Construct directional difference operators along the principal axis and in directions orthogonal to the principal axis, respectively; Based on the principal axis scale, secondary axis scale, anisotropy ratio, and noise estimate, determine the direction regularization weights corresponding to the two directional difference operators respectively. Based on two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and the data fidelity degradation kernel, an image restoration objective function containing a data fidelity term and a directional prior total variation regularization term is constructed and solved to obtain the restored image.

[0007] Optionally, extracting the data fidelity degradation kernel for image restoration from the composite point spread function includes: The composite point spread function is sequentially nonnegated, energy normalized, and the central region is truncated to obtain the data fidelity degradation kernel used for image restoration.

[0008] Optionally, the step of performing radiometric compensation on the degraded image based on scattering compensation parameters to obtain the blurred image to be restored includes: Calculate the blurred image y to be restored using the following formula. c :

[0009] Where y represents the degraded image; A represents ambient light; t represents transmittance; and t0 is the lower limit threshold for transmittance.

[0010] Optionally, the construction of the second-order central moment matrix using the energy distribution of the data fidelity degradation kernel as weights includes: The centroid position of the data fidelity degradation kernel is determined by weighting the energy distribution of the data fidelity degradation kernel; The second-order central moments (second-order) in the transverse direction, the second-order central moments in the longitudinal direction, and the mixed second-order central moments are determined based on the location of the centroid to construct the second-order central moment matrix.

[0011] Optionally, determining the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degradation kernel based on the second-order central moment matrix includes: Eigenvalue decomposition is performed on the second-order central moment matrix to obtain the first and second eigenvalues. The direction of the eigenvector corresponding to the largest eigenvalue among the first and second eigenvalues ​​is taken as the principal axis direction of the data fidelity degradation kernel. The square root of the first eigenvalue and the square root of the second eigenvalue are used as the primary axis scale and secondary axis scale of the data fidelity degradation kernel, respectively. The ratio of the primary axis scale to the secondary axis scale is used as the anisotropy ratio of the data fidelity degradation kernel.

[0012] Optionally, the construction of directional difference operators along the principal axis and in directions orthogonal to the principal axis includes: Obtain the horizontal discrete difference operator and the vertical discrete difference operator; By rotating and combining the horizontal and vertical discrete difference operators along the principal axis, we obtain a first directional difference operator along the principal axis and a second directional difference operator along a direction orthogonal to the principal axis.

[0013] Optionally, the construction of directional difference operators along the principal axis and in directions orthogonal to the principal axis includes: A subpixel interpolation algorithm is used to calculate the forward difference of the image in the principal axis direction and in the direction orthogonal to the principal axis direction, respectively, as the first directional difference operator along the principal axis direction and the second directional difference operator along the direction orthogonal to the principal axis direction.

[0014] Optionally, determining the direction regularization weights corresponding to the two directional difference operators based on the principal axis scale, secondary axis scale, anisotropy ratio, and noise estimate includes: The baseline regularization weights are determined based on the noise estimate, the principal axis scale, and the secondary axis scale. The anisotropy ratio is truncated by upper and lower limits to obtain the truncated anisotropy ratio; The direction regularization weights corresponding to the first directional difference operator and the direction regularization weights corresponding to the second directional difference operator are determined based on the baseline regularization weights and the truncation anisotropy ratio.

[0015] Optionally, the step of constructing and solving an image restoration objective function containing a data fidelity term and a direction prior total variation regularization term based on two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and a data fidelity degradation kernel, to obtain the restored image, includes: Construct the expression for the image restoration objective function F':

[0016] Where x represents the image variable; h c Indicates a data fidelity degradation kernel; Represents two-dimensional convolution; y c This represents a blurred image to be restored. Represents the L2 norm; This represents the direction regularization weight corresponding to the first directional difference operator; This represents the horizontal discrete difference operator; This represents the direction regularization weight corresponding to the second direction difference operator; This represents the vertical discrete difference operator; Denotes the norm 1; Represents the pixel value constraints for the image variable x; The objective function for image restoration is solved by a variable splitting iterative algorithm. When the relative change of the image variables between two adjacent iterations is less than a preset threshold, or both the original residual and the dual residual satisfy the preset threshold, or the number of iterations reaches the upper limit, the iteration stops and the current image variable is used as the restored image.

[0017] Secondly, this application provides an optical image restoration system based on the direction prior of the composite point spread function, comprising: The acquisition module is used to acquire the degraded image and the composite point spread function, scattering compensation parameters, and noise estimate of the degraded image; The extraction module is used to extract the data fidelity degradation kernel for image restoration from the composite point spread function; The radiation compensation module is used to perform radiation compensation on the degraded image according to the scattering compensation parameters to obtain the blurred image to be restored. The first construction module is used to construct a second-order central moment matrix with the energy distribution of the data-fidelity degenerate kernel as the weight; The first determining module is used to determine the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degenerate kernel based on the second-order central moment matrix. The second construction module is used to construct directional difference operators along the principal axis direction and in directions orthogonal to the principal axis direction, respectively. The second determination module is used to determine the direction regularization weights corresponding to the two directional difference operators based on the principal axis scale, secondary axis scale, anisotropy ratio and noise estimate. The image restoration module is used to construct and solve the image restoration objective function, which includes a data fidelity term and a direction prior total variation regularization term, based on two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and the data fidelity degradation kernel, to obtain the restored image.

[0018] Beneficial effects: This application extracts a data-fidelity-preserving degenerate kernel from the composite point spread function and constructs a second-order central moment matrix using the energy distribution of this degenerate kernel as weights, thereby accurately calculating the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the degenerate kernel. Based on this, two directional difference operators are further constructed along the principal axis direction and its orthogonal direction, respectively, enabling the total variation regularization constraint to adaptively rotate with the principal axis direction of the composite point spread function. This ensures that the regularization penalty is always applied to the stretching direction where the degradation is most severe, significantly improving the suppression effect on arbitrary directional fuzziness.

[0019] This application generates directional regularization weights corresponding to the difference operators in two directions based on the calculated principal axis scale, secondary axis scale, anisotropy ratio, and noise estimates. When the anisotropy ratio is large, the regularization weight in the principal axis direction is automatically increased to strengthen the constraint on the stretched blur, while the regularization weight in the orthogonal direction is correspondingly decreased to reduce the penalty intensity for true edges and textures in that direction. When the degradation kernel is close to isotropy, the weights in the two directions tend to be equal. This directional adaptive weight allocation mechanism enables the restoration process to effectively suppress artifacts in the dominant direction while preserving image details in the orthogonal direction to the maximum extent.

[0020] This application utilizes the structural information contained in the composite point spread function, including principal axis scale, secondary axis scale, anisotropy ratio, and noise estimate, to automatically calculate the baseline regularization weights. Furthermore, it adjusts the regularization intensity in both directions by truncating the anisotropy ratio. This mechanism enables key parameters to automatically match the degradation kernel characteristics under the current observation conditions, eliminating the need for repeated manual adjustments and maintaining consistency and repeatability of the restoration results across different platforms and imaging conditions.

[0021] The image restoration objective function constructed in this application introduces two directional prior total variation regularization terms, one along the principal axis and the other in the orthogonal direction, based on the data fidelity term (constraining the restored image to be consistent with the blurred image to be restored after convolution with a data fidelity degradation kernel). This is solved through a variable splitting iterative method. While performing deconvolution on the global blur, it applies structured sparsity constraints to the solution space using directional prior information, effectively suppressing ringing effects and high-frequency noise amplification in the presence of noise, achieving a better balance between deblurring and noise resistance. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating an optical image restoration method based on the direction prior of the composite point diffusion function, provided for an embodiment of this application; Figure 2 A schematic diagram illustrating the extraction of principal axis direction and principal and secondary axis scales based on the second-order central moment of the composite point diffusion function, provided for embodiments of this application; Figure 3 This diagram illustrates a comparison of the restoration effects of isotropic total variation algorithm and anisotropic total variation algorithm provided in this application embodiment on stretched blur. Figure 4 This is a schematic diagram of the structure of an optical image restoration system based on the direction prior of the composite point spread function, provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0025] Example 1 like Figure 1 As shown, this embodiment provides an optical image restoration method based on the direction prior of the composite point spread function, including: Step 101: Obtain the degraded image and the composite point spread function, scattering compensation parameters, and noise estimate of the degraded image.

[0026] In this step, the composite point spread function is a two-dimensional degradation kernel corresponding to the current observation conditions of the (high-altitude high-speed optical) imaging system. Its sources include pre-calibration, online estimation, star or target estimation, blind deconvolution estimation, external simulation module output, or degradation kernel parameters provided by the imaging system. This embodiment does not limit the generation process of the composite point spread function, but rather converts the obtained composite point spread function into a directional prior constraint in the image restoration process. Therefore, it can be used in conjunction with different high-altitude high-speed degradation modeling methods.

[0027] When the composite point spread function originates from an external end-to-end degradation simulation module, this external module is responsible for generating or estimating the degradation kernel under the current operating conditions; this application is responsible for extracting directional priors from the degradation kernel and performing image restoration. These two aspects correspond to two different technical stages: degradation modeling and restoration solution.

[0028] Scattering compensation parameters include transmittance and background radiation intensity; noise estimates are given by the imaging system calibration results, or can be obtained by statistical analysis of flat regions in degraded images or by high-frequency residual estimation.

[0029] Step 102: Extract the data fidelity degradation kernel for image restoration from the composite point spread function.

[0030] In this step, the nonnegativity, energy normalization, and center region truncation operations performed on the composite point spread function yield a data fidelity degradation kernel for image restoration.

[0031] Specifically, the negative values ​​in the composite point diffusion function caused by estimation errors are removed, the sum of all elements in the data fidelity degradation kernel is normalized to 1, and the region centered on the energy centroid and containing the main energy distribution is selected as the data fidelity degradation kernel.

[0032] If the size of the diffusion function at the composite point is larger than the window required for actual restoration, then the central region containing the main energy is truncated before normalization.

[0033] Step 103: Perform radiation compensation on the degraded image according to the scattering compensation parameters to obtain the blurred image to be restored.

[0034] In this step, radiation compensation includes subtracting the background scattering component from the degraded image and avoiding numerical amplification in low-transmittance areas by setting a lower limit for transmittance.

[0035] For example, the blurred image y to be restored is calculated according to the following formula. c :

[0036] Where y represents the degraded image; A represents ambient light; t represents transmittance; and t0 is the lower limit threshold for transmittance.

[0037] Step 104: Construct a second-order central moment matrix using the energy distribution of the data-fidelity degenerate kernel as weights.

[0038] Specifically, the centroid position of the data fidelity degradation kernel is determined by weighting the energy distribution of the data fidelity degradation kernel.

[0039] For example, let the position of the centroid be... :

[0040] Where (u,v) are the pixel coordinates in the discrete sampling plane of the composite point spread function, u represents the horizontal pixel coordinates, and v represents the vertical pixel coordinates; h c (u,v) represents the data fidelity degradation kernel h. c The value at pixel coordinates (u,v).

[0041] The second-order central moments (second-order) in the transverse direction, the second-order central moments in the longitudinal direction, and the mixed second-order central moments are determined based on the location of the centroid to construct the second-order central moment matrix.

[0042] For example, the expression for the second-order central moment matrix is:

[0043] in, It is the second-order central moment in the transverse direction; It is the second-order central moment in the longitudinal direction; It is a mixed second-order central moment.

[0044] The second-order central moment matrix is ​​essentially a covariance matrix, used to describe the energy distribution of the composite point diffusion function in a two-dimensional plane.

[0045] Step 105: Determine the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degenerate kernel based on the second-order central moment matrix.

[0046] Specifically, the second-order central moment matrix is ​​decomposed into eigenvalues ​​to obtain the first and second eigenvalues.

[0047] Of the first and second eigenvalues, the larger eigenvalue characterizes the variance (i.e., the degree of dispersion) of the energy distribution of the degenerate nucleus along the principal axis. The larger the value, the more severe the stretching of the degenerate nucleus along the principal axis, and the stronger the ambiguity. The smaller eigenvalue characterizes the variance of the energy distribution of the degenerate nucleus in the direction orthogonal to the principal axis (the secondary axis). The smaller the value, the more concentrated the energy of the degenerate nucleus in the secondary axis, and the weaker the ambiguity.

[0048] The direction of the eigenvector corresponding to the largest eigenvalue among the first and second eigenvalues ​​is taken as the principal axis direction of the data fidelity degradation kernel.

[0049] For example, such as Figure 2 As shown, the principal axis direction θ of the data fidelity degradation kernel is calculated according to the following formula. h :

[0050] The square root of the first eigenvalue and the square root of the second eigenvalue are used as the primary axis scale and secondary axis scale of the data fidelity degradation kernel, respectively.

[0051] For example, the main axis dimension is calculated according to the following formula. :

[0052] Calculate the secondary axis dimension using the following formula. :

[0053] The ratio of the primary axis scale to the secondary axis scale is used as the anisotropy ratio of the data fidelity degradation kernel.

[0054] For example, the anisotropy ratio γ is calculated according to the following formula:

[0055] in, To avoid the stability constant of division by zero; when γ is less than or equal to 1, it indicates that the diffusion function at the recombination point is approximately isotropic; when γ is greater than 1, it indicates that the diffusion function at the recombination point has directional stretching.

[0056] Step 106: Construct directional difference operators along the principal axis and in directions orthogonal to the principal axis.

[0057] Specifically, obtain the horizontal discrete difference operator D. x and the vertical discrete difference operator D y .

[0058] By rotating and combining the horizontal and vertical discrete difference operators along the principal axis, we obtain the first directional difference operator D along the principal axis. || and the second directional difference operator D along the direction orthogonal to the principal axis direction ⊥ The first directional difference operator is used to characterize the grayscale change along the principal axis of the diffusion function of the composite point, and the second directional difference operator is used to characterize the grayscale change in a direction orthogonal to the principal axis.

[0059] For example, the following rotational combinations can be used:

[0060] Alternatively, a subpixel interpolation algorithm can be used to calculate the forward difference of the image in the principal axis direction and in the direction orthogonal to the principal axis direction, respectively, as the first directional difference operator along the principal axis direction and the second directional difference operator along the direction orthogonal to the principal axis direction.

[0061] For example, sub-pixel interpolation algorithms are used to calculate the values ​​along θ. h and Forward difference in direction: For any pixel position (i,j), along the principal axis direction θ h Forward differences are divided into:

[0062] Where x(i,j) is the gray value of image variable x at pixel position (i,j).

[0063] Along the orthogonal direction The forward difference can be written as:

[0064] when or When the pixel position is not an integer, the corresponding grayscale value can be obtained through bilinear interpolation.

[0065] Step 107: Determine the directional regularization weights corresponding to the two directional difference operators based on the principal axis scale, secondary axis scale, anisotropy ratio, and noise estimate.

[0066] Specifically, the baseline regularization weights are determined based on the noise estimate, the principal axis scale, and the secondary axis scale.

[0067] For example, the baseline regularization weight λ0 is calculated according to the following formula:

[0068] in, This is the proportionality coefficient; This is a noise estimate.

[0069] The anisotropy ratio is truncated by upper and lower limits to obtain the truncated anisotropy ratio.

[0070] For example, the cutoff anisotropy ratio is calculated according to the following formula. :

[0071] Where, γ max γ is the upper limit of the anisotropy ratio; min This is the lower limit of the anisotropy ratio.

[0072] The direction regularization weights corresponding to the first directional difference operator and the direction regularization weights corresponding to the second directional difference operator are determined based on the baseline regularization weights and the truncation anisotropy ratio.

[0073] In this step, the regularization weights of the first and second directions are adjusted according to the truncation anisotropy ratio to match the regularization constraint strengths of the two directions with the directionality of the current composite point diffusion function.

[0074] For example, the direction regularization weights corresponding to the first direction difference operator are calculated according to the following formula. (As the first direction regularization weight) and the direction regularization weight corresponding to the second direction difference operator. (As a second-direction regularization weight):

[0075] Where q is the directional regularization weight adjustment index; the first directional regularization weight and the second directional regularization weight satisfy the following: when the diffusion function of the composite point is close to isotropic, the first directional regularization weight and the second directional regularization weight tend to be equal; when the anisotropy ratio of the diffusion function of the composite point increases, the difference between the first directional regularization weight and the second directional regularization weight increases.

[0076] Step 108: Based on the two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and the data fidelity degradation kernel, construct and solve the image restoration objective function containing the data fidelity term and the directional prior total variation regularization term to obtain the restored image.

[0077] In this step, for example, the expression for the image restoration objective function F' is constructed:

[0078] Where x represents the image variable; h c Indicates a data fidelity degradation kernel; Represents two-dimensional convolution; y c This represents a blurred image to be restored. Represents the L2 norm; This represents the direction regularization weight corresponding to the first directional difference operator; This represents the horizontal discrete difference operator; This represents the direction regularization weight corresponding to the second direction difference operator; This represents the vertical discrete difference operator; Denotes the norm 1; This represents the pixel value constraints for the image variable x.

[0079] In the image restoration objective function, the first term is the data fidelity term, and the second and third terms are the total variation regularization terms in the principal axis direction and orthogonal direction, respectively; Ω represents the preset grayscale or radiance intensity range. The fourth term is the pixel value constraint term. If explicit pixel constraints are not required, this term can be omitted or the value range can be projected after each iteration.

[0080] The data fidelity term is used to constrain the restored image to be consistent with the blurred image to be restored after convolution with the data fidelity degradation kernel; the first direction total variation regularization term and the second direction total variation regularization term are used to constrain the gradient sparsity of the restored image in the principal axis direction and the orthogonal direction, respectively; the pixel value constraint term is used to limit the restored image to a preset grayscale or radiation intensity range.

[0081] in, It can be defined as:

[0082] In the normalized image, Ω = [0,1]; in the 8-bit image, Ω = [0,255].

[0083] The objective function for image restoration is solved by a variable splitting iterative algorithm. When the relative change of the image variables between two adjacent iterations is less than a preset threshold, or both the original residual and the dual residual satisfy the preset threshold, or the number of iterations reaches the upper limit, the iteration stops and the current image variable is used as the restored image.

[0084] The variable splitting iteration method includes introducing a first auxiliary variable z corresponding to the difference result in the first direction. || =D || x and the first dual variable u || A second auxiliary variable z corresponding to the second direction difference result is introduced. ⊥ =D ⊥ x and the second dual variable u ⊥ In each iteration, the image variables are updated by using soft threshold operators corresponding to the first and second direction regularization weights to update the first and second auxiliary variables respectively, and the first and second dual variables are also updated. The iteration is stopped based on the relative change of the image variables between two adjacent iterations, the original residual, the dual residual, or the maximum number of iterations.

[0085] For example, the solution is obtained using the alternating direction multiplier method. Here, H is denoted as H, H', H'', H'''''''''''''''''''''''''''''''''""""""' ... and These represent the frequency domain representations of the data fidelity degradation kernel, the first-direction difference operator, and the second-direction difference operator, respectively, where ρ is the penalty parameter, and F and F0 are the values. -1 Let represent the Fourier transform and the inverse Fourier transform, respectively. Then, the image variables can be updated in the frequency domain:

[0086] Among them, X (k+1) This represents the frequency domain representation of the image variable x at the (k+1)th iteration; superscript Y represents complex conjugation; c For the frequency domain representation of the observed image (the blurred image to be restored) or the image after scattering compensation; When it is the kth iteration Frequency domain representation; This represents the first auxiliary variable during the k-th iteration; When it is the kth iteration Frequency domain representation; Let represent the first dual variable at the k-th iteration; During the kth iteration Frequency domain representation; This represents the second auxiliary variable during the k-th iteration; During the kth iteration Frequency domain representation; Let x represent the second dual variable in the k-th iteration; (k+1) This represents the spatial domain restored image obtained in the (k+1)th iteration; k = 1, 2, ..., K max ;K max This represents the maximum number of iterations.

[0087] If the pixel value range is set to Ω, then for x (k+1) Perform projection:

[0088] Ω as When the subscript is used, it indicates the target set corresponding to the projection operation, which is not a parameter for ordinary multiplication or division operations. It participates in the calculation by limiting the projection range, where...

[0089] in, This represents the projection operator over the range Ω of pixel values; Ω min Indicates the lower limit of pixel values; Ω max This indicates the upper limit of pixel values.

[0090] If Ω = [0, 1], then:

[0091] Auxiliary variables can be updated using the soft threshold operator:

[0092] in, This represents the first auxiliary variable in the (k+1)th iteration; Indicates the threshold is Soft threshold operator; This represents the second auxiliary variable during the (k+1)th iteration; Indicates the threshold is The soft threshold operator.

[0093] The dual variable is updated using the residual accumulation method:

[0094] in, Let represent the first dual variable at the (k+1)th iteration; Let represent the second dual variable at the (k+1)th iteration.

[0095] The above-described variable splitting iteration method is merely one specific solution implementation in this application. The focus of this application is not on applying a specific optimizer to an image, but rather on generating directional difference operators and directional regularization weights from the directional features of the composite point spread function, and introducing them into the restoration objective function.

[0096] When the degraded image is a color image, a multispectral image, or a multichannel image containing a panchromatic channel, the image can first be converted into a luminance-chrominance representation, and the aforementioned directional a priori total variation restoration can be performed on the luminance channel or panchromatic channel. For the chrominance channel or other channels not directly involved in the restoration, color or radiometric consistency reconstruction is performed based on the restored luminance channel or panchromatic channel. The chrominance channel can be reconstructed for color consistency based on the restored luminance channel. Alternatively, restoration can be performed separately for each channel, and inter-channel radiometric consistency constraints can be applied after restoration. The above processing method enables this application to adapt to different types of high-altitude high-speed optical imaging data.

[0097] Conventional isotropic total variation restoration typically employs fixed horizontal and vertical differences, applying the same or manually set regularization weights to both directions. However, when the principal axes of the composite point spread function do not align with the image coordinate axes, fixed horizontal and vertical constraints are insufficient to accurately reflect the true directionality of the degradation kernel.

[0098] like Figure 3 As shown, Figure 3 This is a comparison of the restoration effects of isotropic total variation algorithm and anisotropic total variation algorithm on stretched blur. Figure 3 In the image, (a) is the stretched and blurred image, (b) is the image restored using the isotropic total variation method with a peak signal-to-noise ratio of 24.30 dB, and (c) is the image restored using the method of this application with a peak signal-to-noise ratio of 25.61 dB.

[0099] This application first calculates the principal axis direction from the composite point spread function, and then constructs a directional difference operator around this principal axis direction, causing the regularization constraint to rotate with the degeneracy kernel direction. Simultaneously, it adjusts the regularization weights in both directions according to the anisotropy ratio, enabling the restoration model to automatically adapt to stretching blur of different directions and intensities. Therefore, this application does not simply change the type of image to be processed, but rather alters the basis for generating the regularization constraint and its directional structure.

[0100] When the composite point spread function changes with the field of view, the degraded image can be divided into multiple overlapping image blocks, and a corresponding local composite point spread function can be matched for each image block. Second-order central moment analysis, directional difference operator construction, directional weight generation, and directional prior total variation restoration are performed on each image block. Then, adjacent image blocks are weighted and fused using a Hanning window, cosine window, or distance weight to obtain the entire restored image. This application can handle spatially varying degradation under the condition of local isohalos or local approximate spatial invariance.

[0101] In summary, the optical image restoration method based on the direction prior of the composite point spread function provided in this application extracts the data fidelity degradation kernel from the composite point spread function and constructs a second-order central moment matrix with its energy distribution as weight. This determines the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the degradation kernel, enabling the subsequent restoration process to accurately capture and utilize the inherent directional structure of the degradation kernel. This overcomes the shortcomings of traditional methods where fixed horizontal / vertical difference operators cannot adapt to stretching blur in arbitrary directions.

[0102] Building upon this, this application further constructs directional difference operators along the principal axis and its orthogonal direction. This feature enables the total variation regularization constraint to adaptively rotate with the principal axis direction of the degradation kernel, ensuring that the regularization penalty is always applied to the stretching direction where image degradation is most severe. This significantly improves the ability to suppress arbitrary directional blur caused by aero-optical effects, platform vibrations, and other factors, effectively solving the problem that conventional deconvolution methods are prone to ringing effects or high-frequency noise amplification under oblique blur.

[0103] Furthermore, this application determines the directional regularization weights corresponding to the two directional difference operators based on the principal axis scale, secondary axis scale, anisotropy ratio, and noise estimate, achieving differentiated and automated allocation of regularization weights: when the anisotropy ratio is large, the regularization weight in the principal axis direction is automatically increased to strengthen the constraint on stretched blur, while the regularization weight in the orthogonal direction is correspondingly decreased to reduce the penalty intensity for true edges and textures in that direction; when the degradation kernel is close to isotropy, the weights in the two directions tend to be equal. The resulting technical effect is that it not only effectively suppresses artifacts in the dominant direction but also preserves the original image details and edge information in the orthogonal direction to the maximum extent, overcoming the shortcomings of isotropic total variation models that cannot distinguish the degradation differences between the principal axis and orthogonal directions, and the information loss caused by excessive smoothing.

[0104] Finally, this application constructs and solves an image restoration objective function that includes a data fidelity term and a direction prior total variation regularization term. The data fidelity term constrains the consistency between the restored image and the blurred image to be restored, while the two total variation regularization terms along the principal axis and orthogonal directions impose structured gradient sparsity constraints. Combined with the baseline regularization weights automatically calculated from the principal axis scale, secondary axis scale, anisotropy ratio, and noise estimates, key parameters can be automatically matched to the degradation kernel features under the current observation conditions, eliminating the need for repeated manual adjustments. This achieves a better balance between deblurring and noise resistance, maintaining consistency and repeatability of restoration results across different platforms and imaging conditions, and significantly improving the automation and robustness of optical image restoration on high-altitude, high-speed platforms.

[0105] Example 2 Based on the same inventive concept as Embodiment 1, this embodiment provides an optical image restoration system based on the direction prior of the composite point spread function. Since the principle of this system in solving the problem is similar to that of the optical image restoration method based on the direction prior of the composite point spread function described in Embodiment 1, the implementation of this system can refer to the implementation of the optical image restoration method based on the direction prior of the composite point spread function.

[0106] like Figure 4 As shown, this embodiment provides an optical image restoration system based on the direction prior of the composite point spread function, including: The acquisition module 10 is used to acquire the degraded image and the composite point spread function, scattering compensation parameters and noise estimate of the degraded image.

[0107] Extraction module 20 is used to extract the data fidelity degradation kernel for image restoration from the composite point spread function.

[0108] The radiation compensation module 30 is used to perform radiation compensation on the degraded image according to the scattering compensation parameters to obtain the blurred image to be restored.

[0109] The first construction module 40 is used to construct a second-order central moment matrix with the energy distribution of the data-fidelity degenerate kernel as weights.

[0110] The first determining module 50 is used to determine the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degenerate kernel based on the second-order central moment matrix.

[0111] The second construction module 60 is used to construct directional difference operators along the principal axis direction and in directions orthogonal to the principal axis direction.

[0112] The second determining module 70 is used to determine the direction regularization weights corresponding to the two directional difference operators based on the principal axis scale, secondary axis scale, anisotropy ratio and noise estimate.

[0113] The image restoration module 80 is used to construct and solve the image restoration objective function containing the data fidelity term and the direction prior total variation regularization term based on two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and the data fidelity degradation kernel, so as to obtain the restored image.

[0114] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0115] Example 3 This embodiment provides a computer device, including a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the steps of the optical image restoration method based on the direction prior of the composite point spread function described in Embodiment 1.

[0116] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0117] Example 4 This embodiment provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the optical image restoration method based on the direction prior of the composite point spread function described in Embodiment 1.

[0118] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0119] Example 5 This embodiment provides a computer program product, including computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, they implement the steps of the optical image restoration method based on the direction prior of the composite point spread function described in Embodiment 1.

[0120] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0121] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems, devices, storage media, and computer program products disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0122] Those skilled in the art will clearly understand that the techniques in the embodiments of this application can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application or some parts of the embodiments.

[0123] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0124] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0125] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0126] The present application has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present application. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and implementation methods of the present application without departing from the spirit and scope of the present application, and all such modifications and improvements fall within the scope of the present application. The scope of protection of the present application is determined by the appended claims.

Claims

1. An optical image restoration method based on the direction prior of the composite point spread function, characterized in that, include: Obtain the point spread function, scattering compensation parameters, and noise estimate of the degraded image and its composite image; Extract the data fidelity degradation kernel for image restoration from the composite point spread function; Radiometric compensation is performed on the degraded image based on the scattering compensation parameters to obtain the blurred image to be restored. A second-order central moment matrix is ​​constructed using the energy distribution of the data-fidelity degenerate kernel as weights; The principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degenerate kernel are determined based on the second-order central moment matrix. Construct directional difference operators along the principal axis and in directions orthogonal to the principal axis, respectively; Based on the principal axis scale, secondary axis scale, anisotropy ratio, and noise estimate, determine the direction regularization weights corresponding to the two directional difference operators respectively. Based on two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and the data fidelity degradation kernel, an image restoration objective function containing a data fidelity term and a directional prior total variation regularization term is constructed and solved to obtain the restored image.

2. The optical image restoration method according to claim 1, characterized in that, The extraction of the data fidelity degradation kernel for image restoration from the composite point spread function includes: The composite point spread function is sequentially nonnegated, energy normalized, and the central region is truncated to obtain the data fidelity degradation kernel used for image restoration.

3. The optical image restoration method according to claim 1, characterized in that, The step of performing radiometric compensation on the degraded image based on scattering compensation parameters to obtain the blurred image to be restored includes: Calculate the blurred image y to be restored using the following formula. c : Where y represents the degraded image; A represents ambient light; t represents transmittance; and t0 is the lower limit threshold for transmittance.

4. The optical image restoration method according to claim 1, characterized in that, The construction of the second-order central moment matrix using the energy distribution of the data fidelity degradation kernel as weights includes: The centroid position of the data fidelity degradation kernel is determined by weighting the energy distribution of the data fidelity degradation kernel; The second-order central moments (second-order) in the transverse direction, the second-order central moments in the longitudinal direction, and the mixed second-order central moments are determined based on the location of the centroid to construct the second-order central moment matrix.

5. The optical image restoration method according to claim 1, characterized in that, The determination of the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degradation kernel based on the second-order central moment matrix includes: Eigenvalue decomposition is performed on the second-order central moment matrix to obtain the first and second eigenvalues. The direction of the eigenvector corresponding to the largest eigenvalue among the first and second eigenvalues ​​is taken as the principal axis direction of the data fidelity degradation kernel. The square root of the first eigenvalue and the square root of the second eigenvalue are used as the primary axis scale and secondary axis scale of the data fidelity degradation kernel, respectively. The ratio of the primary axis scale to the secondary axis scale is used as the anisotropy ratio of the data fidelity degradation kernel.

6. The optical image restoration method according to claim 1, characterized in that, The construction of directional difference operators along the principal axis and in directions orthogonal to the principal axis includes: Obtain the horizontal discrete difference operator and the vertical discrete difference operator; By rotating and combining the horizontal and vertical discrete difference operators along the principal axis, we obtain a first directional difference operator along the principal axis and a second directional difference operator along a direction orthogonal to the principal axis.

7. The optical image restoration method according to claim 1, characterized in that, The construction of directional difference operators along the principal axis and in directions orthogonal to the principal axis includes: A subpixel interpolation algorithm is used to calculate the forward difference of the image in the principal axis direction and in the direction orthogonal to the principal axis direction, respectively, as the first directional difference operator along the principal axis direction and the second directional difference operator along the direction orthogonal to the principal axis direction.

8. The optical image restoration method according to claim 6 or 7, characterized in that, The step of determining the direction regularization weights corresponding to the two directional difference operators based on the principal axis scale, secondary axis scale, anisotropy ratio, and noise estimate includes: The baseline regularization weights are determined based on the noise estimate, the principal axis scale, and the secondary axis scale. The anisotropy ratio is truncated by upper and lower limits to obtain the truncated anisotropy ratio; The direction regularization weights corresponding to the first directional difference operator and the direction regularization weights corresponding to the second directional difference operator are determined based on the baseline regularization weights and the truncation anisotropy ratio.

9. The optical image restoration method according to claim 8, characterized in that, The process involves constructing and solving an image restoration objective function, which includes a data fidelity term and a direction prior total variation regularization term, based on two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and a data fidelity degradation kernel, to obtain the restored image. This includes: Construct the expression for the image restoration objective function F': Where x represents the image variable; h c Indicates a data fidelity degradation kernel; Represents two-dimensional convolution; y c This represents a blurred image to be restored. Represents the L2 norm; This represents the direction regularization weight corresponding to the first directional difference operator; This represents the horizontal discrete difference operator; This represents the direction regularization weight corresponding to the second direction difference operator; This represents the vertical discrete difference operator; Denotes the norm 1; Represents the pixel value constraints for the image variable x; The objective function for image restoration is solved by a variable splitting iterative algorithm. When the relative change of the image variables between two adjacent iterations is less than a preset threshold, or both the original residual and the dual residual satisfy the preset threshold, or the number of iterations reaches the upper limit, the iteration stops and the current image variable is used as the restored image.

10. An optical image restoration system based on the direction prior of the composite point spread function, characterized in that, include: The acquisition module is used to acquire the degraded image and the composite point spread function, scattering compensation parameters, and noise estimate of the degraded image; The extraction module is used to extract the data fidelity degradation kernel for image restoration from the composite point spread function; The radiation compensation module is used to perform radiation compensation on the degraded image according to the scattering compensation parameters to obtain the blurred image to be restored. The first construction module is used to construct a second-order central moment matrix with the energy distribution of the data-fidelity degenerate kernel as the weight; The first determining module is used to determine the principal axis direction, principal axis scale, secondary axis scale, and anisotropy ratio of the data fidelity degenerate kernel based on the second-order central moment matrix. The second construction module is used to construct directional difference operators along the principal axis direction and in directions orthogonal to the principal axis direction, respectively. The second determination module is used to determine the direction regularization weights corresponding to the two directional difference operators based on the principal axis scale, secondary axis scale, anisotropy ratio and noise estimate. The image restoration module is used to construct and solve the image restoration objective function, which includes a data fidelity term and a direction prior total variation regularization term, based on two directional difference operators and their corresponding directional regularization weights, the blurred image to be restored, and the data fidelity degradation kernel, to obtain the restored image.

Images