Sparse representation correction method for aerodynamic thermal radiation effect images

Abstract

The present invention discloses a sparse representation correction method for aerodynamic thermal radiation effect images, belonging to the field of digital image processing. The method comprises: constructing an energy functional optimization model with the goal of minimizing the sum of a data fidelity term, an image prior regularization term, and a bias field prior regularization term; the data fidelity term is used to measure the difference between the degraded image and the original clear image and the bias field; the image prior regularization term is used to measure the gradient of the original clear image; the bias field B is fitted using a dictionary as B = Wa, where W and a are the dictionary and coefficient vector, respectively, and the bias field prior regularization term is used to measure the L1 norm of the coefficient vector a; solving the energy functional optimization model under the constraint that the number of nonzero elements in the coefficient vector a does not exceed a preset threshold to obtain the coefficient vector a; reconstructing the bias field B according to B = Wa, and subtracting the bias field B from the degraded image to obtain the original clear image. The present invention can improve the correction accuracy and image quality of aerodynamic thermal radiation effect images.

Description

Technical Field

[0001] The present invention belongs to the field of digital image processing, and more particularly, relates to a sparse representation correction method for aerodynamic thermal radiation effect images. Background Art

[0002] When an aircraft equipped with an imaging system flies within the atmosphere, friction between the optical hood and the high-speed incoming airflow causes aerodynamic thermal radiation effects similar to "halos" on the image plane, degrading the signal-to-noise ratio of the target image captured by the imaging system. According to Planck's blackbody radiation law, aerodynamic thermal radiation effects exist in both infrared and visible light bands. Therefore, correcting for these effects is crucial to improving image quality.

[0003] The aerodynamic thermal radiation degradation mechanism model can be expressed as:

[0004] Z=X+B

[0005] Where Z is the known degraded image, X is the unknown original clear image, and B is the unknown thermal radiation bias field superimposed on the clear image.

[0006] Based on the aforementioned aerothermal radiation degradation mechanism model, it can be seen that after reconstructing the bias field B, the degraded image Z can be corrected to obtain the original clear image X. Therefore, existing correction technologies focus on statistical modeling of the bias field degradation characteristics of the aerothermal radiation effect. Currently, research results have proposed fitting bias field surfaces using methods such as spline surfaces (CN114529481A), Chebyshev polynomials (CN114359093A), and pixel coordinate polynomials (CN105118037B).

[0007] The aforementioned thermal radiation bias field correction methods all suffer from the following issues: 1) The surface morphology of the thermal radiation bias field is typically similar to the shape of the aircraft's hood, exhibiting smooth overall variations with minimal localized mutations. However, existing correction methods are susceptible to interference from the imaging scene, and the estimated bias field often contains background information with localized mutations, resulting in loss of corrected image information. 2) Existing fitting methods only impose smoothing constraints on image gradients, with no constraints on image grayscale. This can easily lead to uneven grayscale between corrected frames in an image sequence, causing screen flicker and resulting in poor corrected image quality. Summary of the Invention

[0008] In response to the defects and improvement needs of the existing technology, the present invention provides a sparse representation correction method for aerodynamic thermal radiation effect images, which aims to improve the correction accuracy of aerodynamic thermal radiation effect images and improve the quality of the corrected images.

[0009] To achieve the above objectives, according to one aspect of the present invention, a sparse representation correction method for an aerodynamic thermal radiation effect image is provided, which is used to correct a degraded image captured by an aircraft equipped with an imaging system into an original clear image. The method comprises:

[0010] An energy functional optimization model is constructed with the goal of minimizing the sum of the data fidelity term, the image prior regularization term, and the bias field prior regularization term. The data fidelity term is used to measure the difference between the degraded image and the corrected original clear image and the bias field. The image prior regularization term is used to measure the gradient of the original clear image. The bias field is fitted using a dictionary as B=Wa, where B is the bias field, W and a are the dictionary and the corresponding coefficient vector, respectively. The bias field prior regularization term is used to measure the L1 norm of the coefficient vector a.

[0011] After applying a sparse constraint to the coefficient vector a, the energy functional optimization model is solved to obtain the coefficient vector a; the sparse constraint is used to limit the number of non-zero elements in the coefficient vector a to not exceed a preset threshold;

[0012] Reconstruct the bias field B according to B=Wa;

[0013] The reconstructed bias field B is subtracted from the degraded image to obtain the corrected original clear image.

[0014] Furthermore, the expression of the sparse constraint is: ||a||1<τ;

[0015] Here, || ||1 represents the L1 norm, and τ is a preset positive constant.

[0016] Furthermore, the expression of the data fidelity term is:

[0017] Where Z represents the degraded image, X represents the original clear image, and |‖‖|2 represents the two-norm.

[0018] Furthermore, the expression of the image prior regularization term is:

[0019] in, represents the gradient of the original clear image, and λ is the regularization hyperparameter.

[0020] Furthermore, a semi-quadratic variable separation algorithm is used to solve the energy functional optimization model.

[0021] Furthermore, the bias field is fitted using a polynomial dictionary.

[0022] According to another aspect of the present invention, a computer program product is provided, comprising a computer program; when the computer program is executed by a processor, the sparse representation correction method for aerodynamic thermal radiation effect images provided by the present invention is implemented.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, comprising a stored computer program; when the computer program is executed by a processor, the computer-readable storage medium is controlled to execute the sparse representation correction method for aerodynamic thermal radiation effect images provided by the present invention.

[0024] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0025] a computer-readable storage medium for storing a computer program;

[0026] and a processor for reading a computer program stored in a computer-readable storage medium and executing the sparse representation correction method for aerodynamic thermal radiation effect images provided by the present invention.

[0027] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0028] (1) The sparse representation correction method for aerodynamic thermal radiation effect images provided by the present invention imposes a sparse constraint on the coefficient vector used to fit the bias field when solving it, and limits the number of non-zero elements in the sparse vector, thereby avoiding the noise points in the coefficient vector from causing overfitting of the bias field surface, thereby effectively avoiding the inclusion of background information in the reconstructed bias field, improving the reconstruction accuracy of the bias field, and ultimately effectively separating the thermal radiation bias field from the degraded image, achieving the purpose of high-precision protection of the image background and avoiding image information loss; the energy functional optimization model established by the present invention, while constraining the image gradient through the image prior regularization term, constrains the image grayscale through the data fidelity term, avoiding obvious jumps in the bias field between adjacent frames, and ensuring that the grayscale of the reconstructed image sequence is stable and free of screen flicker, thereby effectively improving the image quality. In general, the present invention constructs an energy functional optimization model with the goal of minimizing the weighted sum of data fidelity terms, image prior regularization terms, and bias field prior regularization terms, and solves the model while imposing sparse constraints on the coefficient vector to achieve bias field reconstruction and image correction. This can improve the correction accuracy of aerodynamic thermal radiation effect images and improve the quality of the reconstructed images.

[0029] (2) Directly constraining the number of non-zero elements in the coefficient vector will make the model solution more difficult. As a preferred embodiment, the present invention relaxes the sparse constraint imposed on the coefficient vector and converts it into a constraint on the L1 norm of the coefficient vector, thereby indirectly achieving the purpose of limiting the number of non-zero elements in the coefficient vector. While ensuring the effectiveness of the constraint, it effectively reduces the difficulty of solving the model.

[0030] (3) Half-Quadratic Splitting (HQS) is a method for solving optimization problems, especially when the objective function contains non-convex or difficult-to-handle terms. By introducing auxiliary variables and alternating optimization, HQS decomposes the original problem into a series of simpler quadratic optimization problems, thereby simplifying the calculation and approximating the solution to the original problem. The energy functional optimization model established in this invention can be expressed as In the preferred embodiment of the present invention, a semi-quadratic variable separation algorithm is used to solve the model, which can effectively improve the solution efficiency while ensuring the solution accuracy.

[0031] (4) The bias field of a degraded image usually appears as a globally smoothly varying curved surface field. In the preferred embodiment of the present invention, a polynomial dictionary is used to fit the bias field, which can accurately characterize the spatial variation of the bias field, further improve the reconstruction accuracy of the bias field, and thus improve the image correction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of a sparse representation correction method for aerodynamic thermal radiation effect images provided by an embodiment of the present invention;

[0033] Figure 2 A block diagram of a sparse representation correction method for aerodynamic thermal radiation effect images provided by an embodiment of the present invention;

[0034] Figure 3 Schematic diagram of image correction effect provided by an embodiment of the present invention; wherein (a) is a schematic diagram of infrared image correction effect, and (b) is a schematic diagram of near-infrared image correction effect;

[0035] Figure 4 Schematic diagram of the background preservation effect of the corrected image provided by an embodiment of the present invention; wherein (a) is a true clear image, (b) is a degraded image, (c) is the original clear image corrected using the correction method in CN105118037B, and (d) is the original clear image corrected using the correction method provided by an embodiment of the present invention;

[0036] Figure 5 Schematic diagram comparing the screen flicker suppression effects of the corrected image sequences provided in an embodiment of the present invention; wherein, (a) is the grayscale mean curve of the image sequence corrected using the correction method in CN105118037B, and (b) is the grayscale mean curve of the image sequence corrected using the correction method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0038] In the present invention, the terms "first", "second", etc. (if any) in the present invention and the drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0039] In order to solve the technical problems that the thermal radiation bias field reconstructed in the prior art contains background information, resulting in the loss of corrected image information and uneven grayscale on the corrected image sequence, the present invention provides a sparse representation correction method for aerodynamic thermal radiation effect images. The overall concept is to conduct an in-depth analysis of the generation mechanism of related problems. Based on the analysis results, sparse constraints are applied to the coefficient vectors used to fit the bias field accordingly to avoid the inclusion of background information in the bias field due to overfitting of the surface caused by noise points in the coefficient vectors. At the same time, items for imposing constraints on the image grayscale are set in the established optimization model to ensure that the grayscale of the finally reconstructed image sequence is stable and there is no screen flicker.

[0040] Before explaining the technical solution of the present invention in detail, starting from the above technical concept, the setting of relevant constraints and the establishment of the overall model in the present invention are analyzed as follows:

[0041] The aerodynamic thermal radiation degradation mechanism model can be expressed as:

[0042] Z=X+B

[0043] Where Z is the known degraded image, X is the unknown original clear image, and B is the unknown thermal radiation bias field superimposed on the clear image.

[0044] Based on the above degradation model, the optimization framework can be established based on the maximum a posteriori probability method as follows:

[0045]

[0046] Among them, Ψ(Z,X,B) is the data fidelity term, Pr(X) is the image prior regularization term, Pr(B) is the bias field prior regularization term, and λ and β are the corresponding regularization hyperparameters.

[0047] Using any dictionary such as spline surface, Chebyshev polynomial, pixel coordinate polynomial, etc. to fit the surface of the bias field, the bias field can be expressed as:

[0048] B=Wa

[0049] Among them, B is the bias field, W and a are the dictionary and the corresponding coefficient vector respectively.

[0050] The present invention finds that the bias field contains background information, mainly because the noise points in the coefficient vector a will cause the surface of the bias field to overfit. Based on this discovery, the present invention proposes to impose a sparse constraint on the sparse vector a of the bias field to limit the number of non-zero elements in the coefficient vector a. Specifically, the number of non-zero elements in the coefficient vector a is limited to not exceed a preset threshold. This sparse constraint will be used as a constraint condition in the subsequent model solution. Considering that directly limiting the number of non-zero elements in the coefficient vector when solving the model will make the model solution more difficult, to address this problem, the present invention further proposes to relax the sparse constraint imposed on the coefficient vector and convert it into a constraint on the L1 norm of the coefficient vector, thereby indirectly achieving the purpose of limiting the number of non-zero elements in the coefficient vector, while ensuring the validity of the constraint, effectively reducing the difficulty of model solution. The sparse constraint after conversion can be expressed as:

[0051] ||a||1<τ

[0052] Here, || ||1 represents the L1 norm, and τ is a preset positive constant.

[0053] In practical applications, any dictionary can be used to fit the surface of the bias field. Considering that the bias field of a degraded image usually presents a globally smoothly varying surface field, fitting the bias field with a polynomial dictionary can accurately characterize the spatial variation of the bias field, which helps to improve the reconstruction accuracy of the bias field. Without loss of generality, in the following embodiments, a polynomial dictionary is used to fit the bias field. After applying a sparse constraint to the coefficient vector, the bias field can be expressed as:

[0054]

[0055] Among them, x and y are the horizontal and vertical coordinates of the image respectively, D is the highest order of the preset polynomial, and a t,s are the polynomial coefficients.

[0056] In order to solve the problem of screen flicker caused by uneven grayscale of the corrected image sequence, the present invention constructs an image gradient prior for constraining the gradient and a data item for constraining the image grayscale when establishing the corresponding optimization model.

[0057] The data item expression is as follows:

[0058]

[0059] The image gradient prior expression is as follows:

[0060]

[0061] The correction framework is as follows:

[0062]

[0063] Refer to the above optimization framework By using the sparsity constraint imposed on the coefficient vector a, the above correction framework can be further transformed into the following solvable energy functional optimization model:

[0064]

[0065] In the formula and β||a||1 correspond to the data fidelity term, image prior regularization term, and bias field prior regularization term in the optimization framework, respectively.

[0066] By solving the above energy functional optimization model, the coefficient vector a can be solved. Then, the bias field B can be reconstructed based on the fitting expression B = Wa. After subtracting the bias field B from the degraded image, correction can be achieved to obtain the original clear image X.

[0067] Based on the above analysis, the present invention constructs an energy functional optimization model with the goal of minimizing the weighted sum of the data fidelity term, the image prior regularization term, and the bias field prior regularization term, and solves the model while imposing a sparse constraint on the coefficient vector to achieve bias field reconstruction and image correction. This can improve the correction accuracy of the aerodynamic thermal radiation effect image and improve the quality of the reconstructed image.

[0068] The following are examples.

[0069] Example 1:

[0070] A sparse representation correction method for aerodynamic thermal radiation effect images is used to correct the degraded images taken by an aircraft equipped with an imaging system into original clear images; Figure 1 and Figure 2 As shown, this embodiment includes:

[0071] Step S1: Construct an energy functional optimization model with the goal of minimizing the sum of a data fidelity term, an image prior regularization term, and a bias field prior regularization term; the data fidelity term is used to measure the difference between the degraded image and the corrected original clear image and the bias field; the image prior regularization term is used to measure the gradient of the original clear image; the bias field is fitted using a dictionary as B=Wa, where B is the bias field, W and a are the dictionary and the corresponding coefficient vector, respectively, and the bias field prior regularization term is used to measure the L1 norm of the coefficient vector a;

[0072] Preferably, in this embodiment, the expression of the energy functional optimization model is:

[0073]

[0074] Step S2: Solving the energy functional optimization model after applying a sparse constraint to the coefficient vector a to obtain the coefficient vector a; the sparse constraint is used to limit the number of non-zero elements in the coefficient vector a to not exceed a preset threshold;

[0075] Preferably, in this embodiment, the expression of the sparse constraint is:

[0076]

[0077] Optionally, in this embodiment, a semi-quadratic variable separation algorithm is used to solve the energy functional optimization model; the specific process is as follows:

[0078] The energy functional optimization model is separated by variables, resulting in the following two sub-problems:

[0079]

[0080] (1) For the X subproblem, the closed-form solution of the original clear image X is:

[0081]

[0082] Where t is the iteration index and I is the identity matrix;

[0083] (2) For subproblem a, introduce auxiliary variable u:

[0084]

[0085] Using the semi-quadratic variable separation algorithm to solve the problem, we get the following sub-problems:

[0086]

[0087] in, The subproblem is a least squares problem, and the closed-form solution is obtained by differentiation:

[0088]

[0089] Use the soft threshold shrinkage operator to solve the u-subproblem:

[0090]

[0091] Where k is the index of the inner loop iteration number for solving subproblem a, 1 is a matrix with all element values ​​1, and 0 is a matrix with all element values ​​0; the regularization parameter η of the auxiliary variable is updated with the iteration:

[0092] η k+1 =2·η k

[0093] After multiple iterations of inner and outer loops, the outer loop number t reaches the maximum outer loop number (10 in this embodiment), and the inner loop k reaches the maximum inner loop number (5 in this embodiment), and the final estimated coefficient vector a is obtained;

[0094] Step S3: Reconstruct the bias field B according to B=Wa;

[0095] Step S4: Subtract the reconstructed bias field B from the degraded image to obtain the corrected original clear image.

[0096] In general, the sparse constraints imposed on the polynomial dictionary coefficient vector in this embodiment can accurately separate the thermal radiation bias field from the degraded image, thereby achieving the purpose of high-precision protection of the image background; the constraints imposed on the image grayscale can ensure that the grayscale of the corrected image sequence is stable and free of screen flicker.

[0097] The following further analyzes and illustrates the beneficial effects that can be achieved by this embodiment in combination with actual calibration examples.

[0098] Figure 3 Figure 2 is a schematic diagram showing the correction effect of aerodynamic thermal radiation effect image correction based on the sparse representation correction method for aerodynamic thermal radiation effect image provided by this embodiment, wherein (a) is a schematic diagram showing the correction effect of an infrared image, and (b) is a schematic diagram showing the correction result of a near-infrared image. Figure 3 In (a) and (b), the first row from left to right is the degraded image, the corrected original clear image and the thermal radiation bias field, and the second row is the corresponding three-dimensional grayscale coordinate diagram. Figure 3 It can be seen from the results that this embodiment can effectively achieve correction for different images.

[0099] The correction method disclosed in CN105118037B and this embodiment both use a polynomial dictionary to fit the bias field, but this method does not impose a sparse constraint on the coefficient vector. The established model and yoke only impose a smooth constraint on the image gradient. The correction method disclosed in CN105118037B is used as a comparison method for this embodiment. The background preservation effect and grayscale preservation effect of the original clear image corrected by the two correction methods are compared. The schematic diagrams are shown as follows: Figure 4 and Figure 5 shown.

[0100] Figure 4 In the figure, (a) is a true clear image, (b) is a degraded image, (c) is the original clear image corrected using the correction method in CN105118037B, and (d) is the original clear image corrected using the correction method provided by an embodiment of the present invention. By comparing (a), (c) and (d), it can be seen that this embodiment can more cleanly remove the bias field from the degraded image and maintain the image background by applying a sparse constraint to the coefficient vector.

[0101] Figure 5 In the figure, (a) is the grayscale mean curve of the image sequence corrected by the correction method in CN105118037B, and (b) is the grayscale mean curve of the image sequence corrected by the correction method provided by the embodiment of the present invention. Figure 5 As can be seen from the grayscale mean curves shown in (a) and (b), the grayscale mean curve of the image sequence obtained by the comparison method has a large number of burrs, indicating that the bias field reconstructed on the image sequence frequently jumps, while the grayscale mean curve of the image sequence obtained by this embodiment changes smoothly without burrs, indicating that the reconstructed bias field has no obvious jumps between adjacent frames. Figure 5 As can be seen from (a) and (b) in FIG, this embodiment can effectively ensure that the grayscale of the image sequence does not jump, screen flickering does not occur, and the reconstruction quality is high.

[0102] Example 2:

[0103] A computer program product includes a computer program; when the computer program is executed by a processor, the sparse representation correction method for aerodynamic thermal radiation effect images provided in the above-mentioned embodiment 1 is implemented.

[0104] Example 3:

[0105] A computer-readable storage medium includes a stored computer program. When the computer program is executed by a processor, the computer-readable storage medium is controlled to execute the sparse representation correction method for aerodynamic thermal radiation effect images provided in the above-mentioned embodiment 1.

[0106] Example 4:

[0107] An electronic device, comprising:

[0108] a computer-readable storage medium for storing a computer program;

[0109] and a processor configured to read a computer program stored in a computer-readable storage medium and execute the sparse representation correction method for aerodynamic thermal radiation effect images provided in the first embodiment.

[0110] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A sparse representation correction method for aerodynamic thermal radiation effect images, used to correct degraded images taken by an aircraft equipped with an imaging system into original clear images; characterized in that: The method comprises: An energy functional optimization model is constructed with the goal of minimizing the sum of the data fidelity term, the image prior regularization term, and the bias field prior regularization term; the data fidelity term is used to measure the difference between the degraded image and the corrected original clear image and the bias field; the image prior regularization term is used to measure the gradient of the original clear image; the bias field is fitted using a dictionary as , is the bias field, and are dictionaries and corresponding coefficient vectors, respectively, and the bias field prior regularization term is used to measure the coefficient vector The L1 norm of For the coefficient vector After applying the sparse constraint, the energy functional optimization model is solved to obtain the coefficient vector ; The sparse constraint is used to limit the coefficient vector The number of non-zero elements in does not exceed the preset threshold; according to Reconstruct the bias field ; Subtract the reconstructed bias field from the degraded image , and get the corrected original clear image.

2. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: The expression of the sparse constraint is: ; in, represents the L1 norm, The default positive number.

3. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: The expression of the data fidelity term is: ; in, represents the degraded image, represents the original clear image, represents the two-norm.

4. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: The expression of the image prior regularization term is: ; in, represents the gradient of the original clear image, is the regularization hyperparameter.

5. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: When solving the energy functional optimization model, a semi-quadratic variable separation algorithm is used.

6. The sparse representation correction method for aerodynamic thermal radiation effect images according to any one of claims 1 to 5, characterized in that: The bias field is fitted using a polynomial dictionary.

7. A computer program product, characterized in that The invention comprises a computer program; when the computer program is executed by a processor, the sparse representation correction method of the aerodynamic thermal radiation effect image according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that The invention comprises a stored computer program; when the computer program is executed by a processor, the sparse representation correction method of the aerodynamic thermal radiation effect image according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: a computer-readable storage medium for storing a computer program; and a processor configured to read the computer program stored in the computer-readable storage medium and execute the sparse representation correction method for aerodynamic thermal radiation effect images according to any one of claims 1 to 6.

Images