Sparse representation correction method for pneumatic thermal radiation effect image

Abstract

The invention discloses a sparse representation correction method for an aerodynamic thermal radiation effect image, which belongs to the field of digital image processing, and comprises the following steps: constructing an energy functional optimization model with the goal of minimizing the sum of a data fidelity term, an image prior regularization term and a bias field prior regularization term; the data fidelity item is used for measuring the difference between the degraded image and the original clear image and the difference between the degraded image and the bias field; the image prior regularization item is used for measuring the gradient of the original clear image; the bias field B adopts dictionary fitting as B = Wa, W and a are a dictionary and a coefficient vector respectively, and a prior regularization item of the bias field is used for measuring an L1 norm of the coefficient vector a; solving the energy functional optimization model under the constraint that the number of non-zero elements in the coefficient vector a does not exceed a preset threshold value to obtain the coefficient vector a; and reconstructing a bias field B according to B = Wa, and subtracting the bias field B from the degraded image to obtain an original clear image. The correction precision and the corrected image quality of the pneumatic thermal radiation effect image can be improved.

Description

Technical Field

[0001] The invention belongs to the field of digital image processing, and more specifically, relates to a sparse representation correction method for aerodynamic thermal radiation effect images. Background Art

[0002] When an aircraft equipped with an imaging system flies in the atmosphere, the friction between the optical hood and the high-speed incoming flow will cause the temperature to rise, which will induce an aerodynamic thermal radiation effect similar to a "halo" on the image plane, resulting in a degradation of the signal-to-noise ratio of the target image taken by the imaging system. According to Planck's blackbody radiation law, aerodynamic thermal radiation effects exist in both infrared and visible light bands. Therefore, it is of great significance to correct the aerodynamic thermal radiation effect to improve image quality.

[0003] The aerodynamic thermal radiation degradation mechanism model can be expressed as:

[0004] Z=X+B

[0005] Among them, Z is the known degraded image, X is the unknown original clear image, and B is the unknown thermal radiation bias field superimposed on the clear image.

[0006] Based on the above aerodynamic thermal radiation degradation mechanism model, it can be known that after reconstructing the bias field B, the degraded image Z can be corrected to obtain the original clear image X. Therefore, the focus of existing correction technology is on the statistical modeling of the degradation characteristics of the bias field of the aerodynamic thermal radiation effect. At present, existing research results have proposed to fit the bias field surface by means of spline surfaces (CN114529481A), Chebyshev polynomials (CN114359093A), pixel coordinate polynomials (CN105118037B), etc.

[0007] The above thermal radiation bias field correction methods all have the following problems: 1) The surface morphology of the thermal radiation bias field is usually similar to the shape of the aircraft hood, showing smooth changes as a whole and small local mutations. However, the existing correction methods are easily disturbed by the imaging scene, and the estimated bias field often contains background information with local mutations, resulting in loss of corrected image information. 2) The existing fitting methods only impose smooth constraints on image gradients, and have no constraints on image grayscale. This can easily cause uneven grayscale between frames after correction in the image sequence, resulting in screen flicker and poor quality of the corrected image. Summary of the invention

[0008] In view of the defects and improvement needs of the prior art, the present invention provides a sparse representation correction method for aerodynamic thermal radiation effect images, which aims to improve the correction accuracy of aerodynamic thermal radiation effect images and improve the quality of the corrected images.

[0009] To achieve the above object, according to one aspect of the present invention, a sparse representation correction method for an aerodynamic thermal radiation effect image is provided, which is used to correct a degraded image taken by an aircraft equipped with an imaging system into an original clear image; the method comprises:

[0010] An energy functional optimization model is constructed with the goal of minimizing the sum of the data fidelity term, the image prior regularization term and the bias field prior regularization term; the data fidelity term is used to measure the difference between the degraded image and the corrected original clear image and the bias field; the image prior regularization term is used to measure the gradient of the original clear image; the bias field is fitted using a dictionary as B=Wa, where B is the bias field, W and a are the dictionary and the corresponding coefficient vector respectively, and the bias field prior regularization term is used to measure the L1 norm of the coefficient vector a;

[0011] After applying a sparse constraint to the coefficient vector a, the energy functional optimization model is solved to obtain the coefficient vector a; the sparse constraint is used to limit the number of non-zero elements in the coefficient vector a to not exceed a preset threshold;

[0012] Reconstruct the bias field B according to B=Wa;

[0013] Subtract the reconstructed bias field B from the degraded image to obtain the corrected original clear image.

[0014] Furthermore, the expression of the sparse constraint is: ||a||1<τ;

[0015] Among them, || ||1 represents the L1 norm, and τ is a preset positive constant.

[0016] Furthermore, the expression of the data fidelity term is:

[0017] Among them, Z represents the degraded image, X represents the original clear image, and |‖‖|2 represents the binary norm.

[0018] Furthermore, the expression of the image prior regularization term is:

[0019] in, represents the gradient of the original clear image, and λ is the regularization hyperparameter.

[0020] Furthermore, a semi-quadratic variable separation algorithm is used to solve the energy functional optimization model.

[0021] Furthermore, the bias field is fitted using a polynomial dictionary.

[0022] According to another aspect of the present invention, a computer program product is provided, comprising a computer program; when the computer program is executed by a processor, the sparse representation correction method for the aerodynamic thermal radiation effect image provided by the present invention is implemented.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, comprising a stored computer program; when the computer program is executed by a processor, the computer-readable storage medium is controlled to execute the sparse representation correction method for the aerodynamic thermal radiation effect image provided by the present invention.

[0024] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0025] A computer-readable storage medium for storing a computer program;

[0026] and a processor, which is used to read a computer program stored in a computer-readable storage medium and execute the sparse representation correction method for aerodynamic thermal radiation effect images provided by the present invention.

[0027] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0028] (1) The sparse representation correction method for aerodynamic thermal radiation effect images provided by the present invention imposes a sparse constraint on the coefficient vector used to fit the bias field when solving it, and limits the number of non-zero elements in the sparse vector, thereby avoiding the noise points in the coefficient vector from causing overfitting of the bias field surface, thereby effectively avoiding the inclusion of background information in the reconstructed bias field, improving the reconstruction accuracy of the bias field, and ultimately effectively separating the thermal radiation bias field from the degraded image, achieving the purpose of high-precision protection of the image background and avoiding image information loss; the energy functional optimization model established by the present invention, while constraining the image gradient through the image prior regularization term, constrains the image grayscale through the data fidelity term, avoiding obvious jumps in the bias field between adjacent frames, and ensuring that the grayscale of the reconstructed image sequence is stable and free of screen flicker, thereby effectively improving the image quality. In general, the present invention constructs an energy functional optimization model with the goal of minimizing the weighted sum of data fidelity terms, image prior regularization terms and bias field prior regularization terms, and solves the model while imposing sparse constraints on the coefficient vector to achieve bias field reconstruction and image correction. This can improve the correction accuracy of aerodynamic thermal radiation effect images and improve the quality of reconstructed images.

[0029] (2) Directly constraining the number of non-zero elements in the coefficient vector will make it more difficult to solve the model. As a preferred implementation, the present invention relaxes the sparse constraint imposed on the coefficient vector and converts it into a constraint on the L1 norm of the coefficient vector, thereby indirectly achieving the purpose of limiting the number of non-zero elements in the coefficient vector. While ensuring the effectiveness of the constraint, it effectively reduces the difficulty of solving the model.

[0030] (3) Half-Quadratic Splitting (HQS) is a method for solving optimization problems, especially when the objective function contains non-convex or difficult-to-handle terms. By introducing auxiliary variables and alternating optimization, HQS decomposes the original problem into a series of simpler quadratic optimization problems, thereby simplifying the calculation and approximating the solution of the original problem. The energy functional optimization model established in the present invention can be expressed as In the preferred embodiment of the present invention, a semi-quadratic variable separation algorithm is used to solve the model, which can effectively improve the solution efficiency while ensuring the solution accuracy.

[0031] (4) The bias field of a degraded image usually appears as a globally smoothly varying curved surface field. In the preferred embodiment of the present invention, a polynomial dictionary is used to fit the bias field, which can accurately characterize the spatial variation of the bias field, further improve the reconstruction accuracy of the bias field, and thus improve the image correction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flow chart of a sparse representation correction method for aerodynamic thermal radiation effect images provided by an embodiment of the present invention;

[0033] Figure 2 A block diagram of a sparse representation correction method for aerodynamic thermal radiation effect images provided by an embodiment of the present invention;

[0034] Figure 3 Schematic diagram of image correction effect provided by an embodiment of the present invention; wherein (a) is a schematic diagram of infrared image correction effect, and (b) is a schematic diagram of near-infrared image correction effect;

[0035] Figure 4 Schematic diagram of the background preservation effect of the corrected image provided by the embodiment of the present invention; wherein (a) is a real clear image, (b) is a degraded image, (c) is the original clear image corrected by the correction method in CN105118037B, and (d) is the original clear image corrected by the correction method provided by the embodiment of the present invention;

[0036] Figure 5 A schematic diagram comparing the screen flicker suppression effect of the corrected image sequence provided in an embodiment of the present invention; wherein (a) is a grayscale mean curve of the image sequence corrected by the correction method in CN105118037B, and (b) is a grayscale mean curve of the image sequence corrected by the correction method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] In the present invention, the terms "first", "second", etc. (if any) in the present invention and the accompanying drawings are used to distinguish similar objects but not necessarily to describe a specific order or sequence.

[0039] In order to solve the technical problems that the thermal radiation bias field reconstructed in the prior art contains background information, resulting in loss of corrected image information and uneven grayscale on the corrected image sequence, the present invention provides a sparse representation correction method for aerodynamic thermal radiation effect images. The overall concept is to conduct an in-depth analysis of the generation mechanism of related problems, and based on the analysis results, impose sparse constraints on the coefficient vector used to fit the bias field accordingly to avoid background information being included in the bias field due to overfitting of the surface caused by noise points in the coefficient vector. At the same time, items for imposing constraints on the image grayscale are set in the established optimization model to ensure that the grayscale of the image sequence finally reconstructed is stable and there is no screen flicker.

[0040] Before explaining the technical solution of the present invention in detail, starting from the above technical concept, the setting of relevant constraints and the establishment of the overall model in the present invention are analyzed as follows:

[0041] The aerodynamic thermal radiation degradation mechanism model can be expressed as:

[0042] Z=X+B

[0043] Among them, Z is the known degraded image, X is the unknown original clear image, and B is the unknown thermal radiation bias field superimposed on the clear image.

[0044] Based on the above degradation model, the optimization framework can be established based on the maximum a posteriori probability method as follows:

[0045]

[0046] Among them, Ψ(Z,X,B) is the data fidelity term, Pr(X) is the image prior regularization term, Pr(B) is the bias field prior regularization term, and λ and β are the corresponding regularization hyperparameters.

[0047] Using any dictionary such as spline surface, Chebyshev polynomial, pixel coordinate polynomial, etc. to fit the surface of the bias field, the bias field can be expressed as:

[0048] B=Wa

[0049] Among them, B is the bias field, W and a are the dictionary and the corresponding coefficient vector respectively.

[0050] The present invention finds that the bias field contains background information, mainly because the noise points in the coefficient vector a will cause the surface of the bias field to overfit. Based on this finding, the present invention proposes to impose a sparse constraint on the sparse vector a of the bias field to limit the number of non-zero elements in the coefficient vector a. Specifically, the number of non-zero elements in the coefficient vector a is limited to not exceed a preset threshold. The sparse constraint will be used as a constraint in the subsequent model solution. Considering that directly limiting the number of non-zero elements in the coefficient vector during model solution will lead to greater difficulty in model solution, in response to this problem, the present invention further proposes to relax the sparse constraint imposed on the coefficient vector and convert it into a constraint on the L1 norm of the coefficient vector, thereby indirectly achieving the purpose of limiting the number of non-zero elements in the coefficient vector, while ensuring the effectiveness of the constraint, effectively reducing the difficulty of model solution. The sparse constraint after conversion can be expressed as:

[0051] ||a||1<τ

[0052] Among them, || ||1 represents the L1 norm, and τ is a preset positive constant.

[0053] In practical applications, any dictionary can be used to fit the surface of the bias field. Considering that the bias field of the degraded image is usually presented as a globally smoothly changing surface field, the bias field can be accurately characterized by using a polynomial dictionary to fit the bias field, which helps to improve the reconstruction accuracy of the bias field. Without loss of generality, in the following embodiments, a polynomial dictionary is used to fit the bias field. After applying a sparse constraint to the coefficient vector, the bias field can be expressed as:

[0054]

[0055] Among them, x and y are the horizontal and vertical coordinates of the image, D is the highest order of the preset polynomial, and a t,s are the polynomial coefficients.

[0056] In view of the uneven grayscale of the corrected image sequence, there will be a problem of screen flickering. When establishing a corresponding optimization model, the present invention constructs an image gradient prior for constraining the gradient and constructs a data item for constraining the image grayscale;

[0057] The data item expressions are as follows:

[0058]

[0059] The image gradient prior expression is as follows:

[0060]

[0061] The correction framework is as follows:

[0062]

[0063] Refer to the above optimization framework By using the sparse constraint imposed on the coefficient vector a, the above correction framework can be further transformed into the following solvable energy functional optimization model:

[0064]

[0065] In the formula and β||a||1 correspond to the data fidelity term, image prior regularization term, and bias field prior regularization term in the optimization framework, respectively.

[0066] By solving the above energy functional optimization model, the coefficient vector a can be solved, and then the bias field B can be reconstructed based on the fitting expression B=Wa. After subtracting the bias field B from the degraded image, correction can be achieved to obtain the original clear image X.

[0067] Based on the above analysis, the present invention can improve the correction accuracy of aerodynamic thermal radiation effect images and improve the quality of reconstructed images by constructing an energy functional optimization model with the goal of minimizing the weighted sum of data fidelity terms, image prior regularization terms and bias field prior regularization terms, and solving the model while imposing sparse constraints on the coefficient vector to achieve bias field reconstruction and image correction.

[0068] The following are examples.

[0069] Embodiment 1:

[0070] A sparse representation correction method for an aerodynamic thermal radiation effect image is used to correct a degraded image taken by an aircraft equipped with an imaging system into an original clear image; Figure 1 and Figure 2 As shown, this embodiment includes:

[0071] Step S1: construct an energy functional optimization model with the goal of minimizing the sum of a data fidelity term, an image prior regularization term, and a bias field prior regularization term; the data fidelity term is used to measure the difference between the degraded image and the corrected original clear image and the bias field; the image prior regularization term is used to measure the gradient of the original clear image; the bias field is fitted using a dictionary as B=Wa, where B is the bias field, W and a are the dictionary and the corresponding coefficient vector, respectively, and the bias field prior regularization term is used to measure the L1 norm of the coefficient vector a;

[0072] Preferably, in this embodiment, the expression of the energy functional optimization model is:

[0073]

[0074] Step S2: Solving the energy functional optimization model after applying a sparse constraint to the coefficient vector a to obtain the coefficient vector a; the sparse constraint is used to limit the number of non-zero elements in the coefficient vector a to not exceed a preset threshold;

[0075] Preferably, in this embodiment, the expression of the sparse constraint is:

[0076]

[0077] Optionally, in this implementation, a semi-quadratic variable separation algorithm is used when solving the energy functional optimization model; the specific process is as follows:

[0078] The energy functional optimization model is separated by variables, and the following two sub-problems are obtained:

[0079]

[0080] (1) For the subproblem X, the closed-form solution of the original clear image X is:

[0081]

[0082] Where t is the iteration index, and I is the identity matrix;

[0083] (2) For subproblem a, introduce auxiliary variable u:

[0084]

[0085] Using the semi-quadratic variable separation algorithm to solve the problem, we get the following sub-problems:

[0086]

[0087] in, The subproblem is a least squares problem, and the closed-form solution is obtained by differentiation:

[0088]

[0089] Use the soft threshold shrinkage operator to solve the u-subproblem:

[0090]

[0091] Where k is the index of the number of iterations of the inner loop to solve the a subproblem, 1 is a matrix with all element values ​​1, and 0 is a matrix with all element values ​​0; the regularization parameter η of the auxiliary variable is updated with the iteration:

[0092] η k+1 =2·η k

[0093] After multiple iterations of inner and outer loops, the outer loop number t reaches the maximum outer loop number (10 in this embodiment), and the inner loop k reaches the maximum inner loop number (5 in this embodiment), and the final estimated coefficient vector a is obtained;

[0094] Step S3: Reconstruct the bias field B according to B=Wa;

[0095] Step S4: Subtract the reconstructed bias field B from the degraded image to obtain a corrected original clear image.

[0096] In general, the sparse constraints imposed on the polynomial dictionary coefficient vector in this embodiment can accurately separate the thermal radiation bias field from the degraded image, thereby achieving the purpose of protecting the image background with high precision; the constraints imposed on the image grayscale can ensure that the grayscale of the corrected image sequence is stable and has no screen flicker.

[0097] The beneficial effects that can be achieved by this embodiment are further analyzed and explained below in combination with actual correction examples.

[0098] Figure 3 FIG. 1 is a schematic diagram showing the correction effect of aerodynamic thermal radiation effect image correction based on the sparse representation correction method for aerodynamic thermal radiation effect image provided by this embodiment, wherein (a) is a schematic diagram showing the correction effect of an infrared image, and (b) is a schematic diagram showing the correction result of a near-infrared image. Figure 3 In (a) and (b), the first row from left to right is the degraded image, the corrected original clear image and the thermal radiation bias field, and the second row is the corresponding three-dimensional grayscale coordinate diagram. Figure 3 It can be seen from the results that this embodiment can effectively implement correction for different images.

[0099] The correction method disclosed in CN105118037B and this embodiment both use a polynomial dictionary to fit the bias field, but this method does not impose a sparse constraint on the coefficient vector. The established model and yoke only impose a smooth constraint on the image gradient. The correction method disclosed in CN105118037B is used as a comparison method of this embodiment to compare the background preservation effect and grayscale preservation effect of the original clear image corrected by the two correction methods. The schematic diagrams are shown as follows: Figure 4 and Figure 5 shown.

[0100] Figure 4 In the figure, (a) is a real clear image, (b) is a degraded image, (c) is the original clear image corrected by the correction method in CN105118037B, and (d) is the original clear image corrected by the correction method provided by the embodiment of the present invention. By comparing (a), (c) and (d), it can be seen that this embodiment can more cleanly remove the bias field from the degraded image and maintain the image background by applying sparse constraints to the coefficient vector.

[0101] Figure 5 In the figure, (a) is a grayscale mean curve of an image sequence corrected by the correction method in CN105118037B, and (b) is a grayscale mean curve of an image sequence corrected by the correction method provided in an embodiment of the present invention. Figure 5 From the grayscale mean curves shown in (a) and (b), it can be seen that the grayscale mean curve of the image sequence obtained by the comparison method has a large number of burrs, indicating that the bias field reconstructed on the image sequence frequently jumps, while the grayscale mean curve of the image sequence obtained in this embodiment changes smoothly without burrs, indicating that the reconstructed bias field has no obvious jumps between adjacent frames. Figure 5 As can be seen from (a) and (b) in the figure, this embodiment can effectively ensure that the grayscale of the image sequence does not jump, no screen flickering occurs, and the reconstruction quality is high.

[0102] Embodiment 2:

[0103] A computer program product includes a computer program. When the computer program is executed by a processor, the sparse representation correction method for aerodynamic thermal radiation effect images provided in the above-mentioned embodiment 1 is implemented.

[0104] Embodiment 3:

[0105] A computer-readable storage medium includes a stored computer program. When the computer program is executed by a processor, the computer-readable storage medium is controlled to execute the sparse representation correction method for aerodynamic thermal radiation effect images provided in the above-mentioned embodiment 1.

[0106] Embodiment 4:

[0107] An electronic device, comprising:

[0108] A computer-readable storage medium for storing a computer program;

[0109] And a processor, used to read the computer program stored in the computer-readable storage medium, and execute the sparse representation correction method of the aerodynamic thermal radiation effect image provided in the above-mentioned embodiment 1.

[0110] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A sparse representation correction method for aerodynamic thermal radiation effect images, used to correct the degraded images taken by an aircraft equipped with an imaging system into original clear images; characterized in that: The method comprises: An energy functional optimization model is constructed with the goal of minimizing the sum of a data fidelity term, an image prior regularization term, and a bias field prior regularization term; the data fidelity term is used to measure the difference between the degraded image and the corrected original clear image and the bias field; the image prior regularization term is used to measure the gradient of the original clear image; the bias field is fitted using a dictionary as B=Wa, where B is the bias field, W and a are the dictionary and the corresponding coefficient vector, respectively, and the bias field prior regularization term is used to measure the L1 norm of the coefficient vector a; After applying a sparse constraint to the coefficient vector a, the energy functional optimization model is solved to obtain the coefficient vector a; the sparse constraint is used to limit the number of non-zero elements in the coefficient vector a to not exceed a preset threshold; Reconstructing the bias field B according to B=Wa; The reconstructed bias field B is subtracted from the degraded image to obtain a corrected original clear image.

2. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: The expression of the sparse constraint is: ||a||1<τ; Among them, || ||1 represents the L1 norm, and τ is a preset positive constant.

3. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: The expression of the data fidelity term is: Wherein, Z represents the degraded image, X represents the original clear image, and || ||2 represents the binary norm.

4. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: The expression of the image prior regularization term is: in, represents the gradient of the original clear image, and λ is a regularization hyperparameter.

5. The sparse representation correction method for aerodynamic thermal radiation effect images according to claim 1, characterized in that: When solving the energy functional optimization model, a semi-quadratic variable separation algorithm is used.

6. The sparse representation correction method for aerodynamic thermal radiation effect images according to any one of claims 1 to 5, characterized in that: The bias field is fitted using a polynomial dictionary.

7. A computer program product, characterized in that It comprises a computer program; when the computer program is executed by a processor, it implements the sparse representation correction method of aerodynamic thermal radiation effect image as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: It comprises a stored computer program; when the computer program is executed by a processor, it controls the computer-readable storage medium to execute the sparse representation correction method for aerodynamic thermal radiation effect images as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A computer-readable storage medium for storing a computer program; and a processor, configured to read the computer program stored in the computer-readable storage medium and execute the sparse representation correction method for aerodynamic thermal radiation effect images as described in any one of claims 1 to 6.

Images