An infrared image de-streaking method
By constructing a structure tensor matrix and an edge-preserving operator matrix, splitting the objective function, and iteratively solving it using the augmented Lagrangian function and Fourier transform, the problem of image quality degradation after long-term on-orbit operation of infrared imaging systems was solved, achieving high-quality stripe noise removal and edge preservation.
Patent Information
- Application Number
- CN202511352623.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-22
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2045-09-22
AI Technical Summary
After long-term operation in orbit, the image quality of infrared imaging systems is affected by environmental interferences such as detector non-uniformity, circuit noise, and temperature control stability of the refrigerator, resulting in blurred images and low contrast. Existing noise reduction schemes are difficult to effectively distinguish between stripes and edges, resulting in poor image quality after stripe removal.
By constructing a structure tensor matrix and an edge-preserving operator matrix, the objective function is split into multiple auxiliary variables. The fringe noise is solved iteratively using the augmented Lagrangian function and Fourier transform. The ADMM algorithm is used to optimize the solution process, ensuring edge preservation and noise removal.
It significantly improves the detail and edge fidelity of infrared images, and the image quality is improved after removing stripe noise, making it suitable for infrared imaging systems that operate in orbit for long periods of time.
Smart Images
Figure CN120852245B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and particularly relates to a method for removing stripes from infrared images. Background Technology
[0002] In practical engineering, cooled infrared detectors employ a long-term power-on approach to achieve stable temperature control, reduce thermal cycling, and improve imaging performance. However, after long-term on-orbit operation, the image quality of infrared imaging systems is affected by environmental interferences such as detector non-uniformity, circuit noise, and the temperature control stability of the cooler, leading to problems such as image blurring and low contrast, severely impacting high-sensitivity detection capabilities. Conventional denoising schemes are based on traditional variational and low-rank models, or on deep learning methods; however, most methods are insufficient in preserving image edges and details, failing to distinguish between stripes and edges, such as... Figure 2 As shown, this results in smooth edges in the destriated image, but the image quality after removing stripe noise is poor. Summary of the Invention
[0003] In view of this, the present invention aims to provide a method for removing stripes from infrared images, which is beneficial for obtaining infrared images with better quality after stripe noise removal.
[0004] To achieve the above objectives, the technical solution created by this invention is implemented as follows:
[0005] This invention provides a method for removing stripes from infrared images, comprising: Step 1, providing an infrared image I, which consists of a clear image B and stripe noise S; Step 2, establishing a structure tensor matrix based on the infrared image I, and obtaining an edge-preserving operator matrix based on the structure tensor matrix. Step 3: Based on the edge-preserving operator matrix A target function Y(S) is constructed to solve for the stripe noise S. The solution of the target function Y(S) is decomposed into solving multiple auxiliary variables. Step four, the target function Y(S) is transformed into an augmented Lagrangian function, and each auxiliary variable and its corresponding Lagrange multiplier are iteratively solved to obtain multiple parameter combinations. Step five, the augmented Lagrangian function is subjected to a Fourier transform to obtain a function of the stripe noise S. The function of the stripe noise S is iteratively solved using multiple parameter combinations to obtain the stripe noise S. Step six, the stripe noise S in the infrared image I is removed to obtain a clear image B.
[0006] Furthermore, establishing the structure tensor matrix based on the infrared image I includes: establishing the structure tensor matrix corresponding to each pixel unit of the infrared image. , ,in, This represents the gradient in the horizontal direction of the corresponding pixel unit. is the gradient of the corresponding pixel unit in the vertical direction, i is the number of the pixel unit; based on the structural tensor matrix obtaining an edge-preserving operator matrix comprising: solving the coefficients of each structural tensor matrix trace is the trace of the structural tensor matrix is the rank of the structural tensor matrix constructing a corresponding edge-preserving operator , all the edge-preserving operators constitute an edge-preserving operator matrix .
[0007] Further, a target function Y(S) constructed based on the edge-preserving operator matrix is as follows: are weight factors, ; solving the target function Y(S) is split into solving a plurality of auxiliary variables including: , and .
[0008] Further, the target function Y(S) is converted into an augmented Lagrangian function including: converting Y(S) into formula 1, and the formula 1 is as follows:
[0009] ; wherein, is a first penalty term parameter, is a second penalty term parameter, is a third penalty term parameter, is a first Lagrange multiplier, is a second Lagrange multiplier, is a third Lagrange multiplier.
[0010] Further, each auxiliary variable and the corresponding Lagrange multiplier of each auxiliary variable are iteratively solved to obtain a plurality of parameter combinations including: from formula 1, extracting the related term of , introducing a shrinkage operator based on the related term of to obtain formula 3 to iteratively solve , based on The related terms of formula 4 are introduced into the shrinkage operator to obtain formula 5 for iterative solution , based on The related terms of formula 5 are introduced into the shrinkage operator to obtain formula 6 for iterative solution , based on formula 7 for iterative solution , based on formula 8 for iterative solution , based on formula 9 for iterative solution , , , , , and are obtained by solving formula 10 as a parameter combination.
[0011] Further, formula 3 is as follows:
[0012] ;
[0013] Formula 4 is as follows:
[0014] ;
[0015] Formula 5 is as follows:
[0016] ; wherein t is the iteration number, is a sign function for determining the positive and negative, if the parameter in the back bracket is less than 0, then the function is equal to -1, if the parameter in the back bracket is equal to 0, then the function is equal to 0, if the parameter in the back bracket is greater than 0, then the function is equal to 1.
[0017] Further, formula 6 is as follows: ; formula 7 is as follows: ; formula 8 is as follows: .
[0018] Further, the Fourier transform of the augmented Lagrange function is obtained to obtain the function of the stripe noise S, including: the Fourier transform of formula 1 is obtained to obtain formula 2, and formula 2 is as follows:
[0019] ;
[0020] Wherein, , , respectively represent the Fourier transform, conjugate and inverse transform, represents the matrix point product, , .
[0021] Further, the function of the stripe noise S is iteratively solved by using multiple parameter combinations to obtain the stripe noise S, each parameter combination is introduced into formula 2 to iteratively solve the stripe noise S, and when a stop condition is met, the iteration is stopped, and the stripe noise S is output; the stop condition is as follows: ; wherein, , , is the current noise residual, is the last round noise residual, is the last round stripe noise, is the current stripe noise, the initial value of , indicates the relative error of two consecutive iterations, when is less than 0.0001, the iteration is stopped, and is output as the stripe noise S.
[0022] Compared with the prior art, the present application can achieve the following beneficial effects: the present application can effectively improve the problem of poor details of infrared images of an infrared imaging system after long-term work in orbit, some methods are not enough to maintain the details and edges of infrared images, there is no distinction between stripes and image edges, leading to the problem of smoothing of image edges, the present application can better remove the stripe noise in the infrared image, and the quality of the infrared image after removing the stripe noise is higher.
[0023] Specifically, the edge-preserving operator matrix constructed based on the structure tensor can suppress smoothing in the stripe direction (suppress the stripes at the same time) while suppressing smoothing in the real image edge direction (preserve the edges), thereby significantly reducing the blurring of image details, compared with simple global TV (Total Variation) or simple frequency domain filtering, the objective function of the present application can distinguish which high frequency is the stripe to be removed and which high frequency is the edge to be preserved, so that the restored infrared image has high edge fidelity and few artifacts. By introducing auxiliary variables, the complex coupled objective function is decomposed into easy-to-solve subproblems, improving the controllability and numerical stability of the solution, and the solution of each step is independent, facilitating parallelization or acceleration on different hardware; and ADMM (Alternating Direction Method of Multipliers) is adopted, which is conducive to ensuring good convergence experience, through precise decomposition and edge constraint, the infrared image after denoising has high preservation degree at the edges and details, and the stripe residual energy is low, the improved visual effect and quantitative index are good, and the present application is particularly suitable for long-term on-orbit working infrared imaging systems. BRIEF DESCRIPTION OF DRAWINGS
[0024] The accompanying drawings, which form a part of the present application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification. The illustrations are shown for the purpose of enabling one of ordinary skill in the art to make and use the application.
[0025] Figure 1 A schematic diagram of an infrared image with stripe noise according to an embodiment of the present application;
[0026] Figure 2 A schematic diagram of an infrared image after removing stripe noise according to the related art;
[0027] Figure 3 A schematic diagram of an infrared image after removing stripe noise according to the infrared image de-stripping method according to an embodiment of the present application. DETAILED DESCRIPTION
[0028] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not constitute limitations on the present application.
[0029] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0030] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second" and the like are only used for description purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited by "first", "second" and the like can explicitly or implicitly include one or more features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.
[0031] In the description of the present application, it should be noted that unless specifically defined and limited, the terms "mounting", "connection", "linking" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through intermediate medium, or internal connection of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.
[0032] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
[0033] Reference Figure 1 And Figure 3 , the present application provides an infrared image de-streaking method, comprising:
[0034] Step one, providing an infrared image I, the infrared image I is composed of a clear image B and a streak noise S;
[0035] Step two, establishing a structure tensor matrix based on the infrared image I, and obtaining an edge-preserving operator matrix based on the structure tensor matrix ;
[0036] Step three, constructing a target function Y(S) for solving the streak noise S based on the edge-preserving operator matrix , and splitting the solving of the target function Y(S) into solving a plurality of auxiliary variables;
[0037] Step four, converting the target function Y(S) into an augmented Lagrangian function, and iteratively solving each auxiliary variable and the Lagrange multiplier corresponding to each auxiliary variable to obtain a plurality of parameter combinations;
[0038] Step five, performing Fourier transform on the augmented Lagrangian function to obtain a function of the streak noise S, and iteratively solving the function of the streak noise S with the plurality of parameter combinations to obtain the streak noise S;
[0039] Step six, removing the streak noise S in the infrared image I to obtain the clear image B.
[0040] In some embodiments, step six is to subtract the streak noise S from the infrared image I to obtain the clear image B after de-noising.
[0041] In some embodiments, establishing the structure tensor matrix based on the infrared image I comprises: establishing a structure tensor matrix corresponding to each pixel unit of the infrared image , , wherein, is the gradient of the corresponding pixel unit in the horizontal direction, is the gradient of the corresponding pixel unit in the vertical direction, i is the number of the pixel unit.
[0042] In some embodiments, the structure tensor matrix is obtained based on the edge preserving operator matrix comprises: solving the coefficients of each structure tensor matrix , wherein, trace is the trace of the structure tensor matrix , and det is the rank of the structure tensor matrix ; and constructing the corresponding edge preserving operator , , all of the edge preserving operators constitute the edge preserving operator matrix .
[0043] In some embodiments, the objective function Y(S) constructed based on the edge preserving operator matrix is as follows: wherein, , and are weight factors, , Solving the objective function Y(S) is to iteratively find an optimal solution S such that Y(S) is minimum.
[0044] In some embodiments, solving the objective function Y(S) is split into solving a plurality of auxiliary variables including: , and , and , , , Solving the objective function Y(S) into solving a plurality of auxiliary variables is to convert the unconstrained optimization problem into a constrained optimization problem, and introducing the three auxiliary variables can split the complex objective function constructed initially, and the constraints are performed respectively, is used to constrain the overall image vertical gradient, is used to ensure the vertical sparsity, remove the vertical stripes, and keep the edge clear, is used to perform sparse processing on the noise, is used to constrain the overall image horizontal gradient.
[0045] In some embodiments, converting the objective function Y(S) into an augmented Lagrangian function comprises: converting Y(S) into formula 1, and formula 1 is as follows:
[0046] ;
[0047] wherein, is a first penalty term parameter, is a second penalty term parameter, is a third penalty term parameter, is a first Lagrange multiplier, is a second Lagrange multiplier, is a third Lagrange multiplier, in some examples, .
[0048] In some embodiments, iteratively solving each auxiliary variable and the corresponding Lagrange multiplier of each auxiliary variable to obtain the plurality of parameter combinations comprises: extracting the relevant terms of , , from Formula 1, introducing a shrinkage operator based on the relevant terms of to obtain Formula 3 to iteratively solve , introducing a shrinkage operator based on the relevant terms of to obtain Formula 4 to iteratively solve , introducing a shrinkage operator based on the relevant terms of to obtain Formula 5 to iteratively solve , , , , , , , as a parameter combination.
[0049] wherein the relevant terms of are as follows: ;
[0050] the relevant terms of are as follows:
[0051] the relevant terms of are as follows.
[0052] In some embodiments, Formula 3 is as follows:
[0053] ;
[0054] Formula 4 is as follows:
[0055] ;
[0056] Formula 5 is as follows:
[0057] ;
[0058] wherein t is the iteration number, t = 1, 2, 3, …, 100.
[0059] is a sign function, used to determine the positive and negative, if the parameter in the back bracket is less than 0, then the function is equal to -1, if the parameter in the back bracket is equal to 0, then the function is equal to 0, if the parameter in the back bracket is greater than 0, then the function is equal to 1, with the parameter in the back bracket being For example, if is less than 0, then is equal to -1, if is equal to 0, then is equal to 0, if is greater than 0, then is equal to 1.
[0060] In some embodiments, formula 6 is as follows: ;
[0061] Formula 7 is as follows: ;
[0062] Formula 8 is as follows: . The initial values of are all zero matrices of the image size.
[0063] In some embodiments, the Fourier transform of the augmented Lagrangian function to obtain the function of the stripe noise S includes: Fourier transform of formula 1 to obtain formula 2, formula 2 is as follows:
[0064] ;
[0065] wherein , , respectively represent the Fourier transform, conjugate and inverse transform, “·” represents the matrix point product, , , and T represents the transpose.
[0066] In some embodiments, the function of the stripe noise S is iteratively solved by using multiple parameter combinations to obtain the stripe noise S, which includes: introducing each parameter combination into formula 2 iteratively to solve the stripe noise S, stopping iteration when the stopping condition is met, and outputting the stripe noise S;
[0067] The stopping condition is as follows: ;
[0068] wherein, , , is the current noise residue, is the last round noise residue, is the last round stripe noise, is the current stripe noise, the initial value of is equal to , denotes the relative error of two consecutive iterations, when is less than 0.0001, the iteration is stopped and is output as the stripe noise S. The maximum iteration number is set to 1000 to avoid falling into a dead loop.
[0069] It should be understood that the various forms of flow shown above can be reordered, steps added or deleted. For example, the steps described in the present disclosure can be performed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which is not limited herein.
[0070] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements within the spirit and principles of the present application shall be included in the scope of protection of the present application.
Claims
1. An infrared image de-striping method, characterized by, Comprising: Step one, providing an infrared image I, the infrared image I is composed of a clear image B and a stripe noise S; Step two, establishing a structure tensor matrix based on the infrared image I, and obtaining an edge preserving operator matrix based on the structure tensor matrix The step of establishing the structure tensor matrix based on the infrared image I comprises: establishing a structure tensor matrix corresponding to each pixel unit of the infrared image , wherein, is a gradient in a horizontal direction of the corresponding pixel unit, is a gradient in a vertical direction of the corresponding pixel unit, and i is a number of the pixel unit. based on a structure tensor matrix obtaining an edge preserving operator matrix comprising solving coefficients of each structure tensor matrix , wherein trace is a trace of the structure tensor matrix and det is a rank of the structure tensor matrix based on constructing the corresponding edge-preserving operator , all the edge-preserving operators form an edge-preserving operator matrix ; Step three, constructing the objective function Y(S) for solving the stripe noise S based on the edge preserving operator matrix constructing the objective function Y(S) for solving the stripe noise S based on the edge preserving operator matrix The constructed objective function Y(S) is as follows: wherein , and are weight factors, , solving the objective function Y(S) is split into solving a plurality of auxiliary variables; Step four, converting the target function Y(S) into an augmented Lagrangian function, iteratively solving each auxiliary variable and the Lagrange multiplier corresponding to each auxiliary variable to obtain a plurality of parameter combinations; Step five, Fourier transforming the augmented Lagrangian function to obtain a function of the stripe noise S, and iteratively solving the function of the stripe noise S using a plurality of parameter combinations to obtain the stripe noise S; Step six, removing the stripe noise S in the infrared image I to obtain the clear image B.
2. The method of claim 1, wherein, Splitting the solution of the objective function Y(S) into solving a plurality of auxiliary variables comprises introducing auxiliary variables , and , another , , then .
3. The method of claim 2, wherein, Converting the target function Y(S) into an augmented Lagrangian function comprises: converting Y(S) into formula 1, formula 1 is as follows: ； wherein is a first penalty term parameter, is a second penalty term parameter, is a third penalty term parameter, is a first Lagrange multiplier, is a second Lagrange multiplier, is a third Lagrange multiplier.
4. The method of claim 3, wherein, solving each auxiliary variable and the Lagrange multiplier corresponding to each auxiliary variable to obtain a plurality of parameter combinations comprises: extracting the related terms of , and from formula 1, introducing a shrinkage operator based on the related terms of to obtain formula 3 to iteratively solve , introducing a shrinkage operator based on the related terms of to obtain formula 4 to iteratively solve , introducing a shrinkage operator based on the related terms of to obtain formula 5 to iteratively solve , iteratively solving based on formula 6, based on formula 7, based on formula 8, , , , , and as a parameter combination in the same iteration number.
5. The method of claim 4, wherein, Formula 3 is as follows: ； Formula 4 is as follows: ； Formula 5 is as follows: ； where t is the iteration number, is a sign function for determining the positive or negative, if the parameter in the back bracket is less than 0, then the function equals -1, if the parameter in the back bracket is equal to 0, then the function equals 0, if the parameter in the back bracket is greater than 0, then the function equals 1.
6. The infrared image stripe removal method according to claim 5, wherein, Equation 6 is as follows: ; Equation 7 is as follows: ; Equation 8 is as follows: .
7. The method of claim 3, wherein, Fourier transforming the augmented Lagrangian function to obtain a function of the stripe noise S comprises: Fourier transforming formula 1 to obtain formula 2, formula 2 is as follows: ； wherein , , denote the Fourier transform, the conjugate and the inverse transform, respectively, ” represents the matrix point product, , .
8. The method of claim 7, wherein, Iteratively solving the function of the stripe noise S using a plurality of parameter combinations to obtain the stripe noise S comprises: sequentially introducing each parameter combination into formula 2 to iteratively solve the stripe noise S, stopping iteration when a stop condition is met, and outputting the stripe noise S; The stop condition is as follows: ; wherein, , , is the current noise residual, is the previous noise residual, is the previous stripe noise, is the current stripe noise, the initial value of , denotes the relative error of two successive iterations, and when is less than 0.0001, the iteration is stopped and is output as the stripe noise S.