Synchronous progressive correction method for thermal effect and stripe noise based on RBF surface
Through the synchronous progressive correction method based on RBF surface, the image denoising problem of aerodynamic heating and stripe noise in aircraft optical equipment was solved, efficient and simplified image correction was achieved, and the signal-to-noise ratio and image clarity were improved.
Patent Information
- Application Number
- CN202511065074.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2045-07-31
AI Technical Summary
The existing technology for eliminating aerodynamic heating radiation effects and stripe noise in aircraft optical equipment is complex and has the problem of residual effects on image clarity.
A synchronous progressive correction method for thermal effect and stripe noise based on RBF surface is adopted. Through grayscale processing, guided filtering, RBF kernel matrix construction, stripe operator estimation and synchronous progressive correction optimization algorithm, the algorithm is dynamically adjusted to remove noise and generate a clear image.
It effectively removes aero-optical effects and stripe noise, improves image signal-to-noise ratio, simplifies operations, reduces computational complexity, and improves image quality.
Smart Images
Figure CN120563363B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of aerodynamic optical thermal radiation effect correction, and in particular relates to a synchronous progressive correction method for thermal effect and strip noise based on RBF surface. Background Art
[0002] When an aircraft flies at high speed in the atmosphere, the optical hood of the detector will be affected by the airflow. The kinetic energy of the airflow in the boundary layer near the hood surface is dissipated and converted into heat energy, causing the temperature of the optical window to rise rapidly. This convective heat exchange between the high-speed incoming flow and the hood surface is called aerodynamic heating. The thermal radiation effect generated by aerodynamic heating seriously affects the detection efficiency of the imaging detection system, significantly reducing the signal-to-noise ratio, and thus leading to subsequent target detection and recognition failures. In addition, there is also stripe noise caused by damage to internal components caused by the vibration of the hardware during flight. That is, during the actual flight of the aircraft, stripe noise is likely to be generated along with the oscillation of the components, which also poses a great challenge to the current rapid target recognition.
[0003] To eliminate the effects of thermal radiation on optical imaging, several effective methods have been developed, including variable integration time infrared imaging technology, replacing infrared window materials with high transmission and low emissivity, window cooling technology, and optical filtering. While these technologies can effectively prevent optical device saturation, these physically implemented methods are complex and expensive to implement, and they still retain varying degrees of residual thermal radiation effects. Furthermore, while many methods exist for removing banding noise, most only address the occurrence of a single band of noise and are not well-suited for practical applications.
[0004] At present, the method of removing strips first and then removing thermal radiation is usually used to remove noise and restore the image. However, this method often makes the image unclear because the removal of a single noise often affects the removal of the next noise, making the image unclear and unable to meet the requirements of target recognition. Summary of the Invention
[0005] In response to the above-mentioned defects in the prior art, the present invention provides a synchronous progressive correction method for thermal effects and stripe noise based on RBF surfaces, which can simply and efficiently remove aero-optical effects and stripe noise, improve the image signal-to-noise ratio, and facilitate subsequent target recognition and detection operations.
[0006] In a first aspect of the present invention, a synchronous progressive correction method for thermal effect and stripe noise based on an RBF surface is provided, the method comprising:
[0007] Acquire a degraded image;
[0008] Performing grayscale processing on the degraded image to obtain a grayscale degraded image;
[0009] Perform guided filtering on the grayscale degraded image to obtain a filtered thermal radiation effect estimation image;
[0010] The RBF kernel matrix is constructed based on the thermal radiation effect estimation image and the RBF kernel function, and the weight coefficient is solved by combining the pixel grayscale vector of the thermal radiation effect estimation image;
[0011] Multiply the weight coefficient obtained by the solution with the RBF kernel matrix to generate the RBF thermal radiation effect surface;
[0012] Based on the degraded image and RBF thermal radiation effect surface, the stripe noise is estimated through the stripe operator;
[0013] According to the RBF thermal radiation effect surface and stripe noise, the potential clear image is restored through the synchronous progressive correction optimization algorithm.
[0014] In the above scheme, the grayscale degraded image is subjected to guided filtering to obtain a filtered thermal radiation effect estimation image, including:
[0015] For each pixel in the grayscale degraded image , define a point centered on it and of size Window , and calculate the local mean:
[0016]
[0017] Where, For the The local mean of pixels; is the total number of pixels in the window, equal to ; Indicated in pixels i The center window j The grayscale value of each pixel;
[0018] Calculate the local variance based on the local mean:
[0019]
[0020] Where, For the The local variance of pixels;
[0021] Calculate the local linear coefficient based on the local mean and local variance:
[0022]
[0023]
[0024] Where, and are two local linear coefficients, is the smoothing parameter;
[0025] Will and smooth:
[0026]
[0027]
[0028] Where, and are the two local linear coefficients after smoothing;
[0029] Output filtered thermal radiation effect estimation image s :
[0030]
[0031] Where, Estimate the thermal radiation effect in the image The pixel value of each pixel, is the grayscale degraded image The pixel value of pixels.
[0032] In the above scheme, the RBF kernel matrix is constructed based on the thermal radiation effect estimation image and the RBF kernel function, and the weight coefficient is solved in combination with the pixel grayscale vector of the thermal radiation effect estimation image, including:
[0033] Define the pixel position coordinate matrix of the thermal radiation effect estimation image :
[0034]
[0035] Where, Estimate the thermal radiation effect in the image The position coordinates of the pixels, and Estimate the height and width of the image for thermal radiation effects, The total number of pixels in the image is estimated for thermal radiation effects;
[0036] Constructing RBF kernel matrix based on RBF kernel function , the elements in the matrix The definition is as follows:
[0037]
[0038] Where, is an exponential function, ; is the shape parameter; is the square of the Euclidean distance, indicating the pixels Hedi pixels the distance between them;
[0039] Estimate the image based on the thermal radiation effect and determine its pixel grayscale vector ;in, represents vectorization, The pixel gray value matrix of the image is used to estimate the thermal radiation effect;
[0040] The weight coefficients are solved by minimizing the following regularized objective function :
[0041]
[0042] Weight coefficient Analytical solution of for:
[0043]
[0044] Where, is the fitting error, which means that the combination of basis functions To approximate the observed data G Error; Represents a regular term to prevent overfitting, is the regularization coefficient; I is the identity matrix.
[0045] In the above scheme, the weight coefficient obtained by the solution is multiplied by the RBF kernel matrix to generate the RBF thermal radiation effect surface, including:
[0046] The weight coefficient obtained and RBF kernel matrix Multiply to generate a one-dimensional estimate vector of thermal radiation :
[0047]
[0048] The thermal radiation one-dimensional estimate vector Restore to image matrix ,Right now:
[0049]
[0050] Where, The matrix Rank The value of the element at column position, is the one-dimensional estimated value vector of thermal radiation Middle elements;
[0051] The RBF thermal radiation effect surface is generated and the thermal radiation effect layer is obtained after surface fitting.
[0052] In the above scheme, based on the degraded image and the RBF thermal radiation effect surface, the stripe noise is estimated through the stripe operator, including:
[0053] Based on grayscale degraded image And RBF thermal radiation effect surface , calculate the residual image ; The residual image Contains stripe noise S and high-frequency details of the image itself;
[0054] In order to extract the stripe noise S, the optimization problem form using the nuclear norm plus the quadratic term as the regularization term is as follows:
[0055]
[0056] Where, is the nuclear norm, which represents the sum of the singular values of the matrix; It represents the square root of the sum of the squares of the matrix elements, which is the extension of the Euclidean norm in the matrix space. Its purpose is to measure the fitting error between the current estimate and the observation. is the regularization coefficient of the nuclear norm, is the penalty parameter; It is a dual variable, which is an all-zero matrix initially and will be updated gradually during the iterative update.
[0057] Convert the above formula to standard form, that is, by introducing the matrix and Convert the above equation into the following standard optimization problem:
[0058]
[0059] The optimal solution is given by the following formula:
[0060]
[0061] Where, is the singular value soft threshold operator, which represents the SVD decomposition of the matrix R;
[0062] Then perform soft threshold processing on the singular values, the steps are:
[0063] Perform singular value decomposition on matrix R: ;in, A represents the left singular vector of the matrix R, Vrepresents the right singular vectors of the matrix R, is a diagonal matrix of singular values;
[0064] This gives all the singular values , and then soft threshold it:
[0065]
[0066] Where, is the singular value after soft thresholding, is the nuclear norm regularization weight, which controls the degree of compression of the strip;
[0067] Get the new singular value diagonal matrix :
[0068]
[0069] Reconstruction Matrix S :
[0070]
[0071] This results in stripe noise S .
[0072] In the above scheme, based on the RBF thermal radiation effect surface and stripe noise, a synchronous progressive correction optimization algorithm is used to restore the potential clear image, including:
[0073] The original problem to be solved is determined to be:
[0074]
[0075] Where, For potential clear images;
[0076] Clarify the problem goal, that is, the original optimization problem:
[0077]
[0078] Where, and To constrain B to be smooth and constrain S to be low-rank, and are two regularization parameters, where is the regularization coefficient of the nuclear norm;
[0079] Introducing variable splitting, that is, introducing dual variables and , construct the Lagrangian function, and the optimization objective of the subproblem of F is:
[0080]
[0081] The above formula is a quadratic optimization problem with a closed-form solution. Derivative F is calculated and its gradient is set to 0. The first gradient is set to , the second gradient is , the third gradient is , add up the three terms to make them equal to 0:
[0082]
[0083] That is:
[0084]
[0085] Determine whether the convergence conditions are met as follows:
[0086] In the k In the iteration of round, let the original residual for:
[0087]
[0088] Dual residual for:
[0089]
[0090] in Indicates stacking two matrices into a long matrix by row;
[0091] The convergence condition is:
[0092]
[0093] and:
[0094]
[0095] in n is the total number of image pixels, is the matrix norm (Frobenius norm), and For the set absolute and relative tolerances, Indicates taking the maximum value, is the penalty parameter;
[0096] If this iteration meets the above convergence conditions, the potential clear image F obtained in this iteration is output; if the above convergence conditions are not met, the iteration is continued;
[0097] In each iteration, the dual variables are updated:
[0098]
[0099] In the k+1th iteration, the current residual and dual variables are used to construct the subproblem of B:
[0100]
[0101] in, is RBF surface fitting; is the current residual image;
[0102] The update of S has the following subproblems:
[0103]
[0104] In each round, B and S are re-estimated and updated by minimizing the subproblem. Then, based on B and S in each round, the corresponding latent clear image F is obtained, and it is judged whether the convergence condition is met. The process continues until the convergence condition is met and the latent clear image F obtained in this iteration is output.
[0105] In the above scheme, based on the RBF thermal radiation effect surface and stripe noise, a synchronous progressive correction optimization algorithm is used to restore the potential clear image, which also includes:
[0106] The potential clear image output for each iteration , calculate its NIQE index, recorded as , if:
[0107]
[0108] Then take the potential clear image output by this iteration as the new grayscale degraded image, use the same method to obtain the corresponding potential clear image, and continue to judge whether it satisfies the above formula;
[0109] Otherwise, the potential clear image output by the previous iteration is output as the final output :
[0110]
[0111] Where, is the minimum threshold, For the n The convergence condition is satisfied and the potential clear image output is obtained; and the NIQE formula is as follows:
[0112]
[0113] Where, For images F The characteristic mean vector of For images F The characteristic covariance matrix of is the mean vector of the natural image model, is the covariance matrix of the natural image model;
[0114] Finally Perform normalization processing to obtain the final clear image; the normalization formula is:
[0115]
[0116] Where, For images No. Rank The pixel value of the column, For images The maximum pixel value in For images The minimum pixel value in For the final clear image.
[0117] In the above solution, the method further includes:
[0118] Before constructing the RBF kernel matrix, the thermal radiation effect estimation image is downsampled;
[0119] After generating the RBF thermal radiation effect surface, the RBF thermal radiation effect surface is upsampled to restore it to its original size.
[0120] According to a second aspect of the present invention, a computer device is provided, comprising: a processor and a memory, the memory storing programs or instructions executable on the processor, wherein when the programs or instructions are executed by the processor, the steps of the method for synchronous progressive correction of thermal effects and stripe noise based on RBF surfaces according to any one of the first aspects are implemented.
[0121] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the synchronous progressive correction method for thermal effects and stripe noise based on RBF surfaces described in any one of the first aspects are implemented.
[0122] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:
[0123] The present invention provides a synchronous and progressive correction method for thermal effects and stripe noise based on RBF surfaces. By continuously determining whether the thermal radiation effect and stripe noise of iteratively degraded images are significant, the algorithm is dynamically adjusted, significantly reducing the time required for calculation. The advantage of RBF surface fitting lies in its flexibility, especially for handling non-uniformly distributed data and high-frequency components. These data precisely meet the non-uniform distribution characteristics of thermal radiation, resulting in excellent denoising effects. Furthermore, the choice of RBF kernel function can adjust the smoothness of the fit to accommodate different thermal radiation patterns.
[0124] Furthermore, the optimization algorithm employed is suitable for solving distributed optimization problems. It decomposes the optimization objectives of RBF and SVD, enabling parallel processing to improve efficiency. Traditional methods may process thermal radiation and striping separately, leading to error accumulation. However, alternating direction optimization allows for simultaneous processing, reducing mutual interference and improving overall accuracy. While RBF is computationally intensive, combining this method with downsampling can significantly reduce computational complexity.
[0125] In addition, the present invention is simple to implement and has generalizability. It can complete the task of removing thermal radiation and stripe noise effects with high quality, greatly improving the image signal-to-noise ratio. The residual thermal radiation effect and stripe noise residual of the corrected image are no longer obvious, solving the problem that the existing aerodynamic-optical effect correction methods are either complicated to implement or still have relatively obvious residual effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0126] Figure 1 A schematic flow chart of a synchronous progressive correction method for thermal effects and stripe noise based on RBF surfaces provided in an embodiment of the present invention;
[0127] Figure 2 An RBF surface fitting effect diagram provided by an embodiment of the present invention; wherein, Figure 2 (a) is the thermal radiation and stripe degradation image after grayscale processing. Figure 2 (b) in the figure is the image after guided filtering. Figure 2 (c) in the figure is the estimated strip operator. Figure 2 (d) is the thermal radiation effect image after RBF surface fitting;
[0128] Figure 3 A downsampled image provided by an embodiment of the present invention;
[0129] Figure 4 A thermal radiation effect and stripe noise correction diagram provided by an embodiment of the present invention with progressive iterations from 1 to 30 times;
[0130] Figure 5 An aerodynamic optical thermal radiation correction effect diagram provided by an embodiment of the present invention; wherein, Figure 5(a) is the thermal radiation and stripe degradation image. Figure 5 (b) is the clear original image. Figure 5 (c) in the figure is the correction image after gradual iteration. Figure 5 (d) in the figure is the low-rank curve graph during the image correction process;
[0131] Figure 6 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0132] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments provided by the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present invention.
[0133] Obviously, the drawings described below are merely examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar scenarios based on these drawings without inventive effort. Furthermore, it is understood that while the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the disclosure of the present invention, any design, manufacturing, or production changes based on the technical content disclosed in the present invention are merely conventional technical means and should not be construed as an inadequacy of the disclosure of the present invention.
[0134] References to "embodiments" in this disclosure mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the disclosure. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this disclosure may be combined with other embodiments, unless there is a conflict.
[0135] Unless otherwise defined, technical or scientific terms used in this disclosure shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this disclosure pertains. As used herein, the terms "a," "an," "an," "the," and similar expressions do not denote limitations on quantity and may refer to either the singular or the plural. As used herein, the terms "comprise," "include," "have," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. As used herein, the terms "connect," "connected," "coupled," and similar expressions are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, the term "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may represent: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in the present invention are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0136] This invention provides a synchronous and progressive correction method for thermal effects and stripe noise based on RBF surfaces. The method relates to the fields of surface fitting, stripe noise, and aero-optical thermal radiation effect correction. Specifically, it provides a synchronous and progressive correction method for non-uniform thermal radiation degradation and stripe noise based on RBF surface fitting. This method avoids the risk of existing methods, where removing a single noise often affects the removal of subsequent noise types. Furthermore, it achieves the same effect when removing a single noise type, without compromising the final image quality.
[0137] The invention discloses a synchronous progressive correction method for thermal effect and stripe noise based on RBF surface, comprising the following steps: grayscale processing of a degraded image, recording the image size, and adjusting the image; applying guided filtering to the adjusted image to obtain a low-frequency degraded image; downsampling the image to obtain a smaller image; calculating the RBF kernel matrix, solving the RBF weights, and calculating stripe-related parameters; generating a target network for interpolation to generate an RBF thermal radiation surface, and obtaining a thermal radiation effect layer after surface fitting; restoring the surface to the original image size through interpolation, and similarly estimating the stripe noise through a stripe operator; restoring a preliminary potential clear image through synchronous progressive correction of the restored thermal radiation surface and the stripe noise; if some thermal radiation remains, using the potential clear image obtained by this iteration and its filtered image as input for the next iteration, continuing surface fitting and stripe removal, updating the thermal radiation effect layer and the potential clear image until the algorithm converges and the original image size is restored, thereby obtaining a clear image without residual thermal radiation effect and stripe noise. The invention is simple to implement and efficient, and can perform image correction on aero-optical effect degradation images. By implementing this method, the thermal radiation effect of the image and the possible accompanying stripe noise can be effectively removed, thereby improving the image quality and signal-to-noise ratio.
[0138] like Figure 1 As shown, the synchronous progressive correction method for thermal effect and stripe noise based on RBF surface in an embodiment of the present invention includes the following steps:
[0139] S1. Obtain a degraded image under thermal radiation conditions, perform grayscale processing on the degraded image, record the image size, and adjust the image to facilitate calculation;
[0140] S2, performing guided filtering on the processed image to obtain a filtered image;
[0141] S3, downsampling the filtered image to facilitate calculation using less data;
[0142] S4, calculate the RBF kernel matrix, solve the RBF weight and calculate the strip-related parameters;
[0143] S5, generating a target network for interpolation to generate an RBF thermal radiation effect surface;
[0144] S6, restoring the surface to a thermal effect surface of the same size as in step S2 through interpolation, and estimating the stripe noise through a stripe operator;
[0145] S7. Restore the preliminary potential clear image by fitting the restored thermal radiation surface and the stripe noise synchronous progressive correction optimization algorithm f ;
[0146] S8. If some thermal radiation still remains, the latent clear image obtained in this iteration is used as the input for the next iteration. RBF surface fitting and strip clipping are continued, and the thermal radiation effect layer and the latent clear image are updated until the algorithm converges and is restored to the original size to obtain a clear image f without residual thermal radiation effect and strip noise.
[0147] In some embodiments, the method in step S1 is specifically as follows:
[0148] For degraded images z Perform grayscale processing to obtain a grayscale degraded image ,like Figure 2 As shown in (a) in the figure, its size is recorded as H×W and it is scaled to a square matrix with both length and width equal to the short side for ease of calculation. The subsequent verification in this embodiment continues to use H×W to make the height and width clear.
[0149] In some embodiments, the method in step S2 is specifically:
[0150] Grayscale degradation map (Size is ) Use guided filtering for smoothing to obtain the filtered thermal radiation effect estimation image s, as shown in Figure 2 As shown in (b) in .
[0151] The degradation map is used as both the input image and the guidance image. , define a window centered on it as , assuming the size is , for the pixels in it, define a local mean :
[0152]
[0153] in, is the total number of pixels in the window, equal to , For the window j The grayscale value of a pixel.
[0154] Then calculate the local variance .
[0155] Under the premise that the guide image is equal to the input image z, the local linear coefficient is (there are two):
[0156]
[0157]
[0158] in, is the smoothing parameter, which is 50 here.
[0159] For all pixels in the entire image, and smooth:
[0160]
[0161]
[0162] Where, and are the two local linear coefficients after smoothing.
[0163] The final output filtered image s:
[0164]
[0165] Where, Estimate the thermal radiation effect in the image The pixel value of each pixel, is the grayscale degraded image The pixel value of pixels.
[0166] In some embodiments, the method of downsampling in step S3 is specifically:
[0167] remember s Size , downsample it as Figure 3 As shown. First determine the coordinates of each pixel value of the target image, the target image The coordinates of are extracted proportionally from the original image, where the pixel positions are ,in Represents the row index, Represents the column index. Let the pixel position of the target image be , then the corresponding original image position calculation formula is as follows:
[0168]
[0169]
[0170] Among them, [] means rounding. Then interpolation calculation is performed, for each pixel in the target image , bilinear interpolation determines the value of the target pixel based on the weighted average of the four surrounding original image pixels.
[0171] Given the pixel location of the target image , which corresponds to the original image pixel position is , let the coordinates of the four adjacent pixels in the original image be: top left: , upper right: , lower left: , lower right: , then the bilinear interpolation formula is:
[0172]
[0173] in, are the four neighboring pixel values of the original image. x 、 y is the pixel in the original image corresponding to the target image The floating point position of , , , Interpolation estimates the pixel value in the target image by taking a weighted sum of these four pixel values.
[0174] In this embodiment, the image is effectively downsampled to 1 / 6 of its original size in this way while retaining the image details, such as Figure 3 shown.
[0175] In some embodiments, the method of step S4 is specifically as follows:
[0176] The estimated image of thermal radiation effect after downsampling in step S3 is recorded as ,in h and w are 1 / 6 of the length and width of the original degraded image, that is, , .
[0177] In order to represent the two-dimensional position relationship of each pixel, a position coordinate matrix is defined again:
[0178]
[0179] in, Represents the spatial coordinates of the i-th pixel.
[0180] Constructing Gaussian kernel matrix based on RBF kernel function , the elements in this matrix are defined as follows:
[0181]
[0182] Among them, exp is the exponential function, which means , is the square of the Euclidean distance, indicating the i Hedi j The position difference between pixels, x , y is the coordinate. It is a shape parameter that controls the width of the kernel function and takes a value according to the specific situation (a larger value indicates local correlation, a smaller value indicates global correlation, and a smaller value is taken in this method). This formula reflects the Indicates the j The RBF center is i The response value of each pixel decreases exponentially as the distance between the two points increases.
[0183] Given a pixel grayscale vector , the weight coefficient is solved by minimizing the following regularized objective function :
[0184]
[0185] In the formula, argmin refers to the variable that makes the function achieve the minimum value The value of . is the fitting error, which means that the combination of basis functions To approximate the observed data G The error also represents the square of the Euclidean distance and the square of the L2 norm. Represents a regular term to prevent overfitting, is the regularization coefficient, which is used to control the weight.
[0186] The weight coefficient The analytical solution is:
[0187]
[0188] Where, I is the identity matrix.
[0189] Pixel grayscale vector middle, is an operation as follows:
[0190]
[0191] It converts the matrix into a one-dimensional vector in column-major order.
[0192] During the solution process, due to the global smoothness of the RBF kernel function, the estimated thermal radiation weight also implicitly includes the low-frequency structure of the strip noise, providing prior support for subsequent strip modeling.
[0193] In some embodiments, the method in step S5 is specifically as follows:
[0194] The weights obtained above Multiplying with the kernel matrix, reconstructing the thermal radiation effect surface, and generating a one-dimensional thermal radiation estimation value vector .
[0195] Then the estimated heat radiation vector Restore to image matrix ,Right now:
[0196]
[0197] Where, The matrix Rank The value of the element at column position, is the one-dimensional estimated value vector of thermal radiation Middle elements.
[0198] The estimated heat radiation vector Restore to image matrix The column-major recovery rule is used:
[0199] First column: pixels are ;
[0200] Second column: ; ......
[0201] Column w: .
[0202] The final restored thermal radiation image matrix is:
[0203]
[0204] The RBF thermal radiation effect surface is generated and the thermal radiation effect layer is obtained after surface fitting.
[0205] In some embodiments, the method in step S6 is specifically:
[0206] By bilinear interpolation as in step S3 Upsample back to the original size to obtain the thermal radiation estimation image B, as shown in Figure 2 As shown in (d) in the figure.
[0207] First, for each pixel of the target image , mapping it to the location of the low-resolution image:
[0208]
[0209] in, i 、 j For floating point bits, that is, they may fall on between two pixels.
[0210] Then interpolate to find the position The four surrounding pixel values (upper left, upper right, lower left, lower right) are weighted averaged, with the weights determined by the distance:
[0211]
[0212] in, is the target pixel coordinate of the high-resolution image, that is, the coordinate of B, Represents relative offset, used to enumerate the four surrounding pixels (upper left, upper right, lower left, lower right); represents the interpolation weight, representing the low-resolution image Medium pixels The influence weight on the target pixel.
[0213] For example, suppose , that is, the floating point bit is offset to the upper left, where due to i and j For floating point bits, is the coordinate of the upper left integer pixel point after rounding down, that is , Represents round down. Upper left, For the upper right, is the lower left, is the lower right, these are The value of .
[0214] Then the thermal radiation estimation image B is transformed from the grayscale degraded image The residual image is deducted from This residual image contains the stripe noise S and the high-frequency details of the image itself.
[0215] In order to extract the stripe noise component S, the optimization problem form using the nuclear norm (low rank constraint) plus the quadratic term as the regularization term is as follows:
[0216]
[0217] in, is the sum of the singular values of the nuclear norm representation matrix; It represents the square root of the sum of the squares of the matrix elements, which is the expansion of the Euclidean norm in the matrix space. Its purpose is to measure the fitting error between the current estimate and the observation.
[0218] Then convert it to standard format and import and (Nuclear norm regularization weight, adaptive parameter adjustment, initial value is 15) converted to the following standard optimization problem:
[0219]
[0220] The above formula represents a least squares problem with nuclear norm regularization, which has a closed-form solution, namely the singular value soft threshold decomposition.
[0221] The optimal solution can be given by the following formula:
[0222]
[0223] in, It is a singular value soft threshold operator, which represents the SVD decomposition of the matrix R.
[0224] Then perform soft threshold processing on the singular values, the steps are:
[0225] Perform singular value decomposition on R: ;in U represents the left singular vector of the matrix R (not the residual image mentioned above), and , r is the rank of the matrix R; is a diagonal matrix of singular values, V is a right singular vector, and .
[0226] Then we get all the singular values , and then soft threshold it:
[0227]
[0228] Where, is the nuclear norm regularization weight, which controls the degree of compression of the strip. This formula represents if the singular value Greater than , just subtract Then keep it, if it is less than , it is considered as noise and set it to 0.
[0229] Then it is processed and reassembled into a diagonal matrix:
[0230]
[0231] Finally, the matrix S is reconstructed to obtain the stripe noise, such as Figure 2 As shown in (c):
[0232]
[0233] In the above formula, To optimize the dual variables in the algorithm, the initial value is an all-zero matrix, which will be gradually updated in subsequent iterative updates. It is the penalty parameter in the optimization algorithm, and a fixed value of 50 is taken here. is the regularization coefficient of the nuclear norm, which is usually 10.
[0234] In some embodiments, the method in step S7 is specifically:
[0235] First we need to solve the original problem:
[0236]
[0237] Where, F For a potentially clear image.
[0238] Then clarify the problem goal, that is, the original optimization problem:
[0239]
[0240] in ， and To constrain B to be smooth and constrain S to be low-rank, this algorithm has been completed previously. and Are two regularization parameters. In this algorithm, is 1. Same as above , is the regularization coefficient of the nuclear norm.
[0241] Then variable splitting is introduced, that is, the dual variable is introduced and , construct the Lagrangian function, and the optimization objective of the subproblem of F is:
[0242]
[0243] This is a quadratic optimization problem with a closed-form solution. F Take the derivative and set its gradient to 0, and set the first gradient to , the second gradient is , the third gradient is .
[0244] Add the three terms together to equal 0:
[0245]
[0246] That is:
[0247]
[0248] Now we have obtained the potential clear image F, and then we determine whether it meets the convergence conditions.
[0249] The inner iteration in this optimization algorithm, i.e., the convergence judgment, has the following conditions:
[0250] If this round is k Round, let the original residual for:
[0251]
[0252] Dual residual for:
[0253]
[0254] in Indicates stacking two matrices row by row into one long matrix.
[0255] The convergence condition is:
[0256]
[0257] and:
[0258]
[0259] in n is the total number of image pixels, is the matrix norm (Frobenius norm), and For the set absolute and relative tolerances, Indicates taking the maximum value, As mentioned above.
[0260] If the above conditions are met, the potential clear image F is output; if not, the optimization algorithm is used to perform synchronization correction through iteration:
[0261] In each iteration, the dual variable is updated first:
[0262]
[0263] In the k+1th iteration, the current residual and duality are used to construct the subproblem of B:
[0264]
[0265] in, For RBF surface fitting, as in the above steps. is the current residual image.
[0266] The update of S has the following subproblems:
[0267]
[0268] Update the dual variables and After obtaining B and S, the same method is used to obtain the potential clear image F of this iteration, and then whether the convergence condition is met is continuously judged until the potential clear image F of this iteration is output.
[0269] B and S are re-estimated in each round and updated by minimizing the sub-problems. The output image of the specific iteration is as follows: Figure 4 After the iteration is completed, the strip noise image recorded in each iteration is compressed into a low-rank representation to obtain a low-rank curve, which reflects the low-rank characteristics of the strip, as shown in Figure 5 (d) in.
[0270] In some embodiments, the method in step S8 is specifically as follows:
[0271] Determine the iterative convergence of the latent clear image F obtained in each iteration. That is, determine whether the latent clear image F outputted by this iteration still has residual thermal radiation or striping traces. If residual thermal radiation or striping traces are still present, use the current latent clear image F as the new input image and repeat steps S2 to S7 until convergence is achieved.
[0272] The NIQE indicator is used for the convergence judgment of the latent clear image F (i.e., whether the latent clear image F still has residual thermal radiation or stripe traces), and the optimization algorithm is iterated at least once internally to meet the generation of the stripe matrix. , calculate the current potential clear image The NIQE index is denoted as , if:
[0273]
[0274] Then, the potential clear image output by the current iteration is used as a new input image, and steps S2 to S7 are repeated.
[0275] Otherwise, the iteration ends and the image of the previous iteration is output as the final output:
[0276]
[0277] in, is the minimum threshold, set to , and the NIQE formula is as follows:
[0278]
[0279] Where, is the feature mean vector of the input image F, is the feature covariance matrix of image F, is the mean vector of the natural image model, is the covariance matrix of the natural image model.
[0280] The convergence of the latent clear image F is determined by the two potential clear images that meet the convergence conditions. The clear image obtained in step S7 is judged by the NIQE indicator. When the indicator stops optimizing or even degrades compared with the previous value, the loop stops and the image is judged to have been restored. The restored clear image is normalized and restored to its original size (real H×W). The final output clear image f is as follows: Figure 5 As shown in (c), compared with Figure 5 The degraded image shown in (a) and Figure 5 The original clear image shown in (b) shows that this method has a significant correction effect. The normalization formula is:
[0281]
[0282] in, The first i Rank j The pixel value of the column. is the minimum pixel value in the image. Is the maximum pixel value in the image. The normalized pixel value will be scaled to the range [0, 255].
[0283] The above method is used to determine whether the final clear image still has residual thermal radiation or stripe traces; if so, the final clear image is subjected to guided filtering processing to obtain a new thermal radiation effect estimation image, and this process is repeated until the final clear image has no residual thermal radiation or stripe traces.
[0284] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0285] In addition, combined Figure 1 The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to the embodiment of the present invention can be implemented by a computer device. Figure 6 FIG. 1 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Figure 6 As shown, the device may include a processor 301 and a memory 302 storing computer program instructions.
[0286] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present invention.
[0287] Memory 302 may include a large-capacity memory for data or instructions. By way of example, and not limitation, memory 302 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the data processing device. In certain embodiments, memory 302 is non-volatile memory. In certain embodiments, memory 302 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.
[0288] The memory 302 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 301 .
[0289] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any one of the synchronous progressive correction methods for thermal effect and stripe noise based on the RBF surface in the above embodiments.
[0290] In some embodiments, the computer device may further include a communication interface 303 and a bus 300. Figure 6 As shown, the processor 301 , the memory 302 , and the communication interface 303 are connected via a bus 300 and communicate with each other.
[0291] The communication interface 303 is used to implement communication between the various modules, devices, units, and / or equipment in the embodiments of the present invention. The communication interface 303 can also implement data communication with other components such as external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.
[0292] The bus 300 includes hardware, software, or both, and couples the components together. The bus 300 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example, and not limitation, bus 300 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 300 may comprise one or more buses, where appropriate. Although embodiments of the present invention describe and illustrate a particular bus, the present invention contemplates any suitable bus or interconnect.
[0293] The computer device can execute the synchronous progressive correction method of thermal effect and strip noise based on RBF surface in the embodiment of the present invention, thereby realizing the combination of Figure 1 A synchronous progressive correction method for thermal effect and stripe noise based on RBF surface is described.
[0294] In addition, in conjunction with the synchronous progressive correction method for thermal effects and stripe noise based on RBF surfaces in the above-mentioned embodiments, embodiments of the present invention may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when executed by a processor, the computer program instructions implement any of the synchronous progressive correction methods for thermal effects and stripe noise based on RBF surfaces in the above-mentioned embodiments.
[0295] It should be noted that the various technical features of the above-described embodiments can be combined in any manner. To simplify the description, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there are no contradictions in the combination of these technical features, they should be considered to be within the scope of this specification. In addition, as needed for implementation, the various steps / components described in the present invention can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the objectives of the present invention.
[0296] Those skilled in the art will readily understand that the above-described embodiments merely represent several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the patent. It should be noted that those skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, and these modifications and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.
Claims
1. A synchronous progressive correction method for thermal effect and stripe noise based on RBF surface, characterized in that: The method includes: Acquire a degraded image; Performing grayscale processing on the degraded image to obtain a grayscale degraded image; Perform guided filtering on the grayscale degraded image to obtain a filtered thermal radiation effect estimation image; The RBF kernel matrix is constructed based on the thermal radiation effect estimation image and the RBF kernel function, and the weight coefficient is solved by combining the pixel grayscale vector of the thermal radiation effect estimation image; Multiply the weight coefficient obtained by the solution with the RBF kernel matrix to generate the RBF thermal radiation effect surface; Based on the degraded image and RBF thermal radiation effect surface, the stripe noise is estimated through the stripe operator; According to the RBF thermal radiation effect surface and stripe noise, the potential clear image is restored through the synchronous progressive correction optimization algorithm.
2. The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to claim 1, characterized in that: Perform guided filtering on the grayscale degraded image to obtain a filtered thermal radiation effect estimation image, including: For each pixel in the grayscale degraded image , define a point centered on it and of size Window , and calculate the local mean: Where, For the The local mean of pixels; is the total number of pixels in the window, equal to ; Indicated in pixels i The center window j The grayscale value of each pixel; Calculate the local variance based on the local mean: Where, For the The local variance of pixels; Calculate the local linear coefficient based on the local mean and local variance: Where, and are two local linear coefficients, is the smoothing parameter; Will and smooth: Where, and are the two local linear coefficients after smoothing; Output filtered thermal radiation effect estimation image s : Where, Estimate the thermal radiation effect in the image The pixel value of pixels, is the grayscale degraded image The pixel value of pixels.
3. The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to claim 1, characterized in that: The RBF kernel matrix is constructed based on the thermal radiation effect estimation image and the RBF kernel function, and the weight coefficient is solved by combining the pixel grayscale vector of the thermal radiation effect estimation image, including: Define the pixel position coordinate matrix of the thermal radiation effect estimation image : Where, Estimate the thermal radiation effect in the image The position coordinates of the pixels, and Estimate the height and width of the image for thermal radiation effects, The total number of pixels in the image is estimated for thermal radiation effects; Constructing RBF kernel matrix based on RBF kernel function , the elements in the matrix The definition is as follows: Where, is an exponential function, ; is the shape parameter; is the square of the Euclidean distance, indicating the pixels Hedi pixels the distance between them; Estimate the image based on the thermal radiation effect and determine its pixel grayscale vector ;in, represents vectorization, The pixel gray value matrix of the image is used to estimate the thermal radiation effect; The weight coefficients are solved by minimizing the following regularized objective function : Weight coefficient Analytical solution of for: Where, is the fitting error, which means that the combination of basis functions To approximate the observed data G Error; Represents a regular term to prevent overfitting, is the regularization coefficient; I is the identity matrix.
4. The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to claim 3, characterized in that: Multiply the weight coefficient obtained by the solution with the RBF kernel matrix to generate the RBF thermal radiation effect surface, including: The weight coefficient obtained and RBF kernel matrix Multiply to generate a one-dimensional estimate vector of thermal radiation : The thermal radiation one-dimensional estimate vector Restore to image matrix ,Right now: Where, The matrix Rank The value of the element at column position, is the one-dimensional estimated value vector of thermal radiation Middle elements; The RBF thermal radiation effect surface is generated and the thermal radiation effect layer is obtained after surface fitting.
5. The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to claim 1, characterized in that: Based on the degraded image and the RBF thermal radiation effect surface, the stripe noise is estimated through the stripe operator, including: Based on grayscale degraded image And RBF thermal radiation effect surface , calculate the residual image ; The residual image Contains stripe noise S and high-frequency details of the image itself; In order to extract the stripe noise S, the optimization problem form using the nuclear norm plus the quadratic term as the regularization term is as follows: Where, is the nuclear norm, which represents the sum of the singular values of the matrix; It represents the square root of the sum of the squares of the matrix elements, which is the extension of the Euclidean norm in the matrix space. Its purpose is to measure the fitting error between the current estimate and the observation. is the regularization coefficient of the nuclear norm, is the penalty parameter; It is a dual variable, which is an all-zero matrix initially and will be updated gradually during the iterative update. Convert the above formula to standard form, that is, by introducing the matrix and Convert the above equation into the following standard optimization problem: The optimal solution is given by the following formula: Where, is the singular value soft threshold operator, which represents the SVD decomposition of the matrix R; Then perform soft threshold processing on the singular values, the steps are: Perform singular value decomposition on matrix R: ;in, A represents the left singular vector of the matrix R, V represents the right singular vector of the matrix R, is a diagonal matrix of singular values; This gives all the singular values , and then soft threshold it: Where, is the singular value after soft thresholding, is the nuclear norm regularization weight, which controls the degree of compression of the strip; Get the new singular value diagonal matrix : Reconstruction Matrix S : This results in stripe noise S .
6. The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to claim 5, characterized in that: Based on the RBF thermal radiation effect surface and stripe noise, a potential clear image is restored through a synchronous progressive correction optimization algorithm, including: The original problem to be solved is determined to be: Where, For potential clear images; Clarify the problem goal, that is, the original optimization problem: Where, and To constrain B to be smooth and constrain S to be low-rank, and are two regularization parameters, where is the regularization coefficient of the nuclear norm; Introducing variable splitting, that is, introducing dual variables and , construct the Lagrangian function, and the optimization objective of the subproblem of F is: Derivative F, and set its gradient to 0, let the first gradient be , the second gradient is , the third gradient is , add up the three terms to make them equal to 0: Determine whether the convergence conditions are met as follows: In the k In the iteration of round, let the original residual for: Dual residual for: in Indicates stacking two matrices into a long matrix by row; The convergence condition is: and: in n is the total number of image pixels, is the matrix norm, and For the set absolute and relative tolerances, Indicates taking the maximum value, is the penalty parameter; If this iteration meets the above convergence conditions, the potential clear image F obtained in this iteration is output; if the above convergence conditions are not met, the iteration is continued; In each iteration, the dual variables are updated: In the k+1th iteration, the current residual and dual variables are used to construct the subproblem of B: in, is RBF surface fitting; is the current residual image; The update of S has the following subproblems: In each round, B and S are re-estimated and updated by minimizing the subproblem. Then, based on B and S in each round, the corresponding latent clear image F is obtained, and it is judged whether the convergence condition is met. The process continues until the convergence condition is met and the latent clear image F obtained in this iteration is output.
7. The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to claim 6, characterized in that: Based on the RBF thermal radiation effect surface and stripe noise, a synchronous progressive correction optimization algorithm is used to restore the potential clear image, including: The potential clear image output for each iteration , calculate its NIQE index, recorded as , if: Then take the potential clear image output by this iteration as the new grayscale degraded image, use the same method to obtain the corresponding potential clear image, and continue to judge whether it satisfies the above formula; Otherwise, the potential clear image output by the previous iteration is output as the final output : Where, is the minimum threshold, For the n The convergence condition is satisfied and the potential clear image output is obtained; and the NIQE formula is as follows: Where, For images F The characteristic mean vector of For images F The characteristic covariance matrix of is the mean vector of the natural image model, is the covariance matrix of the natural image model; Finally Perform normalization processing to obtain the final clear image; the normalization formula is: Where, For images No. Rank The pixel value of the column, For images The maximum pixel value in For images The minimum pixel value in For the final clear image.
8. The synchronous progressive correction method for thermal effect and stripe noise based on RBF surface according to claim 1, characterized in that: The method further includes: Before constructing the RBF kernel matrix, the thermal radiation effect estimation image is downsampled; After generating the RBF thermal radiation effect surface, the RBF thermal radiation effect surface is upsampled to restore it to its original size.
9. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the synchronous progressive correction method for thermal effects and stripe noise based on RBF surfaces as described in any one of claims 1 to 8 are implemented.
10. A computer-readable storage medium, characterized in that Programs or instructions are stored thereon, and when the programs or instructions are executed by the processor, the steps of the synchronous progressive correction method for thermal effects and stripe noise based on RBF surfaces as described in any one of claims 1 to 8 are implemented.