Variational optimization multi-type noise removal method based on harmonic analysis
A variational optimization method using harmonic analysis and statistical moment matching addresses stripe noise and color aberration in hyperspectral images, enhancing image quality and detail preservation.
Patent Information
- Application Number
- CN202411101253.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-12-03
AI Technical Summary
Current methods struggle to effectively remove complex noise, such as stripe noise and uneven color differences, from hyperspectral remote sensing images, which affects image quality and subsequent tasks like segmentation and classification.
A variational optimization method based on harmonic analysis is employed, using gradient constraints, harmonic decomposition, and statistical moment matching to correct stripe noise and color aberration while preserving image details.
The method effectively removes stripe noise and color aberration, enhancing image quality and ensuring the retention of spatial and spectral details, improving the performance of hyperspectral image processing.
Smart Images

Figure CN119068204A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of satellite remote sensing technology, and in particular to a variational optimization multi-type noise removal method and device based on harmonic analysis. Background Art
[0002] With the rapid development of imaging spectrometers, hyperspectral remote sensing images have become an important means of obtaining information on the earth's surface because they can continuously cover the visible light to short-wave infrared bands, usually contain hundreds of narrow bands, can accurately depict the spectral characteristic curves of ground objects, and provide rich spectral information. However, in the process of acquiring and processing hyperspectral remote sensing images, the sources of noise in hyperspectral remote sensing images are complex, including multiple links such as the atmosphere, sensors, and data transmission. These noises can cause image distortion, spectral shape changes, and reduced classification accuracy, which seriously affect the accurate interpretation of surface cover types, biological parameters, etc. Therefore, improving the noise removal of hyperspectral remote sensing images has important scientific value.
[0003] How to use the mathematical characteristics and physical meaning of hyperspectral remote sensing images and degradation noise, consider the difference between the target object image information and the noise interference information, accurately construct a highly robust complex noise removal model, and achieve complex noise removal of hyperspectral remote sensing images is a current research problem for improving the quality of remote sensing images based on their own information. Existing deep learning methods have strong spectral preservation capabilities, but weak ability to enhance the spatial details of the image interior and edges.
[0004] Hyperspectral data collected by imaging spectrometers are often affected by banding noise and inconsistent pixel responses. These challenges hinder subsequent tasks such as segmentation, classification, and quantitative inversion. Current optimization-based variational methods have difficulty in effectively removing complex irregular noise, especially image banding caused by CCD stitching.
[0005] In summary, there is currently a lack of a noise removal method to solve or partially solve the above-mentioned problems. Summary of the invention
[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a variational optimization multi-type noise removal method based on harmonic analysis to solve or partially solve the problems of stripe noise pollution and uneven color difference of image blocks in order to overcome the defects of the above-mentioned prior art.
[0007] The purpose of the present invention can be achieved by the following technical solutions:
[0008] One aspect of the present invention provides a variational optimization multi-type noise removal method based on harmonic analysis, comprising the following steps:
[0009] Step S1, obtaining the original image to be denoised, and calculating the step distance corresponding to the original image;
[0010] Step S2, based on the gradient constraints of the ideal noise-free image in the horizontal and vertical directions, and the difference constraint between the column mean curve of the ideal noise-free image and the column mean curve of the harmonic decomposition and the elimination of oscillation, solve and obtain the order moment corresponding to the ideal noise-free image;
[0011] Step S3, obtaining a denoised image by distance matching based on the distances of the original image and the ideal noise-free image.
[0012] As a preferred technical solution, in step S2, the gradient constraint R1(X) of the ideal noise-free image in the horizontal direction is:
[0013]
[0014] Where X is an ideal noise-free image, is the partial differential operator in the horizontal direction, represents the gradient value of an ideal noise-free image in the horizontal direction, and ||·||1 represents the l1 norm.
[0015] As a preferred technical solution, in step S2, the gradient constraint R2(X) in the vertical direction of the ideal noise-free image is:
[0016]
[0017] Where X is an ideal noise-free image, is the partial differential operator in the vertical direction, and They represent the gradient values of the ideal noise-free image and the original image in the horizontal direction, respectively, and ||·||1 represents the l1 norm.
[0018] As a preferred technical solution, in step S2, the difference constraint G(X) between the column mean curve of the ideal noise-free image and the column mean curve of the harmonic decomposition and oscillation elimination is:
[0019]
[0020] g = f(t)
[0021] Among them, g is the column mean curve after eliminating oscillation, f∈R n×1 is a column vector where all elements are 1 / n, Xf is the column mean curve of the ideal noise-free image, X is the ideal noise-free image, and ||·||2 is the L2 norm.
[0022] As a preferred technical solution, in step S2, the process of solving the ideal noise-free image includes the following steps:
[0023] Step S201, performing augmented Lagrangian expansion on the gradient constraint and the difference constraint to simplify them into a ZTX subproblem, where Z and T are auxiliary variables, X is an ideal noise-free image, and solving the expression of the auxiliary variables by soft threshold shrinkage;
[0024] Step S202, solving the expression of an ideal noise-free image;
[0025] Step S203, updating the Lagrange multiplier, and iteratively calculating to obtain an ideal noise-free image.
[0026] As a preferred technical solution, in step S202, the expression of the ideal noise-free image is solved by fast Fourier transform, analytical solution, gradient descent method or conjugate gradient method.
[0027] As a preferred technical solution, the step moment includes a first-order moment and a second-order moment.
[0028] As a preferred technical solution, in step S3, the denoised image is obtained using the following formula:
[0029]
[0030] Among them, X ij Represents the element value of the original image in the jth column and the ith row, s_origin ij and m_o raw ij Represents the first-order moment and second-order moment value of the original image at the corresponding position, s_new ij and m_new ij represents the optimized first-order moment and second-order moment values, is the element value of the denoised image in the jth column and the ith row.
[0031] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned variational optimization multi-type noise removal method based on harmonic analysis.
[0032] Another aspect of the present invention provides a computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned variational optimization multi-type noise removal method based on harmonic analysis.
[0033] Compared with the prior art, the present invention has at least one of the following beneficial effects:
[0034] Realize multi-type noise removal: To address the problems of stripe noise pollution and uneven color difference in image blocks, the present invention establishes a framework based on statistical moment matching, introduces harmonic analysis to guide the fluctuating image to a relatively uniform fundamental level, and uses a variational optimization algorithm to mine the mathematical characteristics of the image itself, thereby realizing the synchronous correction of stripe noise and uneven color difference in banded blocks of multi-source remote sensing images, while ensuring the retention of feature details in remote sensing images. Analysis of qualitative visual effect evaluation and quantitative indicators shows that this method exhibits strong performance in processing various images and is robust and practical. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a flow chart of the variational optimization multi-type noise removal method based on harmonic analysis in the embodiment. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0037] Example 1
[0038] In view of the mixed pollution problem of banded block chromatic aberration and strip noise in remote sensing images, this embodiment provides a variational optimization multi-type noise removal method based on harmonic analysis, introduces harmonic analysis technology, and constructs a variational optimization method of the moment matching framework. Based on the statistical distribution matching principle, the consistency of image spatial features is guaranteed, a gradient domain prior constraint is created to construct an ideal image energy functional, and harmonic decomposition is used as a constraint to guide the algorithm correction performance, while correcting chromatic aberration and strip noise.
[0039] See also Figure 1 , this method first establishes the framework of the moment matching method, making full use of the advantages of statistical distribution matching, which not only ensures the uniformity of the image processing results, but also reduces the computational complexity of variational optimization. Secondly, the harmonic analysis technology is introduced in the variational optimization process and the corresponding constraints are constructed, and the block chromatic aberration is regarded as sinusoidal interference for separation. Finally, combined with the gradient analysis method, the gradient prior constraint is constructed according to the spatial domain characteristics of the image, while eliminating the strip noise and maintaining the integrity of the image details. Through the iterative solution method, the energy functional is optimized to complete the prediction of the ideal noise-free image. The following will explain this method in detail.
[0040] (1) Construct the original image degradation model.
[0041] Hyperspectral data collected by imaging spectrometers are often affected by banding noise and inconsistent pixel responses. These challenges hinder subsequent tasks such as segmentation, classification, and quantitative inversion. Current optimization-based variational methods are difficult to effectively remove complex irregular noise, especially the image banding phenomenon caused by CCD stitching. In response to the above problems, this embodiment adopts a framework based on the moment matching method and uses statistical characteristics to solve these problems. In addition, the algorithm of this embodiment introduces a more rigorous mathematical model for the variational method and combines harmonic analysis in signal processing. The latter helps eliminate abnormal grayscale fluctuations between bands through harmonic decomposition, thereby obtaining the basic amplitude of the image grayscale and restoring its grayscale value to the basic level, solving the problem of banding inhomogeneity. The noise degradation model of the proposed method can be expressed as:
[0042] I=X+Y
[0043] Where I∈R m×n Represents the original image, the dimension is m rows and n columns, X∈R m×n refers to the noise-free image data, and Y is the degraded part of the original image. The noise part can be further expressed as follows:
[0044] Y=S+N
[0045] In the above formula, S represents the degraded part of the stripe noise in the image, and N represents the degradation factor of the image heterogeneity.
[0046] (2) Construct a variational optimization model.
[0047] By combining the gradient characteristics of the image space domain with the characteristics of the processing results of the harmonic analysis method, the final energy functional is generated by creating prior constraints as follows:
[0048] X = argminE(X)
[0049]
[0050] The specific construction process is as follows:
[0051] 1) Construction of spatial domain constraints.
[0052] Considering that the features in an ideal image usually have spatial correlation in natural space, the image elements in adjacent areas in the image space often have similar reflectivity or radiance, and the gradient changes of similar objects are usually slow. In this embodiment, considering that the ideal noise-free image X has a smooth characteristic in the horizontal direction, the constraints are constructed as follows:
[0053]
[0054] In the above formula, the l1 norm can be used to constrain the prior conditions of the ideal image horizontal gradient. Considering that the existence of stripe noise will cause large grayscale mutations on both sides of the noise position, the l1 norm is more tolerant to the impact of mutations than other norm constraints, and therefore can better meet the requirements of this experiment.
[0055] Considering that the stripe noise in the image is along the vertical direction, its existence destroys the overall smoothness of the image in the horizontal direction, but the image still maintains a certain smoothness in the vertical direction. At the same time, the ideal noise-free image is also smooth in the vertical direction, so the following prior constraints are constructed:
[0056]
[0057] In the formula, as well as They represent the partial differential operators in the horizontal and vertical directions, respectively, which are used to calculate the gradient. It represents the gradient value of the ideal image in the horizontal direction. and They represent the gradient values of the ideal image and the original image in the horizontal direction respectively. Through this constraint, the ideal image finally solved can obtain the horizontal gradient information of the original image that is not affected by the stripe noise.
[0058] 2) Construction of constraints guided by harmonic analysis.
[0059] The image to be corrected has obvious uneven striping phenomenon, and this uneven phenomenon is reflected in the segmented sudden change of the image column average curve. At the same time, the influence of striping noise on the column average curve is manifested as obvious up and down oscillations in the curve. The column average curve of an ideal image will be relatively smooth. Taking the above characteristics into consideration, the harmonic analysis method is introduced to decompose the segmented sudden change in the column average curve into a sine wave at the fundamental level and its multiple harmonic frequencies that interfere. Through the harmonic decomposition operation, the curve will be uniformly restored to the fundamental level, which is manifested in the image as the correction of the image block color aberration phenomenon is initially completed. The processing flow can be described as follows:
[0060] g = f(t)
[0061] In the formula, g is the column mean curve after eliminating oscillation, and f∈R n×1 is a column vector, in which all elements are 1 / n, and Xf is the column mean curve of the predicted ideal image. And this function can be expressed by Fourier series through harmonic decomposition:
[0062]
[0063] Where ω=2π / T, n is a positive integer. When n=1, c1 is the fundamental amplitude of the curve. Through harmonic analysis, the oscillation on the curve is removed and restored to a smooth column average curve. In addition, the obtained curve undergoes a simple filtering process to eliminate the noise fluctuations on the curve. The column average curve obtained after processing can be used to guide the removal of bands by constructing the corresponding constraints as follows:
[0064]
[0065] This constraint can effectively reduce the difference between the column mean curve of the ideal noise-free image to be solved and the column mean curve after harmonic decomposition to eliminate oscillation.
[0066] 3) Iterative optimization of energy functional.
[0067] Combined with the previously constructed constraints, the energy functional of the entire variational optimization can be completed. Since the solution to this problem belongs to an equation-constrained optimization problem with multiple optimization variables, it can be quickly solved by the alternating direction method of multipliers (ADMM). The specific steps are as follows:
[0068] First, introduce auxiliary variables Z and T, then the above formula can be rewritten as follows:
[0069]
[0070] Introducing the Lagrange multiplier p and the penalty parameter ρ, the augmented Lagrangian expansion of the above formula can be written as:
[0071]
[0072] At this point, the original problem can be simplified to the sub-problems of ZTX, and the solutions to these sub-problems can be solved by alternating calculations. The problems of Z and T can be solved by soft threshold shrinkage (reference), and the specific solution process is as follows:
[0073]
[0074]
[0075] The solution to the ideal image X is a quadratic minimization problem, which can be solved by analytical methods, gradient descent method, conjugate gradient method, etc. This paper uses fast Fourier transform to solve it, and the corresponding calculation process is as follows:
[0076]
[0077] During the iterative calculation process, the Lagrange multiplier also needs to be iteratively updated. The specific calculation formula is as follows:
[0078]
[0079] (4) Construction of moment matching method framework.
[0080] In an ideal image, since it is not contaminated by stripe noise, most of its information can be concisely represented by fewer important features or structures. It is reasonable to assume that each column of the ideal image shows smooth characteristics. At the same time, the ideal image also has a certain smoothness in the direction perpendicular to the direction of stripe noise. In contrast, an image that is severely disturbed by stripe noise will experience perturbations in its first-order moment distribution. The mean and variance of pixels used in the calculation are usually used to represent the first-order and second-order moments in the moment matching process. Considering that stripe noise is distributed in columns and the values of adjacent columns may show significant fluctuations, the first-order and second-order moments of the matrix are extracted. The process involves using the pixel averages of n rows and 1 column in the original matrix as the element values of the first-order matrix to generate the feature matrix of the moment. The second-order matrix is generated similarly.
[0081] In summary, in the actual implementation process, considering the unique characteristics of the ideal image and the image contaminated by noise, the newly generated first-order matrix and second-order matrix are optimized by the aforementioned variational method. This optimization process produces ideal first-order and second-order matrices, which will be used to guide the original image to denoise using the moment matching algorithm. The specific solution process is as follows:
[0082]
[0083] In the above formula, X ij Represents the element value of the original image in the jth column and the ith row, s_origin ij and m_original ij It represents the first-order moment and second-order moment value of the original image at that location. Similarly, s_new ij and m_new ij It represents the optimized first-order moment and second-order moment values. That is, the element value of the denoised image in the jth column and the ith row.
[0084] In summary, this method considers the causes of different types of noise in hyperspectral remote sensing images and their impact on image quality, thereby constructing a more efficient and accurate denoising algorithm and method. The optimized hyperspectral images will be improved in terms of spectral response, spatial resolution and signal-to-noise ratio, providing high-quality data for subsequent target detection, inversion analysis, etc. This method has the following advantages: This method establishes a framework based on statistical moment matching, introduces harmonic analysis to guide the fluctuating image to a relatively uniform fundamental level, and uses variational optimization algorithm to mine the mathematical characteristics of the image itself, realizing the synchronous correction of multi-source remote sensing image strip noise and band-shaped block chromatic aberration inhomogeneity, while ensuring the retention of feature details in remote sensing images. The analysis of qualitative visual effect evaluation and quantitative indicators shows that this method has strong performance in processing various images and is robust and practical.
[0085] Example 2
[0086] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the variational optimization multi-type noise removal method based on harmonic analysis as described in Example 1.
[0087] Example 3
[0088] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the variational optimization multi-type noise removal method based on harmonic analysis as described in Example 1.
[0089] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A variational optimization method for multi-type noise removal based on harmonic analysis, It is characterized in that The steps include: Step S1, obtaining the original image to be denoised, and calculating the step distance corresponding to the original image; Step S2, based on the gradient constraints of the ideal noise-free image in the horizontal and vertical directions, and the difference constraint between the column mean curve of the ideal noise-free image and the column mean curve of the harmonic decomposition and the elimination of oscillation, solve and obtain the order moment corresponding to the ideal noise-free image; Step S3, obtaining a denoised image by distance matching based on the distances of the original image and the ideal noise-free image.
2. According to claim 1, a variational optimization multi-type noise removal method based on harmonic analysis, It is characterized in that In step S2, the gradient constraint R of the ideal noise-free image in the horizontal direction is 1 (X) is: Where X is an ideal noise-free image, is the partial differential operator in the horizontal direction, represents the gradient value of an ideal noise-free image in the horizontal direction, ||·|| 1 Indicates l 1 Norm.
3. According to claim 1, a variational optimization multi-type noise removal method based on harmonic analysis, It is characterized in that In step S2, the vertical gradient constraint R of the ideal noise-free image 2 (X) is: Where X is an ideal noise-free image, is the partial differential operator in the vertical direction, and Represent the horizontal gradient values of the ideal noise-free image and the original image, respectively, ||·|| 1 Indicates l 1 Norm.
4. According to claim 1, a variational optimization multi-type noise removal method based on harmonic analysis, It is characterized in that In step S2, the difference constraint G(x) between the column mean curve of the ideal noise-free image and the column mean curve of the harmonic decomposition and oscillation elimination is: g = f(t) Among them, g is the column mean curve after eliminating oscillation, f∈R n×1 is a column vector, in which all elements are 1 / n, Xf is the column mean curve of the ideal noise-free image, x is the ideal noise-free image, ||·|| 2 is the L2 norm.
5. According to claim 1, a variational optimization multi-type noise removal method based on harmonic analysis, It is characterized in that In step S2, the process of solving the ideal noise-free image includes the following steps: Step S201, performing augmented Lagrangian expansion on the gradient constraint and the difference constraint to simplify them into a ZTX subproblem, where Z and T are auxiliary variables, X is an ideal noise-free image, and solving the expression of the auxiliary variables by soft threshold shrinkage; Step S202, solving the expression of an ideal noise-free image; Step S203, updating the Lagrange multiplier, and iteratively calculating to obtain an ideal noise-free image.
6. According to claim 5, a variational optimization multi-type noise removal method based on harmonic analysis, It is characterized in that In step S202, the expression of the ideal noise-free image is solved by fast Fourier transform, analytical solution, gradient descent method or conjugate gradient method.
7. The variational optimization multi-type noise removal method based on harmonic analysis according to claim 1, It is characterized in that The order moments include first-order moments and second-order moments.
8. The variational optimization multi-type noise removal method based on harmonic analysis according to claim 1, It is characterized in that In step S3, the denoised image is obtained using the following formula: Among them, X ij Represents the element value of the original image in the jth column and the ith row, s_original ij and m_original ij Represents the first-order moment and second-order moment value of the original image at the corresponding position, s_new ij and m_new ij represents the optimized first-order moment and second-order moment values, is the element value of the denoised image in the jth column and the ith row.
9. An electronic device, It is characterized in that include: One or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the variational optimization multi-type noise removal method based on harmonic analysis as described in any one of claims 1-8.
10. A computer-readable storage medium, It is characterized in that It comprises one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs comprise instructions for executing the variational optimization multi-type noise removal method based on harmonic analysis as claimed in any one of claims 1-8.
Citation Information
Patent Citations
Stripe removing method and device, electronic equipment and storage medium
CN110689494A
Stripe noise removal method based on multi-level image decomposition and multi-term sparse constraint representation
CN117152024A
Method and device for removing stripe noise of remote sensing data based on unidirectional variation, and medium
CN117173042A
