A hyperspectral image denoising method, device and storage medium
Through the improved weighted variational tensor decomposition method, multiple mixed noises in hyperspectral images are processed, and the problem of ignoring spatial correlation in the prior art is solved, and better image recovery and visual effects are achieved.
Patent Information
- Application Number
- CN202210365831.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-08
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-04-08
AI Technical Summary
Existing hyperspectral image denoising methods are difficult to effectively deal with a variety of mixed noises, especially ignoring spatial correlation, resulting in unsatisfactory recovery results.
Using the improved weighted variational tensor decomposition method, the hyperspectral image data is converted into third-order tensors, using low-rank tensor decomposition and weighted full variational regularizer, combined with L1 norm and F norm, through augmentation The Lagrangian multiplication method is iteratively optimized to build an efficient denoising model.
It significantly improves the image recovery effect, improves structural similarity and peak signal-to-noise ratio, enhances the smoothness of the spectral space, and significantly improves the visual effect of the image.
Smart Images

Figure CN114820352B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image processing, and relates to a hyperspectral image denoising method, a device and a storage medium based on improved weighted variational tensor decomposition. Background Art
[0002] Hyperspectral images have rich spatial and spectral structural information and are widely used in military, urban, aerospace and other fields. However, during the acquisition process, images are contaminated by various types of noise, such as Gaussian, salt and pepper, and stripe noise, which seriously degrades the quality of hyperspectral images. Therefore, it is necessary to denoise hyperspectral images and restore images close to the original clear images from degraded images.
[0003] A natural way to denoise hyperspectral images is to treat each band as a grayscale image, and then use traditional two-dimensional or one-dimensional denoising methods to denoise band by band; subsequent methods use the similarities and spatial characteristics of adjacent image pixels to achieve spatial segmentation smoothness through total variation regularization methods, process the edge information of the image, and improve the accuracy of image restoration. These methods are all aimed at removing one or two types of noise, namely Gaussian noise, impulse noise, etc. However, in the process of hyperspectral acquisition, it is usually corrupted by several different types of noise, such as Gaussian noise, impulse noise, dead lines, stripes, etc. Although a method for eliminating noise mixing has been proposed based on low-rank matrix modeling, the restoration effect is not ideal. Summary of the invention
[0004] Objective: Most of the image denoising algorithms in the prior art use low-rank restoration through the similarity of adjacent bands, which are second-order and ignore spatial correlation. In order to solve this problem, the present invention provides a hyperspectral image denoising method, device and storage medium based on improved weighted variational tensor decomposition, which aims to solve various mixed noises in the hyperspectral image acquisition process.
[0005] Converting multidimensional hyperspectral data into vectors or matrices usually destroys the spectral spatial structural correlation. Tensor-based modeling technology has more advantages than matrix technology. The third-order tensor is used to simultaneously capture the non-local similarity and spectral correlation of the spectral space. The tensor-based method basically maintains the inherent structural correlation and has better recovery results.
[0006] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:
[0007] In a first aspect, a hyperspectral image denoising method is provided, comprising:
[0008] Acquire the hyperspectral image data to be denoised;
[0009] The hyperspectral image data to be denoised is input into the pre-trained and optimized hyperspectral image denoising model to obtain the output denoised hyperspectral image result;
[0010] The method for constructing the hyperspectral image denoising model includes:
[0011] 1) Use low-rank tensor decomposition to separate the noise term and obtain a hyperspectral degradation model;
[0012] 2) Adding an improved weighted total variation regularizer (w-SSTV) to the hyperspectral degradation model obtained in step 1) to fully characterize the spatial and spectral correlations between the hyperspectral image bands;
[0013] 3) In the model obtained in step 2), the sparse noise is regulated by the L1 norm, the Gaussian noise is regulated by the F norm, and the constraint conditions are determined;
[0014] 4) The model obtained in step 3) is solved by using the augmented Lagrange multiplier method, and the model is iteratively optimized by inputting training set images with sparse noise and Gaussian noise to obtain a trained and optimized hyperspectral image denoising model.
[0015] In some embodiments, the hyperspectral degradation model comprises:
[0016] Y=X+N+S
[0017] Where Y represents the noisy third-order tensor hyperspectral cube Y = {Y 1 ,Y 1 ,…Y B}, where Y i ∈R h×w , h represents height, i represents frequency band, width is w, B represents the number of frequency bands; X represents a clean image, N represents Gaussian noise, S represents sparse noise, and X, N, S and Y have the same tensor size.
[0018] In some embodiments, the noise term is separated using low-rank tensor decomposition, including:
[0019] The hyperspectral image is divided into overlapping three-dimensional blocks. For an n-order tensor, it is decomposed into n factor matrices and a core tensor by Tucker decomposition to make full use of the spectral and spatial low-rank characteristics of the image; the factor matrix on each mode is called the basis matrix or principal component of the tensor; the Tucker decomposition equation is expressed as:
[0020] X=C×1U1×2U2×…× n U n ,U n T U n =I
[0021] Where C is the control factor matrix The core tensor of the interaction between them; U is the coefficient matrix.
[0022] In some embodiments, an improved weighted total variation regularizer is added to the hyperspectral degradation model, including:
[0023] ||X|| SSTV =w1||D x X||+w2||D y X||+w3||D z X||
[0024] Among them, ||X|| SSTV is the weighted total variation model, D x ,D y ,D z They represent the first-order forward finite difference operators along the horizontal direction x, the vertical direction y and the spectral direction z respectively; w1, w2 and w3 represent the weighted difference operators in the three directions x, y and z respectively;
[0025] D x =X(i+1,j,k)-X(i,j,k)
[0026] D y =X(i,j+1,k)-X(i,j,k)
[0027] D z =X(i,j,k+1)-X(i,j,k)
[0028] In X(i,j,k), i and j represent the spatial positions of the image X in the horizontal and vertical directions respectively, and k represents the kth band of the image X.
[0029] In some embodiments, the sparse noise is regulated by the L1 norm, the Gaussian noise is regulated by the F norm, and the constraint conditions are determined, including:
[0030]
[0031] stY=X+S+N,
[0032] X=C×1U1×2U2×3U3,U n T U n =I
[0033] Where X represents a clean image, N represents Gaussian noise, S represents sparse noise, τ, λ, β represent the control coefficient factors of X, S, and N respectively, st represents the constraints of the model; C is the core tensor of the interaction between the control factor matrices U1, U2, and U3; U is the coefficient matrix.
[0034] In some embodiments, the augmented Lagrange multiplier method is used to solve, including:
[0035] Let X = Z, D w (Z)=F,D w It is a weighted three-dimensional difference operator. There are three first-order difference operators in different directions. Z and F both represent auxiliary variables introduced in the augmented Lagrangian multiplier method. The augmented Lagrangian function L is expressed as follows:
[0036]
[0037] Where μ is the penalty parameter, μ1, μ2, μ3 represent the augmented Lagrange multipliers.
[0038] In some embodiments, the model is iteratively optimized by inputting training set images with sparse noise and Gaussian noise, including:
[0039] First, input the training set images with sparse noise and Gaussian noise, the matrix rank r, the weight values w1, w2, w3, initialize the hyperspectral image X, and according to the augmented Lagrangian multiplier law, X=Z=S=N=0, μ1=μ2=μ3=0, k=0, and then update X, Z, F, S, N and the augmented Lagrangian multipliers μ1, μ2, μ3, respectively, and optimize the model for k+1 iterations until the iteration stop condition is met.
[0040] In a second aspect, the present invention provides a hyperspectral image denoising device, including a processor and a storage medium;
[0041] The storage medium is used to store instructions;
[0042] The processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.
[0043] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.
[0044] Beneficial effects: The hyperspectral image denoising method and device provided by the present invention have the following advantages: the hyperspectral image denoising method based on weighted variational tensor decomposition has better image restoration effect in removing multiple mixed noises compared to the image denoising method, and the structural similarity and peak signal-to-noise ratio are significantly improved, which can improve the smoothness of the spectral space and obtain better effect in image vision. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 The figure is a flow chart of a method for constructing a hyperspectral image denoising model according to one embodiment of the present invention. DETAILED DESCRIPTION
[0046] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0047] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0048] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0049] Example 1
[0050] A hyperspectral image denoising method is provided, comprising:
[0051] Acquire the hyperspectral image data to be denoised;
[0052] The hyperspectral image data to be denoised is input into the pre-trained and optimized hyperspectral image denoising model to obtain the output denoised hyperspectral image result;
[0053] The method for constructing the hyperspectral image denoising model includes:
[0054] 1) Use low-rank tensor decomposition to separate the noise term and obtain a hyperspectral degradation model;
[0055] 2) Adding an improved weighted total variation regularizer (w-SSTV) to the hyperspectral degradation model obtained in step 1) to fully characterize the spatial and spectral correlations between the hyperspectral image bands;
[0056] 3) In the model obtained in step 2), the sparse noise is regulated by the L1 norm, the Gaussian noise is regulated by the F norm, and the constraint conditions are determined;
[0057] 4) The model obtained in step 3) is solved by using the augmented Lagrange multiplier method, and the model is iteratively optimized by inputting training set images with sparse noise and Gaussian noise to obtain a trained and optimized hyperspectral image denoising model.
[0058] In some embodiments, Figure 1 An improved hyperspectral image denoising method based on weighted total variation tensor decomposition is shown, comprising the following steps:
[0059] Step S1, using low-rank tensor decomposition to separate noise terms and obtain a degraded hyperspectral model; specifically,
[0060] First, the hyperspectral image is segmented into overlapping three-dimensional blocks. For an n-order tensor, it is decomposed into n factor matrices and a core tensor by Tucker decomposition to fully utilize the spectral and spatial low-rank characteristics of the image. The factor matrix on each modality is called the basis matrix or principal component of the tensor. Tucker decomposition can also be considered as a high-order PCA, which is used to describe the low-rank tensor approximation. The Tucker decomposition equation is expressed as:
[0061] X=C×1U1×2U2×…× n U n ,U n T U n =I
[0062] C is the control factor matrix The core tensor of the interaction between them. U is the coefficient matrix. The classical high-order orthogonal iteration (HOOI) algorithm is used to achieve low-rank Tucker decomposition without increasing the amount of calculation, and the hyperspectral degradation model is obtained:
[0063] Y=X+N+S
[0064] Y represents the noisy third-order tensor of the hyperspectral cube Y = {Y 1 ,Y 1 ,…Y B}, where Y i ∈R h×w, h represents height, i represents frequency band, width is w, and B represents the number of frequency bands; X represents a clean image, N and S represent Gaussian noise and sparse noise respectively, and they have the same tensor size as Y.
[0065] Step S2: adding an improved weighted total variation regularizer (w-SSTV); specifically,
[0066] Hyperspectral images have a strong local smooth structure in their spectral modes. That is, the differences between adjacent frequency bands in the spectral domain are mostly close to zero, and the number of zeros is significantly greater than that in the spatial domain. The SSTV regularization algorithm is used to study the piecewise smooth structure in the spatial and spectral domains.
[0067] Introducing the anisotropic total variation model ||X|| SSTV , where ||X|| SSTV =||D x X||+||D y X||+||D z X||,D x ,D y ,D z represent the first-order forward finite difference operators along the horizontal direction, vertical direction and spectral direction of space, respectively.
[0068] D x =X(i+1,j,k)-X(i,j,k)
[0069] D y =X(i,j+1,k)-X(i,j,k)
[0070] D z =X(i,j,k+1)-X(i,j,k)
[0071] In X(i,j,k), i and j represent the horizontal and vertical spatial positions of image X respectively, and k represents the kth band of image X;
[0072] We weight the model w to get: ||X|| SSTV =w1||D x X||+w2||D y X||+w3||D z X||. The weight controls the regularization speed of X. w1, w2, w3 represent the weighted difference operators in the x, y, and z directions respectively.
[0073] Step S3, use L1 norm to normalize sparse noise, and use F norm to normalize Gaussian noise; specifically, the S sparse noise is modeled by L1 norm, and the N Gaussian noise is modeled by F norm to obtain the following expression:
[0074]
[0075] stY=X+S+N,
[0076] X=C×1U1×2U2×3U3,U n T U n =I
[0077] τ, λ, β represent the control coefficient factors of X, S, and N respectively, and st represents the constraint conditions of the model, which fully obtains the similarity of spectral image pixels and the piecewise smoothness of spatial spectrum, reduces artifacts, and removes mixed noise.
[0078] Step S4: image denoising is performed by using the augmented Lagrange multiplier method. Specifically,
[0079] Optimize the model obtained in step S3 above, and let X = Z, D w (Z)=F,D w It is a weighted three-dimensional difference operator. There are three first-order difference operators in different directions. Its ALM model expression is as follows:
[0080]
[0081] μ is the penalty parameter, μ1, μ2, μ3, represents the augmented Lagrange multiplier. Denoising algorithm: First input the image with Gaussian and sparse noise, the matrix rank r, the weight value w, the iteration stop criterion, etc., initialize the hyperspectral image X, according to the augmented Lagrange multiplier law X = Z = S = N = 0, μ1 = μ2 = μ3 = 0, k = 0, and then update X, Z, F, S, N and update the Lagrange multipliers μ1, μ2, μ3, etc., fix other variables, and iterate the model k+1 times.
[0082] The mixed noise experiment is carried out on the Indian dataset, and the final output is the hyperspectral image restoration result.
[0083] Tables 1-2 below show the comparison of the effects of the image denoising method of the present invention and the traditional prior art under two experiments:
[0084] Experiment 1: Gaussian noise and salt and pepper impulse noise are added to all bands of the indian dataset. The variance of Gaussian noise (G) is 0.02, 0.06, and 0.1 respectively. At the same time, salt and pepper impulse noise (P) is also added to all bands to simulate sparse noise. The percentage of impulse noise (P) is 0.04, 0.12, and 0.2 respectively.
[0085] Experiment 2: Based on Experiment 1, deadline noise is added, and other parameters remain unchanged.
[0086] The proposed method and the comparative method are used to restore three different simulated observation data. The SSIM and PSNR of all channels of the restoration results are averaged, recorded as MSSIM and MPSNR, and these two means are used as the evaluation criteria for the final restoration effect. Peak signal-to-noise ratio (PSNR) is an error-sensitive image quality evaluation. Given a clean image X and a noisy image Y of a certain size, PSNR is defined as:
[0087]
[0088]
[0089] M, N represent the spatial size of the hyperspectral image, i.e., width and height, i, j represent the spatial position, and B represents the number of bands of the hyperspectral image. The larger the value of the peak signal-to-noise ratio (PNSR), the smaller the image distortion, and the closer the restored image is to the real image.
[0090] Structural similarity SSIM is defined as:
[0091]
[0092]
[0093] μ u ,μ u′ Represents the pixel averages of image Y and image X, σ u ,σ u′ represents the variance of the image, C1 and C2 are constants, and B represents the number of bands of the hyperspectral image. The value range of structural similarity SSIM is [0,1]. The larger the value, the smaller the image distortion and the better the image restoration effect.
[0094] Table 1 Experiment 1 results on the Indian dataset
[0095]
[0096]
[0097] Table 2 Experiment 2 results on the Indian dataset
[0098]
[0099] Among them, the model in the present invention is replaced by OURS, and the evaluation criteria are MSSIM and MPSNR (structural similarity and peak signal-to-noise ratio). Table 1 shows different image denoising methods respectively. As Gaussian noise and impulse noise increase, the image restoration effect will decrease. It can be seen that the model of the present invention performs well in the Indian dataset. Table 2 shows the restoration effect of three kinds of mixed noise in hyperspectral images. It can be seen that the model in this paper performs well, indicating that the influence of multiple mixed noises on the image is well solved. Compared with traditional restoration methods, the method provided by the present invention has significant competitive advantages, so the method performs well in solving the hyperspectral dataset with mixed noise and has certain significance.
[0100] Example 2
[0101] In a second aspect, this embodiment provides a hyperspectral image denoising device, including a processor and a storage medium;
[0102] The storage medium is used to store instructions;
[0103] The processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.
[0104] Example 3
[0105] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.
[0106] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.
[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0110] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A hyperspectral image denoising method, characterized in that: include: Acquire the hyperspectral image data to be denoised; The hyperspectral image data to be denoised is input into the pre-trained and optimized hyperspectral image denoising model to obtain the output denoised hyperspectral image result; The method for constructing the hyperspectral image denoising model includes: 1) Use low-rank tensor decomposition to separate the noise term and obtain a hyperspectral degradation model; 2) Adding an improved weighted total variation regularizer to the hyperspectral degradation model obtained in step 1) to fully characterize the spatial and spectral correlations between the hyperspectral image bands; 3) In the model obtained in step 2), the sparse noise is regulated by the L1 norm, the Gaussian noise is regulated by the F norm, and the constraint conditions are determined; 4) The model obtained in step 3) is solved by using the augmented Lagrange multiplier method, and the model is iteratively optimized by inputting training set images with sparse noise and Gaussian noise to obtain a trained and optimized hyperspectral image denoising model.
2. The hyperspectral image denoising method according to claim 1, characterized in that: The hyperspectral degradation model comprises: Y=X+N+S Where Y represents the noisy third-order tensor hyperspectral cube Y = {Y 1 ,Y 1 ,…Y B }, where Y i ∈R h×w , h represents height, i represents frequency band, width is w, B represents the number of frequency bands; X represents a clean image, N represents Gaussian noise, S represents sparse noise, and X, N, S and Y have the same tensor size.
3. The hyperspectral image denoising method according to claim 1 or 2, characterized in that: The noise terms are separated using low-rank tensor decomposition, including: The hyperspectral image is divided into overlapping three-dimensional blocks. For an n-order tensor, it is decomposed into n factor matrices and a core tensor by Tucker decomposition to make full use of the spectral and spatial low-rank characteristics of the image; the factor matrix on each mode is called the basis matrix or principal component of the tensor; the Tucker decomposition equation is expressed as: X=C×1U1×2U2×…× n U n ,U n T U n =I Where C is the control factor matrix The core tensor of the interaction between them; U is the coefficient matrix.
4. The hyperspectral image denoising method according to claim 1, characterized in that: An improved weighted total variation regularizer is added to the hyperspectral degradation model, including: ||X|| SSTV =w1||D x X||+w2||D y X||+w3||D z X|| Among them, ||X|| SSTV is the weighted total variation model, D x ,D y ,D z They represent the first-order forward finite difference operators along the horizontal direction x, the vertical direction y and the spectral direction z respectively; w1, w2 and w3 represent the weighted difference operators in the three directions x, y and z respectively; D x =X(i+1,j,k)-X(i,j,k) D y =X(i,j+1,k)-X(i,j,k) D z =X(i,j,k+1)-X(i,j,k) In X(i,j,k), i and j represent the spatial positions of the image X in the horizontal and vertical directions respectively, and k represents the kth band of the image X.
5. The hyperspectral image denoising method according to claim 1, characterized in that: Use the L1 norm to normalize sparse noise, use the F norm to normalize Gaussian noise, and determine the constraints, including: stY=X+S+N, X=C×1U1×2U2×3U3,U n T U n =I Where X represents a clean image, N represents Gaussian noise, S represents sparse noise, τ, λ, β represent the control coefficient factors of X, S, and N respectively, st represents the constraints of the model; C is the core tensor of the interaction between the control factor matrices U1, U2, and U3; U is the coefficient matrix.
6. The hyperspectral image denoising method according to claim 5, characterized in that: The solution is obtained by using the augmented Lagrange multiplier method, including: Let X = Z, D w (Z)=F,D w It is a weighted three-dimensional difference operator. There are three first-order difference operators in different directions. Z and F both represent auxiliary variables introduced in the augmented Lagrangian multiplier method. The augmented Lagrangian function L is expressed as follows: Where μ is the penalty parameter, μ1, μ2, μ3 represent the augmented Lagrange multipliers.
7. The hyperspectral image denoising method according to claim 6, characterized in that: The model is iteratively optimized by inputting training set images with sparse noise and Gaussian noise, including: First, input the training set images with sparse noise and Gaussian noise, the matrix rank r, the weight values w1, w2, w3, initialize the hyperspectral image X, and according to the augmented Lagrangian multiplier law, X=Z=S=N=0, μ1=μ2=μ3=0, k=0, and then update X, Z, F, S, N and the augmented Lagrangian multipliers μ1, μ2, μ3 respectively, and optimize the model for k+1 iterations until the iteration stop condition is met.
8. A hyperspectral image denoising device, characterized in that: include: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 7.
9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Hyperspectral image denoising method and system based on adaptive rank correction, and medium
CN110458777A
Hyperspectral image denoising method and device based on tensor loop decomposition
CN111598795A
Hyperspectral image denoising method based on non-convex low-rank matrix approximation and total variation regularization
CN112069919A
Method of reconstructing magnetic resonance image data
US20190346522A1
