Hyperspectral image denoising method based on multi-task convolutional sparse coding network

Abstract

The application discloses a hyperspectral image denoising method based on a multi-task convolution sparse coding network, and comprises the following steps: S1, acquiring a hyperspectral image to be denoised; S2, modeling a denoising task through a convolution sparse representation model, and adding an L1 regular term to constrain the sparsity; S3, solving the denoising model through an iterative soft thresholding algorithm (ISTA) to obtain a solving formula; and S4, converting the solving formula into an end-to-end learnable deep neural network through a deep unfolding mode for optimization, wherein a loss function of the network is a mean square loss function, and a denoised image is obtained. The application proposes a key observation that images in different bands can share convolution sparse coefficients, realizes joint modeling of the spatial-spectral relationship of the hyperspectral image, overcomes the problem of needing to divide image blocks, and can directly take the whole image as input, thereby realizing hyperspectral image denoising. The application has the advantages of strong denoising capability, good applicability and the like.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing, and specifically relates to a method for denoising hyperspectral images. Background Technology

[0002] Hyperspectral imaging utilizes sensors with extremely high spectral resolution to capture a large number of narrow-bandwidth spectral images within the visible to near-infrared wavelength range. These images range in dimension from tens to hundreds of dimensions and contain significantly more object information compared to the common RGB three-band images. They play a crucial role in forestry, agriculture, geological surveys, environmental monitoring, and military reconnaissance. However, during image acquisition, factors such as imaging mechanisms, equipment errors, and the imaging environment can lead to noise in the acquired hyperspectral data, limiting the accuracy of subsequent tasks such as target detection, classification, and demixing. Therefore, hyperspectral image denoising is an indispensable preprocessing step in hyperspectral image processing.

[0003] Sparsity is an important property of images. Sparse representation utilizes this property to recover a clean image using linear combinations of atoms from an overcomplete dictionary. However, a common approach is to divide the image into small image patches and perform sparse representation separately in each patch, ignoring the spatial relationships between the patches. Furthermore, to utilize the information relationships between bands in hyperspectral images, stronger constraints need to be added to the sparse representation coefficients to achieve better results in hyperspectral image denoising. Summary of the Invention

[0004] To address the aforementioned shortcomings, this invention proposes a hyperspectral image denoising method based on a multi-task convolutional sparse coding network.

[0005] The technical solutions to achieve the above objectives are as follows:

[0006] A hyperspectral image denoising method based on a multi-task convolutional sparse coding network includes the following steps:

[0007] S1. Acquire a noisy hyperspectral image. Y is a hyperspectral image with B bands, length h, and width w, and noise level of [value missing]. Corresponding clean image The degradation model corresponding to the observed noisy image is:

[0008] Y = X + N#(1)

[0009] S2. Model the denoising task using a convolutional sparse representation model, and add an L1 regularization term to constrain the sparsity.

[0010] S3. The solution formula is obtained by solving the denoising model using the iterative soft-thresholding algorithm (ISTA).

[0011] S4. The solution formula is transformed into an end-to-end learnable deep neural network through deep unfolding to optimize it and obtain the denoised image;

[0012] Furthermore, the specific implementation method of step S2 is as follows:

[0013] For each band of the image, it can be sparsely represented by the convolution kernels in the convolution dictionary, and the formula is as follows:

[0014]

[0015]

[0016] in Represents the convolution operation. d represents a dictionary bm It is the m-th convolutional kernel corresponding to the b-th band image. The coefficients represent sparse representations, and different bands share the same convolutional sparse code S, ||s m ||1 represents the pair of s m Adding an L1 regularization term ensures sparsity. Substituting it into equation (1) yields the objective expression for hyperspectral image denoising under multi-task convolutional sparse prior:

[0017]

[0018] Y b λ represents the b-th band image of the input noisy image, B represents the number of bands in the hyperspectral image, and λ is the regularization coefficient.

[0019] Furthermore, the specific implementation method of step S3 is as follows:

[0020] With the dictionary D fixed, equation (3) is solved using the iterative soft-thresholding algorithm (ISTA):

[0021]

[0022] Where C is the step size, and soft(·) is the soft threshold shrinkage operator, which is defined as soft θ (x) = sign(x)·max(|x|-θ,0). After K iterations, we obtain S. K After that, the image after denoising. The image of band b It can be obtained from formula (5):

[0023]

[0024] Furthermore, the specific implementation method of step S4 is as follows:

[0025] make

[0026] Set parameter W d =[D1,…,D B ], W f =[D1,…,D B Update formulas (4) and (5) to obtain the multi-task convolutional sparse coding network:

[0027]

[0028]

[0029] Among them W d and W f It is implemented using convolution, while W e Implemented using transposed convolution. via ReLu(s-θ) k )-ReLu(-s-θ k The network uses the mean square error between the denoised and clean images as its loss function. The parameters are updated through backpropagation of the network, and finally the denoised hyperspectral image is output.

[0030]

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] (1) The present invention can extract features from a complete hyperspectral image without dividing the image into blocks, thereby making use of the spatial relationship between image blocks at different locations in the image.

[0033] (2) The present invention shares convolutional sparse representation coefficients between hyperspectral bands, which can realize the sharing of global information between bands, thereby improving the noise reduction capability. Attached Figure Description

[0034] Figure 1 This is a network structure diagram of the multi-task convolutional sparse coding network of the present invention.

[0035] Figure 2 It is a noisy image of band 20 of image ulm_0328-1118 in the ICVL dataset.

[0036] Figure 3 This is the denoising result corresponding to the noisy image of band 20 of image ulm_0328-1118 in the ICVL dataset.

[0037] Figure 4 This is a flowchart of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings.

[0039] like Figure 1 As shown, a hyperspectral image denoising method based on a multi-task convolutional sparse coding network includes the following steps:

[0040] S1. Acquire a noisy hyperspectral image. like Figure 2 The image shown is a noisy image of band 20 of image ulm_0328-1118 in the ICVL dataset. Y is a hyperspectral image with band number B, length h, and width w, and the noise level is [value missing]. Corresponding clean image The degradation model corresponding to the noisy image is:

[0041] Y = X + N#(1)

[0042] The greatest advantage of deep learning methods lies in learning more information from datasets. To train the network, the ICVL dataset was used. The ICVL dataset consists of a training set of 100 images and a test set of 50 images. Gaussian noise of different intensities was added to each band of the clean hyperspectral images to generate noise-clean image pairs, and the images were augmented by scaling, flipping, and other operations.

[0043] S2. Model the denoising task using a convolutional sparse representation model, and add an L1 regularization term to constrain the sparsity.

[0044] For each band of the image, it can be sparsely represented by the convolution kernels in the convolution dictionary, and the formula is as follows:

[0045]

[0046]

[0047] in Represents the convolution operation. d represents a dictionary bm It is the m-th convolutional kernel corresponding to the b-th band image. The coefficients represent sparse representations, and different bands share the same convolutional sparse code S, ||sm ||1 represents the pair of s m Adding an L1 regularization term ensures sparsity. Substituting it into equation (1) yields the objective expression for hyperspectral image denoising under multi-task convolutional sparse prior:

[0048]

[0049] Y b λ represents the b-th band image of the input noisy image, B represents the number of bands in the hyperspectral image, and λ is the regularization coefficient.

[0050] S3. The solution formula is obtained by solving the denoising model using the iterative soft-thresholding algorithm (ISTA).

[0051] With the dictionary D fixed, equation (3) is solved using the iterative soft-thresholding algorithm (ISTA):

[0052]

[0053] Where C is the step size, and soft(·) is the soft threshold shrinkage operator, which is defined as soft θ (x) = sign(x)·max(|x|-θ,0). After K iterations, we obtain S. K After that, the image after denoising. The image of band b It can be obtained from formula (5):

[0054]

[0055] S4. The solution formula is transformed into an end-to-end learnable deep neural network through deep unfolding to optimize it and obtain the denoised image;

[0056] make Set parameter W d =[D1,…,D B ], W f =[D1,…,D B Update formulas (4) and (5) to obtain the multi-task convolutional sparse coding network:

[0057]

[0058]

[0059] Among them Wd and W f It is implemented using convolution, while W e Implemented using transposed convolution. via ReLu(s-θ) k )-ReLu(-s-θ k The network structure is shown in the attached diagram. Figure 4 As shown, the network contains multiple encoding layers and one decoding layer. Setting the number of encoding layers to 7 yields the best results. The loss function used by the network is the mean squared error between the denoised and clean output images.

[0060]

[0061] After obtaining the loss, the parameters are updated through backpropagation of the network, and finally the denoised hyperspectral image is output. Figure 3 This is band 20 in the denoised image. It can be seen that the denoised image is clearer than the original noisy image, proving the effectiveness of this method.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1.A method for hyperspectral image denoising based on multi-task convolutional sparse coding, characterized in that, The method comprises the following steps: S1, obtaining a hyperspectral image containing noise Y is a hyperspectral image with B bands, length h, and width w, and the noise size is The corresponding clean image The degradation model corresponding to the noise image is: Y = X + N (1) S2, modeling the denoising task by a convolution sparse representation model, and adding an L1 regularization term to constrain the sparsity; S3, solving the denoising model by an iterative soft threshold shrinkage algorithm ISTA to obtain a solving formula; S4, converting the solving formula into an end-to-end learnable deep neural network by a deep unfolding method for optimization to obtain a denoised image; The specific implementation method of the step S2 is: For each waveband image, the image is represented by sparse representation of convolution kernels in a convolution dictionary, and the formula is as follows: wherein represents a convolution operation, denotes a set of M dictionaries, d bm is the mth convolution kernel corresponding to the bth band image, denotes a sparse representation coefficient, and the same convolution sparse code S is shared by different bands, ‖s m ‖1 denotes that s m An L1 regularization term is added to ensure sparsity; The formula (1) is brought into to obtain a target expression of hyperspectral image denoising under a multi-task convolution sparse prior: Y b B represents the b-th band image of the input noise image, B represents the number of bands of the hyperspectral image, and λ is a regularization term coefficient. The specific implementation method of the step S3 is: In the case of a fixed dictionary D, the formula (3) is solved by an iterative threshold shrinkage ISTA algorithm: where C is the step size, soft(·) is a soft threshold shrinkage operator defined as soft θ (x) = sign(x) · max(|x| - θ,0); S is obtained after K iterations K After denoising the image Where the b-th band image From equation (5) 2. The hyperspectral image denoising method based on multi-task convolution sparse coding according to claim 1, characterized in that, The specific implementation method of the step S4 is: Let Setting parameter W d = [D1,...,D B ], W f = [D1,...,D B ] update formula (4) (5) to get multi-task convolution sparse coding network: where W d and W f is implemented using convolution, e is implemented using transpose convolution, is implemented using ReLu(s-θ k )-ReLu(-s-θ k ), the loss function used by the network is the mean square error between the output denoised image and the clean image, the parameters are updated through the back propagation of the network, and finally the hyperspectral image after denoising is output. The loss function used is:

Images