Multispectral image denoising algorithm based on deep 3D convolutional sparse coding

Abstract

The present invention discloses a multispectral image denoising algorithm based on deep 3D convolutional sparse coding, comprising the following steps: step 1, according to the inter-spectral information of the multispectral image, the convolutional sparse coding model is expanded into a 3D form, and a 3D-CSC mathematical model is constructed; step 2, the constructed 3D-CSC model is iteratively solved using an iterative shrinkage soft threshold algorithm; step 3, a deep network is constructed: according to the idea of ​​deep expansion, the iterative solution form of the 3D-CSC in step 2 is expanded into a corresponding deep network; step 4, the network is trained after constructing a data set; step 5, the performance of the network is tested. The present invention uses 3D convolution and takes into account the inter-spectral characteristics of the image, so that the present invention achieves a better denoising effect in processing multispectral image problems.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a multispectral image denoising algorithm based on deep 3D convolutional sparse coding. Background Art

[0002] A multispectral image is an image containing multiple bands. It uses a specific imaging technology to divide the incident full-band or wide-band light signal into several narrow-band light beams, and then uses the same multi-band sensor for imaging to obtain a multispectral band image. Compared with grayscale images and ordinary color images, multispectral images have more bands and contain richer spectral information. They are widely used in many scenarios, such as crop detection, medical tumor positioning, industrial detection, etc. However, during the imaging process of multispectral images, they are often interfered by equipment and external environmental noise, resulting in noise in the collected images, which affects the accuracy of subsequent research, such as soil composition detection errors, inaccurate tumor positioning, and abnormal component inspection. Therefore, multispectral image denoising has important practical application value.

[0003] The task of image denoising is to reconstruct a clean image from a noisy image. Existing image denoising methods are mainly divided into model-based methods and deep learning-based methods. The main idea of ​​the model-based method is to transform the image denoising problem into an optimization problem by introducing different prior information. Commonly used prior information includes total variation constraints and sparse prior constraints. The model-based method has strong interpretability, but its performance depends largely on manual parameter design and has poor flexibility. In recent years, with the rise of deep learning, denoising models based on deep neural networks have received widespread attention, such as DnCNN and CNLNet. The core idea of ​​these networks is to train neural networks through end-to-end strategies to map noisy images to clean images. Compared with model-based methods, image denoising algorithms based on deep learning have achieved great performance improvements. However, existing deep denoising networks are mainly artificially constructed and have poor interpretability.

[0004] In order to improve the interpretability of deep neural networks, researchers have proposed a deep neural network based on deep expansion, which originated from a learnable iterative shrinkage soft threshold algorithm (LISTA). LISTA expands the iterative solution of the soft threshold algorithm to be solved for the sparse model into a deep network, where each layer in the deep network corresponds to each iteration. Based on the LISTA idea, deep expansion has also been applied to image denoising, such as CSCNet, GroupSC, etc. Existing experimental results show that while these interpretable networks have similar performance to the classic deep denoising networks, they also have fewer parameters and stronger interpretability. However, these interpretable networks do not well characterize the inter-spectral features of the image itself. Therefore, the present invention designs a multispectral image denoising network based on a deep 3D convolutional sparse coding model. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a multispectral image denoising algorithm based on deep 3D convolutional sparse coding, which can improve the interpretability of deep neural networks.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] The present invention is a multispectral image denoising algorithm based on deep 3D convolutional sparse coding, comprising the following steps:

[0008] Step 1: According to the inter-spectral information of the multispectral image, the convolutional sparse coding model is expanded into a 3D form to build a 3D-CSC mathematical model;

[0009] Step 2, using an iterative shrinkage soft threshold algorithm to iteratively solve the constructed 3D-CSC model;

[0010] Step 3, build a deep network: According to the idea of ​​deep expansion, the iterative solution form of 3D-CSC in step 2 is expanded into the corresponding deep network 3D-CSCNet;

[0011] Step 4, construct the data set and train the network;

[0012] Step 5: Test the performance of the network.

[0013] A further improvement of the present invention is that the specific operation of step 1 is:

[0014] Step 1.1, the convolutional sparse coding model represents X as a linear combination of m convolutional features. According to the spatial structure of the multispectral image, it is expanded to 3D form:

[0015]

[0016] Where X represents the clean multispectral image, represents m filters of size h×h×h, * represents three-dimensional convolution, represents the i-th filter d i The corresponding convolution features;

[0017] Step 1.2, Convolutional Sparse Coding Assumption is sparse, that is, contains a small number of non-zero elements, and is obtained through the following optimization problem:

[0018]

[0019] where ||·||1 represents l 1 norm, and λ is the regularization parameter.

[0020] A further improvement of the present invention is that the specific process of iterative solution in step 2 is: the iterative update formula of formula (2) is:

[0021]

[0022] in A is the corresponding sparse code, the superscript t represents the tth (t=0, 1, 2, ..., K-1) iteration value, T is the transposed symbol, L is the Laplace constant, λ t is the regularization parameter corresponding to the tth iteration, is a soft threshold operator, defined as:

[0023]

[0024] The further improvement of the present invention is that: Step 3 specifically comprises the following steps: by introducing the variable B=D converts formula (3) into:

[0025]

[0026] Expand formula (5) into a deep neural network, where the tth layer of the neural network corresponds to the tth iteration of the formula, It plays the role of activation function in the network, while C and B are implemented through convolution and are continuously learned and optimized during the network training process.

[0027] The further improvement of the present invention is that the specific steps of step 4 are as follows:

[0028] Step 4.1, construct the data set;

[0029] Step 4.2, initialization parameters;

[0030] Step 4.3, train 3D-CSCNet;

[0031] Step 4.4, reconstruct the output image.

[0032] The beneficial effects of the present invention are as follows: the present invention uses 3D convolution, takes into account the inter-spectral characteristics of multispectral images, expands the original convolutional sparse coding to a 3D form, and builds a corresponding 3D denoising deep network, so that the network built by the present invention has a certain degree of interpretability, and also achieves better denoising effect in processing multispectral image problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is the network structure diagram of 3D-CSCNet. DETAILED DESCRIPTION

[0034] In order to more clearly illustrate the technical solution of the present invention, the technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0035] The present invention is a multispectral image denoising algorithm based on deep 3D convolutional sparse coding. Considering that multispectral images have a certain spatial structure, the present invention proposes a convolutional sparse coding model based on a 3D convolutional dictionary and solves it using an alternating iterative optimization algorithm. Then, the iterative optimization solution is converted into 3D-CSCNet using a deep expansion strategy. The corresponding network diagram is shown in the figure below. Figure 1 shown.

[0036] A common noisy image can be expressed as:

[0037] Y=X+N (1) in, represents a clean multispectral image, For noise, the present invention only considers Gaussian noise. represents the noise image, M×N represents the spatial size of the image, and C represents the number of bands of the image. Image denoising is how to recover X from Y. The denoising method of the present invention specifically includes the following steps:

[0038] Step 1: According to the inter-spectral information of the multispectral image, the convolutional sparse coding model is expanded into a 3D form to build a 3D-CSC mathematical model;

[0039] Step 2, using the iterative shrinkage soft threshold algorithm (ISTA) to iteratively solve the constructed 3D-CSC model;

[0040] Step 3, build a deep network: According to the idea of ​​deep expansion, the iterative solution form of 3D-CSC in step 2 is expanded into the corresponding deep network;

[0041] Step 4, construct the data set and train the network;

[0042] Step 5: Test the performance of the network.

[0043] In step 1, the convolutional sparse coding model represents X as a linear combination of m convolutional features. According to the spatial structure of the multispectral image, it is expanded to 3D form:

[0044]

[0045] in represents m filters of size h×h×h, * represents three-dimensional convolution, represents the i-th filter d i Corresponding convolutional features. Convolutional sparse coding assumption is sparse, that is, contains a small number of non-zero elements, and can be obtained through the following optimization problem:

[0046]

[0047] where ||·|| 1 Representative 1 norm, and λ is the regularization parameter.

[0048] In step 2, the ISTA algorithm is applied to iteratively solve the original problem in step 1 to obtain the sparse representation vector of each layer;

[0049] The solution process is:

[0050] Formula (3) The iterative update formula of ISTA is:

[0051]

[0052] in A is the corresponding sparse code, t represents the tth (t=0, 1, 2, ..., K-1) iteration value, is the soft threshold operator, T is the transposed sign, L is the Laplace constant, λ t is the regularization parameter corresponding to the tth iteration, defined as:

[0053]

[0054] In step 3, the deep expansion strategy is adopted to expand the 3D-CSC model into 3D-CSCNet. The specific method is as follows: the deep expansion idea is adopted to expand formula (4) into a deep neural network. First, by introducing the variable B=D converts formula (4) into:

[0055]

[0056] Then, formula (6) is expanded into a deep neural network, where the tth layer of the neural network corresponds to the tth iteration of the formula, that is, each iterative solution corresponds to each layer in the neural network, It plays the role of activation function in the network, while C and B are implemented through convolution and are continuously learned and optimized during the network training process.

[0057] Step 4 specifically includes: constructing a data set, initializing parameters, training 3D-CSCNet, and reconstructing the output image.

[0058] Constructing a data set: The network training of the present invention uses the data set CAVE, which has a total of 32 scenes, each of which has 31 grayscale images of size 512×512, corresponding to the 31 bands of this scene. Therefore, the present invention synthesizes these 31 grayscale images in sequence into a three-dimensional tensor of size 512×512×31. Before reading in the data, the present invention cuts these three-dimensional tensors according to the size of 32×32, and the step size of the cutting interval is 31 pixel values, and saves these synthesized tensors into mat files for subsequent data import. Finally, these cut data are divided into training set and test set in a ratio of 6:4 after normalization, and the grayscale range of the data after normalization is [0, 1].

[0059] Initialization parameters: Input a multispectral image Y, randomly initialize the 3D dictionary D, and the regularization parameter λ;

[0060] Training 3D-CSCNet: The present invention simulates a noisy image by manually adding Gaussian noise. The network parameters of the entire 3D-CSCNet are Θ = {C, B, D, λ}, which can be trained by the network in an end-to-end supervised learning form. The loss function used in the present invention is the mean square error (MSE) loss function, which is defined as

[0061]

[0062] Where M is the number of training samples, f Θ represents a 3D-CSCNet with parameters Θ, Y j represents the jth training sample, f Θ (Y j ) is the estimated value of 3D-CSCNet. The present invention implements 3D-CSCNet on PyTorch, and performs training and testing on RTX2070Super GPU. The number of samples captured in one training (Batch Size) and the number of training rounds (epoch) are set to 8 and 200, the convolution kernel size is 7×7, and the Adam optimizer is used. When training 3D-CSCNet, the heap expansion number K is set. When solving the sparse coding corresponding to the t (t=0, 1, 2, ..., K) layer, formula (6) is applied until the set iteration ends.

[0063] Reconstruct the output image: from the final Ak ,according to Reconstruct the denoised image.

[0064] The specific method of testing the model in step 5 is: evaluate the network performance under three different noise levels of σ = {0.1, 0.2, 0.3}, and the evaluation index is the Mean Peak Signal-to-Noise Ratio (MPSNR). The comparison methods include LRMR, LRTV, LLRGTV and CSCNet. The experimental results are shown in Table 1:

[0065] Table 1 MPSNR results of different methods on the CAVE test set

[0066] algorithm σ=25 σ=50 σ=75 LRMR 33.48 28.44 25.00 LRTV 36.21 32.03 29.41 LLRGTV 34.40 29.71 26.82 CSCNet 35.36 32.50 30.21 3D-CSCNet 37.40 33.53 25.69

[0067] As can be seen from Table 1, the 3D-CSCNet of the present invention has a good denoising performance. It is worth noting that compared with CSCNet, 3D-CSCNet performs better on the CAVE dataset under the common noise with σ values ​​of 25 and 50, which also proves the effectiveness of the model.

Claims

1. A multispectral image denoising algorithm based on deep 3D convolutional sparse coding, characterized by: The steps include: Step 1: According to the inter-spectral information of the multispectral image, the convolutional sparse coding model is expanded into a 3D form to build a 3D-CSC mathematical model; Step 1.1, the convolutional sparse coding model represents the input signal X as a linear combination of m convolutional features. According to the spatial structure of the multispectral image, it is expanded to a 3D form: Where X represents the clean multispectral image, represents m filters of size h×h×h, * represents three-dimensional convolution, represents the i-th filter d i The corresponding convolution features; Step 1.2, Convolutional Sparse Coding Assumption is sparse, that is, contains a small number of non-zero elements, and is obtained through the following optimization problem: Where ‖·‖1 represents the l1 norm and λ is the regularization parameter; Step 2, using an iterative shrinkage soft threshold algorithm to iteratively solve the constructed 3D-CSC model; The specific process of iterative solution in step 2 is: The iterative update formula of formula (2) is: in A is the corresponding sparse code, the superscript t represents the tth (t = 0, 1, 2, ..., K-1) iteration value, T is the transposed symbol, L is the Laplace constant, λ t is the regularization parameter corresponding to the tth iteration, is a soft threshold operator, defined as: Step 3, build a deep network: According to the idea of ​​deep expansion, the iterative solution form of 3D-CSC in step 2 is expanded into the corresponding deep network 3D-CSCNet; Step 4, construct the data set and train the network; Step 5: Test the performance of the network.

2. The multispectral image denoising algorithm based on deep 3D convolutional sparse coding according to claim 1, characterized in that: Step 3: By introducing variables B=D converts formula (3) into: By introducing variables B=D converts formula (3) into: Expand formula (5) into a deep neural network, where the tth layer of the neural network corresponds to the tth iteration of the formula, It plays the role of activation function in the network, while C and B are implemented through convolution and are continuously learned and optimized during the network training process.

3. The multispectral image denoising algorithm based on deep 3D convolutional sparse coding according to claim 1, characterized in that: Step 4 The specific steps are as follows: Step 4.1, construct the data set: adopt the data set CAVE, and synthesize the images in each scene into a three-dimensional tensor, and cut it as required, and divide it into training set and test set in a ratio of 6:4 after normalization; Step 4.2, initialization parameters; Step 4.3, train 3D-CSCNet; Step 4.4, reconstruct the output image.

Images