A satellite hyperspectral data strip noise removal method based on a U-net network

Abstract

The application provides a satellite hyperspectral data strip noise removal method based on a U-net network, which comprises the following steps: step one, preprocessing of the satellite hyperspectral data to be processed; step two, slice processing of the satellite hyperspectral data to be processed; step three, inputting the slice data into a strip noise removal model; and step four, splicing of the slice data after strip noise removal. The application can inhibit noise while retaining the spatial-spectral information of the hyperspectral data. In order to retain the spectral information of the image, a joint loss function is used, and a spectral angle loss function and a root mean square error loss function are superimposed; meanwhile, when the network data set is prepared, high-quality airborne hyperspectral data is selected, and all random noise is added, so that the noise mode of the satellite-borne hyperspectral data is simulated as much as possible, and the precision of the network is improved. Finally, the denoising network model can remove the strip noise of the satellite hyperspectral image with different band numbers, and is more suitable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral remote sensing data processing, in particular to a satellite hyperspectral data strip noise removal method based on a U-net network, belonging to the field of hyperspectral remote sensing. It is suitable for all satellite hyperspectral remote sensing data strip noise removal. BACKGROUND

[0002] Hyperspectral remote sensing data has hundreds of spectral channels, containing rich spectral information and spatial information, and has been widely applied to agricultural monitoring, water quality inversion, mine environmental investigation, resource exploration and other fields. However, due to the instability of hyperspectral sensors and the influence of atmospheric environment and other factors, hyperspectral images are often disturbed by various noises. Unlike other noises, strip noise has obvious geometric anisotropy and anisotropy characteristics. The formation of this noise is closely related to the imaging process of the push-broom imaging spectrometer. Factors such as uniform push-scan mode of pixels and asymmetric width of thin slits at the entrance of the imaging spectrometer can cause strip noise in the image. This noise is usually more obvious in the short-wave infrared spectral interval of domestic hyperspectral satellites, such as resource series and high-resolution five series hyperspectral satellite data, which seriously restricts its application, such as hyperspectral remote sensing mineral identification, because many mineral diagnostic absorption features are in the short-wave infrared spectral interval. The existence of strip noise reduces the mineral identification accuracy and detection rate. Therefore, an effective hyperspectral strip noise removal method is of great significance for subsequent data analysis and application.

[0003] In recent decades, many scholars have proposed different methods for removing strip noise from satellite hyperspectral images. Some original methods directly extend image denoising algorithms to hyperspectral denoising, such as block matching and three-dimensional filtering-based methods, weighted kernel norm minimization, etc. However, due to the data characteristics of hyperspectral images, this traditional wave-by-wave denoising algorithm ignores the correlation between spectra, resulting in spectral distortion. Considering the spatial-spectral information of hyperspectral images and denoising can largely preserve spectral characteristics. For example: wavelet transformation, sparse-based methods, low-rank-based methods, non-local similarity-based methods, and total variation-based methods. This kind of method is greatly affected by pre-selected prior knowledge and regularization parameters, to some extent, limiting the spatial feature representation of the spectrum, and when facing complex noise, its time cost and computational cost are high.

[0004] In contrast, the end-to-end method in deep learning is applied to hyperspectral image strip noise reduction, which is a new idea in recent years. The deep learning method has strong feature learning and modeling ability, and this method has good technical advantages in solving hyperspectral strip noise reduction. However, in practical application, when using convolutional neural network for wave band by wave band denoising, spectral distortion is inevitable. However, when using multi-wave band input, with the increase of the number of input wave bands, the calculation cost increases, and at the same time, information redundancy may also cause the problem of poor denoising effect. At the same time, using deep learning technology to remove strip noise also has the technical difficulty of lacking training samples, and it is difficult to find high-quality satellite hyperspectral data and the corresponding strip noise data pair.

[0005] In order to solve the above problems, a satellite hyperspectral strip noise removal method based on U-Net network is proposed. At the same time, according to the characteristics of satellite hyperspectral remote sensing data and the characteristics of strip noise, an attention module is added to the network, which is composed of channel attention and spatial attention; In order to obtain strip noise while not affecting other information, a residual module is introduced into the network. At the same time, in order to make representative training samples and verify the denoising effect, high-quality airborne hyperspectral data is used to make network training samples. And the denoising performance of different wave band number of training data set in network model training is compared and analyzed, and the optimal image input wave band number is selected. SUMMARY

[0006] The purpose of the present application is to provide a satellite hyperspectral data strip noise removal method based on U-net network, which can realize effective denoising while preserving hyperspectral spatial-spectral information, and can be applied to existing domestic satellite hyperspectral data, and improve the application efficiency of domestic hyperspectral data.

[0007] In order to realize the above technical method, the satellite hyperspectral data strip noise removal method based on U-net network of the present application has the overall technical scheme as shown in the accompanying Figure 1 The specific steps of the method are as follows:

[0008] Step 1: Pretreatment of satellite hyperspectral data to be processed;

[0009] Remove the water vapor absorption band of the satellite hyperspectral DN value data to be processed near 1400nm and 1900nm.

[0010] Step 2: Slice processing of the satellite hyperspectral data to be processed;

[0011] The data obtained in step one is sliced to obtain an image block with a size of 200*200*9, wherein 200*200 is the column and row of the image block, and 9 is the number of bands of the image block. When the number of bands of the satellite hyperspectral data is not an integer multiple of 9, the last 9 bands are not enough, and 9-band data is obtained by supplementing from the front; when the column or row of the satellite hyperspectral data is not an integer multiple of 200, the last column or row is not enough, and a 200*200 image block is obtained by supplementing from the front.

[0012] Step three: slice data input strip noise removal model;

[0013] The slice data obtained in step two is input into the strip noise removal model to obtain the slice after strip noise removal.

[0014] Step four: splicing of slice data after strip noise removal;

[0015] The slice data after strip noise removal obtained in step three is spliced to obtain the satellite hyperspectral data after strip noise removal. When the number of bands of the preprocessed data obtained in step one is not an integer multiple of 9 or the length or width is not an integer multiple of 200, the overlapping data needs to be removed when splicing the slice after strip noise removal obtained in step three, and then the data splicing is completed.

[0016] The strip noise removal model in the application has the following steps:

[0017] Step one: construction of strip noise removal model;

[0018] The strip noise removal model is based on the U-Net network as the overall framework, which is composed of 4 encoding layers, 3 intermediate layers and 4 decoding layers. Figure 2 The encoding layer is composed of a 3D convolution layer, a batch normalization layer, an attention mechanism and a residual block. Figure 3 The intermediate layer is composed of a 3D convolution layer and a batch normalization layer. Figure 4 The decoding layer is composed of an up-sampling layer, a batch normalization layer, an attention mechanism and a residual block. Figure 5); the first encoding layer input is 200*200*9 image data, and 32 feature maps are output, with a size of 98*98*9, the second encoding layer outputs 64 feature maps, with a size of 94*94*9, the third encoding layer outputs 128 feature maps, with a size of 90*90*9, and the fourth encoding layer outputs 256 feature maps, with a size of 86*86*9; the intermediate layers all output 256 feature maps, with sizes of 86*86*9 respectively; the first decoding layer outputs 128 feature maps, with a size of 90*90*9, the second decoding layer outputs 64 feature maps, with a size of 94*94*9, the third decoding layer outputs 32 feature maps, with a size of 98*98*9, and the fourth decoding layer outputs an image with a size of 200*200*9. The first encoding layer and the fourth decoding layer, the second encoding layer and the third decoding layer, the third encoding layer and the second decoding layer, and the fourth encoding layer and the first decoding layer are connected by jump connection. The attention mechanism is composed of spatial attention mechanism and channel attention mechanism. The channel attention is composed of a max pooling layer, an average pooling layer, a multilayer perceptron, and a sigmoid function Figure 6 ). The spatial attention model is composed of a pooling layer, a convolutional layer, and a sigmoid activation function Figure 7 ). The residual block is composed of two groups of convolutional and batch normalization layers, connected by jump connection between the input and the output Figure 8

[0019] Step two: model dataset preparation;

[0020] The airborne hyperspectral data is removed in the waveband near 1400nm and 1900nm, and then the airborne hyperspectral image is cut by a sliding window with a size of 200*200 and a step of 200, that is, no repeated area cutting, to obtain an image block with a size of 200*200*90, where 90 is the number of bands.

[0021] Step three: model dataset preparation;

[0022] From the image block obtained in step two, 9-band image blocks of 200*200*9 are sequentially obtained. The acquisition method is to first obtain the 1st to 9th bands of 200*200*90, then obtain the 2nd to 10th bands, and so on, until the 82nd to 90th bands are obtained. Then these 200*200*9 image blocks are randomly shuffled. In the 200*200*9 image blocks, 1-9 bands are randomly selected, random strip noise is added to the selected 1-9 band data, and Gaussian noise is also added, to finally obtain a dataset containing strip noise. The obtained dataset is randomly shuffled, and then a 4:1 ratio is used to divide the dataset into a training dataset and a test dataset.

[0023] Step four: model training and testing;

[0024] ​The training data set obtained from step three is input into the network model constructed in step one for network training; the corresponding noise-free data set of the training data set is used for back propagation. The number of network iterations is 100 times. The root mean square error and the loss function of the spectral angle are used, which can improve the spectral fidelity. The loss function is represented as:

[0025] loss = loss MSE + loss spectral

[0026] Wherein, loss MSE represents the root mean square error loss function, loss spectral represents the spectral angle loss function:

[0027]

[0028] Wherein, n represents the total number of samples participating in regression, f(x i ) and y i represent the predicted value and the true value of the i-th sample, respectively;

[0029]

[0030] Wherein, y represents the value of network input, and represents the value of network output. After each iteration, the test data set is used to test the denoising performance, and the test indicators are PSNR, SSIM and loss function:

[0031] Wherein, the mathematical expression of PSNR is:

[0032]

[0033] Wherein, Max pixel represents the maximum pixel value in the image, and MSE represents the mean square error before and after denoising. The larger the PSNR, the better the image quality;

[0034] The expression of SSIM is:

[0035]

[0036] Wherein, l(x,y), c(x,y), s(x,y) represent the brightness, contrast and structure between x and y, respectively, μ x is the average value of x, μ y is the average value of y, is the variance of x, is the variance of y, σ xy is the covariance of x and y; C1=(K1L) 2 , C2=(K2L) 2is a constant used to maintain stability, L is the dynamic range of the pixel value, K1=0.01, K2=0.03. The value of SSIM ranges from [0, 1], and the larger the value is, the better the quality of the image is.

[0037] Step five: model saving;

[0038] 100 model parameters obtained from step four are selected, the PSNR, SSIM and loss function results of which are optimal, and the set of model parameters is saved, and finally a strip noise removal model is obtained.

[0039] The application has the advantages and effects that: a satellite hyperspectral data strip noise removal method based on a U-net network can suppress noise while retaining spatial-spectral information of the hyperspectral data. In order to retain the spectral information of the image, a joint loss function is used to superimpose a spectral angle loss function and a root mean square error loss function; meanwhile, when the network data set is made, high-quality airborne hyperspectral data is selected, and all random noise is added to simulate the noise mode of the satellite hyperspectral data as much as possible, thereby improving the accuracy of the network. Finally, the data input of the proposed denoising network model is not a full-spectral image, but an image of 9 wavebands, which makes it have universality in application and can remove strip noise from satellite hyperspectral images with different waveband numbers, and has wider applicability. The network can remove strip noise from domestic satellite hyperspectral data, and improves the application efficiency of the satellite hyperspectral data. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The flowchart of the application is shown.

[0041] Figure 2 The architecture of the strip noise removal model is shown.

[0042] Figure 3 The composition diagram of the encoding layer is shown.

[0043] Figure 4 The composition diagram of the intermediate layer is shown.

[0044] Figure 5 The composition diagram of the decoding layer is shown.

[0045] Figure 6 The composition diagram of the channel attention is shown.

[0046] Figure 7 The composition diagram of the spatial attention is shown.

[0047] Figure 8 The composition diagram of the residual block is shown.

[0048] Figure 9a The image before strip noise removal of the ZY01-02D satellite hyperspectral 147 waveband image block is shown.

[0049] Figure 9b ZY01-02D satellite hyperspectral 147 band image after strip noise removal.

[0050] Figure 10a ZY01-02D satellite hyperspectral 147 band image before strip noise removal.

[0051] Figure 10b ZY01-02D satellite hyperspectral 147 band image after strip noise removal. DETAILED DESCRIPTION

[0052] Figure 1 The implementation process of the method is as follows: in order to better illustrate the implementation and effect of the method, a ZY01-02D satellite hyperspectral data is taken as an example to remove strip noise, and the specific steps are as follows:

[0053] Step one, preprocessing of satellite hyperspectral data to be processed:

[0054] The experimental data is a ZY01-02D satellite hyperspectral data of Liuyuan area in Gansu obtained on September 23, 2020, with an area of about 3600 square kilometers, which is the original DN value data, and the image size is 1999*2051*166, wherein 1999 is the number of columns, 2051 is the number of rows, and 166 is the number of bands. Remove water vapor absorption bands 93-100 and 121-131 near 1400nm and 1900nm to obtain an image size of 1999*2051*147.

[0055] Step two: slice processing of satellite hyperspectral data to be processed

[0056] The data obtained in step one is processed by slicing, and the slicing starts from the first column and the first row of the image. The first slice data size is the first-200 columns and the first-200 rows, and the band is the first-9 bands. In this way, a 200*200*9 image block is obtained, wherein 200*200 is the column and row of the image block, and 9 is the number of bands of the image block. Then the second slice data is the 201-400 columns and the 201-400 rows, and the band is the first-9 bands. In this way, the row and column data are sequentially obtained. When the first-9 band slice data is obtained, the 10-18 band slice data is obtained, that is, the band dimension is sequentially obtained. Finally, 1870 image blocks are obtained.

[0057] Step three: inputting slice data into strip noise removal model

[0058] The 1870 200*200*9 slice data obtained in step two are sequentially input into the strip noise removal model established by the application, the model automatically removes image strips, and finally the slice data after strip noise removal is obtained. Figure 9a , Figure 9b ).

[0059] Step four: splicing of slice data after strip noise removal

[0060] The 1870 slice data after strip noise removal obtained in step three are spliced, the splicing is performed according to the row-column spatial relationship of the slice and the band order relationship, and the satellite hyperspectral data after strip noise removal is obtained. Figure 10a , Figure 10b ).

Claims

1. A satellite hyperspectral data strip noise removal method based on a U-net network, characterized by: The specific steps are as follows: Step one: preprocessing of satellite hyperspectral data to be processed; Remove the water vapor absorption band near 1400nm and 1900nm from the DN value data of the satellite hyperspectral data to be processed; Step two: slice processing of the satellite hyperspectral data to be processed; The data obtained in step one is subjected to slice processing to obtain an image block with a size of 200*200*9, wherein 200*200 is the column and row of the image block, and 9 is the number of bands of the image block; when the number of bands of the satellite hyperspectral data is not an integer multiple of 9, the last 9 bands are not enough, and 9 bands of data are obtained by supplementing from the front; when the column or row of the satellite hyperspectral data is not an integer multiple of 200, the last column or row is not enough, and a 200*200 size image block is obtained by supplementing from the front; Step three: inputting the slice data into the strip noise removal model; The slice data obtained in step two is inputted into the strip noise removal model to obtain the slice after strip noise removal; Step four: splicing of the slice data after strip noise removal; The slice data after strip noise removal obtained in step three is spliced to obtain the satellite hyperspectral data after strip noise removal; when the number of bands of the preprocessed data obtained in step one is not an integer multiple of 9 or the length or width is not an integer multiple of 200, the overlapping data needs to be removed when splicing the slice after strip noise removal obtained in step three, and then the data splicing is completed; The strip noise removal model is constructed as follows: The strip noise removal model is based on the U-Net network as the overall framework, which is composed of 4 encoding layers, 3 intermediate layers and 4 decoding layers; the encoding layer is composed of a 3D convolution layer, a batch normalization layer, an attention mechanism and a residual block; the intermediate layer is composed of a 3D convolution layer and a batch normalization layer; the decoding layer is composed of an up-sampling layer, a batch normalization layer, an attention mechanism and a residual block.

2. The method of claim 1, wherein the U-net network is used to remove strip noise from the satellite hyperspectral data. The input of the first encoding layer is the image data of 200*200*9, and the output is 32 feature maps with a size of 98*98*9; the output of the second encoding layer is 64 feature maps with a size of 94*94*9; the output of the third encoding layer is 128 feature maps with a size of 90*90*9; the output of the fourth encoding layer is 256 feature maps with a size of 86*86*9; the intermediate layer outputs 256 feature maps with a size of 86*86*9.

3. The method of claim 1, wherein the U-net network is used to remove strip noise from the satellite hyperspectral data. The first decoding layer outputs 128 feature maps with a size of 90*90*9; the second decoding layer outputs 64 feature maps with a size of 94*94*9; the third decoding layer outputs 32 feature maps with a size of 98*98*9; the fourth decoding layer outputs an image with a size of 200*200*9.

4. The satellite hyperspectral data strip noise removal method based on the U-net network according to claim 2 or 3, characterized in that: The first encoding layer and the fourth decoding layer, the second encoding layer and the third decoding layer, the third encoding layer and the second decoding layer, and the fourth encoding layer and the first decoding layer are connected by jump connection; The attention mechanism is composed of spatial attention mechanism and channel attention mechanism; the channel attention is composed of max pooling layer, average pooling layer, multi-layer perceptron and Sigmoid function; the spatial attention model is composed of pooling layer, convolution layer and Sigmoid activation function; the residual block is composed of two groups of convolution and batch normalization layer, and is connected by jump connection between input and output.

5. The method of claim 1, wherein the U-net network is used to remove strip noise from the satellite hyperspectral data. The hyperspectral data in the waveband near 1400 nm and 1900 nm is removed, then the hyperspectral image is cut in a sliding window manner, the window size is 200*200, the step distance is 200, that is, no repeated area is cut, and a 200*200*90 image block is obtained, wherein 90 is the number of wavebands.

6. The method of claim 1 or 5, wherein the U-net network is characterized in that: 9-waveband image blocks of 200*200*9 are sequentially obtained from the image block, the obtaining manner is that 1st to 9th wavebands of 200*200*90 are first obtained, then 2nd to 10th wavebands are obtained, and the like is sequentially repeated until 82nd to 90th wavebands are obtained; the image blocks of 200*200*9 are randomly shuffled; 1-9 wavebands are randomly selected from the image blocks of 200*200*9, random strip noise is added to the 1-9 wavebands, and Gaussian noise is added, and finally a data set containing strip noise is obtained; the obtained data set is randomly shuffled, and then is allocated into a training data set and a test data set according to a ratio of 4:

1.

7. The method of claim 6, wherein the U-net network is used to remove strip noise from the satellite hyperspectral data. The obtained training data set is input into the network model constructed, and the network is trained; The corresponding noise-free data set of the training data set is used for back propagation; the number of network iterations is 100 times; the loss function of root mean square error and spectral angle is used, the fidelity of the spectrum is improved, and the loss function is expressed as: loss = loss MSE + los sspectral where loss MSE represents the mean square error loss function, loss spectral represents the spectral angle loss function: where n represents the total number of samples participating in regression, f(x i ) and y i represent the predicted value and the true value of the i-th sample, respectively; where y x represent the values of the network input, represent the values of the network output; after each iteration, the test dataset is used to test the denoising performance, and the test indicators are PSNR, SSIM, and the loss function.

8. The method of claim 7, wherein the U-net network is used to remove strip noise from the satellite hyperspectral data. The mathematical expression of PSNR is: where Max pixel represents the maximum pixel value in the image, and MSE represents the mean square error before and after denoising; the larger the PSNR, the better the image quality.

9. The method of claim 7, wherein the U-net network is used to remove strip noise from the satellite hyperspectral data. The expression of SSIM is: where l(x, y), c(x, y), s(x, y) represent the luminance, contrast and structure between x and y, respectively, μ x is the mean of x, μ y is the mean of y, is the variance of x, is the variance of y, σ xy is the covariance of x and y; C1= (K1L) 2 , C2= (K2L) 2 is a constant used to maintain stability, L is the dynamic range of pixel values, K1=0.01, K2=0.03; the value of SSIM ranges from [0, 1], the larger the value, the better the quality of the image.

Images